Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

convert fixed length file to CSV

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting convert fixed length file to CSV
# 8  
Old 08-06-2008
Question does this appear to reflect rules corectly?
What follows is (a) input file, (b) working model for script, (c) execution of script. So, does it properly classify the records into one of the three type? If so, then all that is left is to break the lines into pieces to write out in a format.

Code:
> cat file1
4002000W1ABCDABCD7821 123456789071001080600W1VUF 34216002902291LSN 1230-1630 +000000+000400+00000000+00009164+000400+0000916450125+00000000
4002000W1JKLMABCD6022 123456789071001080600W1VUF 34360001302894UOM +000000+000000+00000000+00014514+000000+0001451413125+00000000
4002000W1JKLMABCD6022 123456789071001080600W1VUF 34360001302791RFN123456 +000000+008000+00000000+00223280+008000+0022328050125+00000000
4002000W1JKLMABCD6022 123456789071001080600W1VUFC34360001303067UJN +000000+000000+00000000+00015213+000000+0001521313125+00000000

> cat do_file
#! /usr/bin/bash

while read zf
  do
#  echo "$zf"
  fldcnt=$(echo "$zf" | cut -c50- | wc -w)
  fld3cnt=$(echo "$zf" | cut -c50- | cut -d" " -f1 | wc -c)
  flag=$(echo "$fldcnt""$fld3cnt")
  echo "$flag"
# now do the file formats
  if [ "$flag" = "318" ]
     then
     echo "a"
  fi
  if [ "$flag" = "218" ]
     then
     echo "b"
  fi
  if [ "$flag" = "224" ]
     then
     echo "c"
  fi

done <file1

> do_file
318
a
218
b
224
c
218
b

# 9  
Old 08-06-2008
Question About how large, or how many records?
How big is the input file? It will have a little bearing on how to write the rest of the coding.
# 10  
Old 08-06-2008
Tools OK, done by lots of echo & cut commands
The following appears to work, although it might be slow for large record counts. Fields f08 and f21 had some little tweaks to make the work properly.

Code:
> cat do_file
#! /usr/bin/bash

while read zf
  do
#  echo "$zf"
  fcnt=$(echo "$zf" | cut -c50- | wc -w)
  f3cnt=$(echo "$zf" | cut -c50- | cut -d" " -f1 | wc -c)
  flag=$(echo "$fcnt""$f3cnt")
#  echo "$flag"
# common layouts
  f01=$(echo "$zf" | cut -c1-7)
  f02=$(echo "$zf" | cut -c8-13)
  f03=$(echo "$zf" | cut -c14-17)
  f04=$(echo "$zf" | cut -c18-21)
  f05=$(echo "$zf" | cut -c23-31)
  f06=$(echo "$zf" | cut -c32-37)
  f07=$(echo "$zf" | cut -c38-43)
  f08=$(echo "$zf" | cut -c44-49 | tr -d " ")
  f09=$(echo "$zf" | cut -c50-63)
  f10=$(echo "$zf" | cut -c64-66)

  fpre=$(echo "$f01" "$f02" "$f03" "$f04" "$f05" "$f06" "$f07" "$f08" "$f09" "$f10")

# now do the file formats
  if [ "$flag" = "318" ]
     then
     f21=$(echo "$zf" | cut -c68- | cut -d" " -f1)
     f22=$(echo "$zf" | cut -c68- | cut -d" " -f2)
     f31=$(echo "$f22" | cut -d"+" -f2)
     f32=$(echo "$f22" | cut -d"+" -f3)
     f33=$(echo "$f22" | cut -d"+" -f4)
     f34=$(echo "$f22" | cut -d"+" -f5)
     f35=$(echo "$f22" | cut -d"+" -f6)
     f36=$(echo "$f22" | cut -d"+" -f7)
     f37=$(echo "$f22" | cut -d"+" -f8)

     echo "a" "$fpre" "$f21" "+$f31" "+$f32" "+$f33" "+$f34" "+$f35" "+$f36" "+$f37"
  fi
  if [ "$flag" = "218" ]
     then
     f22=$(echo "$zf" | cut -c68- | cut -d" " -f2)
     f31=$(echo "$f22" | cut -d"+" -f2)
     f32=$(echo "$f22" | cut -d"+" -f3)
     f33=$(echo "$f22" | cut -d"+" -f4)
     f34=$(echo "$f22" | cut -d"+" -f5)
     f35=$(echo "$f22" | cut -d"+" -f6)
     f36=$(echo "$f22" | cut -d"+" -f7)
     f37=$(echo "$f22" | cut -d"+" -f8)
     echo "b" "$fpre" "+$f31" "+$f32" "+$f33" "+$f34" "+$f35" "+$f36" "+$f37"
  fi
  if [ "$flag" = "224" ]
     then
     f21=$(echo "$zf" | cut -c67-72)
     f22=$(echo "$zf" | cut -c68- | cut -d" " -f2)
     f31=$(echo "$f22" | cut -d"+" -f2)
     f32=$(echo "$f22" | cut -d"+" -f3)
     f33=$(echo "$f22" | cut -d"+" -f4)
     f34=$(echo "$f22" | cut -d"+" -f5)
     f35=$(echo "$f22" | cut -d"+" -f6)
     f36=$(echo "$f22" | cut -d"+" -f7)
     f37=$(echo "$f22" | cut -d"+" -f8)
     echo "c" "$fpre" "$f21" "+$f31" "+$f32" "+$f33" "+$f34" "+$f35" "+$f36" "+$f37"
  fi
# output data
done <file1

and the output

Code:
> do_file
4002000 W1ABCD ABCD 7821 123456789 071001 080600 W1VUF 34216002902291 LSN 1230-1630 +000000 +000400 +00000000 +00009164 +000400 +0000916450125 +00000000
4002000 W1JKLM ABCD 6022 123456789 071001 080600 W1VUF 34360001302894 UOM +000000 +000000 +00000000 +00014514 +000000 +0001451413125 +00000000
4002000 W1JKLM ABCD 6022 123456789 071001 080600 W1VUF 34360001302791 RFN 123456 +000000 +008000 +00000000 +00223280 +008000 +0022328050125 +00000000
4002000 W1JKLM ABCD 6022 123456789 071001 080600 W1VUFC 34360001303067 UJN +000000 +000000 +00000000 +00015213 +000000 +0001521313125 +00000000

# 11  
Old 08-07-2008
Joeyg

The output is exactly what im looking for. How do I run this?
# 12  
Old 08-07-2008
Tools
If you copy/paste that do_file script to unix, do the chmod +x to make executable. It can be run.
The only other change might be in the filename, as near the very end I have a "<file1" -- substitute your filename for file1.
# 13  
Old 08-12-2008
joejy

I ran the script. Im getting an error message as follows: Fork Function failed, not enough memory. Any suggestion would be greatly appreciated.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert variable length record to fixed length

Hi Team, I have an issue to split the file which is having special chracter(German Char) using awk command. I have a different length records in a file. I am separating the files based on the length using awk command. The command is working fine if the record is not having any... (7 Replies)
Discussion started by: Anthuvan
7 Replies

2. UNIX for Dummies Questions & Answers

Length of a fixed width file

I have a fixed width file of length 53. when is try to get the lengh of the record of that file i get 2 different answers. awk '{print length;exit}' <File_name> The above code gives me length 50. wc -L <File_name> The above code gives me length 53. Please clarify on... (2 Replies)
Discussion started by: Amrutha24
2 Replies

3. Shell Programming and Scripting

how to read fixed length flat file....

Hi Gurus, Thanks in advance... I am new to writing shell scripting and help me out reading a flat file with fixed length. I have a fixed length flat file with storename(lenth 6) , emailaddress(lenth 15), location(10). There is NO delimiters in that file. Like the following str00001.txt... (2 Replies)
Discussion started by: willywilly
2 Replies

4. Shell Programming and Scripting

Need a sort solution for fixed length file

I have a 1250 byte record that I need to sort in column 10-19 and in column 301. I have tried the sort command, but it looks like it needs delimiters to work. The record can have spaces in a lot of its 1250 columns, but 10-19, and 301 are guaranteed. These columns are numeric too. A sample... (1 Reply)
Discussion started by: mb1201
1 Replies

5. UNIX for Dummies Questions & Answers

Convert a tab delimited/variable length file to fixed length file

Hi, all. I need to convert a file tab delimited/variable length file in AIX to a fixed lenght file delimited by spaces. This is the input file: 10200002<tab>US$ COM<tab>16/12/2008<tab>2,3775<tab>2,3783 19300978<tab>EURO<tab>16/12/2008<tab>3,28523<tab>3,28657 And this is the expected... (2 Replies)
Discussion started by: Everton_Silveir
2 Replies

6. UNIX for Dummies Questions & Answers

What the command to find out the record length of a fixed length file?

I want to find out the record length of a fixed length file? I forgot the command. Any body know? (9 Replies)
Discussion started by: tranq01
9 Replies

7. Shell Programming and Scripting

how to convert Fixed length file to delimited file.

I have below fixed lenth file . I have to convert this to delimitted file. File1.txtE116005/29/19930E001E000 E12201/23/19940E001E003 E10406/4/19940E001E003 I want to convert this to : E116,0,05/29/1993,0,E001,E000 E122,0,1/23/1994,0,E001,E003 E104,0,6/4/1994,0,E001,E003 I have a... (7 Replies)
Discussion started by: satyam_sat
7 Replies

8. Shell Programming and Scripting

convert XML file into Text file(fixed length)

If someone out there could help me out with this problem. I would really appreciate it. I am trying to convert xml into text file(fixed length) using Unix Borne shell scripts. My xml file: <root> <header_rec recordtype="00"> <record_id>00</record_id> ... (0 Replies)
Discussion started by: ram2s2001
0 Replies

9. Shell Programming and Scripting

Convert delimited to fixed length

Hi, I have to change a tab delimited file to a fixed length file. For text fields I need to left justify and NULL fill to the right and for number fields I need to right justify and zero fill to the left. If there are spaces between words in a text field I need to keep them as spaces. I am using... (14 Replies)
Discussion started by: nelson553011
14 Replies

10. Shell Programming and Scripting

creating a fixed length output from a variable length input

Is there a command that sets a variable length? I have a input of a variable length field but my output for that field needs to be set to 32 char. Is there such a command? I am on a sun box running ksh Thanks (2 Replies)
Discussion started by: r1500
2 Replies
Login or Register to Ask a Question