Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Awk to convert a flat file to CSV file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Awk to convert a flat file to CSV file
# 8  
Old 07-17-2008
The input data you have pasted doesn't appear to match what you have described.
# 9  
Old 07-17-2008
Apologies for any confusion .

Please let me know any discrepancy.

thanks!
# 10  
Old 07-17-2008
hi let me explain with a quick example:

Input text file
CUST PREM RATE LOS CODE DATE REAS TYPE
========= ======= ==== ==== ==== =========== ==== ====
8174739 7641509 1S NONE CRCD 30-JUN-2008 NPAR N

Output required in CSV file:
8174739 ,7641509,1S ,NONE,CRCD,30-JUN-2008,NPAR,N

In the input file mentionned the field lengths are as below:

1st field --> 9 bit long
2nd field --> 7 bit long
3rd field --> 4 bit long
4th field --> 4 bit long
5th field --> 4 bit long
6th field --> 11 bit long
7th field --> 4 bit long
8th field --> 1 bit long

but the value of the first field CUST in the input record, has only 7 bit. So with the input 7 bit value i will pad 2 bit spaces and then the 10th bit will be a seperator "," in the output csv file.

The next field is the PREM which is actually 7 bit long and the value in this field is also 7 bit long, so the 8th bit will be a seperator "," in the output csv file.

finally my output csv file will be as below:

8174739 ,7641509,1S ,NONE,CRCD,30-JUN-2008,NPAR,N

Please let me know if the explanation is clear.

Thanks !
# 11  
Old 07-17-2008
Try this:

Code:
awk '
        NR==1 { next }
        /^=/ {
                for (i=1; i<NF; i++) { f[i]="%-" length($i) "s," }
                f[NF]="%-" length($NF) "s\n"
                next
        }
        {
                for (i=1; i<=NF; i++) { printf f[i],$i }
        }

' inputfile > outputfile

It builds an array of printf format strings by counting the "=" characters (say "characters" or "bytes", not "bits"), and then uses those to format the subsequent lines.
# 12  
Old 07-17-2008
Thanks Anni!

When i run the script it does not produce any output in the output file, can you please help me.

I was just thinking of, since we know the field lengths of all eight fields, can we use anything like FIELDWIDTH utility of awk. i'am not sure if i'am correct, please correct me in case i'am wrong.
# 13  
Old 07-17-2008
If it didn't work it's probably because the data isn't exactly same as your example. It works for me:

Code:
$ cat rkumudha.dat
CUST PREM RATE LOS CODE DATE REAS TYPE
========= ======= ==== ==== ==== =========== ==== ====
8174739 7641509 1S NONE CRCD 30-JUN-2008 NPAR N
8678696 8091709 1S NONE DDEB 30-JUN-2008 MVD P
8198258 7663210 1S NONE GIRO 30-JUN-2008 NPAR N
8064034 7540836 1S NONE GUAR 30-JUN-2008 NPAR N
$ cat rkumudha.awk
awk '
        NR==1 { next }
        /^=/ {
                for (i=1; i<NF; i++) { f[i]="%-" length($i) "s," }
                f[NF]="%-" length($NF) "s\n"
                next
        }
        {
                for (i=1; i<=NF; i++) { printf f[i],$i }
        }

' rkumudha.dat
$ ./rkumudha.awk
8174739  ,7641509,1S  ,NONE,CRCD,30-JUN-2008,NPAR,N
8678696  ,8091709,1S  ,NONE,DDEB,30-JUN-2008,MVD ,P
8198258  ,7663210,1S  ,NONE,GIRO,30-JUN-2008,NPAR,N
8064034  ,7540836,1S  ,NONE,GUAR,30-JUN-2008,NPAR,N
$

Otherwise it may be due to your operating system or version of awk. What are they?
# 14  
Old 07-17-2008
Hi Anni,

Thanks a lot the script worked fine, i got an issue becaue of some space problems in the server.

There are certain situations in our case that the input file does not have the header details in it, so we are requested to note down the field lengths from the IDD and implement the field separator "," in the input file.

As we know all the field width's as given below,can you please suggest me an approachfor the same:

<9 byte><space><7 byte><space><4 byte><space><4 byte><space><4 byte><space><11 byte><space - 1 byte><4 byte><space><4 byte><space><11 byte><space><11 byte><space><8 byte><space><20 byte>

NOTE: <space> -> 2 bytes and the length of the fields are highlighted in RED.

Thanks !
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Converting csv file to flat file

Hi All, I have a csv file which is comma seperated. I need to convert to flat file with preferred column length country,id Australia,1234 Africa,12399999 Expected output country id Australia 1234 Africa 12399999 the flat file should predefined length on respective... (8 Replies)
Discussion started by: rohit_shinez
8 Replies

2. UNIX for Dummies Questions & Answers

Convert flat file to csv

Hi I have a file like this: a=1 b=2 c=3 a=4 b=2 d=3 a=3 c=4 How can I change this to csv format a,b,c,d 1,2,3,, 4,2,,3 3,,4,, Please use code tags next time for your code and data. Thanks (10 Replies)
Discussion started by: sandip_2014
10 Replies

3. Shell Programming and Scripting

How to convert excel file to csv file or text file?

Hi all, I need to find a way to convert excel file into csv or a text file in linux command. The reason is I have hundreds of files to convert. Another complication is the I need to delete the first 5 lines of the excel file before conversion. so for instance input.xls description of... (6 Replies)
Discussion started by: johnkim0806
6 Replies

4. Shell Programming and Scripting

Awk to convert a text file to CSV file with some string manipulation

Hi , I have a simple text file with contents as below: 12345678900 971,76 4234560890 22345678900 5971,72 5234560990 32345678900 71,12 6234560190 the new csv-file should be like: Column1;Column2;Column3;Column4;Column5 123456;78900;971,76;423456;0890... (9 Replies)
Discussion started by: FreddyDaKing
9 Replies

5. Shell Programming and Scripting

reading a csv file and creating a flat file

hi i have written a script for reading a csv file and creating a flat file, suggest if this script can be optimized #---------------- FILENAME="$1" SCRIPT=$(basename $0) #-----------------------------------------// function usage { echo "\nUSAGE: $THIS_SCRIPT file_to_process\n"... (3 Replies)
Discussion started by: mprakasheee
3 Replies

6. Shell Programming and Scripting

Convert CSV file (with double quoted strings) to pipe delimited file

Hi, could some help me convert CSV file (with double quoted strings) to pipe delimited file: here you go with the same data: 1,Friends,"$3.99 per 1,000 listings",8158here " 1,000 listings " should be a single field. Thanks, Ram (8 Replies)
Discussion started by: Ram.Math
8 Replies

7. Programming

awk script to convert a text file into csv format

hi...... thanks for allowing me to start a discussion i am collecting usb usage details of all users and convert it into csv files so that i can export it into some database.. the input text file is as follows:- USB History Dump by nabiy (c)2008 (1) --- Kingston DataTraveler 130 USB... (2 Replies)
Discussion started by: certteam
2 Replies

8. Shell Programming and Scripting

Flat file to csv conversion

Hi Guy's can someone help me in converting the following I have a flat text file which has several thousand lines which I need to convert to a csv it's got a consistent format but basically want every time it hit's txt to create a new line with the subsequent lines comma delimited for example ... (6 Replies)
Discussion started by: p1_ben
6 Replies

9. Shell Programming and Scripting

Convert case on specified position of flat file

Please help Need a script which will do the following : Search on fixed width file , go to position (25,2) which means 25th and 26th position, Find if there are any char in lower case: For example 25,2 can be (9T) or (9w) or (Ww) or (wW)....The two positions can be numeric or alpha...no... (13 Replies)
Discussion started by: ssantoshss
13 Replies

10. Shell Programming and Scripting

Need help to convert Flat file to HTML

Hello I need help to convert flat file data to HTML Table format. I am generating everyday Flat file and want to convert into HTML Table format. The format of my file is: version host Total YRS NO APPS PSD 10 Sun 30 2 4 6 7 and flat... (11 Replies)
Discussion started by: getdpg
11 Replies
Login or Register to Ask a Question