Setting the number of fields using awk Post: 302998889

Sponsored Content

Top Forums Shell Programming and Scripting Setting the number of fields using awk Post 302998889 by gull04 on Friday 9th of June 2017 07:11:00 AM

06-09-2017

Registered User

Setting the number of fields using awk

Hi Guys,

I've obviously had a senior moment here, what I'm trying to do is set the number of fields to 35 in a csv these should be appended to the end of the line. But what I'm getting is:-

Source Data

Code:

[davem@deneb data]$ head out_file_01.txt
N1000,024,2809003,,,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20090520,20090520,,5,2,V,50003414,,,,,,,,
N1000,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20090520,20090520,1T,5,2,V,50003414,,20090602,,,,,,
N1000,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20090520,20090520,32,5,2,14,50003414,,20091118,,,,,,20110930,
N1000,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20090520,20090520,32,5,2,V,50003414,,20091118,,,,,,
N1000,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20090520,20090520,32,5,2,V,50003414,,20091118,,,,,,20110930,
N1000,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,228120007,1,165,US,20110411,20090520,1T,5,2,25,50003414,,20130711,,,,,,20140430,01032549,,,A0070C,
N1000,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,228120007,1,165,US,20110411,20090520,1T,5,2,V,50003414,,20130711,,,,,,20140430,01032549,,,A0070C,
N1000,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20110411,20090520,32,5,2,V,50003414,,20091118,,,,,,20140430,
N1000,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20110411,20090520,32,5,2,V,50003414,,20091118,,,,,,20140430,01032549,
N1000,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,228120007,1,165,US,20110411,20090520,41,5,2,V,50003414,,20111201,,,,,,20140430,01032549,

What I tried was a nice simple awk command - I'm not great at awk but this was the attempt.

Code:

[davem@deneb data]$ head out_file_01.txt | awk -F"," '{NF=35}1' OFS=","
,,,,,,024,2809003,,,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20090520,20090520,,5,2,V,50003414,,,,,,,,
,,,,,,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20090520,20090520,1T,5,2,V,50003414,,20090602,,,,,,
,,,,0,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20090520,20090520,32,5,2,14,50003414,,20091118,,,,,,20110930,
,,,,,,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20090520,20090520,32,5,2,V,50003414,,20091118,,,,,,
,,,,0,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20090520,20090520,32,5,2,V,50003414,,20091118,,,,,,20110930,
N1000,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,228120007,1,165,US,20110411,20090520,1T,5,2,25,50003414,,20130711,,,,,,20140430,01032549,,,A0070C,
N1000,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,228120007,1,165,US,20110411,20090520,1T,5,2,V,50003414,,20130711,,,,,,20140430,01032549,,,A0070C,
,,,,0,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20110411,20090520,32,5,2,V,50003414,,20091118,,,,,,20140430,
,,,00,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,22812-0007,1,165,US,20110411,20090520,32,5,2,V,50003414,,20091118,,,,,,20140430,01032549,
,,,00,024,2809003,52215,1985,3,DYNAMIC AVLEASE INC,PO BOX 7,,BRIDGEWATER,VA,228120007,1,165,US,20110411,20090520,41,5,2,V,50003414,,20111201,,,,,,20140430,01032549,
[davem@deneb data]$

I know that this is my stupidity - but I'm unsure how I've got it so wrong in such a simple statement.

Regards

Gull04

gull04

View Public Profile for gull04

Visit gull04's homepage!

Find all posts by gull04

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk sed cut? to rearrange random number of fields into 3 fields

I'm working on formatting some attendance data to meet a vendors requirements to upload to their system. With some help on the forums here, I have the data close. But they've since changed what they want. The vendor wants me to submit three fields to them. Field 1 is the studentid field,...

2. UNIX for Dummies Questions & Answers

count number of fields not using SED or AWK

hi forums i need help with a little problem i am having. i need to count the number of fields that are in a saved variable so i can use that number to make a different function work properly. is there a way of doing this without using SED/AWK? anything would be greatly appreciated

3. Shell Programming and Scripting

Number of fields handled by awk

Hi Gurus, Have a file seperated by "~" and no of fields is 104. When i try to run awk, it erros out. awk: record `B~A31~T24_STF~~~2009...' has too many fields Any idea how can i extract a specific filed with this many fields in a row. Kindly help

4. Shell Programming and Scripting

How to (n)awk lines of CSV with certain number of fields?

I have a CSV file with a variable number of fields per record. How do I print lines of a certain number of fields only? Several permutations of the following (including the use of escape characters) have failed to retrieve the line I'm after (1,2,3,4)... $ cat myfile 1,2,3,4 1,2,3 $ # Print...

5. Shell Programming and Scripting

awk split lines without knowing the number of fields a-priori

I want to use awk to split fields and put them into a file but I don't know the number of fields for example, in the following line Ports: 22/filtered/tcp//ssh///, 53/open/tcp//tcpwrapped///, 111/filtered/tcp//rpcbind///, 543/filtered/tcp//klogin///, 544/filtered/tcp//kshell///,...

6. Shell Programming and Scripting

[ksh88 and awk] Number of fields with a value.

Hi, With: # VALUES="one~two~~~" # echo $VALUES | awk 'BEGIN {FS="~"} {print NF}' 5 I can determine the number of fields. How to determine the number of fields with a value ? In this case 2. Thanks in advance, ejdv

7. Shell Programming and Scripting

awk processing of variable number of fields data file

Hy! I need to post-process some data files which have variable (and periodic) number of fields. For example, I need to square (data -> data*data) the folowing data file: -5.34281E-28 -3.69822E-29 8.19128E-29 9.55444E-29 8.16494E-29 6.23125E-29 4.42106E-29 2.94592E-29 1.84841E-29 ...

8. UNIX for Dummies Questions & Answers

Make all records with the same number of fields (awk)

Hi, input: AA|BB|CC DD|EE FF what I am trying to get: AA|BB|CC DD|EE| FF|| I tried to create first an UDF for printing repeats, but I think I have an issue with my END section or my array: function repeat(str, n, rep, i) { for(i=1 ;i<n;i++) rep=rep str return rep } ...

9. Shell Programming and Scripting

Awk: Combine multiple lines based on number of fields

If a file has following kind of data, comma delimited 1,2,3,4 1 1 1,2,3,4 1,2 2 2,3,4 My required output must have only 4 columns with comma delimited 1,2,3,4 111,2,3,4 1,222,3,4 I have tried many awk command using ORS="" but couldnt progress

10. UNIX for Beginners Questions & Answers

Is there a UNIX command that can compare fields of files with differing number of fields?

Hi, Below are the sample files. x.txt is from an Excel file that is a list of users from Windows and y.txt is a list of database account. $ head -500 x.txt y.txt ==> x.txt <== TEST01 APP_USER_PROFILE USER03 APP_USER_PROFILE TEST02 APP_USER_EXP_PROFILE TEST04 APP_USER_PROFILE USER01 ...

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk sed cut? to rearrange random number of fields into 3 fields

Discussion started by: axo959

2. UNIX for Dummies Questions & Answers

count number of fields not using SED or AWK

Discussion started by: strasner

3. Shell Programming and Scripting

Number of fields handled by awk

Discussion started by: srivat79

4. Shell Programming and Scripting

How to (n)awk lines of CSV with certain number of fields?

Discussion started by: cs03dmj

5. Shell Programming and Scripting

awk split lines without knowing the number of fields a-priori

Discussion started by: esolvepolito

6. Shell Programming and Scripting

[ksh88 and awk] Number of fields with a value.

Discussion started by: ejdv

7. Shell Programming and Scripting

awk processing of variable number of fields data file

Discussion started by: radudownload

8. UNIX for Dummies Questions & Answers

Make all records with the same number of fields (awk)

Discussion started by: beca123456

9. Shell Programming and Scripting

Awk: Combine multiple lines based on number of fields

Discussion started by: mdkm

10. UNIX for Beginners Questions & Answers

Is there a UNIX command that can compare fields of files with differing number of fields?

Discussion started by: newbie_01