File manipulation in awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File manipulation in awk
# 1  
Old 01-11-2011
File manipulation in awk

I have got a sample file below(colon(Smilie is the field separator) . The data is like

col1:col2:col3:col4:col5:col6:col7:col8:col9:col10
11:12:012:aa:a a a:10::111:12:
311:321:320:caad::321:31:3333::
2:22:222::bbb::cads::2222:20
:::::12:1234::12:
:5:55::555:5555::::55550

Now I want to find the number of times the column is having no value (null).i.e. the output should be like
col1 - 2
col2 - 1
col3 - 1
col4 - 3
col5 - 2
col6 - 1
col7 - 2
col8 - 3
col9 - 2
col10 - 3

The file can have 'm' number of columns and can have 'n' number of records.
# 2  
Old 01-11-2011
Try:
Code:
awk -F: 'NR==1{for (i=1;i<=NF;i++){a[i]=$i}}NR>1{for (i=1;i<=NF;i++){if ($i==""){b[i]++}}}END{for (i=1;i<=NF;i++){print a[i]" - "b[i]}}' file

# 3  
Old 01-11-2011
Another approach:
Code:
awk -F: '{for(i=1;i<=NF;i++) a[i]+=$i?0:1}
END{for (i=1;i<=NF;i++) print "col" i " - " a[i]}
' file

# 4  
Old 01-11-2011
Quote:
Originally Posted by Franklin52
Another approach:
Code:
awk -F: '{for(i=1;i<=NF;i++) a[i]+=$i?0:1}
END{for (i=1;i<=NF;i++) print "col" i " - " a[i]}
' file

Your approach is OK if the names of the columns in target file are "col1, col2.. etc". I think OP will have different column names Smilie
# 5  
Old 01-11-2011
Quote:
Originally Posted by bartus11
Your approach is OK if the names of the columns in target file are "col1, col2.. etc". I think OP will have different column names Smilie
In that case:
Code:
awk -F: 'NR==1{for(i=1;i<=NF;i++) c[i]=$i}
{for(i=1;i<=NF;i++) a[i]+=$i?0:1}
END{for (i=1;i<=NF;i++) print c[i] " - " a[i]}
' file

# 6  
Old 01-11-2011
Quote:
Originally Posted by Franklin52
In that case:
Code:
awk -F: 'NR==1{for(i=1;i<=NF;i++) c[i]=$i}
{for(i=1;i<=NF;i++) a[i]+=$i?0:1}
END{for (i=1;i<=NF;i++) print c[i] " - " a[i]}
' file

Well, I've found another issue with your code Smilie. It will misbehave if some field contains "0" - it is counting it as an empty field :P
# 7  
Old 01-11-2011
Same idea...
Code:
awk -F: 'NR==1{split($0,H);next} {for(i in H)if($i==x)A[i]++} END{for(i=1;i<=NF;i++)print H[i]" - "A[i]}' infile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk manipulation

hello I have example file AA 11 BB 22 CC 33 And what I expect to have -a AA=11 -a BB=22 -a CC=33 can anyone help how I have this using awk? (1 Reply)
Discussion started by: vikus
1 Replies

2. Shell Programming and Scripting

awk manipulation

Hallo Family, I have csv file which has over a million records in it. All i want to do is to change field 2 to have the same value as field 10. sample file:Now 0860093239,Anonymous,unconditional,+27381230283,Anonymous,unconditional,y,public,,2965511477:0A Desired output: ... (2 Replies)
Discussion started by: kekanap
2 Replies

3. Shell Programming and Scripting

Awk to convert a text file to CSV file with some string manipulation

Hi , I have a simple text file with contents as below: 12345678900 971,76 4234560890 22345678900 5971,72 5234560990 32345678900 71,12 6234560190 the new csv-file should be like: Column1;Column2;Column3;Column4;Column5 123456;78900;971,76;423456;0890... (9 Replies)
Discussion started by: FreddyDaKing
9 Replies

4. UNIX for Dummies Questions & Answers

File manipulation via awk

Hello, I am having issues generating the output file below from this input file: Basically, what I want is if the ID= matches with the line below to print the first value in column 3 and the last value of column 4 for the matching ID's. The ID's can repeat more than twice, however, they... (2 Replies)
Discussion started by: verse123
2 Replies

5. Shell Programming and Scripting

SED/AWK file read & manipulation

I have large number of data files, close to 300 files, lets say all files are same kind and have extension .dat , each file have mulitple lines in it. There is a unique line in each file containing string 'SERVER'. Right after this line there is another line which contain a string 'DIGIT=0',... (4 Replies)
Discussion started by: sal_tx
4 Replies

6. Shell Programming and Scripting

File manipulation with AWK and SED

Hello How do i check that correct input files are used while using AWk and SED for file manipulation? e.g awk '/bin/ {print $0 }' shell.txt sed 's/hp/samsung/' printers.txt how do i ensure that the correct input files I am working with are used? (5 Replies)
Discussion started by: Pauline mugisha
5 Replies

7. Shell Programming and Scripting

$0 manipulation in awk

OK, so if $0 represent the entire record... can I change $2 and will that be reflected back in $0? I think the following answers that YES, it does work. But is there anything I should be thinking about prior to doing this? What I am actually doing is part of 5 pages of scripting and awk... (1 Reply)
Discussion started by: joeyg
1 Replies

8. Shell Programming and Scripting

File manipulation with awk

Could you please help me to achieve the below: In a file I need to convert the multiple lines whose filed 1 and field 5 values are same into a single line but with the field 4 values comma separed as mentioned below. Fileds after 5 shall be discarded. Also here by default all other remaining... (6 Replies)
Discussion started by: dhams
6 Replies

9. Shell Programming and Scripting

File manipulation using AWK

Hi All, I have a file having content, $ cat data1.txt 20060620 142 62310 959400 A 5.00 20060620 142 62310 959400 B 3.00 20060620 143 62310 959401 A 7.00 20060620 143 62310 959401 B 4.00 20060620 144 62310 959402 A 8.00 20060620 144 62310... (6 Replies)
Discussion started by: rinku11
6 Replies

10. Shell Programming and Scripting

awk manipulation

Hi , what a wonderful command but so hard to maintain ! i have a file like that : 03/07/2006 05:58:45 03/07/2006 06:58:45 03/07/2006 07:58:50 03/07/2006 08:58:50 and i want to read it and keep only the lines with 3rd field less than 07:00:00 writing it in a second file ! ... (2 Replies)
Discussion started by: Nicol
2 Replies
Login or Register to Ask a Question