Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Make all records with the same number of fields (awk)

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Make all records with the same number of fields (awk)
# 1  
Old 06-30-2015
Make all records with the same number of fields (awk)
Hi,

input:
Code:
AA|BB|CC
DD|EE
FF

what I am trying to get:
Code:
AA|BB|CC
DD|EE|
FF||

I tried to create first an UDF for printing repeats, but I think I have an issue with my END section or my array:
Code:
function repeat(str, n, rep, i)
{
   for(i=1 ;i<n;i++)
      rep=rep str
      return rep
}

BEGIN{FS=OFS="|"}
{
   a[NR]=$0
   if(NF>max){
      max=NF}
}

END{
   for(i in a){
      if(NF<max){
         x=(max-NF)
         print a[i] repeat("|",x)}
      else{
         print a[i]}
   }
}

With this code it adds the same number of "|" no matter the number of fields the record contains:
Code:
AA|BB|CC|
DD|EE|
FF|

NOTE: I am using gawk (just in case I would have missed a built-in function for printing repeats)
Last edited by beca123456; 06-30-2015 at 02:19 PM..
# 2  
Old 06-30-2015
Try:
Code:
awk -F\| '{NF=3}1' OFS=\| file

# 3  
Old 06-30-2015
Try:
Code:
awk -v n=3 '{$n=$n}1' FS=\| OFS=\| file





--
Quote:
Originally Posted by Yoda
Try:
Code:
awk -F\| '{NF=3}1' OFS=\| file

Setting NF works with gawk and mawk, but not every awk recomputes the record the NF variable. Forcing a recompute with $1=$1 after setting NF works sometimes..
Last edited by Scrutinizer; 06-30-2015 at 11:48 PM..
# 4  
Old 06-30-2015
To be little more flexible, you can store the max number of columns in a variable first

Code:
nmaxcol=$(awk -F\|  '{if (NF > max) max = NF} END{print max}' file )

and then pass this variable to any of the above mentioned awk one liners.

Code:
awk -v n=$nmaxcol  '{$n=$n}1' FS=\| OFS=\| file

Even better, if your file is not too large, this might work too

Code:
awk -F\|  'NR==FNR{ if (NF > max) { max = NF }; next } { $max=$max}1' OFS=\| file file

Last edited by senhia83; 06-30-2015 at 04:33 PM..
# 5  
Old 06-30-2015
Try also
Code:
awk -F\|  '{T[NR]=$0; MAX=(NF>MAX)?NF:MAX} END {for (i=1; i<=NR; i++) {$0=T[i]; $MAX=$MAX; print}}' OFS=\| file
AA|BB|CC
DD|EE|
FF||

This User Gave Thanks to RudiC For This Post:
beca123456
# 6  
Old 07-01-2015
Thanks RudiC !

@Yoda & Scrutinizer: I assumed that max NF was unknown.
@senhia83: it works, but RudiC's one is more concise.
# 7  
Old 07-01-2015
Just for fun...
Code:
awk 'NF>n{n=NF}{$n=$n}NR>FNR' FS=\| OFS=\| file file

Last edited by Scrutinizer; 07-01-2015 at 01:08 PM..
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk for matching fields between files with repeated records

Hello all, I am having trouble with what should be an easy task, but seem to be missing something fundamental. I have two files, with File 1 consisting of a single field of many thousands of records. I also have File 2 with two fields and many thousands of records. My goal is that when $1 of... (2 Replies)
Discussion started by: jvoot
2 Replies

2. UNIX for Beginners Questions & Answers

Awk: group multiple fields from different records

Hi, My input looks like that: A|123|qwer A|456|tyui A|456|wsxe B|789|dfgh Using awk, I am trying to get: A|123;456|qwer;tyui;wsxe B|789|dfgh For records with same $1, group all the $2 in a field (without replicates), and all the $3 in a field (without replicates). What I have tried:... (6 Replies)
Discussion started by: beca123456
6 Replies

3. UNIX for Dummies Questions & Answers

keeping last record among group of records with common fields (awk)

input: ref.1;rack.1;1 #group1 ref.1;rack.1;2 #group1 ref.1;rack.2;1 #group2 ref.2;rack.3;1 #group3 ref.2;rack.3;2 #group3 ref.2;rack.3;3 #group3 Among records from same group (i.e. with same 1st and 2nd field - separated by ";"), I would need to keep the last record... (5 Replies)
Discussion started by: beca123456
5 Replies

4. Shell Programming and Scripting

AWK print number of records, divide this number

I would like to print the number of records of 2 files, and divide the two numbers awk '{print NR}' file1 > output1 awk '{print NR}' file2 > output2 paste output1 output2 > output awl '{print $1/$2}' output > output_2 is there a faster way? (8 Replies)
Discussion started by: programmerc
8 Replies

5. Shell Programming and Scripting

AWK: Cannot read Number of records greater than 1(NR>1)

Hi all, I have a tab-delimited text file of size 10Mb. I am trying to count the number of lines using, grep -c . sample.txtor wc -l < sample.txt or awk 'END {print NR}' sample.txtAll these commands shows the count as 1, which means they are reading only the first header line of the file.... (3 Replies)
Discussion started by: mehar
3 Replies

6. Shell Programming and Scripting

Print records which do not have expected number of fields in a comma delimited file

Hi, I have a comma (,) delimited file, in which few fields are enclosed with in double quotes " ". I have to print the records in the file which donot have expected number of field with the line number. File1 ==== name,desgnation,doj,project #header#... (7 Replies)
Discussion started by: machomaddy
7 Replies

7. Shell Programming and Scripting

nawk -- separation of records on basis of number of fields

A file file1.txt exists having records like The delimiter being "|" X|_|Y|_|Z|_| (number of fields 7) A|_|B|_| (number of fields 5) X|_|Z|_|H|_| (number of fields 7) A|_|D|_|S|_| (number of... (4 Replies)
Discussion started by: centurion_13
4 Replies

8. Shell Programming and Scripting

awk sed cut? to rearrange random number of fields into 3 fields

I'm working on formatting some attendance data to meet a vendors requirements to upload to their system. With some help on the forums here, I have the data close. But they've since changed what they want. The vendor wants me to submit three fields to them. Field 1 is the studentid field,... (4 Replies)
Discussion started by: axo959
4 Replies

9. UNIX for Dummies Questions & Answers

AWK ??-print for fields within records in a file

Hello all, Would appreciate if someone can help me out on the following requirement. INPUT FILE: -------------------------- TPS REPORT abc def ghi jkl mon pqr stu vrs lll END OF TPS REPORT TPS REPORT field1 field2 field3 field4 field5 field6 (8 Replies)
Discussion started by: hyennah
8 Replies

10. Shell Programming and Scripting

awk - Number of records

Hi, Is it possible to find the total number of records processed by awk at begining. NR gives the value at the end. Is there any variable available to find the value at the begining? Thanks ---------- Suman (1 Reply)
Discussion started by: suman_jakkula
1 Replies
Login or Register to Ask a Question