Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Compare Multiple Columns in one file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare Multiple Columns in one file
# 1  
Old 11-05-2013
Compare Multiple Columns in one file
Hello guys, I am quite new to Shell Scripting and I need help for this
I have a CSV file like this:
Code:
Requisition,Order,RequisitionLineNumber,OrderLineNumber
REQ1,Order1,1,1
REQ1,Order1,1,3
REQ2,Order2,1,5

Basically what I want to do is compare the first 3 fields
If all 3 fields are the same then the last field which is OrderLineNumber should be arranged in ascending order and the numbers must be changed:
Example for above: Output should be like this: notice that 3 is changed to 2
Code:
REQ1,Order1,1,1
REQ1,Order1,1,2

if the 3 fields are unique the value of OrderLineNumber should always be 1:
Code:
REQ2,Order2,1,1

I know this is possible with AWK, but I don't know how to do it with multiple columns.

Thanks
# 2  
Old 11-05-2013
Code:
[user@host ~]$ cat file
REQ1,Order1,1,1
REQ1,Order1,1,3
REQ2,Order2,1,5
REQ2,Order2,1,5
REQ2,Order2,1,5
REQ2,Order2,1,5
[user@host ~]$ awk -F',' '{a[$1","$2","$3]++} END {for(x in a) {for(i=1; i<=a[x]; i++) {print x","i}}}' file
REQ1,Order1,1,1
REQ1,Order1,1,2
REQ2,Order2,1,1
REQ2,Order2,1,2
REQ2,Order2,1,3
REQ2,Order2,1,4
[user@host ~]$

Remember, the output will not be sorted.. Meaning.. REQ3 records can come before REQ1
These 2 Users Gave Thanks to balajesuri For This Post:
jeffreybsu RavinderSingh13
# 3  
Old 11-05-2013
That is awesome Bala. Could you please give me a link where array for
awk is explained more with examples. As this was one of the great requirements and I can learn more if you will help me here please.


Thanks,
R. Singh
# 4  
Old 11-05-2013
Link to GNU awk manuals.
# 5  
Old 11-05-2013
Thanks bala.. I think I figured out a way to sort it.
by using this code:
Code:
(head -n 2 try2.csv && tail -n +3 try2.csv | sort -t, -k1n)

I get to this:
Code:
Requisition,Order,RequisitionLineNumber,OrderLineNumber
REQ1,Order1,1,1
REQ1,Order1,1,2
REQ2,Order2,1,1
REQ3,Order1,1,1
REQ3,Order2,1,1

Do you know the solution if there are additional fields after the 4th field? Can I append them?
For example:
Code:
Requisition,Order,RequisitionLineNumber,OrderLineNumber,try
REQ1,Order1,1,1,try2
REQ1,Order1,1,3,try3
REQ2,Order2,1,5,try4
REQ3,Order1,1,2,try5
REQ3,Order2,1,5,try5

# 6  
Old 11-05-2013
Try:
Code:
awk '{$4=++A[$1,$2,$3]}1' FS=, OFS=, file

or to preserve the header:
Code:
awk 'NR>1{$4=++A[$1,$2,$3]}1' FS=, OFS=, file

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare columns in a single file

i have the following files (all separated by tabs): file 1.txt 1 yes 2 no 3 yes 4 yes file 2.txt a no b no c yes d no i combine the above files in file 3 which looks like file 3.txt 1 yes a no 2 no b no 3 yes c yes 4 yes d no now, i need to compare the values between column 2... (3 Replies)
Discussion started by: msonoth
3 Replies

2. Shell Programming and Scripting

Compare two date columns in same file

Hi All, Need to compare two date columns from the filname FinalDate.txt. My data's are like below D_OT_START D_EXP_STR Amount 1/3/2012 1/3/2012 5000 6/21/2011 6/25/2011 6000 2/28/2011 2/28/2011 7000 7/16/2010 8/16/2010 8000 7/14/2010 10/26/2010 9000 ... (3 Replies)
Discussion started by: suresh_target
3 Replies

3. Shell Programming and Scripting

Compare columns of multiple files and print those unique string from File1 in an output file.

Hi, I have multiple files that each contain one column of strings: File1: 123abc 456def 789ghi File2: 123abc 456def 891jkl File3: 234mno 123abc 456def In total I have 25 of these type of file. (5 Replies)
Discussion started by: owwow14
5 Replies

4. Shell Programming and Scripting

Compare multiple columns from 2 files

Hi, I need to compare multiple columns from 2 files. I can, for example, have these 2 files: file1: col1, col2, col3,col4 a,1,4,7 b,2,5,8 c,3,6,9file2: col1, col2, col3,col4 a,2,3,2 b,5,7,5 c,1,9,8As a result, I need for example the difference between the columns 2 and 4: col2,... (3 Replies)
Discussion started by: Subbeh
3 Replies

5. Shell Programming and Scripting

Compare multiple files with multiple number of columns

Hi, input file1 abcd 123 198 xyz1:0909090-0909091 ghij 234 999 xyz2:987654:987655 kilo 7890 7990 xyz3:12345-12357 prem 9 112 xyz5:97-1134 input file2 abcd 123 198 xyz1:0909090-0909091 -9.122 0 abed 88 98 xyz1:98989-090808 -1.234 1.345 ghij 234 999 xyz2:987654:987655 -10.87090909 5... (5 Replies)
Discussion started by: jacobs.smith
5 Replies

6. Shell Programming and Scripting

Awk match multiple columns in multiple lines in single file

Hi, Input 7488 7389 chr1.fa chr1.fa 3546 9887 chr5.fa chr9.fa 7387 7898 chrX.fa chr3.fa 7488 7389 chr21.fa chr3.fa 7488 7389 chr1.fa chr1.fa 3546 9887 chr9.fa chr5.fa 7898 7387 chrX.fa chr3.fa Desired Output 7488 7389 chr1.fa chr1.fa 2 3546 9887 chr5.fa chr9.fa 2... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

7. Shell Programming and Scripting

script to compare two columns in a file

Dear everyone, I need any sort of shell script or perl script would do the following. I have a txt file as follows: ;Stretnumber Resident Resdient (not in file) 16 John Mary 16 Mary Parker 16 Nancy Smith 16 Mary John 18 Trey ... (5 Replies)
Discussion started by: sasharma
5 Replies

8. UNIX for Dummies Questions & Answers

To compare first two columns in an excel file

Hi All, i have a excel sheet with two columns as below. column1 column2 100 100 200 300 300 400 400 400 500 600 i need to compare the values these two columns and the output should be printed in the third column...if these values are equal the output should be green and if these... (2 Replies)
Discussion started by: arunmanas
2 Replies

9. Shell Programming and Scripting

compare file columns

I need help in file comparision. I have two files in below format: FILE_A: ------- COL1 COL2 COL3 COL4 COL5 FILE_B: ------- COL1A COL1B COL1C COL1D COL1E i want to compare for a for each row in FILE_A and FILE_B COL1 of FILE_A with COL1B of FILE_B COL3 of FILE_A with COL1E of... (1 Reply)
Discussion started by: learnoutmore99
1 Replies

10. Shell Programming and Scripting

Compare multiple columns between 2 files

hello I need to compare 2 text files. File 1 has 2 columns and file 2 has 1 to many. Sample: File 1: 111 555 222 666 333 777 444 755 File 2: 000 110 113 114 844 111 555 999 202 777 865 098 023 222 313 499 065 655 333 011 890 777 433 (15 Replies)
Discussion started by: stevesmith
15 Replies
Login or Register to Ask a Question