Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

File comparison using awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File comparison using awk
# 1  
Old 03-31-2014
File comparison using awk
Hi All,

i have two files file1 ,file 2
file 1

Code:
col1|col2|col3|col4|col5|col6|col7|col8
11346925|0|2009-09-20|9999-12-31|100|0
11346925|0|2009-09-20|9999-12-31|120|0
12954311|0|2009-09-11|9999-12-31|100|0
12954311|0|2009-07-23|2999-12-31|120|0
12954312|0|2009-09-11|9999-12-31|100|0
12954312|0|2009-07-23|2999-12-31|120|0

file 2
Code:
col1|col2|col3|col4|col5|col6|col7|col8
12954311|0|2009-09-11|9999-12-31|130|0
12954311|0|2009-07-23|2999-12-31|120|0
12954312|0|2009-09-11|9999-12-31|100|0
12954312|0|2009-07-23|2999-12-31|120|0

My task is to loop through the file1 take the first field say (11346925) grep the lines containing 11346925 from both the files. compare both data's are same if not then write field 1 to a file.if field 1 from file1 not present in file 2 then also need to write field 1.
i tried with while loop but its taking much time when the file has some 16000 lines.
I hope i could do with awk.Please help me to achieve this faster using awk.
# 2  
Old 03-31-2014
Code:
awk -F"|" 'NR==FNR{a[$1]=$0;next}
a[$1]!=$0{print $1;next}
!a[$1]{print $1}' file2 file1

This User Gave Thanks to pravin27 For This Post:
mohanalakshmi
# 3  
Old 03-31-2014
Hi thanks for the quick reply.
when i used the code it just print all the lines i.e all the field 1.
one thing i missed out in the previous post is both the files may not be in a same order like fileld
Code:
11346925

may not be in the file2 sometimes or may be the last record in file 2 or wherever it might be.and just wanted to know about the previous awk code.
kindly help me on this
# 4  
Old 03-31-2014
Could you please let us know desire output ?
# 5  
Old 03-31-2014
field 1 is the desired output under the below conditions
data of field 1 from file 1 and file 2 should be same.i.e two lines from file 1 for field 1 and 2 lines from file 2 should be same. ( because field 1 from both the files may or may not conatins 2 lines with different data except field 1 )
ex:
file 1
129543110|2009-09-11|9999-12-31|130|0
129543110|0|2009-07-23|2999-12-31|120|0
file 2
129543110|0|2009-09-11|9999-12-31|130|0
129543110|0|2009-07-23|2999-12-31|120|0

the above case has a perfect match in two files no need to write anything to a file.

file 1:
Code:
129543112|0|2009-09-11|9999-12-31|110|0
129543112|0|2009-07-23|2999-12-31|120|0

file 2:
Code:
129543112|0|2009-09-11|9999-12-31|100|0
129543112|0|2009-07-23|2999-12-31|120|0

inthis case it is a mismatch then need to write 129543112 to a file.

Only things is that files may not be in a order
# 6  
Old 03-31-2014
Could this help you ?
Code:
awk -F"|" 'NR==FNR{a[$0]=$0;next}
a[$0]!=$0{print $1;next}
!a[$0]{print $1}' file2 file1

This User Gave Thanks to pravin27 For This Post:
mohanalakshmi
# 7  
Old 04-01-2014
Thanks for the reply and its working perfectly as expected.
Thanks again.SmilieCould you please explain how the code works.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File comparison and proccessing using awk

Hi Guys, I am having two requirement in one of my scripts. please help out to find a fast solution using AWK (since there is lot of data to be processed) 1) First snippet - File1 has two columns and file2 has three columns If any value of column 1 of file1 matches with column 1... (4 Replies)
Discussion started by: stormfield
4 Replies

2. Shell Programming and Scripting

File comparison using awk

my files are as follows fileA sepearated by tab /t 00 lieferungen 00 attractiop 01 done 02 forness 03 rasp 04 alwaysisng 04 funny 05 done1 fileB funnymou120112 funnymou234470 mou3raspnhdhv rddfgmoudone1438748 so all those record which are greater than 3 and which are not... (6 Replies)
Discussion started by: rajniman
6 Replies

3. Shell Programming and Scripting

awk file comparison, x lines after matching as output

Hello, I couldn't find anything on the Forum that would help me to solve this problem. Could any body help me process below data using awk? I have got two files: file1: Worker1: Thomas Position: Manager Department: Sales Salary: $5,000 Worker2: Jason Position: ... (5 Replies)
Discussion started by: killerbee
5 Replies

4. Shell Programming and Scripting

Urgent Help Required for File Comparison using Awk

Hello All, I am having a below requirement. File1 contains KEY|VIN|SEQUENCE|COST 101 | XXX111 | 1 | 234.22 234 | XXX111 | 2 | 134.32 444 | ABC234 | 1 | 100.22 555 | DFF611 | 1 | 734.82 FILE 2 Contains only VINs XXX111 DFF611 Now if the VIN from file 1 is present in... (8 Replies)
Discussion started by: dinesh1985
8 Replies

5. Shell Programming and Scripting

awk column comparison big file

Hi all, I would like to compare a column in one file to a column in another file and when there is a match it prints the first column and the corresponding second column. Example File1 ABA ABC ABE ABF File 2 ABA 123 ABB 124 ABD 125 ABC 126 So what I would like printed to a... (6 Replies)
Discussion started by: pcg
6 Replies

6. Shell Programming and Scripting

awk comparison

Hello all, Probably a very simple question, I am stuck with a small part of a code: I am trying to do a comparison to get the maximum value of column 6 if columns 1, 4 and 5 of two or more rows match. Here is what I am doing: awk -F'\t' '{if ($6 > a)a=$6}END{for (i in a) print i"\t"a}' ... (4 Replies)
Discussion started by: jaysean
4 Replies

7. Shell Programming and Scripting

Comparison and editing of files using awk.(And also a possible bug in awk for loop?)

I have two files which I would like to compare and then manipulate in a way. File1: pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2: pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies

8. Shell Programming and Scripting

Looking for AWK Solution for column comparison in a single file

- I am looking for different kind of awk solution which I don't think is mentioned before in these forums. Number of rows in the file are fixed Their are two columns in file1.txt 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 I am looking for 3... (1 Reply)
Discussion started by: softwarekids23
1 Replies

9. Shell Programming and Scripting

awk program for file comparison

Hello there, I'm trying to write an awk program in bash shell with the following three input files: File 1 1001 1 2 3 1002 4 5 6 1003 7 8 9 1004 10 11 12 File 2 1001 11 22 33 1002 44 55 66 1004 100 111 122 ... (4 Replies)
Discussion started by: kbirde
4 Replies

10. Shell Programming and Scripting

Comparison of two files in awk

Hi, I have two files file1 and file2 delimited by semicolon, And I want to compare column 2 and column3 of file1 to column3 and column 4 in file2. file1 -------- abc;cef;155.67;143_34; def;fgh;146.55;123.3; frg;hff;134.67;; yyy;fgh;134.78;35_45; file 2 --------- abc;cef;155.09;;... (12 Replies)
Discussion started by: jerome Sukumar
12 Replies
Login or Register to Ask a Question