compare 2 files based on columns


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting compare 2 files based on columns
# 1  
Old 01-20-2011
compare 2 files based on columns

Hi Experts,

Is there a way to compare 2 files by columns and print matching cases.
I have 2 files as below, I want cases where col1 and col2 in f1 matches col1 and col2 in f2 to be printed as output. The separator is space. I want the output to have col1 col2 col 3 from both files printed in same line. Please see below example.

Code:
> cat f1
a b 1
c d 3
e f 0
f b 4
h h 0
x y 5

> cat f2
Code:
f d 1
h h 0 
f b 4
e f 2
c d 3
a b 1
y x 5

Code:
> Desired output
a b 1 a b 1
c d 3 c d 3
e f 0 e f 2
h h 0 h h 0
f b 4 f b 4


Thanks.

Best Regards,
Mani BR.
# 2  
Old 01-20-2011
Try:
Code:
awk 'NR==FNR{A[$1,$2]=$0;next} A[$1,$2]{print $0,A[$1,$2]}' f2 f1

# 3  
Old 01-20-2011
MySQL

Awesome, It worked as a charm.

(1) Also Is it possible to print the non-matching cases from both files

(2) Interest of learning, If possible can you explain the command used.

Thanks.
# 4  
Old 01-20-2011
1. Yes, the code needs to be expanded a little...
Code:
 awk 'NR==FNR{A[$1,$2]=$0;next}
        {if(A[$1,$2])
             {print $0,A[$1,$2];delete A[$1,$2]}
          else print>ARGV[2]".nomatch"
        }
        END{for(i in A)if(A[i])print A[i]>ARGV[1]".nomatch"}' f2 f1

2. Here is an explanation:
'NR==FNRFor every line do, if we are reading the first file ( that is when FNR and NR are the same) then
A[$1,$2]=$0create an array element with name of field 1 and field 2 ($1,$2) in array A that contains the line ($0)
next} read the next line of the first file and don't do further processing on this line
A[$1,$2]If we are reading the second file then for every line... if Array element with $1 and $2 of the second file exists then
print $0,A[$1,$2]print the current line and A[$1,$2]
f2 f1first read f2, then f1
# 5  
Old 01-20-2011
using ksh/bash/dash/...
Code:
> f3
> f1.notf2
cat f1   |   while read a b c
do
        data=$(grep "^$a $b "   f2    2>/dev/null    )
        [ "$data" != "" ] && echo "$a $b $c $data"    >> f3
        [ "$data" = "" ] && echo "$a $b $c "    >> f1.notf2
done

# 6  
Old 01-21-2011
Hi Scrutinizer,

The script still prints common lines only, Can you please comment.


Code:
$ cat f1
a b 1
c d 2
e f 3

$ cat f2
e f 3
a b 1
g h 1

Code:
$ source awk.csh
a b 1 a b 1
e f 3  e f 3

PHP Code:
Desired Output 
c d 2
g h 1 
Thanks.
# 7  
Old 01-21-2011
If you do
Code:
ls -l *.nomatch

You should see files that contain the non-matchess... If the files are called f2 and f1 then you should see f1.nomatch and f2.nomatch
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to compare two columns in two files?

Hi All, I have a.dat file with content 1,338,30253395122015103,2015103,UB0085000,STMT151117055527002,,, 1,338,30253395122015103,2015103,UB0085000,STMT151117055527001,,, and b.dat having content 1,STMT151117055527001,a1.txt,b1.txt,c1.txt 1,STMT151117055527002,a2.txt,b2.txt,c2.txt ... (13 Replies)
Discussion started by: PRAMOD 96
13 Replies

2. UNIX for Dummies Questions & Answers

Help need to compare columns in files

Hi, Below is my requirement file1 id|cnt 1|1 2|2 3|3 file2 id_1|cnt_1 1|1 2|1 3|1 I want to compare cnt and cnt_1 columns, if they are differ then give the details Am using below awk command, but the output is not as expected. (2 Replies)
Discussion started by: grandhirahuletl
2 Replies

3. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

4. Shell Programming and Scripting

Compare columns in different files

Hi, I have two files like this: 8 1.3 10 1.3 12 1.3 15 1.3 21 1.3 and 1 2 3 4 10 11 15 16 21 22 (3 Replies)
Discussion started by: jamie_123
3 Replies

5. Shell Programming and Scripting

Compare Columns of two files

Hi I have file 1 like this and file 2 like this I need to compare column 3 of both files and delete lines in file1 with same column 3 values in two files. So the output is I tried with perl but didnt work. A perl code will be good as I am learning the language, but any other code would... (1 Reply)
Discussion started by: polsum
1 Replies

6. UNIX for Dummies Questions & Answers

Compare Columns in two files

Hi all, I would like to compare a column in one file to a column in another file and when there is a match it prints the first column and the corresponding second column. Example File1 ABA ABC ABE ABF File 2 ABA 123 ABB 124 ABD 125 ABC 126 So what I would like printed to a file... (0 Replies)
Discussion started by: pcg
0 Replies

7. Shell Programming and Scripting

Compare columns of 2 files based on condition defined in a different file

I have a control file which tells me which are the fields in the files I need to compare and based on the values I need to print the exact value if key =Y and output is Y , or if output is Y/N then I need to print only Y if it matches or N if it does not match and if output =N , then skip the feild... (7 Replies)
Discussion started by: newtoawk
7 Replies

8. Shell Programming and Scripting

How to compare 2 files & get only few columns based on a condition related to both files?

Hiiiii friends I have 2 files which contains huge data & few lines of it are as shown below File1: b.dat(which has 21 columns) SSR 1976 8 12 13 10 44.00 39.0700 70.7800 7.0 0 0.00 0 2.78 0.00 0.00 0 0.00 2.78 0 NULL ISC 1976 8 12 22 32 37.39 36.2942 70.7338... (6 Replies)
Discussion started by: reva
6 Replies

9. Shell Programming and Scripting

How to compare two columns in two files?

Hello all, Could someone please let me know shell script or awk solution to compare two columns in two files? Here is the sample - file1.txt abc/xyz,M1234 ddd/lyg,M2345 cnn/tnt,G0123 file2.txt A,abc/xyz,kk,dd,zz,DCT,G0123,1 A,ddd/lyg,kk,dd,zz,DCT,M1234,1... (17 Replies)
Discussion started by: sncoupons
17 Replies

10. Shell Programming and Scripting

Compare few columns from two files

My Friends, Need your help to find the difference between few columns from two comma delimited files. For example, File1 and File2 has 22 columns, and I want to find the difference in first 12 columns. I have list of file names in MyListOfFiles2Compare.txt. Data is separated with commas.... (5 Replies)
Discussion started by: manish44
5 Replies
Login or Register to Ask a Question