compare two files, selected columns only


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting compare two files, selected columns only
# 1  
Old 11-30-2010
compare two files, selected columns only

hi! i have two files that looks like this

file 1:
ABS 123 456
BCDG 124 542
FGD 459 762

file 2:
ABS 132 456
FGD 459 762

output would be:

from file1:
ABS 132 456
BCDG 124 542

from file 2:
ABS 132 456

i want to compare these 2 files, but columns 2&3 will be compared. the output would be in two parts. FROM FILE 1, those with difference with file 2 and those lines that cant be found on file 2. And FROM FILE 2, those with difference with file 1 and those lines that cant be found on file 1. Thanks!
# 2  
Old 11-30-2010
--- removed ---

Last edited by ctsgnb; 11-30-2010 at 03:50 PM..
# 3  
Old 11-30-2010
Code:
# awk 'NR==FNR{a[$2$3]=$0;next}!a[$2$3]' file2 file1
ABS 123 456
BCDG 124 542
# awk 'NR==FNR{a[$2$3]=$0;next}!a[$2$3]' file1 file2
ABS 132 456

# 4  
Old 11-30-2010
@ 丹

your code has a bug at a[$2$3], if $2 and $3 are (12 3456) and (123 456), awk will think they are same: a[123456]

Here is the fix.

Code:
$ cat file1
BCDG 124 542
FGD 459 762
ABS 13 2456

$ cat file2
ABS 132 456
FGD 459 762


awk 'NR==FNR{a[$2 FS $3]++;next}!a[$2 FS $3]' file1 file2

ABS 132 456


awk 'NR==FNR{a[$2 FS $3]++;next}!a[$2 FS $3]' file2 file1

BCDG 124 542
ABS 13 2456

# 5  
Old 11-30-2010
Quote:
Originally Posted by rdcwayx
@ 丹

your code has a bug at a[$2$3], if $2 and $3 are (12 3456) and (123 456), awk will think they are same: a[123456]
That's correct, Thanks Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Match value in two files and replace values in selected columns

The purpose is to check if values for column 3 and 4 in file1 match with column 1 in file2. If any value match do: 1) Replace values in file2 for column 2 and 3 using the information of file1 columns 5 and 6 2) Replace string ($1,1,5) and string ($1,6,5) in file2 with values of columns 7... (8 Replies)
Discussion started by: jiam912
8 Replies

2. UNIX for Dummies Questions & Answers

How to compare two columns in two files?

Hi All, I have a.dat file with content 1,338,30253395122015103,2015103,UB0085000,STMT151117055527002,,, 1,338,30253395122015103,2015103,UB0085000,STMT151117055527001,,, and b.dat having content 1,STMT151117055527001,a1.txt,b1.txt,c1.txt 1,STMT151117055527002,a2.txt,b2.txt,c2.txt ... (13 Replies)
Discussion started by: PRAMOD 96
13 Replies

3. UNIX for Dummies Questions & Answers

Help need to compare columns in files

Hi, Below is my requirement file1 id|cnt 1|1 2|2 3|3 file2 id_1|cnt_1 1|1 2|1 3|1 I want to compare cnt and cnt_1 columns, if they are differ then give the details Am using below awk command, but the output is not as expected. (2 Replies)
Discussion started by: grandhirahuletl
2 Replies

4. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

5. UNIX for Dummies Questions & Answers

Compare Columns in two files

Hi all, I would like to compare a column in one file to a column in another file and when there is a match it prints the first column and the corresponding second column. Example File1 ABA ABC ABE ABF File 2 ABA 123 ABB 124 ABD 125 ABC 126 So what I would like printed to a file... (0 Replies)
Discussion started by: pcg
0 Replies

6. Shell Programming and Scripting

Compare selected columns of two files and print whole line with mismatch

hi! i researched about comparing two columns here and got an answer. but after examining my two files, i found out that the first columns of the two files are not unique with each other. all i want to compare is the 2nd and 3rd column. FILE 1: ABS 456 315 EBS 923 163 JYQ3 654 237 FILE 2:... (1 Reply)
Discussion started by: engr.jay
1 Replies

7. Shell Programming and Scripting

How to compare 2 files & get only few columns based on a condition related to both files?

Hiiiii friends I have 2 files which contains huge data & few lines of it are as shown below File1: b.dat(which has 21 columns) SSR 1976 8 12 13 10 44.00 39.0700 70.7800 7.0 0 0.00 0 2.78 0.00 0.00 0 0.00 2.78 0 NULL ISC 1976 8 12 22 32 37.39 36.2942 70.7338... (6 Replies)
Discussion started by: reva
6 Replies

8. Shell Programming and Scripting

Compare selected columns from a file and print difference

I have learned file comparison from my previous post here. Then, it is comparing the whole line. Now, i have a new problem. I have two files with 3 columns separated with a "|". What i want to do is to compare the second and third column of file 1, and the second and third column of file 2. And... (4 Replies)
Discussion started by: kingpeejay
4 Replies

9. Shell Programming and Scripting

Compare EDI files by skipping selected Segments

Hi, I wanted to compare EDI files present in Two different Directories which can be related by the file names. While comparing the EDI files i have to skip selected segments such as "ISA" "IEA" and "GS" "GE" since this may have datetime stamp and different "Sender" "Receiver" Qual. and... (3 Replies)
Discussion started by: Sivas
3 Replies

10. Shell Programming and Scripting

Script to place selected columns from a group of files side by side in a new file

Hi Everyone, I need a shell/perl script to bring selected columns from all the files located in a directory and place them in a new file side by side. File1: a b c d 2 3 4 5 f g h i .......... File2: I II III IV w x y z .............. and so on many files are there...... (8 Replies)
Discussion started by: ks_reddy
8 Replies
Login or Register to Ask a Question