Compare two files with different column entries..:-(


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare two files with different column entries..:-(
# 1  
Old 06-13-2013
Compare two files with different column entries..:-(

Dear All,
I would appreciate any help..At the moment, my work is stuck cos of my inability to resolve this issue.
Which is following:
I have two files with the arrngment like this
file-1
Code:
190645 12 3596022
190645 12 3764915
190645 16 3803981
190645 12 3854102
190645 12 4324593
190645 13 4894025


And second file-2 (which I know has less entries then file-1)
Code:
190645 12 3854102
190645 12 3596022
190645 12 3764915
190645 16 3803981
190645 12 4324593

As you can see, the content in both files are SAME except the RED one.
But in this case, the DIFF command wont work..and I am puzzled. As the original such file has thousands of entries..Smilie

Plesae let me know how to get the REd marked files from the file-1 or the differenence of two files based on the matching of three column together.

Please help..Smilie

Thanks in Advance,
emily
# 2  
Old 06-13-2013
Code:
sort file1.txt > file1
sort file2.txt > file2
comm -23 file1 file2


Last edited by Scott; 06-13-2013 at 06:13 PM.. Reason: Code tags
# 3  
Old 06-13-2013
Quote:
Originally Posted by doganaym
sort file1.txt > file1
sort file2.txt > file2
comm -23 file1 file2
Hi doganaym,
Thanks for the quick reply..the first two commands seem to be working...and the files are sorted..
But I am afraid even after neither third command as you provided worked nor the 'diff' command. Smilie

---------- Post updated at 04:17 PM ---------- Previous update was at 04:12 PM ----------

Hi again,
command is working all fine for the example file that I provided here.
Not sure, why its not working on my huge list of files..

Checking ! Thanks again for your help,
emily

---------- Post updated at 04:27 PM ---------- Previous update was at 04:17 PM ----------

I tried that as well, no use..
Somehow the file1 has 1636 entries and the second file has 1538 entries..differnce is of around ~100.
But the comm, diff, and your new command all are giving the final difference file with entries more than 2000..Smilie thats annoying..not sure wats happening..

emily..

Sorry,to be precise, the difference is comign at the moment is 10556 entries..Smilie Smilie
# 4  
Old 06-13-2013
Code:
awk 'FNR==NR{a[$0]++;next} !($0 in a)' file2 file1
190645 13 4894025

# 5  
Old 06-13-2013
Hi,
Thakns for the reply..
Unfortunately, on the main files, it is giving 10663 entries..though the differencce is around 100 in both files.
I did following, jst to mention:
Code:
sort file_1 > file1
sort file_2 > file2
awk 'FNR==NR{a[$0]++;next} !($0 in a)' file2 file1 > differene.txt &

And the difference.txt has 10663 entries.

Even if all lines are different (which i know defnitely not the case is), then also it should be 1064 + 1058 = ~2k...

emily

Last edited by Scott; 06-13-2013 at 06:49 PM.. Reason: Code tags
# 6  
Old 06-14-2013
Check file_2_diff file which will be having entries which is not present in file_1. and file_1_diff file which will be having entries which is not present in file_2.
Code:
awk 'NR==FNR{A[$0]=$0;next}{if(A[$0]){delete A[$0]}else{print > "file_2_diff"}}
    END{for(i in A){if(A[i]){print A[i] > "file_1_diff"}}}' file1 file2

# 7  
Old 06-14-2013
Quote:
Originally Posted by pamu
Check file_2_diff file which will be having entries which is not present in file_1. and file_1_diff file which will be having entries which is not present in file_2.
Code:
awk 'NR==FNR{A[$0]=$0;next}{if(A[$0]){delete A[$0]}else{print > "file_2_diff"}}
    END{for(i in A){if(A[i]){print A[i] > "file_1_diff"}}}' file1 file2

Hi Pamu,
Thanks for replying.
Unfortunately, It did not work either.
Here, I am attaching my actual files
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need awk or Shell script to compare Column-1 of two different CSV files and print if column-1 matche

Example: I have files in below format file 1: zxc,133,joe@example.com cst,222,xyz@example1.com File 2 Contains: hxd hcd jws zxc cst File 1 has 50000 lines and file 2 has around 30000 lines : Expected Output has to be : hxd hcd jws (5 Replies)
Discussion started by: TestPractice
5 Replies

2. Shell Programming and Scripting

Compare two files based on column

Hi, I have two files roughly 1200 fields in length for each row, sorted on the 2nd field. I need to compare based on that 2nd column between file1 and file2 and print lines that exist in both files into separate files (I can't guarantee that every line in file1 is in file2). Example: File1: ... (1 Reply)
Discussion started by: origon
1 Replies

3. Shell Programming and Scripting

Compare and matching column entries in 2 files and

I have 2 files. File 1 has more columns (6 columns but the last column has spaces) than file 2 (file 2 has 4 columns). The entries in file 1 do not change but column 4 in file 2 can be different from the the entry in file 1. I want to create a script that reads in file 1 and then uses column 1 2... (5 Replies)
Discussion started by: kieranfoley
5 Replies

4. Shell Programming and Scripting

Compare 1 column in 2 files

Hi all, I have two two-column tab-separated files with the following input: inputA dog A dog B cat A.... inputB dog C mouse A output dog I need to compare the 1st column of each file and output those shared items. What is the best unix solution for that? (5 Replies)
Discussion started by: owwow14
5 Replies

5. Shell Programming and Scripting

How to compare 2 files column's more than 5?

Hi All I am just trying to compare 2 file using column information using following code awk ' NR==FNR {A=$9; next} {B=A; print $0,B""?B:" Not -In file" } ' OFS="\t" file1 file2if file1 matches with file2 then print $9 content in file 1 along with file2 $0 suppose if I keyed on only $1 in... (17 Replies)
Discussion started by: Akshay Hegde
17 Replies

6. Shell Programming and Scripting

Compare Two Files(Column By Column) In Perl or shell

Hi, I am writing a comparator script, which comapre two txt files(column by column) below are the precondition of this comparator 1)columns of file are not seperated Ex. file1.txt 8888812341181892 1243548895685687 8945896789897789 1111111111111111 file2.txt 9578956789567897... (2 Replies)
Discussion started by: kumar96877
2 Replies

7. Shell Programming and Scripting

Compare files column to column based on keys

Here is my situation. I need to compare two tab separated files (diff is not useful since there could be known difference between files). I have found similar posts , but not fully matching.I was thinking of writing a shell script using cut and grep and while loop but after going thru posts it... (2 Replies)
Discussion started by: blackjack101
2 Replies

8. Shell Programming and Scripting

column compare of files

Hi i want to compare files a.txt 12345,23 34567,76 65456,10 13467,01 b.txt 12346,23 34567,76 23333,90 65456,10 13467,03 i want o/p in 3 files common.txt both have (2 Replies)
Discussion started by: aaysa123
2 Replies

9. Shell Programming and Scripting

awk compare column between 2 files

Hi, I would like to compare file1 and file2 file1 1 2 3 file2 1 a 2 b 3 c 4 d The result should only print out "d" in file 2. Thanks (3 Replies)
Discussion started by: phamp008
3 Replies

10. Shell Programming and Scripting

Compare Column value from Two Different Files

Hi, I need help to write a korn shell script to 1. Check and compare the first file contains single record from the /scp/inbox directory against the badpnt.dat file from the pnt/badfiles directory contains multiple records based on the fam_id column value start at position 38 to 47 from the... (7 Replies)
Discussion started by: hanie123
7 Replies
Login or Register to Ask a Question