Finding differences between 2 text files


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Finding differences between 2 text files
# 1  
Old 11-04-2010
Finding differences between 2 text files

Hi everyone,

I know that's a deep treated issue but I'm actually not able to find the solution. I have 2 plain text files with ~ 2000 rows and ~5 columns. The first column of the shortest file (f1) is fully contained by the first column of the biggest one (f2), but only that column. I want to create a third file that would contain all the rows which first column is in f2 but not in f1. I mean:

f1:
aa 111 111

f2:
aa 555 555
bb 666 666
cc 777 777

f3:
bb 666 666
cc 777 777

I've tried to do it with GREP but it doesn't work (I think it's too much sensitive to the spaces so the f3 is not complete). Exactly, what I tried was:

grep -f f1 -v -w f2 > out.list and many others variants like without -w, with -E, -F...

Any ideas?

PS: sorry because of my latenight English, it's kind a late here in Spain Smilie
# 2  
Old 11-04-2010
Try this

Code:
bash-3.2$ cat f1.txt
aa 111 111
bash-3.2$ cat f2.txt
aa 555 555
bb 666 666
cc 777 777
bash-3.2$ awk '{print "grep -v "$1" f2.txt"}' f1.txt |sh
bb 666 666
cc 777 777

This User Gave Thanks to phoenix_nebula For This Post:
# 3  
Old 11-04-2010
Thanks, but it doesn't work. It seems to be stuck. The size of the output files is extremely large (187 Mb). There are too many rows on it.
# 4  
Old 11-05-2010
Try:
Code:
 awk 'NR==FNR{A[$1];next}!($1 in A)' f1 f2

This User Gave Thanks to Scrutinizer For This Post:
# 5  
Old 11-05-2010
Thank you very much Scrutinizer!! It totally works!

Now there are 1077 galaxies waiting for been examinated.

I owe you one.

Nacho
# 6  
Old 11-05-2010
Quote:
Originally Posted by Scrutinizer
Try:
Code:
 awk 'NR==FNR{A[$1];next}!($1 in A)' f1 f2

@Scruti
I don't get that one, could you please explain the code ?
I mean : f1 and f2 ... how awk does know which one it should take and when ?
What is between the simple quote apply to f1 ? or f2 ? or both ?
Why NR==FNR ? to make sure an associative array is set up for all records ? of f2 ?
# 7  
Old 11-06-2010
Hi ctsgnb, NR is only equal equal to FNR when awk is reading the first file.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ksh / AIX - Differences between lists to a text file

This seems pretty simple, but I cant figure it out. I get stumped on the simple things. I am running two commands 1) take a listing a directory of files, and filter out the doc_name (which is in a series of extracted files), and place it in a file. ls -l | awk '{print $9}' | grep... (5 Replies)
Discussion started by: jeffs42885
5 Replies

2. Shell Programming and Scripting

Finding a text in files & replacing it with unique strings

Hallo Everyone. I have to admit I'm shell scripting illiterate . I need to find certain strings in several text files and replace each of the string by unique & corresponding text. I prepared a csv file with 3 columns: <filename>;<old_pattern>;<new_pattern> ... (5 Replies)
Discussion started by: gordom
5 Replies

3. Shell Programming and Scripting

Finding a text file from a group of zip files without unzipping

HI , There are more than 100 zip files in a directory and i wanted to see if there is a max1157.txt file in any of the zip files without actually unzipping them. Could you please help. Thanks in Advance. Karthik. (6 Replies)
Discussion started by: karthikk0508
6 Replies

4. UNIX for Dummies Questions & Answers

Compare and merging the differences in text file

Hi i have gone through some sdiff command it shows the differences side by side and its really awesome file 1: this tool is for checking the differ merging with flower pots documentation file 2: this t ool is for checking the differ mergin g with flower pots documentation ... (27 Replies)
Discussion started by: rakeshkumar
27 Replies

5. Shell Programming and Scripting

Differences between 2 Flat Files and process the differences

Hi Hope you are having a great weeknd !! I had a question and need your expertise for this : I have 2 files File1 & File2(of same structure) which I need to compare on some columns. I need to find the values which are there in File2 but not in File 1 and put the Differences in another file... (5 Replies)
Discussion started by: newbie_8398
5 Replies

6. Shell Programming and Scripting

Detect differences in two files

All, I have two csv files, the format of which are exactly the same. I would like to find differences between the two files but would like to identify the difference as opposed to just printing a different line. For exmaple File 1 xxx,yyy,zzz,1,2,3 111,222,333,xxx,yyy ... (4 Replies)
Discussion started by: pxy2d1
4 Replies

7. Shell Programming and Scripting

Compare two text files and Only show the differences

Hi experts, I'mvery new to shell scripting and learning it now currently i am having a problem which may look easy to u :) i have two files File 1: Start :Thu Nov 19 10:33:09 2009 ABCDGFSDJ.txt APDemoNew.ppt APDemoOutline.doc ARDemoNew.ppt ARDemoOutline.doc File 2: Start... (10 Replies)
Discussion started by: CelvinSaran
10 Replies

8. Solaris

Differences between jar files

I want to find the difference between two jar files sitting on a sun box. How do I do this? (3 Replies)
Discussion started by: runnerpaul
3 Replies

9. Shell Programming and Scripting

finding duplicate files by size and finding pattern matching and its count

Hi, I have a challenging task,in which i have to find the duplicate files by its name and size,then i need to take anyone of the file.Then i need to open the file and find for more than one pattern and count of that pattern. Note:These are the samples of two files,but i can have more... (2 Replies)
Discussion started by: jerome Sukumar
2 Replies

10. UNIX for Dummies Questions & Answers

Number of differences between 2 files

Hi, "diff" command takes two file names as arguements and gives the difference between the two. How do I get the number of differences between two files ??? (Excluding whitespaces). Don't ask me to count number of lines produced by "diff". Thanks in advance, Sharath (4 Replies)
Discussion started by: sharuvman
4 Replies
Login or Register to Ask a Question