Compare two files and remove all the contents of one file from another


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare two files and remove all the contents of one file from another
# 1  
Old 11-17-2008
Compare two files and remove all the contents of one file from another

Hi,

I have two files, in which the second file has exactly the same contents of the first file with some additional records. Now, if I want to remove those matching lines from file2 and print only the extra contents which the first file does not have, I could use the below unsophisticated command, consider f1 and f2 are the two files
Code:
var=`cat f1`
grep -v "$var" f2

but I need a more optimal solution with fast and reliable with less memory consumption.

I have found these 2 lines of code, but it does not work for files having lengthier lines:

Code:
fgrep -v -x -f f2 f1  
awk 'NR==FNR {b[$0]; next} !($0 in b)' f2 f1


Last edited by royalibrahim; 11-18-2008 at 12:51 AM..
# 2  
Old 11-17-2008
Hammer & Screwdriver Perhpas the diff command will work for you

Code:
> cat file70
abc
def
ghi
jkl
mno
pqr
stu
vwx
yz
123
456
789
0

> cat file71
abc
def
ghi
jkl
mno
pqr
stu
vwx
yz
bash ksh
123
456
789
0
unix.com

> diff file70 file71 | grep "^>" | cut -c3-
bash ksh
unix.com

# 3  
Old 11-17-2008
Hi,

to print the different lines of two files try:

Code:
comm -3 file1 file2

and for further informations

Code:
man comm

Kind regards

Chris
# 4  
Old 11-18-2008
Thanks to you all for the suggestions. But anyone has any awk, perl code to do this task?

And Also, the below perl code will remove duplicate, non-consecutive lines based on the last field without sorting. Now, please tell me, what should I change in this code in order to print unique lines of a file by just not seeing the last field but the entire line (the whole record)?
Code:
perl -ane'print unless $_{$F[-1]}++'

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script (sh file) logic to compare contents of one file with another file and output to file

Shell script logic Hi I have 2 input files like with file 1 content as (file1) "BRGTEST-242" a.txt "BRGTEST-240" a.txt "BRGTEST-219" e.txt File 2 contents as fle(2) "BRGTEST-244" a.txt "BRGTEST-244" b.txt "BRGTEST-231" c.txt "BRGTEST-231" d.txt "BRGTEST-221" e.txt I want to get... (22 Replies)
Discussion started by: pottic
22 Replies

2. Shell Programming and Scripting

How to compare two file contents which created within one hour?

Hi Gurus, I need to compare two file contents and log the difference. the detail as below: in current directory there is file abc20140728_1020.txt abc20140728_1110.txt I need to find the files which created within 60 minutes first then compare the contents. I am able to use find... (12 Replies)
Discussion started by: ken6503
12 Replies

3. UNIX for Dummies Questions & Answers

How to remove // inbetween the files contents?

Hi Am Using unix Aix Have a file name called FILE1 having the content of FILE1 is cat FILE1 01/11/2012 03/11/2012 // // // 04/11/2012 I Need Output as:- cat FILE1 01/11/2012 03/11/2012 04/11/2012 (7 Replies)
Discussion started by: Venkatesh1
7 Replies

4. Shell Programming and Scripting

compare files and remove a line from a file if first column is greater than 25

my files are as follows fileA sepearated by tab /t 00 lieferungen 00 attractiop 01 done 02 forness 03 rasp 04 alwaysisng 04 funny 05 done1 fileB funnymou120112 funnymou234470 mou3raspnhdhv rddfgmoudone1438748 so all those record which are greater than 3 and which are not... (4 Replies)
Discussion started by: rajniman
4 Replies

5. Shell Programming and Scripting

script to grep a pattern from file compare contents with another file and replace

Hi All, Need help on this I have 2 files one file file1 which has several entries as : define service{ hostgroup_name !host1,!host5,!host6,.* service_description check_nrpe } define service{ hostgroup_name !host2,!host4,!host6,.* service_description check_opt } another... (2 Replies)
Discussion started by: namitai
2 Replies

6. Shell Programming and Scripting

Compare contents of two files

Hello, I have two files containing both two columns: $ cat file1 col1 nbr1 col2 nbr2 col3 nbr3 ... $ cat file2 val1 ri1 val2 ri2 val3 ri3 ... I need to compare every nbr (nbr1, nbr2,...) in the second column of file1 with every ri (ri1, ri2,...) in the second column of... (5 Replies)
Discussion started by: newpromo
5 Replies

7. Shell Programming and Scripting

shell script to compare file contents

Hello Has anyone got an example shell script that I can use to compare the contents of two files. The files should contain the same contents, eg. file1.txt apple pear grape file2.txt apple pear grape (2 Replies)
Discussion started by: deedaz
2 Replies

8. UNIX for Dummies Questions & Answers

compare 2 file contents , if same delete 2nd file contents

Give shell script....which takes two file names as input and compares the contents, is both are same delete second file's contents..... I try with "diff"...... but confusion how to use "diff" with if ---else Thanking you (5 Replies)
Discussion started by: krishnampkkm
5 Replies

9. Shell Programming and Scripting

Compare & replace contents within a file

I have 2 files file1 1 TMQUEUE QUE1 STMW633A 100 DMADM DOMGRPSTMW633A STMW633A 100 GWADM GWTGRPSTMW633A STMW633A 100 GWADM GWTGRPSTMW633AA STMW633A 100 GWADM GWTGRPSTMW638A STMW638A 100 TMSYSEVT EVTGRPSTMW633A STMW633A 100 TMSYSEVT ... (2 Replies)
Discussion started by: kaustubh137
2 Replies

10. UNIX for Dummies Questions & Answers

compare array contents with file

I have an array "arrA" with the following contents: A0012 Paint Shop.doc ES001 Contract Signature.doc Budget Plan.pdf TS PWS.pdf My data file "Files.dat" has the same contents: A0012 Paint Shop.doc ES001 Contract Signature.doc Budget Plan.pdf TS PWS.pdf I have a script that compares... (0 Replies)
Discussion started by: orahi001
0 Replies
Login or Register to Ask a Question