compare two files and to remove the matching lines on both the files

shellscripter · #1 (**permalink**) 06-19-2009

I have two files and need to compare the two files and to remove the matching lines from both the files

jim mcnamara · #2 (**permalink**) 06-19-2009

Code:

awk '{arr[$0]++} END {for (i in arr) {if (arr[i]>1) {print $0} }} ' file1 file2 > dupfile
awk 'FILENAME=="dupfile" {arr[$0]++}
       FILENAME=="file1" {if ($0 in arr) {next} else {print $0 >"newfile1"}}
       FILENAME=="file2" {if ($0 in arr) {next} else {print $0 >"newfile2"} }' dupfile file1 file2

Not tested.

King Kalyan · #3 (**permalink**) 06-19-2009

Try this...

Code:

 
awk '{ if (FNR==NR) {arr[$0]++;next}
if ($0 in arr) { arr[$0]--; if (arr[$0] == 0) delete arr[$0]; next}
{print $0 >"newfile2"} } 
END {for (i in arr) {print i >"newfile1"} } ' file1 file2

Jim: The code you have given has potential to remove duplicate lines in one file even though no matching record present in other file because those lines has multiple instances in a single file.

kingpeejay · #4 (**permalink**) 06-20-2009

I also have this kind of problem. The code above deletes the lines which are alike on the two files. How can i do this, on a separate file?
File 3:
#from first file
a3px lm4 xpm
0wmp jga1 wmp6

#from second file
a3px lm4 xpj
0xmp lm4 xpm

thanks a lot!

scottn · #5 (**permalink**) 06-20-2009

maybe?

Code:

awk '! LINE[$0] { print; LINE[$0] = $0}' in_file

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
compare two files for matching in solaris	amir07	Shell Programming and Scripting	8	01-02-2009 06:40 PM
Compare two files and remove all the contents of one file from another	royalibrahim	Shell Programming and Scripting	3	11-18-2008 12:55 AM
compare files by lines and columns	giviut	Shell Programming and Scripting	4	01-17-2008 06:00 AM
Trying to compare lines in 2 files	brdholman	Shell Programming and Scripting	2	09-20-2007 07:46 AM
Find matching lines between 2 files	jojojmac5	UNIX for Dummies Questions & Answers	5	01-18-2007 01:06 PM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread:


	Powered by