Need help on comparison of two csv files

sourav1982 · #1 (**permalink**) 08-06-2008

Dear All,

I want to compare two csv files using shell programming -

File 1 contents

44,,NONE,0,,2/2/1901
66,,NONE,0,,2/3/1901

File 2 Contents

1022,3708268,AUFX,0,100919,3/1/2006
66,,NONE,0,,2/3/1901

After comparing each column/field I want to print the occurances of difference for each field/column i.e suppose if 44 and 1022 doesn't match then it will be 1, suppose if it matches then it will be 0. My ultimate goal is to count the total number of differences for each field. Any help will be highly appreciated.

joeyg · #2 (**permalink**) 08-06-2008

diff = show differences in files
comm = show what is the same in files

cfajohnson · #3 (**permalink**) 08-06-2008

Code:

paste file1 file2 | awk -F '\t' '
{
  fnum1 = split($1,a,",")
  fnum2 = split($2,b,",")

  linediff=0
  f=0
  while ( ++f <= fnum1 ) {
    if ( a[f] != b[f] ) { ++diff[f]; ++linediff }
  }
}
END {
  while ( ++n <= fnum1 ) {
    print n,diff[n]+0
  }
}
'

summer_cherry · #4 (**permalink**) 08-06-2008

Hi,

Try this one.

input:

Code:

a:
a,30,c
b,30,c
d,27,c
b:
a,29,f
b,30,c
e,26,f

output:

Code:

Filed1 mismatch:1

Filed2 mismatch:2

Filed3 mismatch:2

code:

Code:

paste -d"," a b | awk 'BEGIN{FS=","}{
if($1!=$4)
	a++
if($2!=$5)
	b++
if($3!=$6)
	c++
}
END{
print "Filed1 mismatch:"a"\n"
print "Filed2 mismatch:"b"\n"
print "Filed3 mismatch:"c"\n"
}'

sourav1982 · #5 (**permalink**) 08-11-2008

Thanks a lot Summer for this code. This is working fine. I want to put the result in a log file. Thats why I have used echo "">log.txt. But this is not working. I have tried with print "">log.txt also but with no result. Any idea if I want to put this in a log file.

shamrock · #6 (**permalink**) 08-11-2008

Can you post a sample of the input and its output.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
comparison of 2 files	kamal_418	Shell Programming and Scripting	2	08-04-2008 06:05 AM
need some help..Comparison	bluesilo	Shell Programming and Scripting	0	02-23-2008 04:43 PM
Comparison of 2 files in UNIX	Dana Evans	UNIX for Dummies Questions & Answers	32	11-21-2007 07:05 AM
Comparison of two files in awk	jerome Sukumar	Shell Programming and Scripting	12	07-26-2006 08:16 AM
String Comparison between two files using awk	rudoraj	Shell Programming and Scripting	7	07-25-2006 11:04 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode	Rate This Thread: