Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Need help on comparison of two csv files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need help on comparison of two csv files
# 1  
Old 08-06-2008
Java Need help on comparison of two csv files
Dear All,

I want to compare two csv files using shell programming -

File 1 contents

44,,NONE,0,,2/2/1901
66,,NONE,0,,2/3/1901

File 2 Contents

1022,3708268,AUFX,0,100919,3/1/2006
66,,NONE,0,,2/3/1901

After comparing each column/field I want to print the occurances of difference for each field/column i.e suppose if 44 and 1022 doesn't match then it will be 1, suppose if it matches then it will be 0. My ultimate goal is to count the total number of differences for each field. Any help will be highly appreciated.
# 2  
Old 08-06-2008
Question have you looked at diff & comm
diff = show differences in files
comm = show what is the same in files
# 3  
Old 08-06-2008
Code:
paste file1 file2 | awk -F '\t' '
{
  fnum1 = split($1,a,",")
  fnum2 = split($2,b,",")

  linediff=0
  f=0
  while ( ++f <= fnum1 ) {
    if ( a[f] != b[f] ) { ++diff[f]; ++linediff }
  }
}
END {
  while ( ++n <= fnum1 ) {
    print n,diff[n]+0
  }
}
'

Last edited by cfajohnson; 08-06-2008 at 06:35 PM..
# 4  
Old 08-07-2008
Hi,

Try this one.

input:
Code:
a:
a,30,c
b,30,c
d,27,c
b:
a,29,f
b,30,c
e,26,f

output:
Code:
Filed1 mismatch:1

Filed2 mismatch:2

Filed3 mismatch:2

code:

Code:
paste -d"," a b | awk 'BEGIN{FS=","}{
if($1!=$4)
	a++
if($2!=$5)
	b++
if($3!=$6)
	c++
}
END{
print "Filed1 mismatch:"a"\n"
print "Filed2 mismatch:"b"\n"
print "Filed3 mismatch:"c"\n"
}'

# 5  
Old 08-11-2008
Comparing two csv files
Thanks a lot Summer for this code. This is working fine. I want to put the result in a log file. Thats why I have used echo "">log.txt. But this is not working. I have tried with print "">log.txt also but with no result. Any idea if I want to put this in a log file.
# 6  
Old 08-11-2008
Can you post a sample of the input and its output.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Match columns from two csv files and update field in one of the csv file

Hi, I have a file of csv data, which looks like this: file1: 1AA,LGV_PONCEY_LES_ATHEE,1,\N,1,00020460E1,0,\N,\N,\N,\N,2,00.22335321,0.00466628 2BB,LES_POUGES_ASF,\N,200,200,00006298G1,0,\N,\N,\N,\N,1,00.30887539,0.00050312... (10 Replies)
Discussion started by: djoseph
10 Replies

2. Shell Programming and Scripting

Compare 2 csv files in ksh and o/p the difference in a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 The... (7 Replies)
Discussion started by: Naresh101
7 Replies

3. Shell Programming and Scripting

Comparing 2 CSV files and sending the difference to a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 ... (1 Reply)
Discussion started by: Naresh101
1 Replies

4. Shell Programming and Scripting

Comparison of two files

Hi all I have two files which I have to compare that whetehr there is soemthing common or not body, div, table, thead, tbody, tfoot, tr, th, td, p { font-family: "Liberation Sans"; font-size: x-small; } body, div, table, thead, tbody, tfoot,... (2 Replies)
Discussion started by: manigrover
2 Replies

5. Shell Programming and Scripting

csv file comparison with normal file

i will like to give my files as fileB : 02.09.2011:MOU04/KUBIS--R_Lieferung_1/KUBIS_V15.0/vpn_wls_15-0-0b01 02.09.2011:MOU04/KUBIS--R_Lieferung_2/KUBIS_V15.0/apng_wls_15-0-0b02 31 02.09.2011:MOU04/KUBIS--R_Lieferung_2/KUBIS_V15.0/ecc_wls_15-0-0b02 32 ... (2 Replies)
Discussion started by: rajniman
2 Replies

6. Shell Programming and Scripting

CSV file comparison

Hi all, i have two .csv files. i need to compare those two files and if there is any difference that should be moved into third .csv file. example, org.csv and dup.csv when we compare those two files org.csv and dup.csv. if there is any change in dup.csv. it should be capture in third... (7 Replies)
Discussion started by: baskivs
7 Replies

7. Shell Programming and Scripting

csv 4 columns values comparison!

Hi all, i have a csv file which as the following data: 294;F03;2000;40441 294;F03;2000;40443 284;F01;5400;44051 284;F01;5700;45666 the file holds 11689 lines. I was trying to get a script running to output results from this file that for each line with the condition: if a line is found... (9 Replies)
Discussion started by: stryng
9 Replies

8. Shell Programming and Scripting

Comparison of two files (sh)

Hi, I have a problem with comparison of two files file1 20100101 20090101 20080101 20071001 20121229 file2 19990112 12 456 7 20011131 19 20100101 2 567 1 987 17890709 123 555 and, sh script needs to compare of these two files and give out to me result: 20100101 2 567 1 987 it... (5 Replies)
Discussion started by: shizik
5 Replies

9. Shell Programming and Scripting

Need to compare two csv files values and write into another csv file

Hi all, Am new to scripting. So i just need your ideas to help me out. Here goes my requirement. I have two csv files 1.csv 2.csv abc,1.24 abc,1 def,2.13 def,1 I need to compare the first column of 1.csv with 2.csv and if matches then need to compare... (2 Replies)
Discussion started by: chinnahyd
2 Replies

10. Shell Programming and Scripting

comparison of 2 files

Kindly help on follows. I have 2 files. One file contains only one column of mobile numbers. And total records in a file 12 million. Second file contains 2 columns mobile numbers and balance. and total records 30 million. I want to find out balance of each data in file 1 corresponding to file 2.... (2 Replies)
Discussion started by: kamal_418
2 Replies
Login or Register to Ask a Question