Compare 2 csv files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare 2 csv files
# 1  
Old 07-08-2018
Compare 2 csv files

Hello,
I want to compare two csv files expected.csv and actual.csv by fields "f3", "f4", "f5". Field "z" can be differnt.
Each record identified by two fields: "number" +"key".
Records are not sorted.
Can you please help me to get such report:

Error: Mismatch in the record "1,a,33,44,56,333", field "f5". Expected - 56, actual - 55
Error: There is no record "4,d,90,12,56,888" in Expected
Error: There is no record "4,x,90,12,56,333" in Actual

expected.csv:
Code:
number,key,f3,f4,f5,z
1,a,33,44,56,333
1,b,45,56,67,222
2,c,55,56,89,222
3,d,87,88,90,444
4,x,90,12,56,333

actual.csv :
Code:
number,key,f3,f4,f5,z
1,a,33,44,55,999
4,d,90,12,56,888
1,b,45,56,67,777
3,d,87,88,90,777
2,c,55,56,89,777

Thanks in advance

Last edited by Eugenne; 07-08-2018 at 04:33 PM..
# 2  
Old 07-08-2018
It's not remotely clear to me how you get the "expected" output. Did you perhaps forget to include the "input" for that, and a description of how that's to be handled, that would add some context and meaning to the question? Also showing what you've done (code) to get that expected output would be good.
# 3  
Old 07-08-2018
I mean that I want to compare expected.csv and actual.csv, and get report in console or file. So far I can only compare by whole string:
Code:
awk -F',' 'NR==FNR {a[$1]=$0;next;}; !($1 in a) || ($0 != a[$1]);' expected.csv actual.csv

# 4  
Old 07-08-2018
Now, this (or very similar) problem has been solved umpteen times in here. Try
Code:
awk -F, '
function PRERR(VAR)     {print "Error: Mismatch in the record ", $0, ", field f" VAR ". Expected - ", FN[VAR, KEY], " actual - ", $VAR
                        }

                        {KEY = $1 SUBSEP $2
                        }

NR == FNR               {LN[KEY] = $0
                         for (i=3; i<=5; i++) FN[i, KEY] = $i
                         next
                        }

(KEY in LN)             {for (i=3; i<=5; i++) if (FN[i, KEY] != $i) PRERR(i)
                         delete LN[KEY]
                         next
                        }

                        {print "Error: There is no record ", $0, " in Expected."
                         delete LN[KEY]
                        }

END                     {for (l in LN) print "Error: There is no record ", LN[l], " in Actual."
                        }
 
' file[12]
Error: Mismatch in the record  1,a,33,44,55,999 , field f5. Expected -  56  actual -  55
Error: There is no record  4,d,90,12,56,888  in Expected.
Error: There is no record  4,x,90,12,56,333  in Actual.

This User Gave Thanks to RudiC For This Post:
# 5  
Old 07-10-2018
Thank you very much!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare and merge two big CSV files

Hi all, i need help. I have two csv files with a huge amount of data. I need the first column of the first file, to be compared with the data of the second, to have at the end a file with the data not present in the second file. Example File1: (only one column) profile_id 57036226... (11 Replies)
Discussion started by: SirMannu
11 Replies

2. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

3. Shell Programming and Scripting

Compare 2 csv files in ksh and o/p the difference in a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 The... (7 Replies)
Discussion started by: Naresh101
7 Replies

4. Shell Programming and Scripting

How to compare the columns in two .csv files?

Hi I have to compare two .csv files which having 4 columns and i am expecting the output if there is difference in the 3,4columns in two files with respect to the first column. if my statement is not clear please refer the example. Input: ----- File 1 : hostname MAC SWITCH_IP SWITCH_PORT... (7 Replies)
Discussion started by: Kanchana
7 Replies

5. Shell Programming and Scripting

Need Perl script to compare two CSV files

Need perl script to compare the two CSV files and and give out put in CSV format File MsPMTP.csv File ProfileNames.csv MsPMTP.csv is having lines like below JBL_VIJ_A_A962/r01sr4sl12/port#01-#13-Au4P-4c-TMi-PMNETR15 JBL_VIJ_A_A962/r01sr4sl12/port#01-#13-Au4P-4c-TMi-PMFETR15... (9 Replies)
Discussion started by: sreedhargouda
9 Replies

6. Shell Programming and Scripting

Help with script to open and compare csv files

We are testing an application that accesses two tables: A and B. I am to write a script to validate the ouput files of this application.The application marks any account that has become overdue as per rule. When it runs, it updates the overdue flag in the A table according to the following rules: ... (1 Reply)
Discussion started by: inkyponky
1 Replies

7. Shell Programming and Scripting

Need to compare values on two CSV files

:( Hello, Having a problem with reading two files using awk/nawk, am new to both them. I need to compare field values between two csv files and arrange for an appropriate output if both the values are equal or not for each feild. $cat File1.csv... (4 Replies)
Discussion started by: pgop
4 Replies

8. Shell Programming and Scripting

Need to compare two csv files values and write into another csv file

Hi all, Am new to scripting. So i just need your ideas to help me out. Here goes my requirement. I have two csv files 1.csv 2.csv abc,1.24 abc,1 def,2.13 def,1 I need to compare the first column of 1.csv with 2.csv and if matches then need to compare... (2 Replies)
Discussion started by: chinnahyd
2 Replies

9. UNIX for Dummies Questions & Answers

How to compare csv files using perl

I need to compare 2 csv files and report should containg number of matching lines,different lines ,missing lines in one file using perl. I dont want to use read line by line and scan thru the second file for matching line ,as this logic was so time consuming .Can other ideas .please respond asap... (2 Replies)
Discussion started by: kittu1979
2 Replies

10. UNIX for Dummies Questions & Answers

Compare 2 csv files in perl

need to compare 2 csv files and report should containg number of matching lines,different lines ,missing lines in one file using perl. I dont want to use read line by line and scan thru the second file for matching line ,as this logic was so time consuming .Any ideas.i need the soultion badly .... (2 Replies)
Discussion started by: kittu1979
2 Replies
Login or Register to Ask a Question

Featured Tech Videos