Shell Programming and Scripting

Hi All,

My requirement is to write a shell script to compare two large csv files.
I've created sample files for explaining my problem
i.e., a.csv and b.csv

contents of files:
-----------------
a.csv
------

Code:

Type,Memory (Kb),Location HD,Size (Mb),Serial #
XT,640,D402,0,MG0010
386,2048,D403,100,MG0011
486,4096,D404,270,MG0012
386,8192,A423,400,CC0177
486,8192,A424,670,CC0182
286,4096,A423,C100,C0183
286,4096,A425,80,CC0184
Mac,4096,B407,80,EE1027

b.csv
------

Code:

Type,Memory (Kb),Serial #,Location HD
XT,640,MG0010,D402
386,2048,MG0011,D403
486,4096,CC0177,D404
386,8192,MG0012,A423
486,8192,C0183,A424
286,4096,CC0182,A423
286,4096,CC0184,A425
Mac,4096,EE1027,B407

cases to consider:
1. In a.csv and b.csv the column order may differ as shown above and in any of the files few extra columns might be there which are not present in other file.
2.The comparision should be like, for each column data in a.csv we should compare the matching column data in b.csv (ex., data for column "Type" from a.csv with data for column "Type" from b.csv) and write the DATA DIFFERENCE with FILE NAME, COLUMN NAME, COLUMN NUMBER and ROW NUMBER to a log file.

Here in a.csv one column is extra -> Size (Mb), we need to capture the extra columns as well in the same log.

Finally, the log file should contain something like the following data:

Code:

Filename: a.csv, Filename:b.csv    
 
serial #        serial #    
 
3 MG0012        CC0177
4 CC0177        MG0012
5 CC0182        A424
6 C0183        A423
7 CC0184        A425
8 EE1027        B407

Extra columns in a.csv Extra columns in b.csv

Size (Mb) column at field 4

Any help would be greatly appreciated.....

What have you tried so far?

Regards and welcome to the forum,
Alister

Tried with ulitilities like awk, sed etc., but not getting how to sort data based on column names common to both the files alphabetically so that extra cols would be at last in both files which will be easy for comparing.
Any scripts available for references would be of great help.