My requirement is to write a shell script to compare two large csv files.
I've created sample files for explaining my problem
i.e., a.csv and b.csv
contents of files:
-----------------
a.csv
------
b.csv
------
cases to consider:
1. In a.csv and b.csv the column order may differ as shown above and in any of the files few extra columns might be there which are not present in other file.
2.The comparision should be like, for each column data in a.csv we should compare the matching column data in b.csv (ex., data for column "Type" from a.csv with data for column "Type" from b.csv) and write the DATA DIFFERENCE with FILE NAME, COLUMN NAME, COLUMN NUMBER and ROW NUMBER to a log file.
Here in a.csv one column is extra -> Size (Mb), we need to capture the extra columns as well in the same log.
Finally, the log file should contain something like the following data:
Extra columns in a.csv Extra columns in b.csv
Size (Mb) column at field 4
Any help would be greatly appreciated.....
Last edited by vasavi; 06-18-2013 at 04:32 AM..
Reason: code tags
Hi All,
I've two .csv files as below
file1.csv
abc, tdf, 223, tpx
jgsd, tex, 342, rpy
a, jdjdsd, 423, djfkld
Where as file2.csv is the new version of file1.csv with some added fields in the end of each line and some additional lines.
lfj, eru, 98, jkldj, 39, jdkj9
abc, tdf, 223, tpx,... (3 Replies)
EDIT: My problems have been solved thanks to the help of bartus11 and pravin27
This code is just to help me learn. It serves no purpose other than that.
Here's a sample csv that I'm working with -
#listofpeeps.csv
Jackie Chan,1954,M
Chuck Norris,1930,M
Bruce Lee,1940,M
This code is... (13 Replies)
Hi Guys,
I'm a complete shell scripting newbie and need some help with comparing a file against a master file and outputting the results.
master.txt would look something like this:
000123
000345
000341
000927
000762
000235
000155
000452
000846
000623
file.txt would look like... (1 Reply)
Hi!
I'm just new to shell scripting n simple tasks looks so tough in initial stage. i need to write a script which will read a property file, property file will be containing count of the csv files, and in a folder(same folder) there will be respective csv files.
like Property file
data1=100... (3 Replies)
Hi,
I have two very large CSV files, which I want to merge (equi-join) based on a key (column).
One of the file (say F1) would have ~30 MM records and 700 columns. The other file (~f2) would have same # of records and lesser columns (say 50). I want to create an output file joining on a... (3 Replies)
Hello,
I have about 10 csv files which range from csv1 - csv10.
Each csv file has same type/set of tabs and we have around 5-6 tabs for each of the csv file which have slightly different content(data).
A sample of CSV1 is shown below:
Joins: Data related to Joins, it can be any number of... (2 Replies)
I have two csv files and im trying to compare them. e.g.
SAMPLE DATA:
file one:
ZipCode Name
20878 Washington
10023 Missouri
20304 Maryland
file two:
ID Name City ZipCode
11654 ... (11 Replies)
Hello all,
I have searched high and low for a solution to this, many have come really close but not quite what I'm after.
I have 2 files. One contains GUID's, for example:
8121E002-96FE-4C9C-BC5A-6AFF20DACECD
84468F30-F3B7-418B-81F0-0908E80792BF
A second file, contains a path to the... (8 Replies)