Need help with unconventional comparing of CSV


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need help with unconventional comparing of CSV
# 1  
Old 04-24-2010
Need help with unconventional comparing of CSV

Hi All,
Am a newbie at unix, need help for writing a script for comparing two files (csv)and generating two output files (csv)
output1: records from file 1 not found in file 2
output2: records from file 2 not found in file 1 + records from file 2 whose instances (based on column 1) are not equal to instances in file 1

for ex:
file 1
1,1
1,2
2,3
3,3

file 2
1,1
2,3
4,3

output should be
output1:
1,2
3,3

output2:
1,1
4,3

can anyone help???
thanks in advance
-vikaas
# 2  
Old 04-24-2010
output2 example not ok, 1,1 is in the both file.
Split output2, examples about + record ... what you mean.
Code:
#!/bin/ksh or bash or ...
cat file1 | while read line
do
        grep "^$line$" file2 >/dev/null 2>&1 || echo "$line"
done > output1

oifs="$IFS"
cat file2 | while read line
do
        grep "^$line$" file1 >/dev/null 2>&1 || echo "$line" && continue
        # some other rule, if line exist
        IFS=","
        flds=($line)
        IFS="$oifs"
        first=${flds[0]}
        # some rule here
done > output2

# 3  
Old 04-24-2010
for the first part, can be done
Code:
grep -v -f file2 file1 > output1

# 4  
Old 04-24-2010
the line 1,1 is in output file becoz
i need all records from fille2 where the instance (of column 1) is not equql to the instances in file 1.
forex:
in file 1 we have two instances if 1 i.e 1,1 and 1,2
while in file 2 only one instance is found hence 1,1 is in output 2
# 5  
Old 04-25-2010
any help????
# 6  
Old 04-26-2010
Code:
awk -F, '
t<3{++c[t,$1]}
t==1&&(!($0 in f)) {print > "out1" }
t==2{f[$0]}
t==3&&c[1,$1]!=c[2,$1] { print > "out2"}
' t=2 file2 t=1 file1 t=3 file2

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk assistance - Comparing 2 csv files

Hello all, I have searched high and low for a solution to this, many have come really close but not quite what I'm after. I have 2 files. One contains GUID's, for example: 8121E002-96FE-4C9C-BC5A-6AFF20DACECD 84468F30-F3B7-418B-81F0-0908E80792BF A second file, contains a path to the... (8 Replies)
Discussion started by: tirmUK
8 Replies

2. Shell Programming and Scripting

Comparing two CSV files

I have two csv files and im trying to compare them. e.g. SAMPLE DATA: file one: ZipCode Name 20878 Washington 10023 Missouri 20304 Maryland file two: ID Name City ZipCode 11654 ... (11 Replies)
Discussion started by: dan139
11 Replies

3. Shell Programming and Scripting

Comparing two large unsorted csv files

Hi All, My requirement is to write a shell script to compare two large csv files. I've created sample files for explaining my problem i.e., a.csv and b.csv contents of files: ----------------- a.csv ------ Type,Memory (Kb),Location HD,Size (Mb),Serial # XT,640,D402,0,MG0010... (2 Replies)
Discussion started by: vasavi
2 Replies

4. Shell Programming and Scripting

Comparing 2 CSV files and sending the difference to a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 ... (1 Reply)
Discussion started by: Naresh101
1 Replies

5. Shell Programming and Scripting

Comparing 2 difference csv files

Hello, I have about 10 csv files which range from csv1 - csv10. Each csv file has same type/set of tabs and we have around 5-6 tabs for each of the csv file which have slightly different content(data). A sample of CSV1 is shown below: Joins: Data related to Joins, it can be any number of... (2 Replies)
Discussion started by: bobby1015
2 Replies

6. Shell Programming and Scripting

comparing csv files

Hi! I'm just new to shell scripting n simple tasks looks so tough in initial stage. i need to write a script which will read a property file, property file will be containing count of the csv files, and in a folder(same folder) there will be respective csv files. like Property file data1=100... (3 Replies)
Discussion started by: sukhdip
3 Replies

7. Shell Programming and Scripting

Comparing Strings in 2 .csv/txt files?

EDIT: My problems have been solved thanks to the help of bartus11 and pravin27 This code is just to help me learn. It serves no purpose other than that. Here's a sample csv that I'm working with - #listofpeeps.csv Jackie Chan,1954,M Chuck Norris,1930,M Bruce Lee,1940,M This code is... (13 Replies)
Discussion started by: chickeneaterguy
13 Replies

8. Shell Programming and Scripting

Comparing 2 csv files and matching content

Hello, I have the following problem: There are two csv files csv-file #1: aaa1, aaa2, ... aaan aaa1, bbb2, ... bbbn aaa1, ccc2, ... cccn bbb1, bbb2, ... bbbn ... zzz1, zzz2, ... zzzn csv-file #2: aaa1, matchvalue1 ccc1, matchvalue2 (7 Replies)
Discussion started by: ghl10000
7 Replies

9. Shell Programming and Scripting

Help with comparing columns from a csv file

Hi there, I have an csv file. I want to compare the 16th and 18th columns. They contain alpha numeric characters. Some are same and some are different. We have to pick the ones which are different. But with certain rules. 16th col. 18th col. ---------- ... (1 Reply)
Discussion started by: sickboy
1 Replies

10. Shell Programming and Scripting

Last field problem while comparing two csv files

Hi All, I've two .csv files as below file1.csv abc, tdf, 223, tpx jgsd, tex, 342, rpy a, jdjdsd, 423, djfkld Where as file2.csv is the new version of file1.csv with some added fields in the end of each line and some additional lines. lfj, eru, 98, jkldj, 39, jdkj9 abc, tdf, 223, tpx,... (3 Replies)
Discussion started by: ganapati
3 Replies
Login or Register to Ask a Question