Sponsored Content
Top Forums Shell Programming and Scripting Counting difference in two CSV files Post 302434906 by sam40 on Monday 5th of July 2010 04:34:12 PM
Old 07-05-2010
Counting difference in two CSV files

Hi,

I am new to awk and trying to count the difference between the first columns of two CSV files.

--------
Sample input (header is:name, id1,id2):

file1.csv
Code:
name, id1,id2
sss,34,56
yyy,3,56
www,56,78
pppp,43,12

file2.csv
Code:
name,id1,id2
sss,32,56
yyy,12,7
ttt,4,8
uuu,7,9
eee,6,7


Sample Output:
Code:
Common:2
Different:5 (Dropped:2, Added:3)

--------

I have tried with the following code:
Code:
paste -d"," file1.csv file2.csv | awk 'BEGIN{FS=","}{
if($1!=$4)
    diff++
else if($1==$4)
       comm++
}
END{
print "diff:"count"\n"
print "comm:"comm"\n"
}'

The above code does not work if the order of the data in the input file is changed, even if the data is same.

Any idea/solution? Thank you very much, in advance, for your time and cooperation.

Last edited by radoulov; 07-08-2010 at 12:25 PM.. Reason: Added code tags!
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with counting files please

Hi all. If I have a unix directory with multiple files, lets say, I have some with .dat extensions, some with .txt extensions, etc etc. How in a script would I provide a count of all the different file types (so, the different extensions, I guess) in the directory?? So if I had: test.dat... (6 Replies)
Discussion started by: gerard1
6 Replies

2. Solaris

Counting up files

Hi, I have a load of if statements that look for files in a directory, I want to be able to count them up and the total files confirmed in an email? I ahve tried expr but i this does not work and it only reads in the first if and ignores the rest. Please see script, #!/bin/ksh ... (2 Replies)
Discussion started by: Pablo_beezo
2 Replies

3. Shell Programming and Scripting

Need to compare two csv files values and write into another csv file

Hi all, Am new to scripting. So i just need your ideas to help me out. Here goes my requirement. I have two csv files 1.csv 2.csv abc,1.24 abc,1 def,2.13 def,1 I need to compare the first column of 1.csv with 2.csv and if matches then need to compare... (2 Replies)
Discussion started by: chinnahyd
2 Replies

4. Shell Programming and Scripting

Comparing 2 difference csv files

Hello, I have about 10 csv files which range from csv1 - csv10. Each csv file has same type/set of tabs and we have around 5-6 tabs for each of the csv file which have slightly different content(data). A sample of CSV1 is shown below: Joins: Data related to Joins, it can be any number of... (2 Replies)
Discussion started by: bobby1015
2 Replies

5. Shell Programming and Scripting

Difference between two huge .csv files

Hi all, I need help on getting difference between 2 .csv files. I have 2 large . csv files which has equal number of columns. I nned to compare them and get output in new file which will have difference olny. E.g. File1.csv Name, Date, age,number Sakshi, 16-12-2011, 22, 56 Akash,... (10 Replies)
Discussion started by: Dimple
10 Replies

6. Shell Programming and Scripting

Compare two CSV files and put the difference in third file with line no,field no and diff value.

I am having two csv files i need to compare these files and the output file should have the information of the differences at the field level. For Example, File 1: A,B,C,D,E,F 1,2,3,4,5,6 File 2: A,C,B,D,E,F 1,2,4,5,5,6 out put file: (12 Replies)
Discussion started by: karingulanagara
12 Replies

7. Shell Programming and Scripting

Comparing 2 CSV files and sending the difference to a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 ... (1 Reply)
Discussion started by: Naresh101
1 Replies

8. Shell Programming and Scripting

Compare 2 csv files in ksh and o/p the difference in a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 The... (7 Replies)
Discussion started by: Naresh101
7 Replies

9. Shell Programming and Scripting

Match columns from two csv files and update field in one of the csv file

Hi, I have a file of csv data, which looks like this: file1: 1AA,LGV_PONCEY_LES_ATHEE,1,\N,1,00020460E1,0,\N,\N,\N,\N,2,00.22335321,0.00466628 2BB,LES_POUGES_ASF,\N,200,200,00006298G1,0,\N,\N,\N,\N,1,00.30887539,0.00050312... (10 Replies)
Discussion started by: djoseph
10 Replies
COMM(1) 						    BSD General Commands Manual 						   COMM(1)

NAME
comm -- select or reject lines common to two files SYNOPSIS
comm [-123i] file1 file2 DESCRIPTION
The comm utility reads file1 and file2, which should be sorted lexically, and produces three text columns as output: lines only in file1; lines only in file2; and lines in both files. The filename ``-'' means the standard input. The following options are available: -1 Suppress printing of column 1. -2 Suppress printing of column 2. -3 Suppress printing of column 3. -i Case insensitive comparison of lines. Each column will have a number of tab characters prepended to it equal to the number of lower numbered columns that are being printed. For example, if column number two is being suppressed, lines printed in column number one will not have any tabs preceding them, and lines printed in column number three will have one. The comm utility assumes that the files are lexically sorted; all characters participate in line comparisons. ENVIRONMENT
The LANG, LC_ALL, LC_COLLATE, and LC_CTYPE environment variables affect the execution of comm as described in environ(7). EXIT STATUS
The comm utility exits 0 on success, and >0 if an error occurs. SEE ALSO
cmp(1), diff(1), sort(1), uniq(1) STANDARDS
The comm utility conforms to IEEE Std 1003.2-1992 (``POSIX.2''). The -i option is an extension to the POSIX standard. HISTORY
A comm command appeared in Version 4 AT&T UNIX. BUGS
Input lines are limited to LINE_MAX (2048) characters in length. BSD
January 26, 2005 BSD
All times are GMT -4. The time now is 11:28 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy