Visit Our UNIX and Linux User Community

Compare Fields from two text files using key columns

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare Fields from two text files using key columns
# 1  
Old 05-12-2010
Compare Fields from two text files using key columns

Hi All,

I have two files to compare. Each has 10 columns with first 4 columns being key index together. The rest of the columns have monetary values.

Using Perl, I want to read one file into hash; check for the key value availability in file 2; then compare the values in the rest of 6 columns; report the differences found.

The files are comman separated and do not have header

Here is the sample file:
File A:
Row1: abcd,abrd,fun,D000,$15,$236,$217,$200,$200,$200
Row2: dear,dare,tun,D000,$12.00405,$234.08976,$212.09876,$200,$200,$200

File B:
Row1: abcd,abrd,fun,D000,$12,$234,$212,$200,$200,$200
Row2: dear,dare,tun,D000,$12.00405,$234.08976,$212.09876,$200,$200,$200

Difference found for index abcd,abrd,fun,D000 for field 5,6 and 7

Any help would be appreciated. I am able to come up with the script in Bash, but not very comfortable with the concept of Hash in PerlImage and also setting up key index columns.


Last edited by vgersh99; 05-12-2010 at 01:10 PM.. Reason: code tags, please!
# 2  
Old 05-12-2010
perl -F, -lane'
    $_{ join $,, @F[ 0 .. 3 ] } = [ @F[ 4 .. $#F ] ] and next
      if @ARGV;
    if ( ( join $,, @F[ 4 .. $#F ] ) ne join $,, @{ $_{ join $,, @F[ 0 .. 3 ] } } )
        @diff = map ++$_,
          grep { $F[$_] ne $_{ join $,, @F[ 0 .. 3 ] }->[ $_ - 4 ] } 4 .. $#F;
        print "Difference found for index ", ( join ",", @F[ 0 .. 3 ] ),
          " for field(s) ", join ",", @diff;
    }' file[ab]

# 3  
Old 05-12-2010
Just curious why, if you can achieve it in your known comfort zone, you would prefer to accomplish it in an area you're not so comfortable with...and possibly unable to support in the long-term?

Perl is no doubt powerful but it can be a bugbear to support if you leverage it using someone else's snippet, without a firm grasp of what it's doing for you. Additionally, it would make scaling that much more difficult. Are you sure you'd prefer this approach?

Silly me...homework, ha!

Last edited by curleb; 05-12-2010 at 05:40 PM.. Reason: duh...

Previous Thread | Next Thread
Test Your Knowledge in Computers #917
Difficulty: Easy
Unix time is also known as Epoch time.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Is there a UNIX command that can compare fields of files with differing number of fields?

Hi, Below are the sample files. x.txt is from an Excel file that is a list of users from Windows and y.txt is a list of database account. $ head -500 x.txt y.txt ==> x.txt <== TEST01 APP_USER_PROFILE USER03 APP_USER_PROFILE TEST02 APP_USER_EXP_PROFILE TEST04 APP_USER_PROFILE USER01 ... (3 Replies)
Discussion started by: newbie_01
3 Replies

2. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

3. Shell Programming and Scripting

Compare fields in two files

Hi, I am trying to check two files based on certain string and field. cat f1 source=\GREP\" hi this \\ source=\SED\" skdmsmd dnksdns source=\PERL\" cat f2 source=\SED\" source=\GREP\" vlamskds amdksk m source=\AWK\" awk \here\" (3 Replies)
Discussion started by: greet_sed
3 Replies

4. UNIX for Dummies Questions & Answers

Compare 2 fields in 2 files

I am trying to compare two files (separted by a pipe) using 2 fields (field 1,3 from fileA and 1,2 from fileB) if the two files match i want the whole record of fileA adding the extra fields left from fileB. 1. A.txt cat|floffy|12|anything|anythings cat|kitty|15|lala|lalala... (6 Replies)
Discussion started by: sabercats
6 Replies

5. Shell Programming and Scripting

Compare two fields in text files?

Hi, I have two text files, compare column one in both the files and if it matches then the output should contain the id in column one, the number and the description. Both the files are sorted. Is there a one liner to get this done, kindly help. Thank you File 1: NC_000964 92.33 ... (2 Replies)
Discussion started by: pulikoti
2 Replies

6. Shell Programming and Scripting

Compare fields in files

Hi, I need the most efficient way of comparing the following and arriving at the result I have a file which has entries like, File1: 1|2|5|7|8|2|3|6|3|1 File2: 1|2|3|1|2|7|9|2 I need to compare the entries in these two file with those of a general file, 1|2|3|5|2|5|6|9|3|1... (7 Replies)
Discussion started by: pradebban
7 Replies

7. Shell Programming and Scripting

compare fields in different files

HI I'm having some troubles to compare and permut diffrent fields indexed with another filed like the following example `: file1 1 1 2 2 3 3 file2 7 1 9 2 10 3 result------------------- (6 Replies)
Discussion started by: yassinegoth
6 Replies

8. Shell Programming and Scripting

how to convert fields from a text file to excel columns

i have this file which has the following contents: ,-0.3000 ,-0.3000 ,-0.3000 ,-0.9000 ,-0.9000 ,-0.9000 i would like to get this: -0.3-0.9-0.3-0.9-0.3-0.9 so far i am trying: awk '{for(i=1; i<=NF; i++) {printf("%f\n",$i)}}' test1 > test2 any help... (4 Replies)
Discussion started by: npatwardhan
4 Replies

9. Shell Programming and Scripting

Compare 2 huge files wrt to a key using awk

Hi Folks, I need to compare two very huge file ( i.e the files would contain a minimum of 70k records each) using awk or sed. The comparison needs to be done with respect to a 'key'. For example : File1 ********** 1234|TONY|Y75634|20/07/2008 1235|TINA|XCVB56|30/07/2009... (13 Replies)
Discussion started by: Ranjani
13 Replies

10. Shell Programming and Scripting

Joining columns from two files, if the key matches

I am trying to join/paste columns from two files for the rows with matching first field. Any help will be appreciated. Files can not be sorted and may not have all rows in both files. Thanks. File1 aaa 111 bbb 222 ccc 333 File2 aaa sss mmmm ccc kkkk llll ddd xxx yyy Want to... (1 Reply)
Discussion started by: sk_sd
1 Replies

Featured Tech Videos