awk script to parse results from TWO files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk script to parse results from TWO files
# 1  
Old 05-10-2011
awk script to parse results from TWO files

I am trying to parse two files and get data that does not match in one of the columns ( column 3 in my case )

Data for two files are as follows

A.txt
=====
Code:
abc   10    5   0  1  16
xyz    16    1   1  0  18  
efg    30    8   0   2  40
ijk      22    2   0   1  25

B.txt
=====
Code:
abc   10    5   0  1  16
xyz    13    4   1  0  18  
efg    30    8   0   2  40
ijk      17    7   0   1  25

I am trying to get out put like following...if col3 value of B.txt is greater than col3 of A.txt then corresponding col1 and difference between the col3 values.

xyz 3 ---> basically col1R2 (B.col2R2 - A.col2R2)
ijk 5 ---> basically col1R4 (B.col2R4 - A.col2R4)

I tried the following to get the rows that differ..but I am getting all of the B.txt rows..
Code:
awk  'FILENAME=="A.txt" {arr[$0]++} FILENAME=="B.txt"  {if ($3 > arr[$3]) {print $1 "\t" $3}}' A.txt B.txt

I think if condition is not getting evaluated properly....any hint will be appreciated !!

- Roger

Last edited by Franklin52; 05-11-2011 at 03:45 AM.. Reason: Please use code tags
# 2  
Old 05-10-2011
I think that the following would produce the output you expect:

Code:
awk 'FILENAME == "A.txt" { arr[$1]=$3 } FILENAME == "B.txt" && arr[$1] != $3 { print $1 ,$3 - arr[$1] }' A.txt B.txt

What I couldn't understand from your code is this statement:

Code:
arr[$0]++

$0 contains the entire current record (current line). What is your idea here?
This User Gave Thanks to pflynn For This Post:
# 3  
Old 05-10-2011
Code:
awk 'NR==FNR{a[$1]=$3;next}$3>a[$1]{print $1,$3-a[$1]}' A B
xyz 3
ijk 5

This User Gave Thanks to yinyuemi For This Post:
# 4  
Old 05-10-2011
Thanks yinyuemi and pflynn !! Both solutions work !!

pflynn , Yes you are right, I was doing wrong in defining the array....I thought it will yield me just the col3 value when I compare...but I suppose it was taking whole row and trying to compare...

do you guys explain this part of code , { arr[$1]=$3 } ?
# 5  
Old 05-10-2011
Quote:
Originally Posted by roger67
Thanks yinyuemi and pflynn !! Both solutions work !!

pflynn , Yes you are right, I was doing wrong in defining the array....I thought it will yield me just the col3 value when I compare...but I suppose it was taking whole row and trying to compare...

do you guys explain this part of code , { arr[$1]=$3 } ?

Hi roger67,

Code:
arr[$1]=$3

means to build up an array named by "arr", indexed as $1, its corresponding value is $3.

Best,

Y
This User Gave Thanks to yinyuemi For This Post:
# 6  
Old 05-10-2011
Code:
arr[$1]=$3

What we are doing here is creating an array, whose indexes are the contents of the first column of each line ("abc", "xyz", etc), and the values are the corresponding third element of each line. For example, the first element of the array is arr[abc], whose value is 5. Notice that this operation is done when we are reading file A.txt. After we are done reading A.txt, we start reading file B.txt, then we can use column 1 (element $1) of each line as index to retrieve back the corresponding third column of file A.txt from the array, and compare it to the current element from the trhird column ($3). If they are different, we print them.

Code:
arr[$1] != $3

This User Gave Thanks to pflynn For This Post:
# 7  
Old 05-11-2011
Thanks guys !!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Comparing 2 files using awk , not getting any results - C shell

I am using c shell and trying to compare 2 files using awk . But the below awk statement doesnt give any result. Pls. advise why am not getting the desired o/p with the corrected awk script. Need to acheive this solution in awk using C shell. awk 'FNR==NR{a++;next} {for(i in a) {if ( a=$0... (8 Replies)
Discussion started by: reach2khan
8 Replies

2. Shell Programming and Scripting

Parse input of two files to be the same in awk

I have two files that I am going to use diff to find the differences but need to parse them before I do that. I have include the format of each file1 and file2 with the desired output of each (the first 5 fields in each file). The first file has a "chr" before the # that needs to be removed. I... (1 Reply)
Discussion started by: cmccabe
1 Replies

3. Shell Programming and Scripting

awk parse snmp results

i run the command snmptable -v2c -c public myIP IF-MIB::ifTable the result look like this : SNMP table: IF-MIB::ifTable ifIndex ifDescr ifType ifMtu ifSpeed ifPhysAddress ifAdminStatus ifOperStatus 1 Unit: 1 Slot: 0 Port: 1... (7 Replies)
Discussion started by: wanttolearn1
7 Replies

4. Shell Programming and Scripting

awk to parse huge files

Hello All, I have a situation as below: (1) Read a source file (a single file of 1.2 million rows in it ) (2) Read Destination files one by one and replace the content ( few fields in it ) with the corresponding matching field from source file. I tried as below: ( please note I am not... (4 Replies)
Discussion started by: panyam
4 Replies

5. Shell Programming and Scripting

awk help: Match data fields from 2 files & output results from both into 1 file

I need to take 2 input files and create 1 output based on matches from each file. I am looking to match field #1 in both files (Userid) and create an output file that will be a combination of fields from both file1 and file2 if there are any differences in the fields 2,3,4,5,or 6. Below is an... (5 Replies)
Discussion started by: ambroze
5 Replies

6. Shell Programming and Scripting

AWK failing to parse on certain files

Dear Unix Gurus, need your expertise to help troubleshoot a certain problem i'm having. I crated a shell script which will ftp get 1 crash log from multiple servers (listed in a text file). Each log will then be parsed by calling an awk script. The problem is, for certain log its parsing... (7 Replies)
Discussion started by: tarj
7 Replies

7. Shell Programming and Scripting

awk - Matching columns between 2 files and reordering results

I am trying to match 4 colums (first_name,last_name,dob,ssn) between 2 files and when there is an exact match I need to write out these matches to a new file with a combination of fields from file1 and file2. I've managed to come up with a way to match these 2 files based on the columns (see below)... (7 Replies)
Discussion started by: ambroze
7 Replies

8. Shell Programming and Scripting

awk script giving unstable results

Hi all Here I came accross a situation which i am unable to reason out... snippet 1 psg ServTest | grep -v "grep" | grep -v "vi" | awk '{ pgm_name=$8 cmd_name="ServTest" gsub(/]*/,"",pgm_name) if(pgm_name==cmd_name) { print "ServTest Present =" cmd_name} }'... (10 Replies)
Discussion started by: Anteus
10 Replies

9. Shell Programming and Scripting

Can awk do lookups to other files and process results

I know that 'brute-force' scripting could accomplish this with lots of cat/echo/cut/grep and more. But, because my real file has 800k records, and the matching files have 10-20k records, this is not time-possible or efficient. I have input file: > cat file_in... (4 Replies)
Discussion started by: joeyg
4 Replies

10. Shell Programming and Scripting

Shell Script Needed to Parse Results

Raw Results: results|192.168.2|192.168.2.1|general/udp|10287|Security Note|For your information, here is the traceroute from 192.168.2.24 to 192.168.2.1 : \n192.168.2.24\n192.168.2.1\n\n results|192.168.2|192.168.2.1|ssh (22/tcp)|22964|Security Note|An SSH server is running on this port.\n... (2 Replies)
Discussion started by: jroberson
2 Replies
Login or Register to Ask a Question