File Compare at field level


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File Compare at field level
# 1  
Old 02-11-2014
File Compare at field level

Hi,

I am trying to compare two fixed width files as shown below. The file is NOT sorted. The field in bold red is the key field. The comparison needs to be based of key fields and not whole record. But needs to write out the whole record in the output.

Code:
OldFile.txt:
A100135123456789  firstname mi lastname .......
A100112123456677  firstname mi lastname .......
A189135123456777  firstname mi lastname .......

NewFile.txt:
A100135123456789  firstname mi lastname .......
B122112123456666  firstname mi lastname .......
C123135123456444  firstname mi lastname .......


I need to compare the Old file and New file based on their key field in boldred (not the whole record) and generate two files: NewRec.txt and DropRec.txt. These two files will contain the whole record even though the comparison is done only on the key fields.

NewRec.txt: Key field that was not there in Old file but exists in New file. So comparing the key fields in the file above, the output should be
B122112123456666 firstname mi lastname .......
C123135123456444 firstname mi lastname .......

DropRec.txt: Key field that was there in Old file but NOT in new file. So comparing the key fields in the file above, the output should be:
A100112123456677 firstname mi lastname .......
A189135123456777 firstname mi lastname .......

The comparison is on the key field and not the whole file. The file is fixed width. The key field is from character 1 thru 22. Even though the text is only there from char 1 thru 16 and padded with spaces till 22 character.

Will really appreciate any help.

Thanks
# 2  
Old 02-11-2014
Code:
awk 'NR==FNR{A[$1];next}($1 in A)' OldFile.txt NewFile.txt

awk 'NR==FNR{A[$1];next}!($1 in A)' OldFile.txt NewFile.txt

You can modify the sequence of input files to get desired results.
# 3  
Old 02-12-2014
Thanks for suggestion. I tried the suggested code but somehow getting the error

Code:
Below is the code I tried:

#!/bin/ksh

pfile=oldfile.txt
cfile=currentfile.txt

awk 'NR==FNR{A[$1];next}($1 in A)' ${pfile} ${cfile} #> NewRec.txt

awk 'NR==FNR{A[$1];next}!($1 in A)' ${pfile} ${cfile} #> OldRec.txt

Code:
The error I am getting is:
awk: syntax error near line 1
awk: bailing out near line 1
awk: syntax error near line 1
awk: bailing out near line 1

Thanks
# 4  
Old 02-12-2014
Use nawk or /usr/xpg4/bin/awk instead on Solaris / SunOS
# 5  
Old 02-12-2014
Thank you so much....the code below is working perfectly now.

Code:
nawk 'NR==FNR{A[$1];next}!($1 in A)' ${pfile} ${cfile} > NewRec.txt
nawk 'NR==FNR{A[$1];next}!($1 in A)' ${cfile} ${pfile} > DropRec.txt

I am a little beginner in awk. I wanted to check how the nawk statement above is picking up the first field for comparison. Also, just trying to break down the nawk statement above to understand how it is working.

Thanks
# 6  
Old 02-12-2014
The program reads first file and stores the value of first field as key in Associate Array: A

The program then reads second file and check if first field is not a key in Associate Array: A and print the whole record if true.
This User Gave Thanks to Yoda For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare two CSV files and put the difference in third file with line no,field no and diff value.

I am having two csv files i need to compare these files and the output file should have the information of the differences at the field level. For Example, File 1: A,B,C,D,E,F 1,2,3,4,5,6 File 2: A,C,B,D,E,F 1,2,4,5,5,6 out put file: (12 Replies)
Discussion started by: karingulanagara
12 Replies

2. UNIX for Dummies Questions & Answers

Help with AWK - Compare a field in a file to lookup file and substitute if only a match

I have the below 2 files: 1) Third field from file1.txt should be compared to the first field of lookup.txt. 2) If match found then third field, file1.txt should be substituted with the second field from lookup.txt. 3)Else just print the line from file1.txt. File1.txt:... (4 Replies)
Discussion started by: venalla_shine
4 Replies

3. Shell Programming and Scripting

Plz Help. Compare 2 files field by field and get the output in another file.

Hi Freinds, I have 2 files . one is source.txt and second one is target.txt. I want to keep source.txt as baseline and compare target.txt. please find the data in 2 files and Expected output. Source.txt 1|HYD|NAG|TRA|34.5|1234 2|CHE|ESW|DES|36.5|134 3|BAN|MEH|TRA|33.5|234... (5 Replies)
Discussion started by: i150371485
5 Replies

4. Shell Programming and Scripting

Compare two files Field by field and output the result in another file

Hi Friends, Need Help. I have file1.txt as File1.txt |123|A|7267|Hyder|Cross|Sell|7801 |995|A|7051|2008|Lunar|New|Year|Promotion|7801 |996|A|7022|Q108|Targ|Prospect|&|SSCC|Savings|Promo|7801 |997|A|7182|Q1|Feb-Apr|08|Credit|ITA|PA|SBA|Campaign|7801 File2.txt... (7 Replies)
Discussion started by: i150371485
7 Replies

5. Shell Programming and Scripting

Compare a common field in two files and append a column from File 1 in File2

Hi Friends, I am new to Shell Scripting and need your help in the below situation. - I have two files (File 1 and File 2) and the contents of the files are mentioned below. - "Application handle" is the common field in both the files. (NOTE :- PLEASE REFER TO THE ATTACHMENT "Compare files... (2 Replies)
Discussion started by: Santoshbn
2 Replies

6. Shell Programming and Scripting

Compare Field in Current Line with Field in Previous

Hi Guys I have the following file Essentially, I am trying to find the right awk/sed syntax in order to produce the following 3 distinct files from the file above: Basically, I want to print the lines of the file as long as the second field of the current line is equal to the... (9 Replies)
Discussion started by: moutaye
9 Replies

7. Shell Programming and Scripting

AWK: Pattern match between 2 files, then compare a field in file1 as > or < field in file2

First, thanks for the help in previous posts... couldn't have gotten where I am now without it! So here is what I have, I use AWK to match $1 and $2 as 1 string in file1 to $1 and $2 as 1 string in file2. Now I'm wondering if I can extend this AWK command to incorporate the following: If $1... (4 Replies)
Discussion started by: right_coaster
4 Replies

8. UNIX and Linux Applications

How to compare 2 field from 2 separated file

I have a problem here. I'm trying to compare multiple fields. Files are like this: File 1: Email,Account Number,Contact,Status,Date File 2: Name|Address|Contact|Email|Account Number|0000000 #!/bin/bash myFolder=`pwd` TEMPFILE=$myFolder/tempfile APFILE=$myFolder/file 1.csv... (0 Replies)
Discussion started by: micxshinoda
0 Replies

9. Shell Programming and Scripting

Help with scipt to compare first level dir with its subdirectory

I was wondering if someone could help me or atleast push me in the right direction. All my users have a folder with their username (i.e. john.doe ) some of these folders contain a subdirectory with the first letter of there first name and there lastname (i.e. jdoe ) I'm trying to create a... (1 Reply)
Discussion started by: binary-ninja
1 Replies

10. Shell Programming and Scripting

Compare one field

Hello to every one: i have and archive like this: 5552952099|5552952099|VAHC600312UY9|VARGAS HERNANDEZ MA DEL CARMEN|METRO|LOMAS|CC|20||||900661026|1|METRO 5552952115|5552952949|AABF680818FT7|ABASOLO BONILLA FERDINAND|METRO|LOMAS|CC|20||||900671255|1|METRO... (6 Replies)
Discussion started by: xasky
6 Replies
Login or Register to Ask a Question