Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Compare with 2 ref files -awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare with 2 ref files -awk
# 1  
Old 11-15-2015
Compare with 2 ref files -awk
Guys ... need some help with this please
FILE1
Code:
XYZ,123,1234_1122,AQQ
XXX,345,5678,FFG
YYY,123,3344_5566_7788,YYTT
UUU,567,7799_1111,RRR
EEE,333,8866,III

ref1
Code:
 
ZONE1122_A,1234
ZONE_PROD_1155_A,1122
ZONE_DEV33_A,3344
ZONE_QA11_A,1111
ZONE5566_A,8866
ZONE189_A,4455

ref2
Code:
 
ZONE1122_b,1123
ZONE_PROD_1155_B,5566
ZONE_DEV33_B,7788
ZONE_QA11_B,7799
ZONE5566_B,6789
ZONE189_B,2345

output
Code:
XYZ,123,1234_1122,ZONE1122_A,N/A,AQQ
XXX,345,5678,N/A,N/A,FFG
YYY,123,3344_5566_7788,ZONE_DEV33_A,ZONE_PROD_1155_B;ZONE_DEV33_B,YYTT
UUU,567,7799_1111,ZONE_QA11_A,ZONE_QA11_BRRR
EEE,333,8866,ZONE5566_A,N/A,III

$3 from file 1 could be one field or field1_field2_.... ( seperator is "_")
compare each field with $2 of both ref1 and ref2 files
if a match is found in ref1 append $1 from ref1 as column 4 in file1 , if no match is found then put N/A as column 4 , when more than one entry is found use ";" as FS
if a match is found in ref2 append $1 from ref2 as column 5 in file1 , if no match is found then put N/A as column 5 , when more than one entry is found use ";" as FS
for example one of the $3 from file is "3344_5566_7788" , so basically compare 3344 , 5566 and 7788 to $2 of both ref1 and ref2
Thanks
Last edited by greycells; 11-15-2015 at 09:11 PM..
# 2  
Old 11-15-2015
Quote:
Originally Posted by greycells
Guys ... need some help with this please
FILE1
Code:
XYZ,123,1234_1122,AQQ
XXX,345,5678,FFG
YYY,123,3344_5566_7788,YYTT
UUU,567,7799_1111,RRR
EEE,333,8866,III

ref1
Code:
 
ZONE1122_A,1234
ZONE_PROD_1155_A,1122
ZONE_DEV33_A,3344
ZONE_QA11_A,1111
ZONE5566_A,8866
ZONE189_A,4455

ref1
Code:
 
ZONE1122_b,1122
ZONE_PROD_1155_B,5566
ZONE_DEV33_B,7788
ZONE_QA11_B,7799
ZONE5566_B,6789
ZONE189_B,2345

output
Code:
XYZ,123,1234_1122,ZONE1122_A,ZONE1122_b,AQQ
XXX,345,5678,N/A,N/A,FFG
YYY,123,3344_5566_7788,ZONE_DEV33_A,ZONE_PROD_1155_B;ZONE_DEV33_B,YYTT
UUU,567,7799_1111,ZONE_QA11_A,ZONE_QA11_BRRR
EEE,333,8866,ZONE5566_A,N/A,III

Would you tell us why the discrepancy with your request; why this
Code:
XYZ,123,1234_1122,ZONE1122_A,ZONE1122_b,AQQ

instead of
Code:
XYZ,123,1234_1122,ZONE1122_A,ZONE_PROD_1155_A,ZONE1122_b,AQQ

# 3  
Old 11-15-2015
You are right ... my Bad .. I fixed it ..

basically $2 from ref1 and ref2 will never be same

Thanks
# 4  
Old 11-16-2015
What have you tried to solve this problem?

What operating system are you using?

What shell are you using?
# 5  
Old 11-16-2015
Hi,

for now I am able to replace fields in file1 $3 with any one reference file ..which does not fit the output needed though
Code:
$ gawk -F"," 'NR==FNR{a[$2]=$1;next}{for (i in a) gsub(i,a[i],$0)}1' /tmp/ref2 /tmp/f11 |more

XYZ,123,ZONE1122_A_ZONE_PROD_1155_A,AQQ
XXX,345,5678,FFG
YYY,123,ZONE_DEV33_A_5566_7788,YYTT
UUU,567,7799_ZONE_QA11_A,RRR
EEE,333,ZONE5566_A,III

Its on Linux and Bash

Thanks
# 6  
Old 11-16-2015
How about
Code:
awk -F"," '
FNR == 1        {FC++
                }
FC < 3          {a[FC,$2]=$1
                 next
                }
                {X = Y = "N/A"
                 n = split ($3, T, "_")
                 for (i=1; i<=n; i++)   {if ((1,T[i]) in a) X = X ";" a[1,T[i]]
                                         if ((2,T[i]) in a) Y = Y ";" a[2,T[i]]
                                        }
                 $3 = $3 FS X FS Y
                 gsub (/N\/A;/, "", $3)
                }

1
' OFS=","  ref1 ref2 file1
XYZ,123,1234_1122,ZONE1122_A;ZONE_PROD_1155_A,N/A,AQQ
XXX,345,5678,N/A,N/A,FFG
YYY,123,3344_5566_7788,ZONE_DEV33_A,ZONE_PROD_1155_B;ZONE_DEV33_B,YYTT
UUU,567,7799_1111,ZONE_QA11_A,ZONE_QA11_B,RRR
EEE,333,8866,ZONE5566_A,N/A,III

This User Gave Thanks to RudiC For This Post:
greycells
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[awk] Compare two files

HI!! I am trying to compare two files using AWK but I have some problems. I need to count how many times letters are used in two texts. This is my script { long=length($0) for (i=1;i<=long;i++) { aux=substr($0,i,1) if ( aux != " " && aux != "" ) ... (7 Replies)
Discussion started by: ettore8888
7 Replies

2. Shell Programming and Scripting

awk compare files

I have a below requirement and trying to compare the files using awk File 1 - Already stored on a prev day id | text | email id --------------------------------- 89564|this is line 1 | xyz@sample.txt 985384|this is line 2 | abc@sample.txt 657342|this is line 3 |... (3 Replies)
Discussion started by: rakesh_411
3 Replies

3. Shell Programming and Scripting

Compare 2 files, awk maybe?

I have 2 files, file1: alfa numbers numbers vita numbers numbers gama numbers numbers delta numbers numbers epsilon numbers numbers zita numbers numbers ... file2: 'zita' keepnumbers keepnumbers keepnumbers 'gama' keepnumbers keepnumbers keepnumbers 'misc' ... (11 Replies)
Discussion started by: phaethon
11 Replies

4. HP-UX

Awk compare two files

Hi guys, I have 2 files: File1 ABC|2203|115.50 ABC|2288|328.12 ABC|2289|611.09 ABC|2290|698 DEF|1513|721.3 DEF|1514|40 DEF|1515|5 File2 ABC|2288|328.12 ABC|2289|666.08 ABC|2290|698.00 DEF|1513|721.30 (3 Replies)
Discussion started by: Eduardo Aceves
3 Replies

5. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

6. Shell Programming and Scripting

Compare two files with awk

Hello, I have a script which extracts the values from a csv file when a specific date is entered : #!/bin/sh awk 'BEGIN{printf("Entrez la date : "); getline date < "-"} $0 ~ date {f=1;print;next} /^{2}\//{f=0} f' file1.csv This script gives me a number of lines with different values. ... (6 Replies)
Discussion started by: freyr
6 Replies

7. UNIX for Dummies Questions & Answers

Using AWK to compare 2 files

Hi How can I use awk to compare specific columns in 2 files and print the difference. I currently have this: BEGIN { OFS = FS = "," } NR == FNR { b = $3 next } { e = "" for (x in b) { if (match ($1, x)) { if (RSTART == 1 && RLENGTH > length(e)) { e=x (2 Replies)
Discussion started by: ladyAnne
2 Replies

8. Shell Programming and Scripting

compare two files using awk

Hi, I want to compare two files using awk and write an output based on if the records matched. Both the files are space delimitted. File A: 8351 00000000000636 2009044 -00001.000 8351 00000000000637 2009044 -00002.000 8351 00000000000638 2009044 -00001.000 8351 00000000000640... (7 Replies)
Discussion started by: gpaulose
7 Replies

9. Shell Programming and Scripting

Compare two files using awk

Hi. I'm new to awk and have searched for a solution to my problem, but haven't found the right answer yet. I have two files that look like this: file1 Delete,3105551234 Delete,3105551236 Delete,5625559876 Delete,5625556789 Delete,5625553456 Delete,5625551234 Delete,5625556956... (8 Replies)
Discussion started by: paul.o
8 Replies

10. Shell Programming and Scripting

awk compare 2 files

Hi i hope some awk gurus here can help me.. here is what i need i have 2 files: File1 152445 516532 405088.pdf 152445 516533 405089.pdf 152491 516668 405153.jpg 152491 520977 408779.jpg 152491 0 409265.pdf File2 516532 /tmp/MainStreet_Sum09_Front_FNL.pdf 516533... (9 Replies)
Discussion started by: kenray
9 Replies
Login or Register to Ask a Question