Compare two columns and replacing it with value from third column!!


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare two columns and replacing it with value from third column!!
# 1  
Old 08-05-2015
Hammer & Screwdriver Compare two columns and replacing it with value from third column!!

Hi,

I am new to Unix and I am finding it hard to fix a particular logic.

The context is as below.

File 1 :
This contains of 5 fields :
Code:
Type | Bank Code | Account | Name | Date/Time
60|ABC123|TX123456|XXXX|YYYYMMDDHH:MM:SS
72|ABC123|MYD34561|XXXX|YYYYMMDDHH:MM:SS
95|ABC345|AX672456|XXXX|YYYYMMDDHH:MM:SS
82|ABC456|PY123456|XXXX|YYYYMMDDHH:MM:SS

File 2 :
This contains 5 fields :
Code:
Bank Code | Account | Min Val | Max Val | Stream Code | Stream ID
ABC123|TX123456|0000|1111|TXY123|T1234567
ABC345|AX672456|0000|1111|TCT345|T4589561
ABC789|MM123331|0000|1111|TCB789|T1012548

I need to compare field 'Bank Code' and 'Account' of File 1 with 'Bank Code' and 'Account' of File 2.
Then I need to replace the fields 'Bank Code' and "Account' of file 1 with 'Stream Code' and 'Stream ID' of File 2.

So the final output for file 1 should be :
Code:
Type | Stream Code | Stream ID |  Name | Date/Time

I am currently trying this Unix command, but its not working.
Code:
awk 'NR==FNR{A[$1,$2]=($5,$6); next} ($2,$3) in A{($2,$3)=A[$2,$3]; print}' file2 file1 > file3

Please help me fix this issue.

Thanks in advance.
DJ

Last edited by vgersh99; 08-05-2015 at 01:18 PM.. Reason: code tags, please!
# 2  
Old 08-05-2015
Please use code tags as required by forum rules!

You can't use the ($5,$6) construct for assignment, neither as the source nor the target, and you did not define the correct field sparators. Try this corrected version of your script:
Code:
awk '
NR==FNR         {A[$1,$2]=$5 "|" $6
                 next
                }
($2,$3) in A    {split (A[$2,$3], T)
                 $2=T[1]
                 $3=T[2]
                 print
                }
' FS="|" OFS="|" file2 file1

# 3  
Old 08-05-2015
One more:
Code:
awk 'NR==FNR{A[$1,$2]=$5 FS $6; next} ($2,$3) in A{print $1, A[$2,$3], $4, $5}' FS=\| OFS=\| file2 file1

# 4  
Old 08-06-2015
Hammer & Screwdriver

Thanks Guys. Your POC worked perfectly fine. It was really helpful!!!Smilie
# 5  
Old 11-20-2015
Hello,

I need to understand if nawk would perform faster than the awk command in terms of huge volumes of data (say ~ 1GB).

Also, I would be grateful, if someone could share the replacement code for the below awk command in nawk.

Code:
awk 'NR==FNR{A[$1,$2]=$1; next} ($2,$3) in A(print $0}' FS="|" OFS="|" file1 file2 > file3

(The scenario is same as the one found in this thread)

Thanks in advance,
DJose

Last edited by Scrutinizer; 11-20-2015 at 08:25 PM.. Reason: code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

UNIX script to compare 3rd column value with first column and display

Hello Team, My source data (INput) is like below EPIC1 router EPIC2 Targetdefinition Exp1 Expres rtr1 Router SQL SrcQual Exp1 Expres rtr1 Router EPIC1 Targetdefinition My output like SQL SrcQual Exp1 Expres Exp1 Expres rtr1 Router rtr1 Router EPIC1 Targetdefinition... (5 Replies)
Discussion started by: sekhar.lsb
5 Replies

2. Shell Programming and Scripting

Need awk or Shell script to compare Column-1 of two different CSV files and print if column-1 matche

Example: I have files in below format file 1: zxc,133,joe@example.com cst,222,xyz@example1.com File 2 Contains: hxd hcd jws zxc cst File 1 has 50000 lines and file 2 has around 30000 lines : Expected Output has to be : hxd hcd jws (5 Replies)
Discussion started by: TestPractice
5 Replies

3. UNIX for Dummies Questions & Answers

Compare data - Match first column and compare second

Hi guys, looking for some help with a way to compare data in two files but with some conditions. example, File 1 consists of site1,10.1.1.1 site2,20.2.2.2 site3,30.3.3.3 File 2 contains site1,l0.1.1.1 site2,50.1.1.1 site3,30.3.3.3 site4,40.1.1.1 I want to be able to match the... (1 Reply)
Discussion started by: mutley2202
1 Replies

4. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

5. Shell Programming and Scripting

Combine columns from many files but keep them aligned in columns-shorter left column issue

Hello everyone, I searched the forum looking for answers to this but I could not pinpoint exactly what I need as I keep having trouble. I have many files each having two columns and hundreds of rows. first column is a string (can have many words) and the second column is a number.The files are... (5 Replies)
Discussion started by: isildur1234
5 Replies

6. UNIX for Dummies Questions & Answers

Replacing a specific column of a text file with another column

Hi, I have a text file in the following format: Code: 13412 NA06985 0 0 2 46.6432798439 4 4 4 4 13412 NA06991 NA06993 NA06985 2 48.8478948517 4 4 2 4 13412 NA06993 0 0 1 45.8022601455 4 4 2 4 13401 NA06994 0 0 1 48.780669145 4 4 4 4 13401 NA07000 0 0 2 47.7312017846 2 4 4 4 ... (2 Replies)
Discussion started by: evelibertine
2 Replies

7. UNIX for Dummies Questions & Answers

Replacing a specific column of a text file with another column

I have a text file in the following format: 13412 NA06985 0 0 2 46.6432798439 4 4 4 4 13412 NA06991 NA06993 NA06985 2 48.8478948517 4 4 2 4 13412 NA06993 0 0 1 45.8022601455 4 4 2 4 13401 NA06994 0 0 1 48.780669145 4 4 4 4 13401 NA07000 0 0 2 47.7312017846 2 4 4 4 13402 NA07019... (3 Replies)
Discussion started by: evelibertine
3 Replies

8. Ubuntu

How to compare two columns and fetch the common data with additional column

Dear All, I am new to this forum and please ignore my little knowledge :p I have two types of data (a subset is given below) data version 1: 439798 2 1 451209 1 2 508696 2 1 555760 2 1 582757 1 2 582889 1 2 691827... (2 Replies)
Discussion started by: evoll
2 Replies

9. Shell Programming and Scripting

Replacing column with another column of different file by awk

Hi All, I will really appreciate if you kindly lookinto my requirement below and provide me a solution First file format test1.txt qq ww rr tt ee ff qq ww rr tt ee ff Second file format text2.txt aa aa Now o/p I want as text1.txt's 4th column replaced... (5 Replies)
Discussion started by: Pratik4891
5 Replies

10. UNIX for Dummies Questions & Answers

find common lines using just one column to compare and result with all columns

Hi. If we have this file A B C 7 8 9 1 2 10 and this other file A C D F 7 9 2 3 9 2 3 4 The result i´m looking for is intersection with A B C D F so the answer here will be (10 Replies)
Discussion started by: alcalina
10 Replies
Login or Register to Ask a Question