awk script to perform an action similar to vlookup between two csv files in UNIX


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk script to perform an action similar to vlookup between two csv files in UNIX
# 1  
Old 09-03-2014
awk script to perform an action similar to vlookup between two csv files in UNIX

Hi,

I am new to awk/unix and am trying to put together an awk script to perform an action similar to vlookup between the two csv files.

Here are the contents of the two files:

File 1:

Code:
Date,ParentID,Number,Area,Volume,Dimensions
2014-01-01,ABC,247,83430.33,857.84,8110.76
2014-01-01,DEF,295,2904.60,139.33,102.95
2014-01-01,ABC,146,11392.70,5017.07,800.96
2014-01-01,XYZ,465,589.76,2.55,1145.33

File 2:

Code:
ID,DATE__C,Parent__ID
PPPG000000tXZWXIA4,2014-01-01,DEF
TTTG000000tXZWYIA4,2014-01-01,ABC
UUUG000000tXZWbIAO,2014-01-01,XYZ

I am trying to do a lookup between the two files based on the date and the parent ID. If the date AND The parent ID are the same, the "ID" (first column from in file 2) needs to be copied to file 1 (as the first column). If they are not the same, then a blank should be copied to file 1.

For example, the output would be as follows:

File 1:

Code:
ID,Date,ParentID,Number,Area,Volume,Dimensions
TTTG000000tXZWYIA4,2014-01-01,ABC,247,83430.33,857.84,8110.76
PPPG000000tXZWXIA4,2014-01-01,DEF,295,2904.60,139.33,102.95
,2014-02-01,ABC,146,11392.70,5017.07,800.96
UUUG000000tXZWbIAO,2014-01-01,XYZ,465,589.76,2.55,1145.33

Can someone please assist?

Really appreciate any help!

Thank you,
# 2  
Old 09-03-2014
Yes, what have you wrote so far?
# 3  
Old 09-03-2014
code so far

I tried to do something like this:

Code:
awk -F, 'FNR==NR{a[$2]=$1;next}$1 in a {$7=a[$1]}1' OFS="," FS="," file2.csv file1.csv

The output is completely wrong though Smilie

Have not written any awk scripts before and not sure how I could fix my code..
# 4  
Old 09-03-2014
this should work, not if your headings matched in file 2 the hard-coded assign of ID would not be required:

Code:
awk -F, 'FNR==NR{a[$2,$3]=$1;next}
{
  if($1 SUBSEP $2 in a) $1=a[$1,$2] OFS $1
  else $1=OFS $1
  if(FNR==1) $1="ID"
}1' OFS="," file2.csv file1.csv

# 5  
Old 09-03-2014
Another approach:
Code:
awk -F, 'NR==FNR {a[$2,$3]=$1; next} FNR==1 {$1="ID" FS $1} a[$1,$2] {$1=a[$1,$2] FS $1}1' OFS=, file2 file1


Last edited by Franklin52; 09-03-2014 at 06:21 PM..
# 6  
Old 09-03-2014
@Franklin52 - I considered that approach as well, however it doesn't put blank entry in field #1 when lookup fails:

Quote:
If they are not the same, then a blank should be copied to file 1.
This User Gave Thanks to Chubler_XL For This Post:
# 7  
Old 09-03-2014
@Franklin52 and @Chubler_XL thanks for your replies!

File 1 currently does not have a ID column. This ID column has to be inserted. @Franklin52, when I try tried your approach, the outout is exactly file1 without the ID column inserted?

@Chubler_XL, when I try your approach, the output is as follows:

Code:
ID,Date,ParentID,Number,Area,Volume,Dimensions
,2014-01-01,ABC,247,83430.33,857.84,8110.76
,2014-01-01,DEF,295,2904.60,139.33,102.95
,2014-01-01,ABC,146,11392.70,5017.07,800.96
,2014-01-01,XYZ,465,589.76,2.55,1145.33

Am i missing anything?

Thanks again guys!
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to compare two files in UNIX using similar to vlookup?

Hi, I want to compare same column in two files, if values match then display the column or display "NA". Ex : File 1 : 123 abc xyz pqr File 2: 122 aab fdf pqr fff qqq rrr (1 Reply)
Discussion started by: hkoshekay
1 Replies

2. Shell Programming and Scripting

Vlookup using awk non similar files

I need to vlookup and check the server not found. Source file 1 server1 server2 server3 server4 server5_root server6_silver server7 server7-test server7-temp Source file 2 server1_bronze (6 Replies)
Discussion started by: ranjancom2000
6 Replies

3. Shell Programming and Scripting

Script to monitor for new file with ext .err and size > 0 bytes and perform a action or command

Hi All, I need to create a script to monitor a dir for new files with ext .err and also it should b a non empty files. and perform a action or command . We have a new ETL application that runs on a linux server, every times a etl fails it creates a .err file or updates the existing .err... (4 Replies)
Discussion started by: MAKHAN
4 Replies

4. Shell Programming and Scripting

Computing the ratio of similar columns in the two files using awk script

Thanks Bartus11 for your help in the following code to compare the two files "t1" and "t2". awk 'NR==FNR{a=1;next}$2 in a{print $2}' t1 t2 First can anyone explain that what is the purpose of assigning a =1? Second, the current script is printing out the matched columns between the... (4 Replies)
Discussion started by: coder83
4 Replies

5. Shell Programming and Scripting

How to access files from different directories and to perform search action in those files?

Hi, I want to access files from different directories (for example: /home/dir1/file1 , /home/dir2/file2 ...) Like this i have to access these files(file1, file2...). (3 Replies)
Discussion started by: bangarukannan
3 Replies

6. Shell Programming and Scripting

shell script - search a file and perform some action

hi, i have a service on unix platform, it will generate traces in a particular folder i want to check using shell script if traces exist, then perform some action else continue to be in loop. filename is service.tra can you please help? thanks (4 Replies)
Discussion started by: gauravah
4 Replies

7. Shell Programming and Scripting

Need help in searching 2 files for strings then perform an action

I have 2 files. I basically want to search both of them to see if the 1st column ($1) matches and if it matches then check to see if the 2nd column ($2) matches, then execute some code showing the results of the matches. File 1: AAA 123 misc blah BBB 456 CCC 789 File 2: ... (2 Replies)
Discussion started by: streetfighter2
2 Replies

8. Shell Programming and Scripting

Sed or awk script to remove text / or perform calculations from large CSV files

I have a large CSV files (e.g. 2 million records) and am hoping to do one of two things. I have been trying to use awk and sed but am a newbie and can't figure out how to get it to work. Any help you could offer would be greatly appreciated - I'm stuck trying to remove the colon and wildcards in... (6 Replies)
Discussion started by: metronomadic
6 Replies

9. Shell Programming and Scripting

Perform action file name written to the pipe

Hello, I have a script that monitors files uploaded via ftp. After a successful upload, the file name is written to the pipe. There is another program that reads this pipe and allows automatically run any program or script ( say test.sh ) to process the newly uploaded file. cat test.sh... (2 Replies)
Discussion started by: fed.linuxgossip
2 Replies
Login or Register to Ask a Question