Lookup subfields from 3 tables and insert


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Lookup subfields from 3 tables and insert
# 1  
Old 12-29-2014
Lookup subfields from 3 tables and insert

Hello masters,

Please help on the following.
I have a tab delimited file with subfields space delimited.

Code:
1	a b	x y	hhghd ghgf
2	v t	f g	gdgdgdg hghg

I have 3 lookup table files tab delimited, for fields 2,3 and 4 respectively

Code:
Lookup2

a	10
b	20
v	30
t	40

Lookup3

x	50
y	60
f	70
g	80

Lookup4

hhghd	90
ghgf	100
gdgdgdg	110
hghg	120

I want to find the lookup values for cols 2,3 and 4 from the corresponding files and then insert a column next to the looked up subfield value, in the same space delimited format.

Code:
1	a b	10 20	x y	50 60	hhghd ghgf	90 100
2	v t	30 40	f g	70 80	gdgdgdg hghg	110 120

Please note that this is a simplified example, in the original file the columns to be looked up are not next to each other. The are $2, $74 and $126 if that information matters.

Last edited by ritakadm; 12-29-2014 at 01:43 AM..
# 2  
Old 12-29-2014
Code:
 awk -F"\t" '{if (FILENAME != "master.txt" ) { a[$1]=$2;next}}
{split ($2,P," "); split($3,Q," "); split ($4,R," "); print $1"\t"$2"\t"a[P[1]],a[P[2]]"\t"$3"\t"a[Q[1]],a[Q[2]]"\t"$4,a[R[1]],a[R[2]]}' lookup* master.txt

This User Gave Thanks to pravin27 For This Post:
# 3  
Old 12-29-2014
thank you Pravin, I need to make it a little more flexible, there maybe be any number of subfields and not always 2..i`m sorry , i should have indicated this in the input.
# 4  
Old 12-29-2014
Hi Ritakdm,

Could this help you ?

Code:
awk -F"\t" '{if (FILENAME != "master.txt" ) { a[$1]=$2;next}}
{n2=split ($2,P," "); n3=split($3,Q," "); n4=split ($4,R," "); printf "\n"$1FS$2FS;
for (i=1;i<=n2;i++) { printf n2==i? a[P[i]]: a[P[i]]" "} printf FS$3FS ; for (j=1;j<=n3;j++) { printf n3==j?a[P[j]]:a[P[j]]" "}  printf FS$4FS ; for (k=1;k<=n4;k++) { printf n4==k?a[P[k]]:a[P[k]]" "}} END {print ""}'  lookup* master.txt

This User Gave Thanks to pravin27 For This Post:
# 5  
Old 12-29-2014
thank you, i will do the necessary testing and get back to you
# 6  
Old 12-29-2014
You haven't said what OS you're using, but when I tried pravin27's code on OS X, I get a syntax error (probably from missing format arguments for the printf statements).

The following seems to do what you want with any number of fields you want to process (just update the fields[] array initialized in the BEGIN clause AND be sure that you have one lookup file for each field to be updated, one or more subfields in each field being processed, and every subfield appearing in the appropriate Lookupxxx file:
Code:
awk '
BEGIN { FS = OFS = "\t"
        fields[++nf] = 2
        fields[++nf] = 3
        fields[++nf] = 4
}
FNR == 1 {
        f++
}
f <= nf {l[f, $1] = $2
        next
}
{       for(i = nf; i > 0; i--) {
                n = split($fields[i], sf, " ")
                af = OFS
                for(j = 1; j <= n; j++)
                        af = af l[i, sf[j]] ((j < n) ? " " : "")
                $fields[i] = $fields[i] af
        }
}
1' Lookup2 Lookup3 Lookup4 file

If you don't set elements in the fields[] array in increasing numeric order and provide Lookupxxx files in the same order as the order of the elements in the fields[] array, it will not work correctly. If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk.

With your sample Lookup* files and file containing:
Code:
1	a b	x y	hhghd ghgf
2	v t	f g	gdgdgdg hghg
3	a b v t	f	gdgdgdg

it produces the output:
Code:
1	a b	10 20	x y	50 60	hhghd ghgf	90 100
2	v t	30 40	f g	70 80	gdgdgdg hghg	110 120
3	a b v t	10 20 30 40	f	70	gdgdgdg	110

These 2 Users Gave Thanks to Don Cragun For This Post:
# 7  
Old 12-29-2014
Hello Don,

Thanks a lot for the great command, I have no hesitation to say that, one day I want to be like you in techincal knowledge. You are great(can't say below than that). I am lucky to be here and see/follow/understand your posts.

There is one very famous quote in a Bollywood movie named Don itself Smilie.
Quote:
To catch Don is not at all difficult because it is impossible. Smilie

Thanks,
R. Singh
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Lookup first high value

Hello experts, I have a file looking like v1 g1 5.42 v2 g1 2.43 v1 g2 1.24 v3 g2 0.6 I want to lookup the first value in another sorted table which is greater than column 3 value, keying on column 2 on the first table. The sorted table looks like the following,. where I want to find... (3 Replies)
Discussion started by: sheetalk
3 Replies

2. UNIX for Dummies Questions & Answers

Joining and sorting with csvs with subfields

hello masters, I am working with csv files that open just fine in excel, but have sub-fields which are comma separated as well. a 3 column csv looks like a,b,"c,d,e" f,g,h How do I make join or sort believe that "c,d,e" is just 1 field? (8 Replies)
Discussion started by: senhia83
8 Replies

3. Shell Programming and Scripting

Changing exact matches with awk from subfields

To give you some context of my issue the following is some sample dummy data. The field delimiter is "<-->". The 4th field is going to be tags for my notes. The tags should always be unique and sorted alphabetically. 1<-->01/20/12<-->01/20/12<-->1st note<-->1st note<-NL->2 lines... (4 Replies)
Discussion started by: adamreiswig
4 Replies

4. Shell Programming and Scripting

lookup

I have a lookup file in unix say /data/lkp.dat (First line is header and space delimited) and the content is shown below. Another file which contains the job_name and rec_count lets say /data/data_file.dat(no header pipe delimited file). Now i want to do a lookup on job_name and my output should... (3 Replies)
Discussion started by: dr46014
3 Replies

5. Shell Programming and Scripting

Reverse lookup

hey guys, can anybody help me out here on the following: grep '^\{1,3\}\.\{1,3\}\.\{1,3\}\.\{1,3\}$' ravi.txt mary.txt lisa.txt https://www.unix.com/images/misc/progress.gif i.e what i did was found ip addreses from different files and then i want... (1 Reply)
Discussion started by: ravis83
1 Replies

6. UNIX for Advanced & Expert Users

Clueless about how to lookup and reverse lookup IP addresses under a file!!.pls help

Write a quick shell snippet to find all of the IPV4 IP addresses in any and all of the files under /var/lib/output/*, ignoring whatever else may be in those files. Perform a reverse lookup on each, and format the output neatly, like "IP=192.168.0.1, ... (0 Replies)
Discussion started by: choco4202002
0 Replies

7. Shell Programming and Scripting

lookup in unix

Hi All I have got a fixed length file of 80bytes long.The first 4bytes of each record represents a client_number.I need to modify the client number based on another lookup file. The lookup file contains 2 fields and a comma delimited file.The first line of the lookup file contains the header... (5 Replies)
Discussion started by: dr46014
5 Replies

8. Shell Programming and Scripting

Converting tables of row data into columns of tables

I am trying to transpose tables listed in the format into format. Any help would be greatly appreciated. Input: test_data_1 1 2 90% 4 3 91% 5 4 90% 6 5 90% 9 6 90% test_data_2 3 5 92% 5 4 92% 7 3 93% 9 2 92% 1 1 92% ... Output:... (7 Replies)
Discussion started by: justthisguy
7 Replies

9. UNIX for Dummies Questions & Answers

HELP with using a lookup table

Using AIX 5.2, Bourne and Korn Shell. I have two flat text files. One is a main file and one is a lookup table that contains a number of letter codes and membership numbers as follows: 316707965EGM01 315672908ANM92 Whenever one of these records from the lookup appears in the main file... (6 Replies)
Discussion started by: Dolph
6 Replies
Login or Register to Ask a Question