Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Performing lookup quickly

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Performing lookup quickly
# 1  
Old 01-16-2012
Performing lookup quickly
Hello,

Have been trying to develop a script that reads a string from an input file then looks up that string in a reference file to get a number, then inset that number back into the original file at a certain point (_____):
Code:
Input File:
HARRY   JON     _____     AA  M N
PETER   PIE     _____     BB  M Y
SARAHAASJOHNASAS_____     CC  F Y
 
Lookup File:
AA 95321
BB 75330
CC 91211

Becomes:
HARRY   JON     95321     AA  M N
PETER   PIE     75330     BB  M Y
SARAHAASJOHNASAS91211     CC  F Y

I tried rexx first but it was painfully slow probably because the lookup file is 167K lines long!

What is the fastest way to perform this kind of search in unix and can anyone give an example!

Thanks
# 2  
Old 01-16-2012
Hi, see if this works:
Code:
awk 'NR==FNR{A[$1]=$2;next}{sub(/_____/,A[$(NF-2)])}1' lookupfile inputfile

# 3  
Old 01-16-2012
Hello,

Thanks for you reply, sorry should of explained that the ___'s represent spaces where I'd like the ouptut to go! also the ouput seems to add a new line character after the looked up reference:

Code:
HARRY   JON     95321
     AA  M N
PETER   PIE     75330
     BB  M Y
SARAHAASJOHNASAS91211
     CC  F Y

Many thanks for your help, know very little about awk!
# 4  
Old 01-16-2012
hello.. try this out --

Input
Code:
HARRY   JON     _____     AA  M N
PETER   PIE     _____     BB  M Y
SARAHAASJOHNASAS_____     CC  F Y

this is the code below :

Code:
awk -F" " ' BEGIN {
 while (getline < "lookup")
 {
  arr[$1]=$2;
 }
}
{
 for (i in arr) {
  if(substr($0,27,2) == i)
   print substr($0,1,16)""arr[i]""substr($0,22);
 }
}' f6

where lookup file contains the values against AA,BB.. and f6 is the input file.

Output ::

Code:
HARRY   JON     95321     AA  M N
PETER   PIE     75330     BB  M Y
SARAHAASJOHNASAS91211     CC  F Y

Regards,
A!
# 5  
Old 01-16-2012
I see. Quick fix in the UNIX tradition:
Code:
sed 's/\(.\{16\}\)...../\1_____/' inputfile | 
awk 'NR==FNR{A[$1]=$2;next}{sub(/_____/,A[$(NF-2)])}1' lookupfile -

The first sed statement converts the file so that it contains the underscores Smilie The result of which is piped into the awk statement..

-or-

Code:
awk 'NR==FNR{A[$1]=$2;next} {print substr($0,1,16) A[$(NF-2)] substr($0,22)}' lookupfile inputfile

Last edited by Scrutinizer; 01-16-2012 at 09:11 AM..
This User Gave Thanks to Scrutinizer For This Post:
dendright
# 6  
Old 01-17-2012
Many thanks for your replys!

Both techniques work great and fast! How could I change if the lookup file was delimitted? Have tried putting FS="|" before the command but that was just a shot in the dark and suprisingly doesn't work!

Code:
T|AA|95 321| 
Y|BB|75 330| 
J|CC|91 211|

# 7  
Old 01-17-2012
if your lookup file is delimited, then you need to mention the FieldSeparator (FS) i.e. FS="|" in the BEGIN block before the while loop begins and then reassign it to FS = " " after the while loop ends.

Code:
 
awk -F" " ' BEGIN {
FS="|";
 while (getline < "lookup")
{
  arr[$1]=$2;
}
FS=" ";
}

Regards,
A!
This User Gave Thanks to archimedes For This Post:
dendright
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Performing aggregation in File

Hi, I need help in UNIX shell script to handle the following: I have file called File1.txt which contains data something like below: LibraryName|BookName|Flag-Indicator|COUNT LIB1|BOOK1|A|12 LIB1|BOOK1|A|2 LIB2|BOOK2|I|1 LIB2|BOOK1|I|4 LIB1|BOOK2|A|12 LIB1|BOOK1|I|22... (3 Replies)
Discussion started by: vfrg
3 Replies

2. Programming

Performing a while for two files

Hi, This is for Perl. I have a while loop, and would like to process two files, the formats are given below : access access.<previousday>-* How can perform a while loop on both logs, while creating the logic for access.<previousday>-* to get format like "access.20130615-124139" when... (11 Replies)
Discussion started by: susankoperna1
11 Replies

3. Solaris

Service is restarting too quickly

Hi guys, one of my services in solaris 10 is in maintenance mode. When I checked why, the reason is that the service is "restarting too quickly." # svcs -x svc:/application/management/snmpdx:default svc:/application/management/snmpdx:default (Sun Solstice Enterprise Master Agent) State:... (2 Replies)
Discussion started by: cjashu
2 Replies

4. Shell Programming and Scripting

Please Help quickly

Hello everyone! I am new here and by new i don't mean only in this place, but in this shell scripting area. I started learning about Operation Systems only 2 months ago. I want to learn more but i found a problem which I don't really know how to solve. :confused: I have a directory with 4... (2 Replies)
Discussion started by: Stripes07
2 Replies

5. UNIX for Advanced & Expert Users

slow performing system

Friends Greetings. I have a RedHat 5.7 64bit virtual server on VMware ESXi 4.1. This server and other Redhat Servers are running very slow. I did some stats collection on ESXi and looks like Linux is holding the disk IO. I am not sure what is causing this behavior. On Linux I checked the CPU... (4 Replies)
Discussion started by: sdewal
4 Replies

6. UNIX for Dummies Questions & Answers

changing directory quickly without cd ..

Hi, I am in /home/development/project/abc directory. Now I want to remove one file which is kept in /trial/dev/<file> directory. I would like to do it in one command. I tried changing directory in one command like: pwd /home/development/project/abc cd ~/trial/dev/ bash: no command... (2 Replies)
Discussion started by: palak08
2 Replies

7. UNIX for Advanced & Expert Users

Clueless about how to lookup and reverse lookup IP addresses under a file!!.pls help

Write a quick shell snippet to find all of the IPV4 IP addresses in any and all of the files under /var/lib/output/*, ignoring whatever else may be in those files. Perform a reverse lookup on each, and format the output neatly, like "IP=192.168.0.1, ... (0 Replies)
Discussion started by: choco4202002
0 Replies

8. UNIX for Dummies Questions & Answers

need solution for this quickly. please quickly.

Write a nawk script that will produce the following report: ***FIRST QUARTERLY REPORT*** ***CAMPAIGN 2004 CONTRIBUTIONS*** ------------------------------------------------------------------------- NAME PHONE Jan | ... (5 Replies)
Discussion started by: p.palakj.shah
5 Replies

9. Shell Programming and Scripting

please help quickly..

I want to pass arguments to a sh program and I want to use these arguments in the program as variables? Will I use argc and argv in a main function can anyone around me write the full code please for example when I write in th command sh myprogram.sh argument1 argument2 I want to use... (1 Reply)
Discussion started by: walnut
1 Replies
Login or Register to Ask a Question