Shell Programming and Scripting

Hi I'm trying to loop through a small list of id's and then pull out a few columns if the id matches that found in column 2 of the larger file. I managed to get one command to work

Code:

awk -F "   " '{if ($2 == '154080196') print $2,$3,$4}' tst.txt | less

However, when I try it in a for loop I get a problem. Is it because I have the two awk statements?

Code:

for i in `awk '{print $1}' keyIndvs.txt`; do echo $i; awk -F "    " '{if ($2 == $i) print $2,$3,$4}'  tst.txt |less

Any help would be appreciated...

What problem do you get? Be specific.

"for x in `stuff`" is a useless use of backticks, wasteful, and also dangerous because strings in a shell often have a maximum size. Whenever you have that you can do it more safely and more efficiently like:

Code:

awk '...' | while read VALUE
do
        
done

...but you don't even need a loop. You can just feed one instance of awk into another with awk '{...}' | awk '{...}' ...but the best option would probably be to do it all in one instance of awk. It's a powerful enough language.

I guess there are two files: keyIndvs.txt, tst.txt

For first file, you need the column 1 data.

For second file, you need find out the record with same in first file.

If my guess is right, all your script can be replaced by one awk command:

Code:

awk 'NR==FNR{a[$1];next} $2 in a {print $2,$3,$4}' keyIndvs.txt, tst.txt

second, if you need use system var in awk, try this:

Code:

awk -v var=$i '{if ($2 == var) print $2,$3,$4}'  tst.txt

Thanks for both your replies. Sorry about the lack of detail, but rdcwayx was spot on with what I was attempting. Also thanks for the reference to the useless backticks. I'm trying to teach myself how to program and am bound to make some mistakes. I first tried the example:

awk 'NR==FNR{a[$1];next} $2 in a {print $2,$3,$4}' keyIndvs.txt, tst.txt

but nothing printed to screen and I'm still not 100% what FNR etc was doing so I tried to do the following:

awk '{print $1}' keyIndvs.txt | while read i; do echo $i; awk -F ' ' '{if ($2 == $i) print $2,$3,$4}' tst.txt;done

However, I can't get anything to work and I really don't have a clue why.

In the end I've decided to use R to solve my problem as I'm a bit more familiar with it. I was hoping I could use awk to do the job as the files I have to load into R are huge and it really takes a long time... Anyway here's the code I used if anyone is interested

>R
>indv = read.table('keyIndv.txt',header=FALSE)
>dat = read.table('tst.txt',header=FALSE)
>outF=subset(dat,(V2 %in% indv[,1]))
>write.table(outF,'keyDat.txt')

Thanks again for your help, unfortunately I had to go to R for my solution, but I picked up some awk commands and lessons that may come in handy...