UNIX for Dummies Questions & Answers

I have a very large results file, and a list of filters

Code:

grep -wf filterlist.txt datafile.txt > outfile.txt

The above line works but is very slow and I'm wondering how to make it faster.

The items in my filterlist are only relevant to the first column. I don't care if any item in the filterlist matches any of the other columns for the datafile. The filterlist items may not always be found. Duplicate items will not happen in either filterlist.txt or datafile.txt

here are some simple examples to illustrate my file structure

filterlist.txt

Code:

pineapple
banana
cherry

datafile.txt

Code:

mango 6  19  $4.40  G4
pineapple 6  28  $3.59  U7
kiwi 6  12  $1.40  G4
banana 6  72  $0.10  T3
coconut 6  33  $2.10  H2

desiredoutput (I'm not worried which order they are in)

Code:

pineapple 6  28  $3.59  U7
banana 6  72  $0.10  T3

Is there some sed or awk commands I could use that would be faster? I was thinking about using awk to keep only the first column of the datafile to create a smaller file (?therefore faster for unix to work with?), then using grep to return the line numbers of my matches (storing in an array), and then using sed to display only those lines in the original datafile.txt file. But I can't find out how (of possible) sed can be used for multiple items in this case.

I know sed can print certain line numbers eg

Code:

sed -n '1000{p;q}' filename  #prints the 1000th line
sed -n '10,20p;20q' filename #prints lines 10-20

Can sed be used with multiple values in this context? i.e. an array? could someone please provide an example for me? I often get confused with quotation marks when using variables inside other commands. Could sed be used to display lines 3,5,9,17?

Any solutions to these questions much appreciated. I'm using BASH.

---------- Post updated at 05:12 PM ---------- Previous update was at 04:54 PM ----------

I'll answer my own question by refering people to this example

look for this topic: Print lines matching value(s) in other file using awk
sorry can't post a url here as i'm new and it won't let me

Code:

nawk -F, 'FNR==NR{f1[$1];next}$1 in f1' OFS=, file1 file2

I didn't have nawk on my system but I changed the command to gawk and it worked okay. Much faster than using grep -wf! (takes a couple of seconds with this solution vs hours with old method)

If anyone can still tell me if using sed is possible with multiple values, I'd really like to know.

--- removed ---