Shell Programming and Scripting

Awk is most likely not your bottleneck. It can loop through the list super fast and print out the commands without much delay. However, each line of command is a grep for a particular string (name) on ALL files inside there. Now that must take some time.
I think after those few hundred you're seeing delays from the searching...
I'd guess it could be sped up by instructing grep to search for multiple strings at a time, rather than only one name; especially since the files that search is performed on, are the same for all strings.
And I did have the feeling this is an overkill. Much simpler and more effective here:

Code:

grep -io -f  Customers.lst *

-f option tells grep that patterns to search for are specified in file Customers.lst
-o option prints just the part of line that matches pattern (just the name)

This should work much better. No need to spawn a myriad of grep processes.

This may, however produce duplicate lines; in case there is multiple lines that contain the same name. So you might want to pipe it to uniq(1) to get rid of duplicates.
Its output is in form:

Code:

file.log:NameMatched

so you want to flip this around to get what you want:

Code:

sh$ echo file.log:NameMatched | sed 's/\([^:]*\):\([^:]*\)/\2, \1/'
NameMatched, file.log

So to put it all together:

Code:

grep -io -f  Customers.lst *  | uniq | sed 's/\([^:]*\):\([^:]*\)/\2, \1/'

I tried the -F swithc of grep which seems to run much faster, and produce the same output, but I'm not sure what it actually does Man page says:
Since our patterns are in an external file, and separated by newlines, this should not have much effect. But it does speed up things... If anyone can bring some light into -F switch, please enlighten me...

Hi ,

THanks for the reply. I will try these today and will send you an update.

Arun