Quote:
How BIG your lookup file is?
Can you get the approx stats of line? How many thousands?
Also, tell us about the O/S and shell version.
Hi clx, thanks for the interest and apologies for the late reply.
I'm using HP-UX, /bin/ksh.
For the files, we're averaging 300,000 lines per file, * 10 files (gzipped), * 10 dirs, so the need for faster processing.
My lookup file is an isql output, and is around 30,000 records.
I'm currently developing/testing an alternative, wherein instead of embedding another while loop inside a while loop, I will have just 1, and sequentially do the following:
1. while loop to match the records from file against the lookup.
a. for every matched record, output the records from file + records from lookup to a result file.
2. use the awk script to check the vardate if between varstart/varend. (much easier since awk will only be accessing 1 file)
But then again, other faster alternatives are welcome.
Thanks.