UNIX for Advanced & Expert Users

I specified zsh in the first line of thread but if a code works in bash or ksh it usually works in zsh.
File A could be ~1e7 - 1e8 lines (in reality it's a set of files), B ~ e6 lines; really instead of
echo "Hello" | tee p1 > p2 &
it could be something as
uncompress -c A_2006*.Z | tee p1 > p2 &
or cycle as
for f in A_2006*.Z
{uncompress -c $f | tee p1 > p2 &
...

I would write a lot of this in C. If that is not an option, then perl. This program would read A style records and would write them if they passed the test. To do that, it would first preprocess file B. I would read file B and build an array. 1,000,000 elements, each of which is 11 bytes is a very large array but not prohibitive. I would explore reducing the 11 bytes though. Then I would sort the array and have a binary search function for it. To validate each A record, the program then just does one binary search on the arrray. Ideally, enough physical memory should be available that the program fits into core without paging.

A shell script would unzip the data files and feed them to the program. Ideally multiple cpu's would be available. This would allow the unzip and the c program to settle into dedicated cpu's.

Thanks, it's a better way than that I tryed to do, because for use join I have to sort file A originally and it's ~ N log(N) operations, so using join is good if A is sorted.