Shell Programming and Scripting

Hello

i have 2 files file1 and file2 as shown below

file1
110010000000206|567810008161509
110010000000207|567810072227627
110010000000208|567811368851555
110010000000209|567811422513652
110010000000210|567812130217683
110010000000211|567813220211182
110010000000212|567813449322589
110010000000213|567813741319623
110010000000214|567816323171591
110010000000215|567816660521463
110010000000216|567818208711973
110010000000217|567819516604228
110010000000218|567819540685909
110010000000219|567820748714137
110010000000220|567821948536668
110010000000221|567822556413253

file2

110010000000206
110010000000210
110010000000211
110010000000214
110010000000217
110010000000221

Now i want a third file obtained from file1 which will not have the entries from file2.

ie

110010000000207|567810072227627
110010000000208|567811368851555
110010000000209|567811422513652
110010000000212|567813449322589
110010000000213|567813741319623
110010000000215|567816660521463
110010000000216|567818208711973
110010000000218|567819540685909
110010000000219|567820748714137
110010000000220|567821948536668

Now my problem is that file1 has 10 million entries and file2 has half a million entries. So grep -v option is out. Pls suggest an easy way out.

I seem to get stuck with problems working with big files.

Regards
Pradeep

It's going to be difficult to parse that many row in any shell script.

Are you not able to load the files into a DB via isql, bcp, or sqlplus?

nawk -f pra.awk file2 file1

pra.awk:

Code:

BEGIN {
  FS=OFS="|"
}
NR==FNR { arr[$1]; next}
!($1 in arr) && !($2 in arr)

egrep -v -f tmp2 tmp1

Thanks nawk worked!!!

Thats a load off my chest

Thanks a lot man

Regards