UNIX for Dummies Questions & Answers

Using AIX 5.2, Bourne and Korn Shell.

I have two flat text files. One is a main file and one is a lookup table that contains a number of letter codes and membership numbers as follows:

316707965EGM01
315672908ANM92

Whenever one of these records from the lookup appears in the main file then it needs to be removed from the main file.

The problem is in the main file you will see 316707965 (membership number) in the first part of the file, like starting character 6 for 9 but EGM01 (letter code) later on in the file starting char 909 for 5 with other data in between.

I have tried egrep but there doesn't seem to be an and function.

Please can someone assist. Cheers

Can you show sample data from main file?

The main data file would something like this. I have removed lots of fields as it would be to large.

00205679740100316707965 Mr T Test 20 Test Street Test Town TE2 7ST 2203200721032008February '08 MAR N00000000072.500043.500000000000072.50N N EGM01 0000010055
00205679740100315672908 Mr T Test2 40 Test Street Test Town TE2 7ST 2203200721032008February '08 MAR N00000000072.500043.500000000000072.50N N ANM92 0000010055

The file is fixed width so the fields in question will always be in the same place.

Thanks

Code:

sed "s/^\(.\{9\}\)/\1.*/" lookup_file > tmp
grep -f tmp main_file

anbu23

Thanks for that, works like a dream.
Could you please explain how the sed command is working and how the *. effects the grep command if you have time.

Many thanks for your help with this.

Dolph

Here is another one which you can try....
"lookup" is the lookup file and "master" is the master data."updated_master" is the file which contains the updated data.

Code:

awk 'BEGIN {i=1; while((getline line <"lookup")>0) arr[i++]=line;  } { for(j=1;j<i;j++) {  if (match($0,substr(arr[j],1,9)) &&  match($0,substr(arr[j],10,5))) next; }  print $0; } ' master >updated_master

Code:

$ echo 316707965EGM01 | sed "s/^\(.\{9\}\)/\1.*/"
316707965.*EGM01

tmp file contains above patterns.

. matches any single character
* Match any number (or none) of the single character that immediately precedes it
.* match any number of any character

Code:

grep -f tmp main_file

-f tmp The regular expression list is taken from the pattern_file tmp