All,

We receive a file with a large no of records (records can vary) and we have to split it into two files based on another file. e.g.

File1:

UHDR 2008112
"25187","00000022","00",21-APR-1991,"" ,"D",-000000519,+0000000000,"C", ,+000000000,+000000000,000000000,"2","" ,21-APR-1991
"8Y3H4","0000004H","00",16-APR-1992,"" ,"H",-001621119,+0000000000,"C", ,+000000000,+000000000,000000000,"2","" ,21-APR-1991
"95Y8U","02100971","00",03-MAR-1991,"" ,"H",-000004499,+0000000000,"" , ,+000000000,+000000000,000000000,"2","US",21-APR-1991
"24567","02100973","00",26-SEP-1991,"" ,"H",-000000362,+0000000000,"" , ,+000000000,+000000000,000000000,"2","US",21-APR-1991
--
--
--
UTRL 00144700


File2:
2518720080512
2456720080512
1256720080512
8WE7820080512
8Y3H020080512
8Y3H220080512
8Y3H420080512
8Y3H620080512
-
--
--
--

If the first 5 characters of file 2 matched with the chars 2-6 in file1, it should separate those records and put them into another file and rest of the records should be copied into a second file.

I tried cut command but as the file1 is quite large, it was taking a lot of time to put the values into a variable and then compare it.

Is there a way which can do the above task quite fast.

Please help as it is needed urgently.

Thanks in anticipation.

If your grep can read patterns from a file (like GNU grep), try something like this.

cut -c1-5 file2 >patterns
grep -f patterns file1 >matches
grep -f patterns -v file1 >nonmatches

sed 's/^\(.....\).*/^"\1",/' file2 >patterns

With awk:

awk 'NR==FNR{a[substr($0,1,5)];next}
NF<3{next}
substr($1,2,5) in a {print > "file1";next}
{print > "file2"}' file2 file1

Thanks Franklin and era.

Franklin,
I fpossible, can you explain wha tthe cod eis doing. I'm an awk novice and this will help me in interpreting things better.
Also, what do I do if I don't want to modify the original files.

Thanks again.

awk 'NR==FNR{a[substr($0,1,5)];next}

 NF<3{next}

 substr($1,2,5) in a {print > "file1";next}

 {print > "file2"}

Franklin,

You rock.
But when I use this:

awk 'NR==FNR{a[substr($0,1,5)];next}
NF<3{next}
substr($1,2,5) in a {print > "matched.txt";next}
{print > "notmatched.txt"}' file2 file1