I am trying to pull information out of a file based on one column. I want the records that meet a certain criteria to be places in a bad file and removed from the main file. I have them going in to the bad file but since there are more than one record I am not getting them all removed from the good file. What is the best way to read in the information and have each piece removed without overwriting the file and not removing all the bad data.

The CODE:
nawk '{if (length($0) > 106) print $1}' reads | sort -u > bad
cat bad | while read METER
do
cat reads | grep $METER >> bad.daily
cat reads | grep -v $METER > daily
done

I know the problem is the second grep -v is overwriting the file but I need to find the best way to be able to delete multiple ones with out overwriting.

Thanks
Cliff

Not tested: This section of the awk manual may help: Redirecting Output to Files and Pipes

Your requirements are not clear.

My take:
You want two files:
one with all the good data
another file with just unique bad data
Criterion for good/bad:
good data = record length less than equal to 106.

Yes I want two files one with bad data and one with good. The problem is the files with bad data could have other records that are good but would need to be pulled out also based on a unique column. The column could contain multiple records with only some of them being bad but all needing to be removed.

Sorry I should have been more clear.

I have used egrep to solve the problem. Thanks for the assistance.