UNIX for Advanced & Expert Users

I have a '~' delimited file of 6 - 7 million rows. Each row should contain 13 columns delimited by 12 ~'s. Where there are 13 tildes, the row needs to be removed. Each row contains alphanumeric data and occasionally a ~ ends up in a descriptive field and therefore acts as a delimiter, resulting in the row looking like it has 14 columns instead of 13. I have tried a combination of grep and awk but it is running very slowly. I suspect it is the way I am using it.

tried this to print the bad rows with line numbers to a file:
grep -n '~.*~.*~.*~.*~.*~.*~.*~.*~.*~.*~.*~.*~' inputfile | awk {print} > outputfile

also tried this to create a file with only the good rows in it:

grep -v '~.*~.*~.*~.*~.*~.*~.*~.*~.*~.*~.*~.*~' inputfile > outputfile

Both are extremely slow. The input file is approx. 800 meg

thanks

Try this with awk:

Code:

awk -F~ 'NF==13' infile > outfile

Regards

thank-you very much. works perfect. and very fast too. Only took a couple of minutes.

The awk statement works great, thanks. I am now looking for a variation of it. The curretn statement eliminates all rows that do not have 13 occurences but does not trap/write them, so now I would like to find all rows that do not have 13 occurrences of the delimiter.

thanks

awk -F~ 'NF!=13' infile > badfile

ok, so what do you think needs to be changed?
Ah, I see a good Samaritan in our midst....

kpd,
How would you implement getting the lines with the number of '~' occurrences between 10 and 17?

Thanks JoeyG. I appreciate your time. I know it was a dumb question but we don't have any Unix knowledge here. If it looks Greek and you don't speak Greek, ask a Greek. Maybe we can get vgersh99 to come and hold our hands, he seems like a very patient teacher