UNIX for Dummies Questions & Answers

We receive a file which usually has 40 to 50 million rows. I want to set up an audit process by which everytime we receive a file we audit it for # of rows and total number of columns.

So if the # of rows is around 1 million on a particular day, I want to raise a flag or send an email....and if the number of colums is greater than expected then also raise a flag or send an e-mail...

what would be the most optimum way to achieve this

Thanks
Pri

Assuming your data is space-separated and every row has the same number of columns:

Code:

awk 'END { print NF, NR }' < datafile

This will print the number of columns, then the number of rows. So:

Code:

awk 'END { print NF, NR }' < datafile > /tmp/$$
read COLS ROWS </tmp/$$
rm -f /tmp/$$

if [ "$COLS" -gt 100 ] || [ "$ROWS" -lt "40000000" ]
then
        echo "Got unexpected result from flatfile"
fi

The "wc -l <inputfile" counts the no of rows in your file...no of columns depends on the field delimiter in your input file...also post what you have tried to do about it.

Thanks .. I am trying this out...

@Shamrock - I did wc -l but was not able to formulate the IF clause...still learning basic techniques from this forum.