Shell Programming and Scripting

hi,

i have a file with many records and each record may or may not have 6 columns.
for example

file1 :

Code:

first second third fourth fifth sixth
first second third fourth fifth
first second third fourth fifth sixth
first second third fourth fifth sixth seventh eigth

if i cat the file and read each line using a while loop, and count the number of columns in each line, it takes a long time.

Code:

test=0
cat file1 | while read line
do
    count=`echo $line | sed 's/[^ ]//g' | wc -c`
    if [ $count -ne 6 ]; then
         test=1
         break;
    fi
done

in the above code the loop reads each line 1 by 1 and counts the number of columns. if the file has many lines then it will consume time.

is there a way so that all the lines can be checked at the same time and if any line is having less than 6 columns, set test=1. or any way which consumes less time.

You don't need the sed call, and you could also redirect the file directly into the while loop (avoiding the need for cat):

Code:

test=0
while read line
do
    count=`echo $line | wc -w`
    if [ $count -ne 6 ]; then
         test=1
         break;
    fi
done < file1

Alternatively, here's an awk-based 1-liner:

Code:

[[ $(awk '{ if (NF<6) {print; nextfile}}' file1) != "" ]] && test=1;

EDIT: Note that the awk solution checks for less than 6 fields, whereas your shell script checks for more or less than 6 fields

actually i want to check for more or less than 6 fields

---------- Post updated at 08:05 PM ---------- Previous update was at 08:01 PM ----------

Code:

[[ $(awk '{ if (NF<6 || NF>6) {print; nextfile}}' file1) != "" ]] && test=1;

this command works fine. can you explain the above command

Why not just the awk equivalent of -ne - i.e. NF != 6?

awk '{ if (NF!=6) {print; nextfile}}' file1
for every line processed, if number of fields is not 6, print the line (just to get some output to check) and skip to the next file (there isn't one, so it just ends). As Scott points out, you can use != for 'not equal to'.

$(awk '{ if (NF!=6) {print; nextfile}}' file1)
Substitutes the output of the awk command (which will be either the matched line or nothing) for this point in the code.

[[ $(awk '{ if (NF!=6) {print; nextfile}}' file1) != "" ]] - is a test which evaluates to TRUE if the awk command produced no output (an empty string).

[[ $(awk '{ if (NF!=6) {print; nextfile}}' file1) != "" ]] && test=1 - (ab)uses lazy evaluation of logical expressions to set test only if the first part of the expression (the awk+test) returned TRUE. If the first part doesn't return TRUE then the part after && is never executed.

Try also

Code:

awk 'NF != 6 {exit 1}' file1 || test=1

what is the {print; nextfile} nextfile in the command do?