How to check the number of columns in a line??

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to check the number of columns in a line??
# 1  
Old 10-17-2013
How to check the number of columns in a line??


i have a file with many records and each record may or may not have 6 columns.
for example

file1 :
first second third fourth fifth sixth
first second third fourth fifth
first second third fourth fifth sixth
first second third fourth fifth sixth seventh eigth

if i cat the file and read each line using a while loop, and count the number of columns in each line, it takes a long time.

cat file1 | while read line
    count=`echo $line | sed 's/[^ ]//g' | wc -c`
    if [ $count -ne 6 ]; then

in the above code the loop reads each line 1 by 1 and counts the number of columns. if the file has many lines then it will consume time.

is there a way so that all the lines can be checked at the same time and if any line is having less than 6 columns, set test=1. or any way which consumes less time.
# 2  
Old 10-17-2013
You don't need the sed call, and you could also redirect the file directly into the while loop (avoiding the need for cat):
while read line
    count=`echo $line | wc -w`
    if [ $count -ne 6 ]; then
done < file1

Alternatively, here's an awk-based 1-liner:
[[ $(awk '{ if (NF<6) {print; nextfile}}' file1) != "" ]] && test=1;

EDIT: Note that the awk solution checks for less than 6 fields, whereas your shell script checks for more or less than 6 fields
# 3  
Old 10-17-2013
actually i want to check for more or less than 6 fields

---------- Post updated at 08:05 PM ---------- Previous update was at 08:01 PM ----------

[[ $(awk '{ if (NF<6 || NF>6) {print; nextfile}}' file1) != "" ]] && test=1;

this command works fine. can you explain the above command
# 4  
Old 10-17-2013
Why not just the awk equivalent of -ne - i.e. NF != 6?
# 5  
Old 10-17-2013
awk '{ if (NF!=6) {print; nextfile}}' file1
for every line processed, if number of fields is not 6, print the line (just to get some output to check) and skip to the next file (there isn't one, so it just ends). As Scott points out, you can use != for 'not equal to'.

$(awk '{ if (NF!=6) {print; nextfile}}' file1)
Substitutes the output of the awk command (which will be either the matched line or nothing) for this point in the code.

[[ $(awk '{ if (NF!=6) {print; nextfile}}' file1) != "" ]] - is a test which evaluates to TRUE if the awk command produced no output (an empty string).

[[ $(awk '{ if (NF!=6) {print; nextfile}}' file1) != "" ]] && test=1 - (ab)uses lazy evaluation of logical expressions to set test only if the first part of the expression (the awk+test) returned TRUE. If the first part doesn't return TRUE then the part after && is never executed.
# 6  
Old 10-17-2013
Try also
awk 'NF != 6 {exit 1}' file1 || test=1

This User Gave Thanks to RudiC For This Post:
# 7  
Old 10-17-2013
what is the {print; nextfile} nextfile in the command do?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Adding columns from 2 files with variable number of columns

I have two files, file1 and file2 who have identical number of rows and columns. However, the script is supposed to be used for for different files and I cannot know the format in advance. Also, the number of columns changes within the file, some rows have more and some less columns (they are... (13 Replies)
Discussion started by: maya3
13 Replies

2. Shell Programming and Scripting

Script to check for a specific number of columns in a file

Hi All I have a file which has five columns in each rows. cat file.txt a|b|c|d|e 1|2|3|4|5 a1|a2|a3|a4|a5 . . . I need to make sure that there are no less than five or more than five columns (in all the rows) by mistake. I tried this : cat file.txt | awk 'BEGIN{FS="|"};{print... (3 Replies)
Discussion started by: chatwithsaurav
3 Replies

3. Shell Programming and Scripting

Request: How to Parse dynamic SQL query to pad extra columns to match the fixed number of columns

Hello All, I have a requirement in which i will be given a sql query as input in a file with dynamic number of columns. For example some times i will get 5 columns, some times 8 columns etc up to 20 columns. So my requirement is to generate a output query which will have 20 columns all the... (7 Replies)
Discussion started by: vikas_trl
7 Replies

4. Shell Programming and Scripting

How do i find the first number in each line and insert dummy string into the missing columns?

Hi, I have one input file with the following content: MY_inpfile.txt Aname1 Cname1 Cname2 1808 5 Aname2 Cname1 1802 47 Bname1 ? 1819 22 Bname2 Cname1 1784 11 Bname3 1817 9 Zname1 Cname1 1805 59 Zname2 Cname1 Cname2 Cname3 1797 27 Every line in my input file have a 4 digit... (5 Replies)
Discussion started by: Szaffy
5 Replies

5. Shell Programming and Scripting

splitting a huge line of file into multiple lines with fixed number of columns

Hi, I have a huge file with a single line. But I want to break that line into lines of with each line having five columns. My file is like this: code: "hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","you." I want it like this: code:... (1 Reply)
Discussion started by: rajsharma
1 Replies

6. UNIX for Dummies Questions & Answers

How to read contents of a file from a given line number upto line number again specified by user

Hello Everyone. I am trying to display contains of a file from a specific line to a specific line(let say, from line number 3 to line number 5). For this I got the shell script as shown below: if ; then if ; then tail +$1 $3 | head -n $2 else ... (5 Replies)
Discussion started by: grc
5 Replies

7. Shell Programming and Scripting

perform a check based on number of @ in a log line

Hello, I am intending to perform a check based on number of "@" , present in a line in a log file . The idea is basically to perform a check on cc or bcc sender, based on an email log, which shows all the for email address. Say if the number of @ is more than 30, I will consider it as a mass... (12 Replies)
Discussion started by: fed.linuxgossip
12 Replies

8. Shell Programming and Scripting

how to get the data from line number 1 to line number 100 of a file

Hi Everybody, I am trying to write a script that will get some perticuler data from a file and redirect to a file. My Question is, I have a Very huge file,In that file I have my required data is started from 25th line and it will ends in 100th line. I know the line numbers, I need to get all... (9 Replies)
Discussion started by: Anji
9 Replies

9. Shell Programming and Scripting

Adding a columnfrom a specifit line number to a specific line number

Hi, I have a huge file & I want to add a specific text in column. But I want to add this text from a specific line number to a specific line number & another text in to another range of line numbers. To be more specific: lets say my file has 1000 lines & 4 Columns. I want to add text "Hello"... (2 Replies)
Discussion started by: Ezy
2 Replies

10. Shell Programming and Scripting

Appending line number to each line and getting total number of lines

Hello, I need help in appending the line number of each line to the file and also to get the total number of lines. Can somebody please help me. I have a file say: abc def ccc ddd ffff The output should be: Instance1=abc Instance2=def Instance3=ccc Instance4=ddd Instance5=ffff ... (2 Replies)
Discussion started by: chiru_h
2 Replies
Login or Register to Ask a Question