Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Checking number of commas in each line.
Top Forums Shell Programming and Scripting Checking number of commas in each line. Post 302790013 by Don Cragun on Thursday 4th of April 2013 02:04:46 PM
Old 04-04-2013
Quote:
Originally Posted by Anupam_Halder
Hi All,

I am checking whether each line is having "n" number of commas or nor. In case not then I need to exit the process.

I tried

Code:
cat "$TEMP_FILE" | while read LINE
do 
	processing_line=`expr $processing_line + 1`
	no_of_delimiters=`echo "$LINE" | awk -F ',' '{ print NF }'`
	if [ $no_of_delimiters -ne $no_of_expected_fields ]
                echo "Error at line $processing_line"
		exit
	fi
done

It's working fine. However the number of records in the file is around .5 million. So it's taking too much time to process it. Anyway I can improve the performance?

Thanks in advanced.
Note that the variable names $no_of_expected_fields and $no_of_delimiters are not representative of what awk actually does. When you aren't using the default awk field separator (<space>), every occurrence of the field separator separates two fields; it doesn't terminate a field. So for every non-empty line read by awk when the value of FS is a comma (such as by having -F, on the command line), the value of NF (Number of Fields) is the number of delimiters plus 1; not the number of delimiters.
If you want to print an error for any file that does not have $n commas on every line in the file, you need something like:
Code:
awk -F,  -v n="$n" 'NF!=(n-1){print "Error at line "NF;exit 1}' $TEMP_FILE
if [ $? -ne 0 ]
then    exit
fi
# Continue processing $TEMP_FILE...

in your script.

As always, if you are using a Solaris/SunOS system, use /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk instead of awk.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

sequence number checking

Hi there, I'm wanting to produce a shell script that will check through some file names and identify a skip in sequence (four digit seq num in file name). I have played on the idea of havng a file that has a sorted list of file names which I can read line at a time and cut out the sequence... (1 Reply)
Discussion started by: nhatch
1 Replies

2. Shell Programming and Scripting

Appending line number to each line and getting total number of lines

Hello, I need help in appending the line number of each line to the file and also to get the total number of lines. Can somebody please help me. I have a file say: abc def ccc ddd ffff The output should be: Instance1=abc Instance2=def Instance3=ccc Instance4=ddd Instance5=ffff ... (2 Replies)
Discussion started by: chiru_h
2 Replies

3. Shell Programming and Scripting

Adding a columnfrom a specifit line number to a specific line number

Hi, I have a huge file & I want to add a specific text in column. But I want to add this text from a specific line number to a specific line number & another text in to another range of line numbers. To be more specific: lets say my file has 1000 lines & 4 Columns. I want to add text "Hello"... (2 Replies)
Discussion started by: Ezy
2 Replies

4. Shell Programming and Scripting

how to get the data from line number 1 to line number 100 of a file

Hi Everybody, I am trying to write a script that will get some perticuler data from a file and redirect to a file. My Question is, I have a Very huge file,In that file I have my required data is started from 25th line and it will ends in 100th line. I know the line numbers, I need to get all... (9 Replies)
Discussion started by: Anji
9 Replies

5. UNIX for Dummies Questions & Answers

Inserting commas and replacing backslashes with commas

Hi, Newbie here. I have a file that consists of data that I want to convert to a csv file. For example: Jul 20 2008 1111 / visit home / BlackBerry8830/4.2.2 Profile/MIDP-2.0 Configuration/CLOC-1.1 VendorID/105 Jul 21 2008 22222 / add friend / BlackBerry8830/4.2.2 Profile/MIDP-2.0... (3 Replies)
Discussion started by: kangaroo
3 Replies

6. Shell Programming and Scripting

Counting number of commas(,) in a variable

Hi all, I am having problems counting commas (,) from a variable in shell scripting.. the variable contains similiar to: ID@NAME@DESCRIPTION,ID@NAME@DESCRIPTION, ..... It can go on and on.. So i need to count the number of sets i.e.( ID@NAME@DESCRIPTION is one set) and process the... (4 Replies)
Discussion started by: faelric
4 Replies

7. UNIX for Dummies Questions & Answers

How to read contents of a file from a given line number upto line number again specified by user

Hello Everyone. I am trying to display contains of a file from a specific line to a specific line(let say, from line number 3 to line number 5). For this I got the shell script as shown below: if ; then if ; then tail +$1 $3 | head -n $2 else ... (5 Replies)
Discussion started by: grc
5 Replies

8. Shell Programming and Scripting

checking a number

ok im trying to find out how many cars a user enters. Its giving me an error message of "integer expression expected" Basically if i enter any number over 0 (zero) it should continue read -p "How many cars:" carsn test $cars -ge 1 test $? -ne 0 && read -p "Invalid number.... (8 Replies)
Discussion started by: gangsta
8 Replies

9. Shell Programming and Scripting

Write $line number into textfile and read from line number

Hello everyone, I don't really know anything about scripting, but I have to manage to make this script, out of necessity. #!/bin/bash while read -r line; do #I'm reading from a big wordlist instructions using $line done Is there a way to automatically write the $line number the script... (4 Replies)
Discussion started by: bobylapointe
4 Replies

10. Shell Programming and Scripting

Cutting commas after the second occurrence in a line

Hello everyone, I am manipulating a large CSV file and am trying to read it into a program and started running into trouble. The have manually edited the file trying to make it correctly run through the program and have made progress. However, I am know stuck with an issue involving too many... (3 Replies)
Discussion started by: tastybrownies
3 Replies

LEARN ABOUT HPUX

fspec

fspec(4)						     Kernel Interfaces Manual							  fspec(4)

NAME

       fspec - format specification in text files

DESCRIPTION

       It  is  sometimes  convenient  to  maintain  text files on the HP-UX system with non-standard tabs, (meaning tabs that are not set at every
       eighth column).	Generally, such files must be converted to a standard format - frequently by replacing all tabs with the appropriate  num-
       ber  of	spaces - before they can be processed by HP-UX system commands.  A format specification occurring in the first line of a text file
       specifies how tabs are to be expanded in the remainder of the file.

       A format specification consists of a sequence of parameters separated by blanks and surrounded by the brackets and Each parameter  consists
       of a keyletter, possibly followed immediately by a value.  The following parameters are recognized:

	      The	     parameter specifies tab settings for the file.  The value of tabs must be one of the following:

				  1.  A list of column numbers separated by commas, indicating tabs set at the specified columns;

				  2.  A followed immediately by an integer n, indicating tabs at intervals of n columns;

				  3.  A followed by the name of a ``canned'' tab specification.

			     Standard  tabs  are  specified  by  or  equivalently,  etc.   Recognized  canned tabs are defined by the command (see
			     tabs(1)).

	      The	     parameter specifies a maximum line size.  The value of size must be an integer.  Size  checking  is  performed  after
			     tabs have been expanded, but before the margin is inserted at the beginning of the line.

	      The	     parameter	specifies a number of spaces to be inserted at the beginning of each line.  The value of margin must be an
			     integer.

	      The	     parameter takes no value.	Its presence indicates that the line containing the format specification is to be  deleted
			     from the converted file.

	      The	     parameter	takes  no  value.   Its presence indicates that the current format is to prevail only until another format
			     specification is encountered in the file.

       Default values (assumed for parameters not supplied) are and If the parameter is not specified, no size	checking  is  performed.   If  the
       first  line  of	a  file  does not contain a format specification, the above defaults are assumed for the entire file.  The following is an
       example of a line containing a format specification:

       If a format specification can be disguised as a comment, it is not necessary to code the parameter.

       Several HP-UX system commands correctly interpret the format specification for a file.  Among them is which can be used to convert files to
       a standard format acceptable to other HP-UX system commands.

SEE ALSO

       ed(1), newform(1), tabs(1).

																	  fspec(4)
All times are GMT -4. The time now is 10:12 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy