Reading a whole line without betting on size Post: 302363892

Sponsored Content

Top Forums Programming Reading a whole line without betting on size Post 302363892 by Corona688 on Wednesday 21st of October 2009 02:08:57 PM

10-21-2009

Registered User

For line-based I/O, you want a maximum. Otherwise, when fed an endless line, your program will allocate endless amounts of memory. Make it a large maximum if you want, but give it some sort of maximum. Also note that while you can make your limit as large as you feasibly want, there are plenty of UNIX commandline tools, like standard sed and grep, that won't handle lines larger than 2048 bytes. Even things like uuencoding that translate raw binary into ASCII have measures to limit their line length because it's always in your best interest to keep line lengths reasonable.

Furthermore, reading more than one character at a time won't let you "unget" the extra data read beyond the end of line. You can unget one and only one character. So you have to read one-at-a-time if you want to keep all your data.

Also, your algorithm overcomplex, there's no need for a linked list or other data structure. You can resize already-allocated memory with realloc(). It's location may change, but already-defined data will stay defined, while the new area on the end will be undefined until you overwrite it.

I'd just use fgets() with a limit of a few kilobytes. If you really wanted to be sure, make the limit configurable with a commandline switch, just in case.

Last edited by Corona688; 10-21-2009 at 03:20 PM..

Corona688

View Public Profile for Corona688

Visit Corona688's homepage!

Find all posts by Corona688

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

URGENT HELP NEEDED ON -File size reading

All Expert, I am using Sun OS 5.8 and Perl version 5 in One server and Perl 5.8 in another unix server. I am able to read a file using fopen function of perl --file size having more then 3 GB of data.(In the machine where Perl 5.8 install) But when i am running the same perl script --It...

2. Shell Programming and Scripting

Reading a file line by line and processing for each line

Hi, I am a beginner in shell scripting. I have written the following script, which is supposed to process the while loop for each line in the sid_home.txt file. But I'm getting the 'end of file' unexpected for the last line. The file sid_home.txt gets generated as expected, but the script...

3. Shell Programming and Scripting

[Solved] Problem in reading a file line by line till it reaches a white line

So, I want to read line-by-line a text file with unknown number of files.... So: a=1 b=1 while ; do b=`sed -n '$ap' test` a=`expr $a + 1` $here do something with b etc done the problem is that sed does not seem to recognise the $a, even when trying sed -n ' $a p' So, I cannot read...

4. Shell Programming and Scripting

Reading text file, comparing a value in a line, and placing only part of the line in a variable?

I need some help. I would like to read in a text file. Take a variable such as ROW-D-01, compare it to what's in one line in the text file such as PROD/VM/ROW-D-01 and only input PROD/VM into a variable without the /ROW-D-01. Is this possible? any help is appreciated.

5. Shell Programming and Scripting

Reading line by line from live log file using while loop and considering only those lines start from

Hi, I want to read a live log file line by line and considering those line which start from time stamp; Below code I am using, which read line but throws an exception when comparing line that does not contain error code tail -F /logs/COMMON-ERROR.log | while read myline; do...

6. Programming

Copying a file and reading size I get always zero.

I need to copy a file (srcFile) to a new ubication (destFile). Then I calculate the size of the copy file using stat. The problem is that I always get a cero size value although the file has not cero size in the hard disk. The source code that does this is : PROGRAM 1: char srcFile;...

7. UNIX for Beginners Questions & Answers

Reading a file line by line and print required lines based on pattern

Hi All, i want to write a shell script read below file line by line and want to exclude the lines which contains empty value for MOUNTPOINT field. i am using centos 7 Operating system. want to read below file. # cat /tmp/d5 NAME="/dev/sda" TYPE="disk" SIZE="60G" OWNER="root"...

LEARN ABOUT BSD

xargs

XARGS(1)						      General Commands Manual							  XARGS(1)

NAME

       xargs - construct argument list(s) and execute utility

SYNOPSIS

       xargs [ -t ][[ -x ] -n number ][ -s size ][ utility [ arguments...  ]]

DESCRIPTION

       The  xargs utility reads space, tab, newline and end-of-file delimited arguments from the standard input and executes the specified utility
       with them as arguments.

       The utility and any arguments specified on the command line are given to the utility upon each invocation, followed by some number  of  the
       arguments read from standard input.  The utility is repeatedly executed until standard input is exhausted.

       Spaces,	tabs  and  newlines  may  be embedded in arguments using single (`` ' '') or double (``"'') quotes or backslashes (``'').  Single
       quotes escape all non-single quote characters, excluding newlines, up to the matching single quote.  Double quotes  escape  all	non-double
       quote characters, excluding newlines, up to the matching double quote.  Any single character, including newlines, may be escaped by a back-
       slash.

       The options are as follows:

       -n number Set the maximum number of arguments taken from standard input for each invocation of the utility.  An invocation of utility  will
		 use less than number standard input arguments if the number of bytes accumulated (see the s option) exceeds the specified size or
		 there are fewer than number arguments remaining for the last invocation of utility.  The current  default  value  for	number	is
		 5000.

       -s size	 Set  the  maximum number of bytes for the command line length provided to utility.  The sum of the length of the utility name and
		 the arguments passed to utility (including /dev/null terminators) will be less than or equal to this number.  The current default
		 value for size is ARG_MAX - 2048.

       -t	 Echo the command to be executed to standard error immediately before it is executed.

       -x	 Force	xargs  to  terminate  immediately if a command line containing number arguments will not fit in the specified (or default)
		 command line length.

       If no utility is specified, echo(1) is used.

       Undefined behavior may occur if utility reads from the standard input.

       The xargs utility exits immediately (without processing any further input) if a	command  line  cannot  be  assembled,  utility	cannot	be
       invoked, an invocation of the utility is terminated by a signal or an invocation of the utility exits with a value of 255.

       The xargs utility exits with a value of 0 if no error occurs.  If utility cannot be invoked, xargs exits with a value of 127.  If any other
       error occurs, xargs exits with a value of 1.

SEE ALSO

       echo(1), find(1)

STANDARDS

       The xargs utility is expected to be IEEE Std 1003.2("POSIX") compliant.

								   June 6, 1993 							  XARGS(1)