Reading a whole line without betting on size

Thread Tools Search this Thread
Top Forums Programming Reading a whole line without betting on size
# 1  
Old 10-21-2009
Reading a whole line without betting on size

i have been googling, and came to the conclusion that there is not standard C library (or commonly used) that reads a complete line of a file, without a size parameter being involved.

so, as a little exercise i decided to think it over, and make my own

i came up with an idea, and wanted to hear comments (keeping in mind i want to learn from this)

the idea, in pseudo bable would be like this

char * readholeline ( FILE *file, size_t chunk);
   while we dont hit the end of line
      allocate a size of chunk array
      read chunk bytes into array
      add array into a new node of a linked list
   calculate how many chars we have counting all nodes
   allocate memory for all chars
   while we still have have more nodes
       copy string from node to "all chars string"
       next node  
       free previous node
    return "all chars pointer"


i discarded readin one char at a time, because it would be i/o intensive, and added the chunk parameter, because most of the time we have "some idea" of how big a file is, so we can make a "close" guess, but not quite exact ...

im still writing the real code, but wanted to know what other would think, or if someone has a better approach too

thanks Smilie
# 2  
Old 10-21-2009
For line-based I/O, you want a maximum. Otherwise, when fed an endless line, your program will allocate endless amounts of memory. Make it a large maximum if you want, but give it some sort of maximum. Also note that while you can make your limit as large as you feasibly want, there are plenty of UNIX commandline tools, like standard sed and grep, that won't handle lines larger than 2048 bytes. Even things like uuencoding that translate raw binary into ASCII have measures to limit their line length because it's always in your best interest to keep line lengths reasonable.

Furthermore, reading more than one character at a time won't let you "unget" the extra data read beyond the end of line. You can unget one and only one character. So you have to read one-at-a-time if you want to keep all your data.

Also, your algorithm overcomplex, there's no need for a linked list or other data structure. You can resize already-allocated memory with realloc(). It's location may change, but already-defined data will stay defined, while the new area on the end will be undefined until you overwrite it.

I'd just use fgets() with a limit of a few kilobytes. If you really wanted to be sure, make the limit configurable with a commandline switch, just in case.

Last edited by Corona688; 10-21-2009 at 04:20 PM..
# 3  
Old 10-21-2009
thanks for the insight.
i didn't knew about other programs having a line limit, and neither tough of the "endless" line problem

about realloc, never crossed my mind, and it does surely sounds safer than keeping a list tight :P

thanks for the info
Login or Register to Ask a Question

Previous Thread | Next Thread

7 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Reading a file line by line and print required lines based on pattern

Hi All, i want to write a shell script read below file line by line and want to exclude the lines which contains empty value for MOUNTPOINT field. i am using centos 7 Operating system. want to read below file. # cat /tmp/d5 NAME="/dev/sda" TYPE="disk" SIZE="60G" OWNER="root"... (4 Replies)
Discussion started by: balu1234
4 Replies

2. Programming

Copying a file and reading size I get always zero.

I need to copy a file (srcFile) to a new ubication (destFile). Then I calculate the size of the copy file using stat. The problem is that I always get a cero size value although the file has not cero size in the hard disk. The source code that does this is : PROGRAM 1: char srcFile;... (4 Replies)
Discussion started by: clxspain
4 Replies

3. Shell Programming and Scripting

Reading line by line from live log file using while loop and considering only those lines start from

Hi, I want to read a live log file line by line and considering those line which start from time stamp; Below code I am using, which read line but throws an exception when comparing line that does not contain error code tail -F /logs/COMMON-ERROR.log | while read myline; do... (2 Replies)
Discussion started by: ketanraut
2 Replies

4. Shell Programming and Scripting

Reading text file, comparing a value in a line, and placing only part of the line in a variable?

I need some help. I would like to read in a text file. Take a variable such as ROW-D-01, compare it to what's in one line in the text file such as PROD/VM/ROW-D-01 and only input PROD/VM into a variable without the /ROW-D-01. Is this possible? any help is appreciated. (2 Replies)
Discussion started by: xChristopher
2 Replies

5. Shell Programming and Scripting

[Solved] Problem in reading a file line by line till it reaches a white line

So, I want to read line-by-line a text file with unknown number of files.... So: a=1 b=1 while ; do b=`sed -n '$ap' test` a=`expr $a + 1` $here do something with b etc done the problem is that sed does not seem to recognise the $a, even when trying sed -n ' $a p' So, I cannot read... (3 Replies)
Discussion started by: hakermania
3 Replies

6. Shell Programming and Scripting

Reading a file line by line and processing for each line

Hi, I am a beginner in shell scripting. I have written the following script, which is supposed to process the while loop for each line in the sid_home.txt file. But I'm getting the 'end of file' unexpected for the last line. The file sid_home.txt gets generated as expected, but the script... (6 Replies)
Discussion started by: sagarparadkar
6 Replies

7. Shell Programming and Scripting

URGENT HELP NEEDED ON -File size reading

All Expert, I am using Sun OS 5.8 and Perl version 5 in One server and Perl 5.8 in another unix server. I am able to read a file using fopen function of perl --file size having more then 3 GB of data.(In the machine where Perl 5.8 install) But when i am running the same perl script --It... (1 Reply)
Discussion started by: jambesh
1 Replies
Login or Register to Ask a Question