03-11-2008
fopen() + reading in large text files
For reading in large text files (say files over 1kB in size) are there any issues with fopen() that I should be aware of ?
cheers
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi All
I have approximately 10 files that are at least 100+ MB in size. I am importing them into a DB to output them to the web. What i need to do first is clean the files up so i dont have un necessary rows in the DB. Below is what the file looks like:
Ignore the <TAB> annotations as that... (4 Replies)
Discussion started by: caddyjoe77
4 Replies
2. Shell Programming and Scripting
Hello all,
I have 2 files, the first (indexFile1) contains start offset and length for each record inside the second file. The second file can be very large, each actual record start offset and length is defined by the entry in indexFile1. Since there are no records separators wc-l returns 0 for... (1 Reply)
Discussion started by: gio001
1 Replies
3. Shell Programming and Scripting
Hi All!!
I have a large file containing millions of record. My purpose is to extract 7 characters immediately after text '19' from this file (including text '19') and save the result in new file.
So, my OUTPUT would be as under :
191234561
194567894
192789005
198839408
and so on.....
... (7 Replies)
Discussion started by: parshant_bvcoe
7 Replies
4. Shell Programming and Scripting
I have a large CSV files (e.g. 2 million records) and am hoping to do one of two things. I have been trying to use awk and sed but am a newbie and can't figure out how to get it to work. Any help you could offer would be greatly appreciated - I'm stuck trying to remove the colon and wildcards in... (6 Replies)
Discussion started by: metronomadic
6 Replies
5. Shell Programming and Scripting
Hi Everyone,
I am using a centos 5.2 server as an sflow log collector on my network. Currently I am using inmons free sflowtool to collect the packets sent by my switches. I have a bash script running on an infinate loop to stop and start the log collection at set intervals - currently one... (2 Replies)
Discussion started by: lord_butler
2 Replies
6. Shell Programming and Scripting
Hi,
i have more than 1000 data files(.txt) like this
first file format:
178.83 554.545
179.21 80.392
second file:
178.83 990.909
179.21 90.196
etc.
I want to combine them to the following format:
178.83,554.545,990.909,...
179.21,80.392,90.196,... (7 Replies)
Discussion started by: mr_monocyte
7 Replies
7. Solaris
Hello everyone. Need some help copying a filesystem. The situation is this: I have an oracle DB mounted on /u01 and need to copy it to /u02. /u01 is 500 Gb and /u02 is 300 Gb. The size used on /u01 is 187 Gb. This is running on solaris 9 and both filesystems are UFS.
I have tried to do it using:... (14 Replies)
Discussion started by: dragonov7
14 Replies
8. Shell Programming and Scripting
Hello all, newbie here. I've searched the forum and found many "how to split a text file" topics but none that are what I'm looking for.
I have a large text file (~15 MB) in size. It contains a variable number of "paragraphs" (for lack of a better word) that are each of variable length. A... (3 Replies)
Discussion started by: lupin..the..3rd
3 Replies
9. Shell Programming and Scripting
Hi all,
I have certain task to do, which involves reading the first column of 1.txt file.
This is variable "event"
28434710
23456656
3456895
& finding this "event" in some other text file 2.txt, which has information in the following format
#Zgamma: 1 run: 160998 event: ... (7 Replies)
Discussion started by: nrjrasaxena
7 Replies
10. Shell Programming and Scripting
Hi,
I have a large output file (star.log), with many lines of the following type
*** T vavg unburnt: 723.187 / burnt: 2662.000
What I would like to do is pick the values 723.187 and 2662.000 and
What I've got so far is
awk '/unburnt:.*burnt:/{Tu=$6;Tb=$NF}END{print Tu, Tb}'... (6 Replies)
Discussion started by: lost.identity
6 Replies
LEARN ABOUT OPENSOLARIS
fopen
fopen(3UCB) SunOS/BSD Compatibility Library Functions fopen(3UCB)
NAME
fopen, freopen - open a stream
SYNOPSIS
/usr/ucb/cc [ flag ... ] file ...
#include <stdio.h>
FILE *fopen(file, mode)
const char *file, *mode;
FILE *freopen(file, mode, iop)
const char *file, *mode;
register FILE *iop;
DESCRIPTION
The fopen() function opens the file specified by file and associates a stream with it. If the open succeeds, fopen() returns a pointer to
be used to identify the stream in subsequent operations. The file argument points to a character string that contains the name of the file
to be opened. The mode argument is a character string having one of the following values:
r open for reading
w truncate or create for writing
a append: open for writing at end of file, or create for writing
r+ open for update (reading and writing)
w+ truncate or create for update
a+ append; open or create for update at EOF
The freopen() function opens the file specified by file and associates the stream pointed to by iop with it. The mode argument is used just
as in fopen(). The original stream is closed, regardless of whether the open ultimately succeeds. If the open succeeds, freopen() returns
the original value of iop.
The freopen() function is typically used to attach the preopened streams associated withstdin, stdout, and stderr to other files.
When a file is opened for update, both input and output can be performed on the resulting stream. Output cannot be directly followed by
input without an intervening fseek(3C) or rewind(3C). Input cannot be directly followed by output without an intervening fseek(3C) or
rewind(3C). An input operation that encounters EOF will fail.
RETURN VALUES
The fopen() and freopen() functions return a NULL pointer on failure.
USAGE
The fopen() and freopen() functions have transitional interfaces for 64-bit file offsets. See lf64(5).
SEE ALSO
cc(1B), open(2), fclose(3C), fopen(3C), freopen(3C), fseek(3C), malloc(3C), rewind(3C), lf64(5)
NOTES
Use of these functions should be restricted to applications written on BSD platforms. Use of these functions with any of the system
libraries or in multithreaded applications is unsupported.
To support the same number of open files as the system, fopen() must allocate additional memory for data structures using malloc(3C) after
64 files have been opened. This confuses some programs that use their own memory allocators.
The fopen() and freopen() functions differ from the standard I/O functions fopen(3C) and freopen(3C). The standard I/O functions distin-
guish binary from text files with an additional use of 'b' as part of the mode, enabling portability of fopen(3C) and freopen(3C) beyond
SunOS 4.x systems.
SunOS 5.11 30 Oct 2007 fopen(3UCB)