fopen() + reading in large text files

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Reading off values from a large file

Hi, I have a large output file (star.log), with many lines of the following type *** T vavg unburnt: 723.187 / burnt: 2662.000 What I would like to do is pick the values 723.187 and 2662.000 and What I've got so far is awk '/unburnt:.*burnt:/{Tu=$6;Tb=$NF}END{print Tu, Tb}'...

2. Shell Programming and Scripting

reading files and pasting in another text file

Hi all, I have certain task to do, which involves reading the first column of 1.txt file. This is variable "event" 28434710 23456656 3456895 & finding this "event" in some other text file 2.txt, which has information in the following format #Zgamma: 1 run: 160998 event: ...

3. Shell Programming and Scripting

splitting a large text file into paragraphs

Hello all, newbie here. I've searched the forum and found many "how to split a text file" topics but none that are what I'm looking for. I have a large text file (~15 MB) in size. It contains a variable number of "paragraphs" (for lack of a better word) that are each of variable length. A...

4. Solaris

How to safely copy full filesystems with large files (10Gb files)

Hello everyone. Need some help copying a filesystem. The situation is this: I have an oracle DB mounted on /u01 and need to copy it to /u02. /u01 is 500 Gb and /u02 is 300 Gb. The size used on /u01 is 187 Gb. This is running on solaris 9 and both filesystems are UFS. I have tried to do it using:...

5. Shell Programming and Scripting

Need help combining large number of text files

Hi, i have more than 1000 data files(.txt) like this first file format: 178.83 554.545 179.21 80.392 second file: 178.83 990.909 179.21 90.196 etc. I want to combine them to the following format: 178.83,554.545,990.909,... 179.21,80.392,90.196,...

6. Shell Programming and Scripting

Help with splitting a large text file into smaller ones

Hi Everyone, I am using a centos 5.2 server as an sflow log collector on my network. Currently I am using inmons free sflowtool to collect the packets sent by my switches. I have a bash script running on an infinate loop to stop and start the log collection at set intervals - currently one...

7. Shell Programming and Scripting

Sed or awk script to remove text / or perform calculations from large CSV files

I have a large CSV files (e.g. 2 million records) and am hoping to do one of two things. I have been trying to use awk and sed but am a newbie and can't figure out how to get it to work. Any help you could offer would be greatly appreciated - I'm stuck trying to remove the colon and wildcards in...

8. Shell Programming and Scripting

Need to extract 7 characters immediately after text '19' from a large file.

Hi All!! I have a large file containing millions of record. My purpose is to extract 7 characters immediately after text '19' from this file (including text '19') and save the result in new file. So, my OUTPUT would be as under : 191234561 194567894 192789005 198839408 and so on..... ...

9. Shell Programming and Scripting

Reading large file, awk and cut

Hello all, I have 2 files, the first (indexFile1) contains start offset and length for each record inside the second file. The second file can be very large, each actual record start offset and length is defined by the entry in indexFile1. Since there are no records separators wc-l returns 0 for...

10. Shell Programming and Scripting

Large Text Files

Hi All I have approximately 10 files that are at least 100+ MB in size. I am importing them into a DB to output them to the web. What i need to do first is clean the files up so i dont have un necessary rows in the DB. Below is what the file looks like: Ignore the <TAB> annotations as that...

LEARN ABOUT BSD

fdopen

FOPEN(3S)																 FOPEN(3S)

NAME

       fopen, freopen, fdopen - open a stream

SYNOPSIS

       #include <stdio.h>

       FILE *fopen(filename, type)
       char *filename, *type;

       FILE *freopen(filename, type, stream)
       char *filename, *type;
       FILE *stream;

       FILE *fdopen(fildes, type)
       char *type;

DESCRIPTION

       Fopen  opens the file named by filename and associates a stream with it.  Fopen returns a pointer to be used to identify the stream in sub-
       sequent operations.

       Type is a character string having one of the following values:

       "r"  open for reading

       "w"  create for writing

       "a"  append: open for writing at end of file, or create for writing

       In addition, each type may be followed by a "+" to have the file opened for reading and writing.  "r+" positions the stream at  the  begin-
       ning of the file, "w+" creates or truncates it, and "a+" positions it at the end.  Both reads and writes may be used on read/write streams,
       with the limitation that an fseek, rewind, or reading an end-of-file must be used between a read and a write or vice-versa.

       Freopen substitutes the named file in place of the open stream.	It returns the original value of stream.  The original stream is closed.

       Freopen is typically used to attach the preopened constant names, stdin, stdout, stderr, to specified files.

       Fdopen associates a stream with a file descriptor obtained from open, dup, creat, or pipe(2).  The type of the stream must agree  with  the
       mode of the open file.

SEE ALSO

       open(2), fclose(3)

DIAGNOSTICS

       Fopen  and freopen return the pointer NULL if filename cannot be accessed, if too many files are already open, or if other resources needed
       cannot be allocated.

BUGS

       Fdopen is not portable to systems other than UNIX.

       The read/write types do not exist on all systems.  Those systems without read/write modes will probably treat the type as if  the  "+"  was
       not present.  These are unreliable in any event.

       In  order to support the same number of open files as does the system, fopen must allocate additional memory for data structures using cal-
       loc after 20 files have been opened.  This confuses some programs which use their own memory allocators.  An undocumented routine, f_preal-
       loc, may be called to force immediate allocation of all internal memory except for buffers.

4th Berkeley Distribution					   May 27, 1986 							 FOPEN(3S)

Programming