11-16-2008
splitting the large file into smaller files
hi all
im new to this forum..excuse me if anythng wrong.
I have a file containing 600 MB data in that. when i do parse the data in perl program im getting out of memory error.
so iam planning to split the file into smaller files and process one by one.
can any one tell me what is the code will do for spliting the bigger file.
what is the better way to achieve this ?
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
I need to split a large file into small files based on a string.
At different palces in the large I have the string ^Job.
I need to split the file into different files starting from ^Job to the last character before the next ^Job.
Also all the small files should be automatically named.... (4 Replies)
Discussion started by: dncs
4 Replies
2. Shell Programming and Scripting
Hi Everyone,
I am using a centos 5.2 server as an sflow log collector on my network. Currently I am using inmons free sflowtool to collect the packets sent by my switches. I have a bash script running on an infinate loop to stop and start the log collection at set intervals - currently one... (2 Replies)
Discussion started by: lord_butler
2 Replies
3. UNIX for Dummies Questions & Answers
I have a file with a simple list of ids. 750,000 rows. I have to break it down into multiple 50,000 row files to submit in a batch process.. Is there an easy script I could write to accomplish this task? (2 Replies)
Discussion started by: rtroscianecki
2 Replies
4. Shell Programming and Scripting
Hello everyone!
I have 2 types of files in the following format:
1) *.fa
>1234
...some text...
>2345
...some text...
>3456
...some text...
.
.
.
.
2) *.info
>1234 (7 Replies)
Discussion started by: ad23
7 Replies
5. Shell Programming and Scripting
I need to write a shell script for below scenario
My input file has data in format:
qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26
qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28
qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43
qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies
6. Shell Programming and Scripting
Hi,
I'm trying to split a large file into several smaller files
the script will have two input arguments argument1=filename and argument2=no of files to be split.
In my large input file I have a header followed by 100009 records
The first line is a header; I want this header in all my... (9 Replies)
Discussion started by: ramky79
9 Replies
7. Shell Programming and Scripting
Help needed urgently please.
I have a large file - a few hundred thousand lines.
Sample
CP START ACCOUNT
1234556
name 1
CP END ACCOUNT
CP START ACCOUNT
2224444
name 1
CP END ACCOUNT
CP START ACCOUNT
333344444
name 1
CP END ACCOUNT
I need to split this file each time "CP START... (7 Replies)
Discussion started by: frustrated1
7 Replies
8. Shell Programming and Scripting
I will simplify the explaination a bit, I need to parse through a 87m file -
I have a single text file in the form of :
<NAME>house........
SOMETEXT
SOMETEXT
SOMETEXT
.
.
.
.
</script>
MORETEXT
MORETEXT
.
.
. (6 Replies)
Discussion started by: sumguy
6 Replies
9. UNIX for Beginners Questions & Answers
Dears,
Need you help with the below file manipulation. I want to split the file into 8 smaller files but without cutting/disturbing the entries (meaning every small file should start with a entry and end with an empty line). It will be helpful if you can provide a one liner command for this... (12 Replies)
Discussion started by: Kamesh G
12 Replies
10. Shell Programming and Scripting
Hello,
I have some large text files that look like,
putrescine
Mrv1583 01041713302D
6 5 0 0 0 0 999 V2000
2.0928 -0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0
5.6650 0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0
3.5217 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies
FSPLIT(1) General Commands Manual FSPLIT(1)
NAME
fsplit - split a multi-routine Fortran file into individual files
SYNOPSIS
fsplit [ -e efile] ... [ file ]
DESCRIPTION
Fsplit takes as input either a file or standard input containing Fortran source code. It attempts to split the input into separate routine
files of the form name.f, where name is the name of the program unit (e.g. function, subroutine, block data or program). The name for
unnamed block data subprograms has the form blkdtaNNN.f where NNN is three digits and a file of this name does not already exist. For
unnamed main programs the name has the form mainNNN.f. If there is an error in classifying a program unit, or if name.f already exists,
the program unit will be put in a file of the form zzzNNN.f where zzzNNN.f does not already exist.
Normally each subprogram unit is split into a separate file. When the -e option is used, only the specified subprogram units are split
into separate files. E.g.:
fsplit -e readit -e doit prog.f
will split readit and doit into separate files.
DIAGNOSTICS
If names specified via the -e option are not found, a diagnostic is written to standard error.
AUTHOR
Asa Romberger and Jerry Berkman
BUGS
Fsplit assumes the subprogram name is on the first noncomment line of the subprogram unit. Nonstandard source formats may confuse fsplit.
It is hard to use -e for unnamed main programs and block data subprograms since you must predict the created file name.
4.2 Berkeley Distribution April 29, 1985 FSPLIT(1)