11-14-2005
use the split command
split -l 60000 in_file
In case of more complex fragmentation of a file, use the context based split command csplit.
Jerardjay
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
I have a large file with a repeating pattern in it. Now i want the file split into the block of patterns with a specified no. of lines in each file.
i.e. The file is like
1...
2...
2...
3...
1...
2...
3...
1...
2...
2...
2...
2...
2...
3...
where 1 is the start of the block... (5 Replies)
Discussion started by: sudhamacs
5 Replies
2. Shell Programming and Scripting
HI,
i've to split a large file which inputs seems like :
Input file name_file.txt
00001|AAAA|MAIL|DATEOFBIRTHT|.......
00001|AAAA|MAIL|DATEOFBIRTHT|.......
00002|BBBB|MAIL|DATEOFBIRTHT|.......
00002|BBBB|MAIL|DATEOFBIRTHT|.......
00003|CCCC|MAIL|DATEOFBIRTHT|.......... (1 Reply)
Discussion started by: AMARA
1 Replies
3. Shell Programming and Scripting
I have a file containing date/time sorted data of the form
...
2009/06/10,20:59:59.950,XAG/USD,Q,1,1115, 14.3025,100,1,1
2009/06/10,20:59:59.950,XAG/USD,Q,1,1116, 14.3026,125,1,1
2009/06/10,20:59:59.950,XAG/USD,R,0,0, , 0,0,0
2009/06/10,20:59:59.950,XAG/USD,R,1,0, 14.1910,100,1,1... (6 Replies)
Discussion started by: asriva
6 Replies
4. Shell Programming and Scripting
Hello Everyone,
I have a large file that needs to be split into many seperate files, however the text in between the blank lines need to be intact. The file looks like
SomeText
SomeText
SomeText
SomeOtherText
SomeOtherText
....
Since the number of lines of text are different for... (3 Replies)
Discussion started by: jwillis0720
3 Replies
5. Shell Programming and Scripting
Hello,
What's the best way to split a large into multiple files based on the last digit in the first column.
input file:
f
2738483300000x0y03772748378831x1y13478378358383x2y23743878383802x3y33787828282820x4y43748838383881x5y5
Desired Output:
f0
3738483300000x0y03787828282820x4y4
f1... (9 Replies)
Discussion started by: alain.kazan
9 Replies
6. Shell Programming and Scripting
I have a 3 GB text file that I would like to split. How can I do this?
It's a giant comma-separated list of numbers. I would like to make it into about 20 files of ~100 MB each, with a custom header and footer. The file can only be split on commas, but they're plentiful.
Something like... (3 Replies)
Discussion started by: CRGreathouse
3 Replies
7. UNIX for Dummies Questions & Answers
hi ,
I have a requirement
input file:
1 1111111111111 108
1 1111111111111 109
1 1111111111111 109
1 1111111111111 110
1 1111111111111 111
1 1111111111111 111
1 1111111111111 111
1 1111111111111 112
1 1111111111111 112
1 1111111111111 112
The output should be, (19 Replies)
Discussion started by: mechvijays
19 Replies
8. UNIX for Beginners Questions & Answers
Dear Users,
Appreciate your help if you could help me with splitting a large file > 1 million lines with sed or awk. below is the text in the file
input file.txt
scaffold1 928 929 C/T +
scaffold1 942 943 G/C +
scaffold1 959 960 C/T +... (6 Replies)
Discussion started by: kapr0001
6 Replies
9. UNIX for Advanced & Expert Users
Hi,
I have received a file which is 20 GB. We would like to split the file into 4 equal parts and process it to avoid memory issues.
If the record delimiter is unix new line, I could use split command either with option l or b.
The problem is that the line terminator is |##|
How to use... (5 Replies)
Discussion started by: Ravi.K
5 Replies
10. UNIX for Beginners Questions & Answers
Trying to split a 35gb file into 1000mb parts. My research shows I should you this. split -b 1000m file.txt and my return is "split: cannot open 'crunch1.txt' for reading: No such file or directory" so I tried split -b 1000m Documents/Wordlists/file.txt and I get nothing other than the curser just... (3 Replies)
Discussion started by: sub terra
3 Replies
LEARN ABOUT FREEBSD
csplit
CSPLIT(1) BSD General Commands Manual CSPLIT(1)
NAME
csplit -- split files based on context
SYNOPSIS
csplit [-ks] [-f prefix] [-n number] file args ...
DESCRIPTION
The csplit utility splits file into pieces using the patterns args. If file is a dash ('-'), csplit reads from standard input.
Files are created with a prefix of ``xx'' and two decimal digits. The size of each file is written to standard output as it is created. If
an error occurs whilst files are being created, or a HUP, INT, or TERM signal is received, all files previously written are removed.
The options are as follows:
-f prefix
Create file names beginning with prefix, instead of ``xx''.
-k Do not remove previously created files if an error occurs or a HUP, INT, or TERM signal is received.
-n number
Create file names beginning with number of decimal digits after the prefix, instead of 2.
-s Do not write the size of each output file to standard output as it is created.
The args operands may be a combination of the following patterns:
/regexp/[[+|-]offset]
Create a file containing the input from the current line to (but not including) the next line matching the given basic regular
expression. An optional offset from the line that matched may be specified.
%regexp%[[+|-]offset]
Same as above but a file is not created for the output.
line_no
Create containing the input from the current line to (but not including) the specified line number.
{num} Repeat the previous pattern the specified number of times. If it follows a line number pattern, a new file will be created for each
line_no lines, num times. The first line of the file is line number 1 for historic reasons.
After all the patterns have been processed, the remaining input data (if there is any) will be written to a new file.
Requesting to split at a line before the current line number or past the end of the file will result in an error.
ENVIRONMENT
The LANG, LC_ALL, LC_COLLATE and LC_CTYPE environment variables affect the execution of csplit as described in environ(7).
EXIT STATUS
The csplit utility exits 0 on success, and >0 if an error occurs.
EXAMPLES
Split the mdoc(7) file foo.1 into one file for each section (up to 21 plus one for the rest, if any):
csplit -k foo.1 '%^.Sh%' '/^.Sh/' '{20}'
Split standard input after the first 99 lines and every 100 lines thereafter:
csplit -k - 100 '{19}'
SEE ALSO
sed(1), split(1), re_format(7)
STANDARDS
The csplit utility conforms to IEEE Std 1003.1-2001 (``POSIX.1'').
HISTORY
A csplit command appeared in PWB UNIX.
BUGS
Input lines are limited to LINE_MAX (2048) bytes in length.
BSD
February 6, 2014 BSD