Hi,
I have a large file with a repeating pattern in it. Now i want the file split into the block of patterns with a specified no. of lines in each file.
i.e. The file is like
where 1 is the start of the block and 3 is the end of the block.
Now i want the file to be split up in blocks. But each file can have more than one block, but specified no. of lines or less.
What i have done is i split up the file using
But this gives one file for each block. I needed to concat the files to get a file with some specified no. of lines, say 10000.
Please give me an efficient way to do this. I feel we can do it in the awk command only.
Thanks,
CSS
Last edited by Yogesh Sawant; 06-24-2008 at 04:36 AM..
Reason: added code tags
Hi,
I have a large file(csv format) that I need to split into 2 files. The file looks something like
Original_file.txt
first name, family name, address
a, b, c,
d, e, f,
and so on for over 100,00 lines
I need to create two files from this one file. The condition is i need to ensure... (4 Replies)
HI,
i've to split a large file which inputs seems like :
Input file name_file.txt
00001|AAAA|MAIL|DATEOFBIRTHT|.......
00001|AAAA|MAIL|DATEOFBIRTHT|.......
00002|BBBB|MAIL|DATEOFBIRTHT|.......
00002|BBBB|MAIL|DATEOFBIRTHT|.......
00003|CCCC|MAIL|DATEOFBIRTHT|.......... (1 Reply)
I have a file containing date/time sorted data of the form
...
2009/06/10,20:59:59.950,XAG/USD,Q,1,1115, 14.3025,100,1,1
2009/06/10,20:59:59.950,XAG/USD,Q,1,1116, 14.3026,125,1,1
2009/06/10,20:59:59.950,XAG/USD,R,0,0, , 0,0,0
2009/06/10,20:59:59.950,XAG/USD,R,1,0, 14.1910,100,1,1... (6 Replies)
Hello Everyone,
I have a large file that needs to be split into many seperate files, however the text in between the blank lines need to be intact. The file looks like
SomeText
SomeText
SomeText
SomeOtherText
SomeOtherText
....
Since the number of lines of text are different for... (3 Replies)
I have a 3 GB text file that I would like to split. How can I do this?
It's a giant comma-separated list of numbers. I would like to make it into about 20 files of ~100 MB each, with a custom header and footer. The file can only be split on commas, but they're plentiful.
Something like... (3 Replies)
Dear Users,
Appreciate your help if you could help me with splitting a large file > 1 million lines with sed or awk. below is the text in the file
input file.txt
scaffold1 928 929 C/T +
scaffold1 942 943 G/C +
scaffold1 959 960 C/T +... (6 Replies)
Hi,
I have received a file which is 20 GB. We would like to split the file into 4 equal parts and process it to avoid memory issues.
If the record delimiter is unix new line, I could use split command either with option l or b.
The problem is that the line terminator is |##|
How to use... (5 Replies)
Trying to split a 35gb file into 1000mb parts. My research shows I should you this. split -b 1000m file.txt and my return is "split: cannot open 'crunch1.txt' for reading: No such file or directory" so I tried split -b 1000m Documents/Wordlists/file.txt and I get nothing other than the curser just... (3 Replies)
Discussion started by: sub terra
3 Replies
LEARN ABOUT OPENDARWIN
split
SPLIT(1) BSD General Commands Manual SPLIT(1)NAME
split -- split a file into pieces
SYNOPSIS
split [-a suffix_length] [-b byte_count[k|m]] [-l line_count] [-p pattern] [file [name]]
DESCRIPTION
The split utility reads the given file and breaks it up into files of 1000 lines each. If file is a single dash ('-') or absent, split reads
from the standard input.
The options are as follows:
-a Use suffix_length letters to form the suffix of the file name.
-b Create smaller files byte_count bytes in length. If ``k'' is appended to the number, the file is split into byte_count kilobyte
pieces. If ``m'' is appended to the number, the file is split into byte_count megabyte pieces.
-l Create smaller files n lines in length.
-p pattern
The file is split whenever an input line matches pattern, which is interpreted as an extended regular expression. The matching line
will be the first line of the next output file. This option is incompatible with the -b and -l options.
If additional arguments are specified, the first is used as the name of the input file which is to be split. If a second additional argument
is specified, it is used as a prefix for the names of the files into which the file is split. In this case, each file into which the file is
split is named by the prefix followed by a lexically ordered suffix using suffix_length characters in the range ``a-z''. If -a is not speci-
fied, two letters are used as the suffix.
If the name argument is not specified, the file is split into lexically ordered files named with prefixes in the range of ``x-z'' and with
suffixes as above.
SEE ALSO csplit(1), re_format(7)STANDARDS
The split utility conforms to IEEE Std 1003.1-2001 (``POSIX.1'').
HISTORY
A split command appeared in Version 3 AT&T UNIX.
BUGS
For historical reasons, if you specify name, split can only create 676 separate files. The default naming convention allows 2028 separate
files. The -a option can be used to work around this limitation.
The maximum line length for matching patterns is 65536.
BSD April 16, 1994 BSD