I have gone through all the threads in the forum and tested out different things. I am trying to split a 3GB file into multiple files. Some files are even larger than this.
For example:
split -l 3000000 filename.txt
This is very slow and it splits the file with 3 million records in each... (10 Replies)
Hi All,
Can someone please help me write a script for the following requirement in awk, grep, sed or perl.
Buuuu xxx bbb
Kmmmm rrr ssss uuuu
Kwwww zzzz ccc
Roooowwww eeee
Bxxxx jjjj dddd
Kuuuu eeeee nnnn
Rpppp cccc vvvv cccc
Rhhhhhhyyyy tttt
Lhhhh rrrrrssssss
Bffff mmmm iiiii
Ktttt... (5 Replies)
Hi experts,Please help me for the below requirement. i have a source file.(lets say contains 50 columns). I am extarcting five columns from the source file by using pattern file. for example input file:--------a,b,c,d,"a,g","v b",s,koutputfile=======a,"a,g","v b",s,kThanks in advance subhendu (2 Replies)
Hi,
I have a data file xyz.dat similar to the one given below,
2345|98|809||x|969|0
2345|98|809||y|0|537
2345|97|809||x|544|0
2345|97|809||y|0|651
9685|98|809||x|321|0
9685|98|809||y|0|357
9685|98|709||x|687|0
9685|98|709||y|0|234
2315|98|809||x|564|0
2315|98|809||y|0|537... (2 Replies)
Hi,
need help in recognizing the pattern of file name. For e.g.
file name 1: <static file prefix>.<store cd>_<YYYYMMDD>.<ext>
file name 2: <static file prefix>_<YYYYMMDD>.<ext>
I want to know that there are 3 dots "." in the file name1 and one dot "." in file name2. How can I know... (3 Replies)
Good day all
I need some helps,
say that I have data like below, each field separated by a tab
DATE NAME ADDRESS
15/7/2012 LX a.b.c
15/7/2012 LX1 a.b.c
16/7/2012 AB a.b.c
16/7/2012 AB2 a.b.c
15/7/2012 LX2 a.b.c... (2 Replies)
Requirement: Need to split a source file say a1.txt which can be of size upto 150 MB into 25 target files each with a max size of 25 MB along with the header line in each target file.
NOTE: Few target files can be empty also ,but 25 files must be generated for 1 source file( I can expect upto... (4 Replies)
Hello,
I want to split a file based on an input list file that contains the lines each split should have + a corresponding file name.
#!/bin/sh
# sed -n 'start_line_#,end_line_#p' my_input_file > lines_extracted_output_file
while read a b c
do
sed -n '$a,$bp' myLarge.file > $c.split... (2 Replies)
Hi
I need to do a patten match between files .
I am new to shell scripting and have come up with this so far. It take 50 seconds to process files of 2mb size . I need to tune this code as file size will be around 50mb and need to save time.
Main issue is that I need to search the pattern from... (2 Replies)
Dear Users,
Appreciate your help if you could help me with splitting a large file > 1 million lines with sed or awk. below is the text in the file
input file.txt
scaffold1 928 929 C/T +
scaffold1 942 943 G/C +
scaffold1 959 960 C/T +... (6 Replies)
Discussion started by: kapr0001
6 Replies
LEARN ABOUT FREEBSD
split
SPLIT(1) BSD General Commands Manual SPLIT(1)NAME
split -- split a file into pieces
SYNOPSIS
split -d [-l line_count] [-a suffix_length] [file [prefix]]
split -d -b byte_count[K|k|M|m|G|g] [-a suffix_length] [file [prefix]]
split -d -n chunk_count [-a suffix_length] [file [prefix]]
split -d -p pattern [-a suffix_length] [file [prefix]]
DESCRIPTION
The split utility reads the given file and breaks it up into files of 1000 lines each (if no options are specified), leaving the file
unchanged. If file is a single dash ('-') or absent, split reads from the standard input.
The options are as follows:
-a suffix_length
Use suffix_length letters to form the suffix of the file name.
-b byte_count[K|k|M|m|G|g]
Create split files byte_count bytes in length. If k or K is appended to the number, the file is split into byte_count kilobyte
pieces. If m or M is appended to the number, the file is split into byte_count megabyte pieces. If g or G is appended to the num-
ber, the file is split into byte_count gigabyte pieces.
-d Use a numeric suffix instead of a alphabetic suffix.
-l line_count
Create split files line_count lines in length.
-n chunk_count
Split file into chunk_count smaller files.
-p pattern
The file is split whenever an input line matches pattern, which is interpreted as an extended regular expression. The matching line
will be the first line of the next output file. This option is incompatible with the -b and -l options.
If additional arguments are specified, the first is used as the name of the input file which is to be split. If a second additional argument
is specified, it is used as a prefix for the names of the files into which the file is split. In this case, each file into which the file is
split is named by the prefix followed by a lexically ordered suffix using suffix_length characters in the range ``a-z''. If -a is not speci-
fied, two letters are used as the suffix.
If the prefix argument is not specified, the file is split into lexically ordered files named with the prefix ``x'' and with suffixes as
above.
ENVIRONMENT
The LANG, LC_ALL, LC_CTYPE and LC_COLLATE environment variables affect the execution of split as described in environ(7).
EXIT STATUS
The split utility exits 0 on success, and >0 if an error occurs.
SEE ALSO csplit(1), re_format(7)STANDARDS
The split utility conforms to IEEE Std 1003.1-2001 (``POSIX.1'').
HISTORY
A split command appeared in Version 3 AT&T UNIX.
BUGS
The maximum line length for matching patterns is 65536.
BSD May 9, 2013 BSD