I don't think switching to awk will speed up things dramatically if at all, as it won't be faster than the designed-for-the-task split command. In either case you'll need to scan through the entire file to calculate its line count, and then again to do the split.
Why not optimize what you got above, reducing it to the single command
(given your OS and shell versions (which you fail to mention) offer the features used)?
Hello everyone,
I have a big folder in a linux server that contains dozen of big files (total folder size ~ 50 GB)
I have a couple of files of 2.5 GB and some others from 100 MB to 1 GB. (that said it's obvious that it's impossible having two 2.5 GB files in one dvd)
The purpose is to... (7 Replies)
Hey guys,
Sorry for the basic question but I have a lot of files that I want to separate into groups based on filenames which I can then cat together. Eg I have:
(a_b_c.txt)
WB34_2_SLA8.txt
WB34_1_SLA8.txt
WB34_1_DB10.txt
WB34_2_DB10.txt
WB34_1_SLA8.txt
WB34_2_SLA8.txt
77_1_SLA8.txt... (1 Reply)
I have number in file which contains date and serial number:
2013101000.
The last two digits are serial number (00). So maximum of serial number is 100.
After reaching 100 it becomes 00 with incrementing 10 which is day with max 31.
after reaching 31 it becomes 00 and increments 10... (31 Replies)
Hi all:
I have a 5-column tab-separated file.
The only thing that I want to do with it is to split it.
However, I want to split it with a 80/20 proportion -- randomized, if possible.
I know that something like :
awk '{print $0 ""> "file" NR}' RS='' input-file
will work, but it only... (6 Replies)
I Am connected to Whatbox.ca Seed Box Via SSH!!
i have a file named avicii.mp3. I Want to split it into two rar parts as Apart1.rar and Apart2.rar So That When i Download Both the parts to My PC And Extract Them They Come out As Whole Avicii.mp3.
There is also one more problem!! When I Rar A... (18 Replies)
Hi, I have a file like this:
aaa 123
aaa 223
aaa 225
bbb 332
bbb 423
bbb 6755
bbb 324
ccc 112
ccc 234
ccc 897
Which I need to split into several files, something like
split -l 3
but the way that the lines with the same names would only go into one file: (7 Replies)
We have a folder XYZ with large number of files (>350,000). how can i split the folder and create say 10 of them XYZ1 to XYZ10 with 35,000 files each. (doesnt matter which files go where). (12 Replies)
I need help to write shell script to copy files from one server to another server.
Source Directory UAE(inside i have another folder Misc with files inside UAE folder).I have to copy this to another server UAE folder( Files should be copied to UAE folder and Misc files should be copied in target... (3 Replies)
Hi Team,
I am new to shell script and there is a requirement where files should be moved from Subfolder to parent folder.
Eg:
parent folder --> /Interface/data/test/IN
Sub folder -->/Interface/data/test/IN/Invoice20180607233338
Subfolder will be always with timestamp... (6 Replies)
Discussion started by: srivarun15
6 Replies
LEARN ABOUT DEBIAN
flow-split
flow-split(1) General Commands Manual flow-split(1)NAME
flow-split -- Split flow files into smaller files.
SYNOPSIS
flow-split [-gGhn] [-b big|little] [-C comment] [-d debug_level] [-N nflows] [-o outfile_basename] [-T nseconds] [-z z_level]
DESCRIPTION
The flow-split utility will split a flow file into smaller files based on the the number of flows or the ammount of time that has passed.
OPTIONS -b big|little
Byte order of output.
-C Comment
Add a comment.
-d debug_level
Enable debugging.
-g Split on source tag.
-G Split on destination tag.
-h Display help.
-n Use symbols for tag field in filename.
-N nflows Split after processing nflows.
-o outfile_basename
The basename of the resulting files.
-T nsecond
Split after processing an interval of nseconds flows.
-z z_level
Configure compression level to z_level. 0 is disabled (no compression), 9 is highest compression.
EXAMPLES
Create 1 minute flow files from the flow archive in /flows/krc4. Store the results in /flows/krc4.split
flow-cat /flows/krc4 | flow-split -T60 -o /flows/krc4.split/1min.
BUGS
None known.
AUTHOR
Mark Fullmer maf@splintered.net
SEE ALSO flow-tools(1)flow-split(1)