I believe your number of lines which you need in output split files are NOT even. So I am coming up here with an approach where one could mention the line numbers when they want to generate a new output file. Let's say your own example posted one, you need first 20 lines in file1, then next 10 lines to file2, next 10 lines to file3, next 10 lines to file4 and then next 40 lines to file5. If this is the case then could you please try following(I have NOT tested it, I am cooking right now, I am pretty sure it should WORK ).
NOTE: Point to be noted here is split("20,10,10,10,40",array,",") code's part is responsible for mentioning number of lines in output file so please be sure you are providing proper values here.
NOTE2: Also I am closing the output files with close(file) statement to avoid errors like
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 2,288
Thanks Given: 430
Thanked 480 Times in 395 Posts
A meta-answer with a non-standard, local utility
Hi.
I think segment 41-50 is supposed to be skipped: 1-20 ,21-30 ,31-40 ,51-60 ,61-100
Our shop needed something like this long ago so we created a split utility, r3split, to handle these situations. When passing a file of lines f1 ... f100 through this command, we get:
Code:
Results, lines in receiving files:
Expecting first lines, 1(20), 21(10), 31(10), 51(10), 61-100
Note - accepting partial completion on r05, exit.
Edges: 1:0:0 of 20 lines in file "r01"
f1
---
Edges: 1:0:0 of 10 lines in file "r02"
f21
---
Edges: 1:0:0 of 10 lines in file "r03"
f31
---
Edges: 1:0:0 of 10 lines in file "r04"
f51
---
Edges: 1:0:0 of 40 lines in file "r05"
f61
---
And this is the command:
Code:
r3split -c "20,10+2,-10,10," -p "r" $FILE
translating to: write to files beginning with "r", 20 lines, 10 lines repeated twice, skip 10 lines, everthing else to last receiving file.
Sadly we have not yet released any of our library of utilities.
The reason it is called r3split is for the 3 functions it performs. If anyone would like to use it as inspiration, here is the help page:
Code:
r3split - Split text file into Ratio, iRregular, or Random chunks.
usage: r3split [options] -- [files]
options:
Options which allow arguments require either a space or an "=" as
a separator, e.g. "-r 3" or "-r=3".
--help | -h
print this message and quit.
--chunks | -c [-]c1,[-]c2 ... cn
Transfer ci lines to next output file. If preceded by "-" skip
ci lines. For convenience, the ci sequence may be the first
parameter (or second after the --debug option).
The chunks may be integers, ratios ( 1 < ci < 0 ), percentages
(similar to ratios but 100 < ci < 0, followed by "%") Ratios
and percentages will require a preliminary (fast) pass over the
file to find the length in lines (see "estimate:). A count of
zero will be treated as 1.
If the symbol "+" occurs after the chunk, an integer repeat
count may be specified.
Usually the chunks are transferred in complete blocks, and
remaining lines less than a chunk are ignored. However if a
final comma, ",", appears, then the incomplete chunk will be
transferred to the final file. (Essentially a large number is
created for the final chunk.)
Ex: -c=1,1 --chunks 20%,.3,100 --chunks=-4,13,.75
-c 3,3+ -c3,3+,
--random | -r n
Transfer to n files, each file containing a random number of
lines. The total lines will equal the number of lines in the
input file. The NUMBER of lines is random here, the sequence
is as it appears in the input.
--debug | -d
Print debugging output. Must be first option.
--suffix | -s format
Suffix format, the default is "%02d". To specify a ".txt" for
each file use -s="%02d.txt". The format is used to create the
sequence of output filenames.
--prefix -p string
Prefix, default is "xx", like many other split utilities.
--estimate | -e
Allow the file length to be estimated, generally useful only
for "long" files, perhaps 1 GB or more. (This is accomplished
by external routine "esmele" which might not be available on
all systems.)
I Have a large file with 24hrs log in the below format.i need to split the large file in to 24 small files on one hour based.i.e ex:from 09:55 to 10:55,10:55-11:55
can any one help me on this.!
... (20 Replies)
Hi All
Is there a way to export every line into new txt file where by the title of each txt output are same as the line ?
I have this txt files containing names:
Kandra Vanhooser
Rhona Menefee
Reynaldo Hutt
Houston Rafferty
Charmaine Lord
Albertine Poucher
Juana Maes
Mitch Lobel... (2 Replies)
Hi All,
I have more than half million lines of XML file , wanted to split in four files in a such a way that top 7 lines should be present in each file on top and bottom line of should be present in each file at bottom.
from the 8th line actual record starts and each record contains 15 lines... (14 Replies)
Hi
i have requirement like below
M <form_name> sdasadasdMklkM
D ......
D .....
M form_name> sdasadasdMklkM
D ......
D .....
D ......
D .....
M form_name> sdasadasdMklkM
D ......
M form_name> sdasadasdMklkM
i want split file based on line number by finding... (10 Replies)
Can u pls advise the unix command as I have a file which contain the records in the below format
333434
435435
435443
434543
343536
Now the total line count is 89380 , now i want to create a separate
I am trying to split my large big file into small bits using the line... (2 Replies)
I want to add/append the info in the following format to my.txt file.
20130702|abcd20130702.txt FN|SN|DOB
I tried the below script but it throws me some exceptions.
<#!/bin/sh
dt = date '+%y%m%d'members;
echo $dt+|+members+$dt;
/usr/bin/awk -f
BEGIN { FS="|"; OFS="|"; } { print... (6 Replies)
Dear shell experts,
I would like to spilt a txt file into small ones. However, I did not know how to program use shell. If someone could help, it is greatly appreciated!
Specifically, I supposed there is file named A.txt. The content of the file likes this:
Subject run condtion ACC time... (3 Replies)
I have a problem which I would like to solve by using UNIX power and inspired minds around world. Here is the problem
I have a text file and it has data as follows
1X.....................1234567890123456789T1234598765XT1 (header)
1Z01............(sub HEADER)
P100001............
Q1........... (4 Replies)
Hi, Please help on this. i want split the below file(11020111.CLT) to more files with some condition. :b:
1) %s stating of the report
2) %e ending of the report
example starting of the report:
%sAEGONCA| |MUMBAI | :EXPC|N|D
ending of the report
%eAEGONCA| |MUMBAI | :EXPC
3)so the... (10 Replies)
Hello all.
Sorry, I know this question is similar to many others, but I just can seem to put together exactly what I need.
My file is tab delimitted and contains approximately 1 million rows. I would like to send lines 1,4,& 7 to a file. Lines 2, 5, & 8 to a second file. Lines 3, 6, & 9 to... (11 Replies)