Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Split a txt file on the basis of line number Post 303033622 by drl on Tuesday 9th of April 2019 09:02:56 AM
Old 04-09-2019
A meta-answer with a non-standard, local utility

Hi.

I think segment 41-50 is supposed to be skipped: 1-20 ,21-30 ,31-40 ,51-60 ,61-100

Our shop needed something like this long ago so we created a split utility, r3split, to handle these situations. When passing a file of lines f1 ... f100 through this command, we get:
Code:
 Results, lines in receiving files:
 Expecting first lines, 1(20), 21(10), 31(10), 51(10), 61-100
 Note - accepting partial completion on r05, exit.
Edges: 1:0:0 of 20 lines in file "r01"
f1
   ---

Edges: 1:0:0 of 10 lines in file "r02"
f21
   ---

Edges: 1:0:0 of 10 lines in file "r03"
f31
   ---

Edges: 1:0:0 of 10 lines in file "r04"
f51
   ---

Edges: 1:0:0 of 40 lines in file "r05"
f61
   ---

And this is the command:
Code:
r3split -c "20,10+2,-10,10," -p "r" $FILE

translating to: write to files beginning with "r", 20 lines, 10 lines repeated twice, skip 10 lines, everthing else to last receiving file.

Sadly we have not yet released any of our library of utilities.

The reason it is called r3split is for the 3 functions it performs. If anyone would like to use it as inspiration, here is the help page:
Code:
r3split - Split text file into Ratio, iRregular, or Random chunks.

usage: r3split [options] -- [files]

options:

Options which allow arguments require either a space or an "=" as
a separator, e.g. "-r 3" or "-r=3".

--help | -h
  print this message and quit.

--chunks | -c [-]c1,[-]c2 ... cn
  Transfer ci lines to next output file.  If preceded by "-" skip
  ci lines.  For convenience, the ci sequence may be the first
  parameter (or second after the --debug option).

  The chunks may be integers, ratios ( 1 < ci < 0 ), percentages
  (similar to ratios but 100 < ci < 0, followed by "%")  Ratios
  and percentages will require a preliminary (fast) pass over the
  file to find the length in lines (see "estimate:).  A count of
  zero will be treated as 1.

  If the symbol "+" occurs after the chunk, an integer repeat
  count may be specified.

  Usually the chunks are transferred in complete blocks, and
  remaining lines less than a chunk are ignored.  However if a
  final comma, ",", appears, then the incomplete chunk will be
  transferred to the final file. (Essentially a large number is
  created for the final chunk.)

  Ex: -c=1,1    --chunks 20%,.3,100     --chunks=-4,13,.75
  -c 3,3+       -c3,3+,

--random | -r n
  Transfer to n files, each file containing a random number of
  lines.  The total lines will equal the number of lines in the
  input file.  The NUMBER of lines is random here, the sequence
  is as it appears in the input.

--debug | -d
  Print debugging output.  Must be first option.
  
--suffix | -s format
  Suffix format, the default is "%02d".  To specify a ".txt" for
  each file use -s="%02d.txt".  The format is used to create the
  sequence of output filenames.

--prefix -p string
  Prefix, default is "xx", like many other split utilities.

--estimate | -e
  Allow the file length to be estimated, generally useful only
  for "long" files, perhaps 1 GB or more.  (This is accomplished
  by external routine "esmele" which might not be available on
  all systems.)

cheers, drl
This User Gave Thanks to drl For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split File Based on Line Number Pattern

Hello all. Sorry, I know this question is similar to many others, but I just can seem to put together exactly what I need. My file is tab delimitted and contains approximately 1 million rows. I would like to send lines 1,4,& 7 to a file. Lines 2, 5, & 8 to a second file. Lines 3, 6, & 9 to... (11 Replies)
Discussion started by: shankster
11 Replies

2. Shell Programming and Scripting

Split one file to Multiple file with report basis in unix

Hi, Please help on this. i want split the below file(11020111.CLT) to more files with some condition. :b: 1) %s stating of the report 2) %e ending of the report example starting of the report: %sAEGONCA| |MUMBAI | :EXPC|N|D ending of the report %eAEGONCA| |MUMBAI | :EXPC 3)so the... (10 Replies)
Discussion started by: krbala1985
10 Replies

3. Shell Programming and Scripting

KSH script for split a txt file

I have a problem which I would like to solve by using UNIX power and inspired minds around world. Here is the problem I have a text file and it has data as follows 1X.....................1234567890123456789T1234598765XT1 (header) 1Z01............(sub HEADER) P100001............ Q1........... (4 Replies)
Discussion started by: ask.chowhan
4 Replies

4. Shell Programming and Scripting

How to split this txt file into small files?

Dear shell experts, I would like to spilt a txt file into small ones. However, I did not know how to program use shell. If someone could help, it is greatly appreciated! Specifically, I supposed there is file named A.txt. The content of the file likes this: Subject run condtion ACC time... (3 Replies)
Discussion started by: psychmyluo
3 Replies

5. Shell Programming and Scripting

Need to append the date | abcddate.txt to the first line of my txt file

I want to add/append the info in the following format to my.txt file. 20130702|abcd20130702.txt FN|SN|DOB I tried the below script but it throws me some exceptions. <#!/bin/sh dt = date '+%y%m%d'members; echo $dt+|+members+$dt; /usr/bin/awk -f BEGIN { FS="|"; OFS="|"; } { print... (6 Replies)
Discussion started by: harik1982
6 Replies

6. UNIX for Dummies Questions & Answers

Splitting the file basis of Line Number

Can u pls advise the unix command as I have a file which contain the records in the below format 333434 435435 435443 434543 343536 Now the total line count is 89380 , now i want to create a separate I am trying to split my large big file into small bits using the line... (2 Replies)
Discussion started by: punpun66
2 Replies

7. Shell Programming and Scripting

How to split a file based on pattern line number?

Hi i have requirement like below M <form_name> sdasadasdMklkM D ...... D ..... M form_name> sdasadasdMklkM D ...... D ..... D ...... D ..... M form_name> sdasadasdMklkM D ...... M form_name> sdasadasdMklkM i want split file based on line number by finding... (10 Replies)
Discussion started by: bhaskar v
10 Replies

8. Shell Programming and Scripting

Splitting XML file on basis of line number into multiple file

Hi All, I have more than half million lines of XML file , wanted to split in four files in a such a way that top 7 lines should be present in each file on top and bottom line of should be present in each file at bottom. from the 8th line actual record starts and each record contains 15 lines... (14 Replies)
Discussion started by: ajju
14 Replies

9. UNIX for Dummies Questions & Answers

Split Every Line In Txt Into Separate Txt File, Named Same As The Line

Hi All Is there a way to export every line into new txt file where by the title of each txt output are same as the line ? I have this txt files containing names: Kandra Vanhooser Rhona Menefee Reynaldo Hutt Houston Rafferty Charmaine Lord Albertine Poucher Juana Maes Mitch Lobel... (2 Replies)
Discussion started by: Nexeu
2 Replies

10. UNIX for Beginners Questions & Answers

Split large file into 24 small files on one hour basis

I Have a large file with 24hrs log in the below format.i need to split the large file in to 24 small files on one hour based.i.e ex:from 09:55 to 10:55,10:55-11:55 can any one help me on this.! ... (20 Replies)
Discussion started by: Raghuram717
20 Replies
All times are GMT -4. The time now is 02:04 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy