Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Split file based on number of blank lines
Top Forums UNIX for Dummies Questions & Answers Split file based on number of blank lines Post 302895654 by DGPickett on Tuesday 1st of April 2014 03:36:24 PM
Old 04-01-2014
For high performance, you need a language that can seek to half way and then search forward or back for 5 linefeeds (assuming no DOS cr or MAC cr only lines). As I recall, fcntl() can truncate the original file to size after the new file is successfully written, if you like the efficiency of less copying. Using mmap() makes both examining and writing simpler, as the file becomes a char[] in VM you can walk through and write() the tail of in one write(). You only need to mmap the top half if the first half is to be the bigger, and with mmap(), no seek()!

Did it have to be half by items, by lines or by byte count?

---------- Post updated at 03:36 PM ---------- Previous update was at 03:27 PM ----------

Taking a completely different tack, one could split it in sed or awk or such by writing alternate items to two different files. Is the exactly 4 blank lines reliable? Are they there at EOF?
Last edited by DGPickett; 04-01-2014 at 04:35 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How can I get an if statement to execute based on number of lines in a file?

I need to have an if statement in a script to run if there are certain processes running. Easiest way I can see to do this is to run a ps and grep the results based on what I am looking for: $ ps -ef | grep wtrs --- webtrend 5046 1 0 May 12 ? 0:28 /webtrends/versions/6.1/wtrs_ui... (6 Replies)
Discussion started by: LordJezo
6 Replies

2. Shell Programming and Scripting

Split File Based on Line Number Pattern

Hello all. Sorry, I know this question is similar to many others, but I just can seem to put together exactly what I need. My file is tab delimitted and contains approximately 1 million rows. I would like to send lines 1,4,& 7 to a file. Lines 2, 5, & 8 to a second file. Lines 3, 6, & 9 to... (11 Replies)
Discussion started by: shankster
11 Replies

3. Shell Programming and Scripting

Scripting help: Split a file into equal number of lines.

Experts, I have a file datafile.txt that consists of 1732 Line, I want to split the file into equal number of lines with 10 file. (The last file can have 2 line extra to match 1732) Please advise how to do that, Thanks in advance.. (2 Replies)
Discussion started by: rveri
2 Replies

4. Shell Programming and Scripting

Split single file into multiple files based on the number in the column

Dear All, I would like to split a file of the following format into multiple files based on the number in the 6th column (numbers 1, 2, 3...): ATOM 1 N GLY A 1 -3.198 27.537 -5.958 1.00 0.00 N ATOM 2 CA GLY A 1 -2.199 28.399 -6.617 1.00 0.00 ... (3 Replies)
Discussion started by: tomasl
3 Replies

5. Shell Programming and Scripting

Script to split files based on number of lines

I am getting a few gzip files into a folder by doing ftp to another server. Once I get them I move them to another location .But before that I need to make sure each gzip is not more than 5000 lines and split it up . The files I get are anywhere from 500 lines to 10000 lines in them and is in gzip... (4 Replies)
Discussion started by: gubbu
4 Replies

6. Shell Programming and Scripting

split a csv file into specified number of files (not lines)

hi, i really need it ...it's not simple to explain but as it's part of a crontab i can't split the file manually...and the file can change every day so the lines are not a good base. example: how to split 1 csv file in 15 files? thank you very much regards :b: (4 Replies)
Discussion started by: 7stars
4 Replies

7. UNIX for Dummies Questions & Answers

Command to split the files based on the number of lines in it

Hello Friends, Can anyone help me for the below requirement. I am having a file called Input.txt. My requirement is first check the count that is wc -l input.txt If the result of the wc -l Input.txt is less than 10 then don't split the Input.txt file. Where as if Input.txt >= 10 the split... (12 Replies)
Discussion started by: malaya kumar
12 Replies

8. Shell Programming and Scripting

How to split a file based on pattern line number?

Hi i have requirement like below M <form_name> sdasadasdMklkM D ...... D ..... M form_name> sdasadasdMklkM D ...... D ..... D ...... D ..... M form_name> sdasadasdMklkM D ...... M form_name> sdasadasdMklkM i want split file based on line number by finding... (10 Replies)
Discussion started by: bhaskar v
10 Replies

9. Shell Programming and Scripting

Split File based on number of rows

Hi I have a requirement, where i will receive multiple files in a folder (say: /fol1/fol2/). There will be at least 14 to 16 files. The size of the files will different, some may be 80GB or 90GB, some may be less than 5 GB (and the size of the files are very unpredictable). But the names of the... (10 Replies)
Discussion started by: kpk_ds
10 Replies

10. UNIX for Beginners Questions & Answers

Split file into multiple files based on empty lines

I am using below code to split files based on blank lines but it does not work. awk 'BEGIN{i=0}{RS="";}{x="F"++i;}{print > x;}' Your help would be highly appreciated find attachment of sample.txt file (2 Replies)
Discussion started by: imranrasheedamu
2 Replies

LEARN ABOUT HPUX

fmt

fmt(1)							      General Commands Manual							    fmt(1)

NAME

       fmt - format text

SYNOPSIS

       width] [file...]

DESCRIPTION

       The  command is a simple text formatter that fills and joins lines to produce output lines of (up to) the number of characters specified in
       the width option.  The default width is 72.  concatenates the arguments.  If none are given, formats text from the standard input.

       Blank lines are preserved in the output, as is the spacing between words.  does not fill lines beginning with a	period	for  compatibility
       with Nor does it fill lines starting with

       Indentation is preserved in the output and input lines with differing indentation are not joined (unless is used).

       can also be used as an in-line text filter for the command:

       reformats the text between the cursor location and the end of the paragraph.

   Options
       recognizes the following options:

	      Crown margin mode.
		    Preserve the indentation of the first two lines within a paragraph and align the left margin of each subsequent line with that
		    of the second line.  This is useful for tagged paragraphs.

	      Split lines only.
		    Do not join short lines to form longer ones.  This prevents sample lines of code, and other such "formatted" text, from  being
		    unduly combined.

	      Fill output lines to up to
		    width columns.

WARNINGS

       The width option is acceptable for BSD compatibility, but it may go away in future releases.

SEE ALSO

       nroff(1), vi(1).

																	    fmt(1)
All times are GMT -4. The time now is 02:57 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy