Split file based on number of blank lines


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Split file based on number of blank lines
# 1  
Old 04-01-2014
Split file based on number of blank lines

Hello All ,

I have a file which needs to split based on the blank lines

Code:
Name ABC
Address London
Age 32
(4 blank new line)



Name DEF
Address London
Age 30
(4 blank new line)



Name DEF
Address London
Age 30
(4 blank new line)



Name DEF
Address London
Age 30
(4 blank new line)

I need to split the file into two different file ..
the separator is the 4 blank lines however these blank lines should be include in the splitted file

Please let me know how to achieve this in Unix

Thanks
Pratik
# 2  
Old 04-01-2014
Into 2 files, but what is supposed to be in each?
# 3  
Old 04-01-2014
An awk approach:
Code:
awk '
        NF {
                s = s ? s RS $0 : $0
        }
        !(NF) && s {
                A[s]
                s = ""
        }
        END {
                for ( k in A )
                {
                        F = "file" ++c
                        print k > F
                        close (F)
                }
        }
' file

This program will create two output files: file1 & file2 for posted input data.
# 4  
Old 04-01-2014
The script suppose to split the main file such a way every file have two records

---------- Post updated at 10:57 AM ---------- Previous update was at 10:46 AM ----------

Thanks Yoda,could you please explain the solution ?
# 5  
Old 04-01-2014
Quote:
Originally Posted by Pratik4891
Thanks Yoda,could you please explain the solution ?
Sure, for each non-zero number of field records, the program construct a variable s with value separated by RS (newline by default)
Code:
s = s ? s RS $0 : $0

Once a zero number of field record is encountered and s value is not null, it stores the record as key into an associative array: A

Finally in the END rule, the program writes each key in the array to a file.
This User Gave Thanks to Yoda For This Post:
# 6  
Old 04-01-2014
For high performance, you need a language that can seek to half way and then search forward or back for 5 linefeeds (assuming no DOS cr or MAC cr only lines). As I recall, fcntl() can truncate the original file to size after the new file is successfully written, if you like the efficiency of less copying. Using mmap() makes both examining and writing simpler, as the file becomes a char[] in VM you can walk through and write() the tail of in one write(). You only need to mmap the top half if the first half is to be the bigger, and with mmap(), no seek()!

Did it have to be half by items, by lines or by byte count?

---------- Post updated at 03:36 PM ---------- Previous update was at 03:27 PM ----------

Taking a completely different tack, one could split it in sed or awk or such by writing alternate items to two different files. Is the exactly 4 blank lines reliable? Are they there at EOF?

Last edited by DGPickett; 04-01-2014 at 04:35 PM..
# 7  
Old 04-01-2014
try also:
Code:
awk '!a[$0]++ {of="file" ++fc; print $0 >> of ; close(of)}' RS= ORS="\n\n\n\n" file

This User Gave Thanks to rdrtx1 For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Split file into multiple files based on empty lines

I am using below code to split files based on blank lines but it does not work. awk 'BEGIN{i=0}{RS="";}{x="F"++i;}{print > x;}' Your help would be highly appreciated find attachment of sample.txt file (2 Replies)
Discussion started by: imranrasheedamu
2 Replies

2. Shell Programming and Scripting

Split File based on number of rows

Hi I have a requirement, where i will receive multiple files in a folder (say: /fol1/fol2/). There will be at least 14 to 16 files. The size of the files will different, some may be 80GB or 90GB, some may be less than 5 GB (and the size of the files are very unpredictable). But the names of the... (10 Replies)
Discussion started by: kpk_ds
10 Replies

3. Shell Programming and Scripting

How to split a file based on pattern line number?

Hi i have requirement like below M <form_name> sdasadasdMklkM D ...... D ..... M form_name> sdasadasdMklkM D ...... D ..... D ...... D ..... M form_name> sdasadasdMklkM D ...... M form_name> sdasadasdMklkM i want split file based on line number by finding... (10 Replies)
Discussion started by: bhaskar v
10 Replies

4. UNIX for Dummies Questions & Answers

Command to split the files based on the number of lines in it

Hello Friends, Can anyone help me for the below requirement. I am having a file called Input.txt. My requirement is first check the count that is wc -l input.txt If the result of the wc -l Input.txt is less than 10 then don't split the Input.txt file. Where as if Input.txt >= 10 the split... (12 Replies)
Discussion started by: malaya kumar
12 Replies

5. Shell Programming and Scripting

split a csv file into specified number of files (not lines)

hi, i really need it ...it's not simple to explain but as it's part of a crontab i can't split the file manually...and the file can change every day so the lines are not a good base. example: how to split 1 csv file in 15 files? thank you very much regards :b: (4 Replies)
Discussion started by: 7stars
4 Replies

6. Shell Programming and Scripting

Script to split files based on number of lines

I am getting a few gzip files into a folder by doing ftp to another server. Once I get them I move them to another location .But before that I need to make sure each gzip is not more than 5000 lines and split it up . The files I get are anywhere from 500 lines to 10000 lines in them and is in gzip... (4 Replies)
Discussion started by: gubbu
4 Replies

7. Shell Programming and Scripting

Split single file into multiple files based on the number in the column

Dear All, I would like to split a file of the following format into multiple files based on the number in the 6th column (numbers 1, 2, 3...): ATOM 1 N GLY A 1 -3.198 27.537 -5.958 1.00 0.00 N ATOM 2 CA GLY A 1 -2.199 28.399 -6.617 1.00 0.00 ... (3 Replies)
Discussion started by: tomasl
3 Replies

8. Shell Programming and Scripting

Scripting help: Split a file into equal number of lines.

Experts, I have a file datafile.txt that consists of 1732 Line, I want to split the file into equal number of lines with 10 file. (The last file can have 2 line extra to match 1732) Please advise how to do that, Thanks in advance.. (2 Replies)
Discussion started by: rveri
2 Replies

9. Shell Programming and Scripting

Split File Based on Line Number Pattern

Hello all. Sorry, I know this question is similar to many others, but I just can seem to put together exactly what I need. My file is tab delimitted and contains approximately 1 million rows. I would like to send lines 1,4,& 7 to a file. Lines 2, 5, & 8 to a second file. Lines 3, 6, & 9 to... (11 Replies)
Discussion started by: shankster
11 Replies

10. Shell Programming and Scripting

How can I get an if statement to execute based on number of lines in a file?

I need to have an if statement in a script to run if there are certain processes running. Easiest way I can see to do this is to run a ps and grep the results based on what I am looking for: $ ps -ef | grep wtrs --- webtrend 5046 1 0 May 12 ? 0:28 /webtrends/versions/6.1/wtrs_ui... (6 Replies)
Discussion started by: LordJezo
6 Replies
Login or Register to Ask a Question