Split a file based on encountering header


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split a file based on encountering header
# 1  
Old 08-02-2016
Split a file based on encountering header

I need to split a file based on headers found

Input file
file1 content:

Code:
ADD
john
mickey

DROP
matt
sam

output of file F1
Code:
john
mickey

output of file F2
Code:
matt
sam

I tried this:

code:

Code:
awk '/ADD/{x="F"++i;}{print > x;}' file1

but this is writing all the records to a single file

Last edited by Scrutinizer; 08-02-2016 at 08:23 AM.. Reason: CODE tags
# 2  
Old 08-02-2016
Try
Code:
awk '$1=="ADD"{f="F1"; next} $1=="DROP"{f="F2"; next} NF{print >f}' file1

or for example:
Code:
awk '/^(ADD|DROP)/{suffix=$1; next} NF{print >(FILENAME "." suffix)}' file1

The use of the NF condition, ensures that empty lines are discarded..
# 3  
Old 08-02-2016
Hi.

Using upper-case letters as the pattern, consider items 2 and 7 from list below:
Code:
Split a text file into pieces, i.e. groups of lines:

        1) split, standard, but many variants exist in systems
           split a file into pieces

        2) csplit, standard
           split a file into sections determined by context lines

        3) r3split (local, Ratio, iRregular, Random,
           specify numbers in groups, percentage, random set of lines)

        4) pasplit (local, approximate, fast parallel-process split)

        5) mmsplit (local, multi-method, by number of lines, groups, pattern))

        6) scatter (local, regular number, like dealing a deck of playing cards)

        7) ppt-split (local, csplit with perl patterns, Perl Power Tools)
           https://github.com/briandfoy/PerlPowerTools/blob/master/bin/split 
           2016.08.02

        8) split_at_colchange (missing-text/local, add empty line when column changes)
        http://www1.cuni.cz/~obo/textutils/

        9) gate (local, Group And Transfer Entries, into files by field pattern)

        10) fsplit (local, split Fortran-77 into files by module name)

        12) mfs (local, split Fortran routines into files by name, handles "module")

        13) xsplit (local, explicit list of line numbers to files)

        14) nsplit (local, into "s" parts - bytes, not lines)

        15) bsplit (3rd/local, binary file with dd, originally named fsplit)

        16) f90split (3rd/local, split Fortran-90 source)

Briefly:
Code:
...
pl " Results, csplit, second file:"
csplit --quiet --prefix=f -z $FILE '/^[A-Z]/' '{*}'
head f01

pl " Results, ppt-split, first file:"
ppt-split -p '^[A-Z]' $FILE
head x.aab

produces:
Code:
-----
 Results, csplit, second file:
DROP
matt
sam

-----
 Results, ppt-split, first file:
ADD
john
mickey

Best wishes ... cheers, drl
# 4  
Old 08-02-2016
Assuming headers are always uppercase:-
Code:
LC_ALL=C awk '/[A-Z]/{F="file."$1;next}NF{print > F}' file1

or
Code:
LC_ALL=C awk '/[A-Z]/{F="F"++i;next}NF{print > F}' file

# 5  
Old 08-02-2016
Quote:
Originally Posted by Scrutinizer
Try
Code:
awk '$1=="ADD"{f="F1"; next} $1=="DROP"{f="F2"; next} NF{print >f}' file1

Hello Scrutinizer,

Not sure if that could be really a case with OP's Input_file but in case OP's Input_file has few lines before either ADD or DROP strings then value of variable f will not be set and it may give following error.
Code:
awk '$1=="ADD"{f="F1"; next} $1=="DROP"{f="F2"; next} NF{print >f}'   Input_file
awk: (FILENAME=Input_file FNR=1) fatal: expression for `>' redirection has null string value

Where I have used Input_file as follows.
Code:
cfcgshg
hjgftyv
ADD
john
mickey

DROP
matt
sam

So I think we could add a small check condition for variable's value existance to as follows for safer side.
Code:
awk '$1=="ADD"{f="F1"; next} $1=="DROP"{f="F2"; next} NF && f{print >f}'   Input_file


Hello Diddy,

Could you please try following too and let me know if this helps you.
Code:
awk '{f=$1=="ADD"?"F1":($1=="DROP"?"F2":f)} NF && f && $1 != "DROP" && $1 != "ADD"{print > f}'   Input_file

Thanks,
R. Singh
# 6  
Old 08-02-2016
or BEGIN {f = "F0"}
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split large xml into mutiple files and with header and footer in file

Split large xml into mutiple files and with header and footer in file tried below it splits unevenly and also i need help in adding header and footer command : csplit -s -k -f my_XML_split.xml extrfile.xml "/<Document>/" {1} sample xml <?xml version="1.0" encoding="UTF-8"?><Recipient>... (36 Replies)
Discussion started by: karthik
36 Replies

2. Shell Programming and Scripting

Find columns in a file based on header and print to new file

Hello, I have to fish out some specific columns from a file based on the header value. I have the list of columns I need in a different file. I thought I could read in the list of headers I need, # file with header names of required columns in required order headers_file=$2 # read contents... (11 Replies)
Discussion started by: LMHmedchem
11 Replies

3. Shell Programming and Scripting

Split file by column value, each with header

Hello all, I have a csv with with different testcase values in column 5. year,min,max,Instrument,Testcase 2016,201,1003,GEOTROPH-02116,TATA7980 2016,53,1011,GEOTROPH-01963,TATA7980 2016,3,1024,GEOTROPH-02067,TATA7980 2016,203,1027,GEOTROPH-02011,TATA7980... (16 Replies)
Discussion started by: senhia83
16 Replies

4. Shell Programming and Scripting

Sort and Split file with header and custom name

Hi, I am using SUN SOLARIS (SunOS sun4v sparc SUNW, T5240). I have a huge data file with header and trailer. This file gets used into an ETL process. ETL skips the header record (which is the first record of the file) and loads the rest of the record. The file can be delimited (comma,... (5 Replies)
Discussion started by: Saanvi1
5 Replies

5. Shell Programming and Scripting

Split and add header and trailer from input file

I need to split the file based on pattern from position 34-37 while retaining the header and trailer records in each individual split file Also is it possible to output the TOM and PAT records in the same output file ? I need the output file names same as xyz_pattern_Datetimestamp.txt ... (23 Replies)
Discussion started by: techedipro
23 Replies

6. Shell Programming and Scripting

substitution of fields based on header from another file

I have two files File1 with position (chromosome:start) and individuals as header, where pos is something like 1:2000 and every individual has a value. I have many columns and rows, here an example (tab separated): pos ind1 ind2 ind3 indn... 1:2000 0 0.1 0.1 1 1:2500 0.99 0.2 0.1 0.2 2:1000... (2 Replies)
Discussion started by: kuin
2 Replies

7. Shell Programming and Scripting

Remove the file content based on the Header of the file

Hi All, I want to remove the content based on the header information . Please find the example below. File1.txt Name|Last|First|Location|DepId|Depname|DepLoc naga|rr|tion|hyd|1|wer|opr Nava|ra|tin|gen|2|wera|opra I have to search for the DepId and remove the data from the... (5 Replies)
Discussion started by: i150371485
5 Replies

8. UNIX for Dummies Questions & Answers

Changing file content based on file header

Hi, I have several text files each containing some data as shown below: File1.txt >DataHeader Data... Data... File2.txt >DataHeader Data... Data... etc. What I want is to change the 'DataHeader' based on the file name. So the output should look like: File1.txt >File1 ... (1 Reply)
Discussion started by: Fahmida
1 Replies

9. Shell Programming and Scripting

Split large file and add header and footer to each small files

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file? (7 Replies)
Discussion started by: ashish4422
7 Replies

10. Shell Programming and Scripting

Split large file and add header and footer to each file

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file? (1 Reply)
Discussion started by: ashish4422
1 Replies
Login or Register to Ask a Question