Split file into multiple files using awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split file into multiple files using awk
# 1  
Old 12-07-2016
Split file into multiple files using awk

I have following file:

Code:
FHEAD0000000001RTLG20161205110959201612055019
THEAD......
TCUST.....
TITEM....
TTEND...
TTAIL...
THEAD......
TCUST.....
TITEM....
TITEM.....
TTEND...
TTAIL...
FTAIL<number of lines in file- 10 digits;prefix 0><number of lines in file-2 - 10 digits- perfix 0> Eg: FTAIL00000006420000000640

I need to split the file into multiple files such that
  1. Each file should have 1st record FHEAD record (fixed)
  2. Each file should have records starting from THEAD to TTAIL
  3. Each file should have last record FTAIL with number of lines as mentioned above <number of lines in file- 10 digits;prefix 0><number of lines in file-2 - 10 digits- perfix 0> For example: FTAIL00000006420000000640
  4. file name should configurable

Expected Output

File 1

Code:
FHEAD0000000001RTLG20161205110959201612055019
THEAD......
TCUST.....
TITEM....
TTEND...
TTAIL
FTAIL000000000700000000005

File 2

Code:
FHEAD0000000001RTLG20161205110959201612055019
THEAD......
TCUST.....
TITEM....
TITEM....
TTEND...
TTAIL...
FTAIL000000000800000000006


Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by rbatte1; 12-07-2016 at 07:59 AM.. Reason: RudiC Added CODE tags. rbatte1 converted text numbered list into formatted numbered list
# 2  
Old 12-07-2016
Welcome to the forum.

This is a widespread problem. Ans attempts/ideas/thoughts from your side? Did you search these fora and/or look into the related threads at the bottom of this page, trying to adapt the solutions given?

What is "number of lines in file-" and "number of lines in file-2" ?
# 3  
Old 12-07-2016
Welcome amitdaf

Thanks for the question.

I have a few to questions pose in response first:-
  • Is this homework/assignment? There are specific forums for these.
  • What have you tried so far?
  • What output/errors do you get?
  • What OS and version are you using?
  • What are your preferred tools? (C, shell, perl, awk, etc.)
  • What logical process have you considered? (to help steer us to follow what you are trying to achieve)
Most importantly, What have you tried so far?

There are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.


We're all here to learn and getting the relevant information will help us all.


You could probably use csplit based on string THEAD but you would then need to insert the FHEAD record to each file you create then calculate & append the FTAIL record.


I hope that this helps, but have a try and show us where you get stuck.



Kind regards,
Robin
# 4  
Old 12-08-2016
Hello
I tried below code
Code:
awk '/^FHEAD/{h=$0} 
     /^THEAD/{close("RTLOG_test"f);f++}{print h >"RTLOG_test"f; print $0 >> "RTLOG_test"f}' RTLOG_5019_05122016110959.DAT

But it is not giving desired output

Number of lines in split file = number of lines in the file (wc -l)
Number of lines in split file -2 --> number if lines (wc -l minus 2)


Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 12-08-2016 at 05:41 AM.. Reason: Added CODE tags.
# 5  
Old 12-08-2016
Try
Code:
awk '
/^F/            {HD = $0
                 next
                }

/^THEAD/        {if (FN)        {printf "FTAIL%010d%010d" ORS, LN+2, LN > FN
                                 close (FN)
                                 LN = 0
                                }
                 FN = "RTLOG_test" ++f
                 print HD > FN
                }

                {print >> FN
                 LN++
                }

END             {printf "FTAIL%010d%010d" ORS, LN+2, LN > FN
                }
' file
cf RTLOG_test*
RTLOG_test1:
FHEAD0000000001RTLG20161205110959201612055019
THEAD......
TCUST.....
TITEM....
TTEND...
TTAIL...
FTAIL00000000070000000005
RTLOG_test2:
FHEAD0000000001RTLG20161205110959201612055019
THEAD......
TCUST.....
TITEM....
TITEM.....
TTEND...
TTAIL...
FTAIL00000000080000000006

This User Gave Thanks to RudiC For This Post:
# 6  
Old 12-09-2016
Hello
Thanks for the code..
In the last file, 2 FTAIL records are coming.

---------- Post updated 12-09-16 at 05:29 AM ---------- Previous update was 12-08-16 at 05:53 AM ----------

There are 2 FTAIL records in the last file.
1) FTAIL of original file
2) FTAIL computed from awk

Can we remove the original file FTAIL while printing into the file?

Last edited by amitdaf; 12-09-2016 at 06:38 AM..
# 7  
Old 12-09-2016
The original FTAIL record should be ignored UNLESS the input file's structure is different from what you posted. Is it?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to split file using multiple deliminators

I am trying to use awk to split a input file using multiple delimiters :-|. The input file is just one field and the output is 6 tab-delimited fields. The awk below does run and works as expected until I add the third delimiter |, which gives the current output below. I am not sure what is... (6 Replies)
Discussion started by: cmccabe
6 Replies

2. Shell Programming and Scripting

Split a big file into multiple files using awk

this thread is a continuation from previous thread https://www.unix.com/shell-programming-and-scripting/223901-split-big-file-into-multiple-files-based-first-four-characters.html ..I am using awk to split file and I have a syntax error while executing the below code I am using AIX 7.2... (4 Replies)
Discussion started by: etldev
4 Replies

3. Shell Programming and Scripting

awk script to split file into multiple files based on many columns

So I have a space delimited file that I'd like to split into multiple files based on multiple column values. This is what my data looks like 1bc9A02 1 10 1000 FTDLNLVQALRQFLWSFRLPGEAQKIDRMMEAFAQRYCQCNNGVFQSTDTCYVLSFAIIMLNTSLHNPNVKDKPTVERFIAMNRGINDGGDLPEELLRNLYESIKNEPFKIPELEHHHHHH 1ku1A02 1 10... (9 Replies)
Discussion started by: viored
9 Replies

4. Shell Programming and Scripting

How to split file into multiple files using awk based on 1 field in the file?

Good day all I need some helps, say that I have data like below, each field separated by a tab DATE NAME ADDRESS 15/7/2012 LX a.b.c 15/7/2012 LX1 a.b.c 16/7/2012 AB a.b.c 16/7/2012 AB2 a.b.c 15/7/2012 LX2 a.b.c... (2 Replies)
Discussion started by: alexyyw
2 Replies

5. Shell Programming and Scripting

split file into multiple files

Hi, I have a file of the following syntax that has around 120K records that are tab separated. input.txt abc def klm 20 76 . + . klm_mango unix_00000001; abc def klm 83 84 . + . klm_mango unix_0000103; abc def klm 415 439 . + . klm_mango unix_00001043; I am looking for an awk oneliner... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

6. Shell Programming and Scripting

Split file into multiple files

Hi I have a file that has multiple sequences; the sequence name is the line starting with '>'. It looks like below: infile.txt: >HE_ER tttggtgccttgactcggattgggggacctcccttgggagatcaatcccctgtcctcctgctctttgctc cgtgaaaaggatccacctatgacctctagtcctcagacccaccagcccaaggaacatctcaccaatttca >M7B_Ho_sap... (2 Replies)
Discussion started by: jdhahbi
2 Replies

7. Shell Programming and Scripting

Split line to multiple files Awk/Sed/Shell Script help

Hi, I need help to split lines from a file into multiple files. my input look like this: 13 23 45 45 6 7 33 44 55 66 7 13 34 5 6 7 87 45 7 8 8 9 13 44 55 66 77 8 44 66 88 99 6 I want to split every 3 lines from this file to be written to individual files. (3 Replies)
Discussion started by: saint2006
3 Replies

8. Shell Programming and Scripting

Split a file into multiple files

Hi, i have a file like this: 1|2|3|4|5| 1|2|8|4|6| Trailer1||||| 1|2|3| Trailer2||| 3|4|5|6| 3|4|5|7| 3|4|5|8| Trailer2||| I want to generate 3 files out of this based on the trailer record. Trailer record string can be different for each file or it may be same for one or two. No... (24 Replies)
Discussion started by: pparthji
24 Replies

9. UNIX for Dummies Questions & Answers

split a file into multiple files

Hi All, I have a file ABC.txt and I need to split this file on every 250 rows. And the file name should be ABC1.txt , ABC2.txt and so on. I tried with split command split -l 250 <filename> '<filename>' but the file name returned was ABC.txtaa ABC.txtab. Please... (8 Replies)
Discussion started by: kumar66
8 Replies

10. Shell Programming and Scripting

Split a file into multiple files

I have a file ehich has multiple create statements as create abc 123 one two create xyz 456 four five create nnn 666 six four I want to separte each create statement in seperate files (3 Replies)
Discussion started by: glamo_2312
3 Replies
Login or Register to Ask a Question