Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk file split

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk file split
# 1  
Old 01-24-2014
awk file split
Hi all,

First of all I' like to mention that I'm pretty new to unix scripting. Smilie

I'm trying to split an large xml with awk and rename it based on the values of two attributes.
Example XML
Code:
<RECORD>
<element1>11</element1>
<element2>22</element2>
<element3>33</element3>
<element4>a</element4>
<element5>b</element5>
<element6>c</element6>
</RECORD>
<RECORD>
<element1>44</element1>
<element2>55</element2>
<element3>66</element3>
<element4>a</element4>
<element5>b</element5>
<element6>c</element6>
</RECORD>

The desired output would be :

Code:
a file named 1122.xml
<RECORD>
<element1>11</element1>
<element2>22</element2>
<element3>33</element3>
</RECORD>

a file named 4455.xml
<RECORD>
<element1>44</element1>
<element2>55</element2>
<element3>66</element3>
</RECORD>

Up till know I have been able to split the file using this command
Code:
awk '/<RECORD/{close("row"count".xml");count++}count{f="row"count".xml";print $0 > f}' file.xml

but couldn't figure how to pass the values of the elements 1 & 2 in the filename. Ideally I would like to add xml tags as this is intended to split xml files and produce valid xml format output.

Any ideas on how to do it with awk ?
Thank you in advance.

PS. I know that it has been addressed in a couple of posts, such as shell-210529-xml-split-extract-string-between-chars.html
though I could use the proposed solution, it always produce me only one file with the final record.Smilie

again, thanks
# 2  
Old 01-24-2014
Hello,

Could you please use the following code for same.

Code:
awk '!/\<RECORD\>/ && !/\<\/RECORD\>/ {f=f"\n"$0} /\<\/RECORD\>/ {print f > "Record"++i".txt";f=""}' check_records

It will create 2 files named Record2.txt and Record1.txt.


NOTE: Where check_records is the Input file name.

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
f0usk4s
# 3  
Old 01-24-2014
Hello RavinderSingh13,

Thank you for your quick reply.
Unfortunately, this produces pretty much the same output with the code I posted earlier Smilie, n files where the file will have a counter in the filename as output.
However, I was trying to get the value of two tags in each filename Smilie

Thanks again
# 4  
Old 01-24-2014
Give this a try:
Code:
awk '/<\/RECORD/{
                 print raw""$0 > name".xml"
                 name=""
                 raw=""
                 next}
    /element[1-2]/{name=sprintf("%s%s",name,$3)}
    {raw=sprintf("%s%s\n",raw,$0)}' FS="(>)|(<)" file.xml

This User Gave Thanks to Klashxx For This Post:
f0usk4s
# 5  
Old 01-24-2014
Hello,

Could you please try following.

Code:
awk '!/\<RECORD\>/ && !/\<\/RECORD\>/ && !/\<element4\>/ && !/\<element5\>/ && !/\<element6\>/ {f=f"\n"$0} /\<\/RECORD\>/ {print f > "Record"++i".xml";f=""}' check_records

It will create 2 output files as follows Record2.txt and Record1.txt.

Output files will be as follows.

Code:
$ cat Record1.xml
<element1>11</element1>
<element2>22</element2>
<element3>33</element3>
$
 
 
$ cat Record2.xml
<element1>44</element1>
<element2>55</element2>
<element3>66</element3>
$



Thanks,
R. Singh
Last edited by RavinderSingh13; 01-24-2014 at 06:55 AM.. Reason: adding output files
This User Gave Thanks to RavinderSingh13 For This Post:
f0usk4s
# 6  
Old 01-24-2014
Hi Klashxx,

Unfortunately the size of the file is too large in order to be handled with sprintf
Error: awk: program limit exceeded: sprintf buffer size=2040

---------- Post updated at 12:05 PM ---------- Previous update was at 11:57 AM ----------

Quote:
Originally Posted by RavinderSingh13
Hello,

Could you please try following.

Code:
awk '!/\<RECORD\>/ && !/\<\/RECORD\>/ && !/\<element4\>/ && !/\<element5\>/ && !/\<element6\>/ {f=f"\n"$0} /\<\/RECORD\>/ {print f > "Record"++i".xml";f=""}' check_records

It will create 2 output files as follows Record2.txt and Record1.txt.

Output files will be as follows.

Code:
$ cat Record1.xml
<element1>11</element1>
<element2>22</element2>
<element3>33</element3>
$
 
 
$ cat Record2.xml
<element1>44</element1>
<element2>55</element2>
<element3>66</element3>
$



Thanks,
R. Singh
Hi RavinderSingh13,

Yes this creates the output you describe, however I would like to have the following output
Code:
$ 1122.xml
<element1>11</element1>
<element2>22</element2>
<element3>33</element3>
$
 
 
$ 4455.xml
<element1>44</element1>
<element2>55</element2>
<element3>66</element3>
$

without the string Record or a counter Smilie. Thanks again for your helpSmilie
# 7  
Old 01-24-2014
Could you please try followign and let me know.

Code:
awk -vi=1122 '!/\<RECORD\>/ && !/\<\/RECORD\>/ && !/\<element4\>/ && !/\<element5\>/ && !/\<element6\>/ {f=f"\n"$0} /\<\/RECORD\>/ {print f > i".xml" ;f="";i=4455}' check_records

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
f0usk4s
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Split file using awk

I need to split the incoming source file in to multiple files using awk. Split position is (6,13) : 8 positions All the records that are greater than 20170101 and less than or equal to 20181231 should go in a split file with file name as source... (11 Replies)
Discussion started by: rosebud123
11 Replies

2. Shell Programming and Scripting

Split File by Pattern with File Names in Source File... Awk?

Hi all, I'm pretty new to Shell scripting and I need some help to split a source text file into multiple files. The source has a row with pattern where the file needs to be split, and the pattern row also contains the file name of the destination for that specific piece. Here is an example: ... (2 Replies)
Discussion started by: cul8er
2 Replies

3. Shell Programming and Scripting

Split a file with awk

Hi! I have a file like this: a,b,c,12,d,e a,b,c,13,d,e a,b,c,14,d,e a,b,c,15,d,e a,b,c,16,d,e a,b,c,17,d,e I need to split that file in two: If field 4 is equal or higher than 14 that row goes to one file and if it is equal or higher than 15 to another. Can anyone point me in the... (2 Replies)
Discussion started by: Tr0cken
2 Replies

4. Shell Programming and Scripting

AWK File Split

Hi All, Input.txt XYZONEABC　　　　　　　　　　　　　　　　　 CZXTWOJJJ KKKSIXOOO asdfhajlsdhfajs asdfasfasdf Output Files: ONE.txt XYZONEABC　　　　　　　　　　　　　　　　 TWO.txt CZXTWOJJJ SIX.txt KKKSIXOOO I had a script (2 Replies)
Discussion started by: kmsekhar
2 Replies

5. Shell Programming and Scripting

How to split a file using AWK?

Hello, I have a file like the following: david,a,b,c,20,r thomas,a,b,c,30,r willaiam,a,b,c,80,r barbara,a,b,c,100,r I would like to split the file into other files using a condition for the contents of column 5. The condition should be a if the contents of column 5 is in a range... (4 Replies)
Discussion started by: keenboy100
4 Replies

6. Shell Programming and Scripting

split a file using awk

Hi , I just need to split a file and outputfiles are redirected to gzip file need: Input file - A.gz content of A.gz is 100|sfdds|dffdds|200112|sdfdf 100|sfdds|dffdds|200112|sdfdf 100|sfdds|dffdds|200112|sdfdf 100|sfdds|dffdds|200212|sdfdf 100|sfdds|dffdds|200212|sdfdf... (3 Replies)
Discussion started by: mohan_xunil
3 Replies

7. Shell Programming and Scripting

awk - split file

How can I split a text file (in awk) in n others with number of record given in input? Thanks (6 Replies)
Discussion started by: pinguc
6 Replies

8. Shell Programming and Scripting

split file with awk

I did a lot of search on this forum on spiting file; found a lot, but my requirement is a bit different, please guide. Master file: x:start:5 line1:23 line2:12 2:90 x:end:5 x:start:2 45:56 22:90 x:end:2 x:start:3 line1:23 line2:12 x:end:3 x:start:2 line5:23 (1 Reply)
Discussion started by: uwork72
1 Replies

9. UNIX for Dummies Questions & Answers

Split a file with no pattern -- Split, Csplit, Awk

I have gone through all the threads in the forum and tested out different things. I am trying to split a 3GB file into multiple files. Some files are even larger than this. For example: split -l 3000000 filename.txt This is very slow and it splits the file with 3 million records in each... (10 Replies)
Discussion started by: madhunk
10 Replies

10. Shell Programming and Scripting

Split file using awk

I am trying to read a file and split the file into multiple files. I need to create new files with different set of lines from the original file. ie, the first output file may contain 10 lines and the second 100 lines and so on. The criteria is to get the lines between two lines starting with some... (8 Replies)
Discussion started by: pvar
8 Replies
Login or Register to Ask a Question