Splitting xml file into several xml files using perl


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Splitting xml file into several xml files using perl
# 1  
Old 07-30-2015
Splitting xml file into several xml files using perl

Hi Everyone,

I'm new here and I was checking this old post:
/shell-programming-and-scripting/180669-splitting-file-into-several-smaller-files-using-perl.html
(cannot paste link because of lack of points)

I need to do something like this but understand very little of perl.

I also check that maybe xml_split should do what I want but didn't really work, only split the first file and then crashed.


what I want is the following I have file X and need to split it into different files (file names are not relevant).

this would be my file content:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:pain.001.002.03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:iso:std:iso:20022:tech:xsd:pain.001.002.03"><CstmrCdtTrfInitn><GrpHdr><MsgId>2504009000100320100060</MsgId></GrpHdr><PmtInf><PmtInfId>2014-10-06-09-39-23-305460</PmtInfId></PmtInf></CstmrCdtTrfInitn></Document><?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:pain.001.002.03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:iso:std:iso:20022:tech:xsd:pain.001.002.03 pain.001.002.03.xsd"><CstmrCdtTrfInitn><GrpHdr><MsgId>WedDec0415:24:58CET2013</MsgId></GrpHdr></CstmrCdtTrfInitn></Document><?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:pain.001.003.03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:iso:std:iso:20022:tech:xsd:pain.001.003.03 pain.001.003.03.xsd"><CstmrCdtTrfInitn><GrpHdr><MsgId>MonOct0609:38:36CEST2014</MsgId></GrpHdr></CstmrCdtTrfInitn></Document>

what I expect at the end is to have something like this:
file1
Code:
<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:pain.001.002.03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:iso:std:iso:20022:tech:xsd:pain.001.002.03"><CstmrCdtTrfInitn><GrpHdr><MsgId>2504009000100320100060</MsgId></GrpHdr><PmtInf><PmtInfId>2014-10-06-09-39-23-305460</PmtInfId></PmtInf></CstmrCdtTrfInitn></Document>

file2
Code:
<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:pain.001.002.03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:iso:std:iso:20022:tech:xsd:pain.001.002.03 pain.001.002.03.xsd"><CstmrCdtTrfInitn><GrpHdr><MsgId>WedDec0415:24:58CET2013</MsgId></GrpHdr></CstmrCdtTrfInitn></Document>

file3
Code:
<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:pain.001.003.03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:iso:std:iso:20022:tech:xsd:pain.001.003.03 pain.001.003.03.xsd"><CstmrCdtTrfInitn><GrpHdr><MsgId>MonOct0609:38:36CEST2014</MsgId></GrpHdr></CstmrCdtTrfInitn></Document>

at the moment I have only just started my code (very basic)
Code:
#!/usr/bin/perl -w
use Data::Dumper qw(Dumper);

# (1) quit unless we have the correct number of command-line args
$num_args = $#ARGV + 1;
if ($num_args < 1) {
    print "\nUsage: split.pl filename\n";
    exit;
}
$filename=$ARGV[0];

open INPUT, "<$filename";
@lines = <INPUT>;
close INPUT;

my @parts = split /<?xml version/, @lines; 

print Dumper \@parts;

I was expecting to get maybe only the each line or something like that in my output but I'm getting the following:

Code:
[root@~]$ perl split.pl split.xml
$VAR1 = [
          '16'
        ];

my split.xml file has 16 lines so I guess the split is giving me that it found 16 occurrences but I don't see any split being done. did I forget any parameter?
# 2  
Old 07-30-2015
Code:
perldoc -f split

Splits the string EXPR into a list of strings and returns the list in list context, or the size of the list in scalar context.

In Perl is important to get familiar with the concept of context. You'll live or die by the scalar or list context.

split is for a string. You gave it a list.
Code:
my @parts = split /<?xml version/, @lines;

This snip has the essence of what you were trying to learn.
The result will show the reason why, ultimately, it will not split as you expect in order to have separated files.
Code:
#!/usr/bin/perl

use strict;
use warnings;
use Data::Dumper qw(Dumper);

sub usage {
    print "Usage: $_[0] <filename>\n";
    exit;
}
#
# $0 is always the name of the program
#
my $filename = shift or usage $0;

# Same:
# my $filename = shift @ARGV or usage $0;

open my $fh, '<', $filename or die "$!\n";
     # change the separator to empty and read
     # the whole file as a string
     local $/;
     my @content = split /<?xml version/, <$fh>;
close $fh;

print Dumper \@content;

This User Gave Thanks to Aia For This Post:
# 3  
Old 08-03-2015
hi, thanks for the feedback, I have now something to work with.

it looks going in the good direction the only thing I don't understand is why the output is has follows:

Code:
$VAR1 = [
          '<?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document><?',
          '="1.0" encoding="UTF-8"?>
<Document>...</Document>'
        ];

I probably need an escape character my string no?

because the first <? get's separated from the full thing.

ok I get the split fix, because of the escape character. (/<\?xml version/)

but how do I make the split give me back the matching part has part of the result line?
maybe is just easier if I store it in a variable and then place it later.

code:

Code:
#!/usr/bin/perl

use strict;
use warnings;
use Data::Dumper qw(Dumper);

sub usage {
    print "Usage: $_[0] <filename>\n";
    exit;
}
#
# $0 is always the name of the program
#
my $filename = shift or usage $0;

# Same:
# my $filename = shift @ARGV or usage $0;

open my $fh, '<', $filename or die "$!\n";
     # change the separator to empty and read
     # the whole file as a string
     local $/;
     my @content = split /<\?xml version/, <$fh>;
close $fh;

print Dumper \@content;


Last edited by mcosta; 08-04-2015 at 04:46 AM.. Reason: post first think later....
# 4  
Old 08-03-2015
You did not post your last modified code.
split will consume that part that matches as you found out, except if you use a lookahead or lookbehind. I use that technique on this code.


Code:
#!/usr/bin/perl

use strict;
use warnings;

sub usage {
    print "Usage: $_[0] <filename>\n";
    exit;
}

my $filename = shift or usage $0;

open my $fh, '<', $filename or die "$!\n";
     # read the file as one string
     local $/;
     # split when after it finds </Document>
     my @content = split /(?<=<\/Document>)/, <$fh>;
close $fh;

# dynamically create files with content
my $suffix = 0;
for my $block (@content) {
    if($block ne "\n") {
    open $fh, '>', "$filename." . ++$suffix or die "$!\n";
            print $fh "$block\n";
    close $fh
    }
}

This User Gave Thanks to Aia For This Post:
# 5  
Old 08-04-2015
thank you, this fixes my problem.

I also edited my previous post with the missing information.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Splitting the XML file and renaming the files

Hello Gurus, I have a requirement to split the xml file into different xml files. Can you please help me with that? Here is my Source XML file <jms-system-resource> <name>PS6SOAJMSModule</name> <target>soa_server1</target> <sub-deployment> ... (3 Replies)
Discussion started by: Siv51427882
3 Replies

2. Shell Programming and Scripting

Splitting the XML file into three different files

Hello Shell Guru's I have a requirement to split the source xml file into three different text file. And i need your valuable suggestion to finish this. Here is my source xml snippet, here i am using only one entry of <jms-system-resource>. There may be multiple entries in the source file. ... (5 Replies)
Discussion started by: Siv51427882
5 Replies

3. Shell Programming and Scripting

Splitting CSV into variables then to XML file

I have a text file that looks like this: FIELD1, FIELD2, THIS IS FIELD3, FIELD4 FIELD1, FIELD2, THIS IS FIELD3, FIELD4 FIELD1, FIELD2, THIS IS FIELD3, FIELD4 I need it to turn it into an XML file to run against a custom application. My ultimate goal is for it to look like... (15 Replies)
Discussion started by: jeffs42885
15 Replies

4. Shell Programming and Scripting

Splitting a single xml file into multiple xml files

Hi, I'm having a xml file with multiple xml header. so i want to split the file into multiple files. Sample.xml consists multiple headers so how can we split these multiple headers into multiple files in unix. eg : <?xml version="1.0" encoding="UTF-8"?> <ml:individual... (3 Replies)
Discussion started by: Narendra921631
3 Replies

5. Shell Programming and Scripting

XML Splitting into multi files

Hi , I have a XML file like below file name : sample.xml <?xml version="1.0"?> <catalog> <author>Rajini</author> <title>XML Guide</title> <Text> </Text> <genre>Computer</genre> <price>44.95</price> </catalog> <?xml version="1.0"?> <catalog> ... (5 Replies)
Discussion started by: karthinvk
5 Replies

6. Shell Programming and Scripting

Help required in Splitting a xml file into multiple and appending it in another .xml file

HI All, I have to split a xml file into multiple xml files and append it in another .xml file. for example below is a sample xml and using shell script i have to split it into three xml files and append all the three xmls in a .xml file. Can some one help plz. eg: <?xml version="1.0"?>... (4 Replies)
Discussion started by: ganesan kulasek
4 Replies

7. Shell Programming and Scripting

splitting a file (xml) into multiple files

To split the files Hi, I'm having a xml file with multiple xml header. so i want to split the file into multiple files. Test.xml --------- <?xml version="UTF_8"> <emp: ....> <name>a</name> <age>10</age> </emp> <?xml version="UTF_8"> <emp: ....> <name>b</name> <age>10</age>... (11 Replies)
Discussion started by: sasi_u
11 Replies

8. Shell Programming and Scripting

splitting huge xml into multiple files

hi all i have a some huge html files (500MB to 1GB). Each file has multiple <html></html> tags <html> ................. .................... .................... </html> <html> ................. .................... .................... </html> <html> .................... (5 Replies)
Discussion started by: uttamhoode
5 Replies

9. Shell Programming and Scripting

Splitting huge XML Files into fixsized wellformed parts

Hi, I need to split xml-files with sizes greater than 2 gb into smaler chunks. As I dont want to end up with billions of files, I want those splitted files to have configurable sizes like 250 MB. Each file should be well formed having an exact copy of the header (and footer as the closing of the... (0 Replies)
Discussion started by: Malapha
0 Replies

10. Shell Programming and Scripting

How to parse a XML file using PERL and XML::DOm

I need to know the way. I have got parsing down some nodes. But I was unable to get the child node perfectly. If you have code please send it. It will be very useful for me. (0 Replies)
Discussion started by: girigopal
0 Replies
Login or Register to Ask a Question