Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help with Splitting a Large XML file based on size AND tags

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with Splitting a Large XML file based on size AND tags
# 8  
Old 07-03-2014
You're absolutely right and I believe I got the exact cause behind this problem. The XML for even one record in the table is so large that it's not spooling into the file completely. To give an example, the first row of the table contained

Code:
<URL><A>v1</A><B>v2</B><C>v3</C></URL>

But it's getting spooled into the file as

Code:
<URL><A>v1</A><B>v2</B><C>v3

Is there any limit to the file size in Unix ?

---------- Post updated at 12:54 PM ---------- Previous update was at 12:14 PM ----------

Hi All,

I think this problem is resolved. Before spooling the file from the query, I just set the LONG and LONGCHUNKSIZE to their maximum values, and the script is working fine now. I just don't know what will happen in case the production data gets more massive in future !!!

Thanks to Robin & Chubler_XL.

Chubler_XL, your script is very clean and efficient !! Thanks again !!

- Avik
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Issue splitting file based on XML tags

more a-d.txt1 <a-dets> <a-serv> <aserv>mymac14,mymac15:MYAPP:mydom:/web/domain/mydom/config <NMGR>:MYAPP:/web/bea_apps/perf/NMGR/NMGR1034 <a-rep-string> 11.12.10.01=192.10.00.26 10.20.18.10=192.10.00.27 </a-rep-string> </a-serv> <w-serv>... (2 Replies)
Discussion started by: mohtashims
2 Replies

2. Shell Programming and Scripting

Splitting a single xml file into multiple xml files

Hi, I'm having a xml file with multiple xml header. so i want to split the file into multiple files. Sample.xml consists multiple headers so how can we split these multiple headers into multiple files in unix. eg : <?xml version="1.0" encoding="UTF-8"?> <ml:individual... (3 Replies)
Discussion started by: Narendra921631
3 Replies

3. Shell Programming and Scripting

Splitting xml file into several xml files using perl

Hi Everyone, I'm new here and I was checking this old post: /shell-programming-and-scripting/180669-splitting-file-into-several-smaller-files-using-perl.html (cannot paste link because of lack of points) I need to do something like this but understand very little of perl. I also check... (4 Replies)
Discussion started by: mcosta
4 Replies

4. Shell Programming and Scripting

Split XML file based on tags

Hello All , Please help me with below requirement I want to split a xml file based on tag.here is the file format <data-set> some-information </data-set> <data-set1> some-information </data-set1> <data-set2> some-information </data-set2> I want to split the above file into 3... (5 Replies)
Discussion started by: Pratik4891
5 Replies

5. Shell Programming and Scripting

Sed: Splitting A large File into smaller files based on recursive Regular Expression match

I will simplify the explaination a bit, I need to parse through a 87m file - I have a single text file in the form of : <NAME>house........ SOMETEXT SOMETEXT SOMETEXT . . . . </script> MORETEXT MORETEXT . . . (6 Replies)
Discussion started by: sumguy
6 Replies

6. Shell Programming and Scripting

Help required in Splitting a xml file into multiple and appending it in another .xml file

HI All, I have to split a xml file into multiple xml files and append it in another .xml file. for example below is a sample xml and using shell script i have to split it into three xml files and append all the three xmls in a .xml file. Can some one help plz. eg: <?xml version="1.0"?>... (4 Replies)
Discussion started by: ganesan kulasek
4 Replies

7. Shell Programming and Scripting

Splitting large file and renaming based on field

I am trying to update an older program on a small cluster. It uses individual files to send jobs to each node. However the newer database comes as one large file, containing over 10,000 records. I therefore need to split this file. It looks like this: HMMER3/b NAME 1-cysPrx_C ACC ... (2 Replies)
Discussion started by: fozrun
2 Replies

8. Shell Programming and Scripting

Problem with splitting large file based on pattern

Hi Experts, I have to split huge file based on the pattern to create smaller files. The pattern which is expected in the file is: Master..... First... second.... second... third.. third... Master... First.. second... third... Master... First... second.. second.. second..... (2 Replies)
Discussion started by: saisanthi
2 Replies

9. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

10. Shell Programming and Scripting

awk - splitting 1 large file into multiple based on same key records

Hello gurus, I am new to "awk" and trying to break a large file having 4 million records into several output files each having half million but at the same time I want to keep the similar key records in the same output file, not to exist accross the files. e.g. my data is like: Row_Num,... (6 Replies)
Discussion started by: kam66
6 Replies
Login or Register to Ask a Question

LEARN ABOUT CENTOS

xml::sax::pureperl

SAX::PurePerl(3)					User Contributed Perl Documentation					  SAX::PurePerl(3)

NAME

       XML::SAX::PurePerl - Pure Perl XML Parser with SAX2 interface

SYNOPSIS

	 use XML::Handler::Foo;
	 use XML::SAX::PurePerl;
	 my $handler = XML::Handler::Foo->new();
	 my $parser = XML::SAX::PurePerl->new(Handler => $handler);
	 $parser->parse_uri("myfile.xml");

DESCRIPTION

       This module implements an XML parser in pure perl. It is written around the upcoming perl 5.8's unicode support and support for multiple
       document encodings (using the PerlIO layer), however it has been ported to work with ASCII/UTF8 documents under lower perl versions.

       The SAX2 API is described in detail at http://sourceforge.net/projects/perl-xml/, in the CVS archive, under libxml-perl/docs. Hopefully
       those documents will be in a better location soon.

       Please refer to the SAX2 documentation for how to use this module - it is merely a front end to SAX2, and implements nothing that is not in
       that spec (or at least tries not to - please email me if you find errors in this implementation).

BUGS

       XML::SAX::PurePerl is slow. Very slow. I suggest you use something else in fact. However it is great as a fallback parser for XML::SAX,
       where the user might not be able to install an XS based parser or C library.

       Currently lots, probably. At the moment the weakest area is parsing DOCTYPE declarations, though the code is in place to start doing this.
       Also parsing parameter entity references is causing me much confusion, since it's not exactly what I would call trivial, or well documented
       in the XML grammar. XML documents with internal subsets are likely to fail.

       I am however trying to work towards full conformance using the Oasis test suite.

AUTHOR

       Matt Sergeant, matt@sergeant.org. Copyright 2001.

       Please report all bugs to the Perl-XML mailing list at perl-xml@listserv.activestate.com.

LICENSE

       This is free software. You may use it or redistribute it under the same terms as Perl 5.7.2 itself.

perl v5.16.3							    2014-06-09							  SAX::PurePerl(3)