Sponsored Content
Top Forums Shell Programming and Scripting Help with Splitting a Large XML file based on size AND tags Post 302907992 by Chubler_XL on Thursday 3rd of July 2014 12:43:27 AM
Old 07-03-2014
Sorry I should have tried my code on more than 1 large URL as I have forgotten to reset the bytes variable please accept this updated version:

Code:
#!/bin/bash
export ORACLE_HOME=.........
export ORACLE_SID=...........
export PATH=........
. ./params        # contains the parameter sizelimit
...

if [ $(stat -c%s $FILE) -gt $sizelimit ]
then
    awk -v limit=$sizelimit '
        BEGIN { num=1 }
        {
          if ((bytes+=length)>limit) {
             close(FILENAME "." num)
             bytes=length
             num++
          }
          printf "%s%s",$0,RS > FILENAME "." num
        } ' RS="</URL>" $FILE
else
   echo "$FILE: already less than the limit of $sizelimit"
fi

This User Gave Thanks to Chubler_XL For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk - splitting 1 large file into multiple based on same key records

Hello gurus, I am new to "awk" and trying to break a large file having 4 million records into several output files each having half million but at the same time I want to keep the similar key records in the same output file, not to exist accross the files. e.g. my data is like: Row_Num,... (6 Replies)
Discussion started by: kam66
6 Replies

2. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

3. Shell Programming and Scripting

Problem with splitting large file based on pattern

Hi Experts, I have to split huge file based on the pattern to create smaller files. The pattern which is expected in the file is: Master..... First... second.... second... third.. third... Master... First.. second... third... Master... First... second.. second.. second..... (2 Replies)
Discussion started by: saisanthi
2 Replies

4. Shell Programming and Scripting

Splitting large file and renaming based on field

I am trying to update an older program on a small cluster. It uses individual files to send jobs to each node. However the newer database comes as one large file, containing over 10,000 records. I therefore need to split this file. It looks like this: HMMER3/b NAME 1-cysPrx_C ACC ... (2 Replies)
Discussion started by: fozrun
2 Replies

5. Shell Programming and Scripting

Help required in Splitting a xml file into multiple and appending it in another .xml file

HI All, I have to split a xml file into multiple xml files and append it in another .xml file. for example below is a sample xml and using shell script i have to split it into three xml files and append all the three xmls in a .xml file. Can some one help plz. eg: <?xml version="1.0"?>... (4 Replies)
Discussion started by: ganesan kulasek
4 Replies

6. Shell Programming and Scripting

Sed: Splitting A large File into smaller files based on recursive Regular Expression match

I will simplify the explaination a bit, I need to parse through a 87m file - I have a single text file in the form of : <NAME>house........ SOMETEXT SOMETEXT SOMETEXT . . . . </script> MORETEXT MORETEXT . . . (6 Replies)
Discussion started by: sumguy
6 Replies

7. Shell Programming and Scripting

Split XML file based on tags

Hello All , Please help me with below requirement I want to split a xml file based on tag.here is the file format <data-set> some-information </data-set> <data-set1> some-information </data-set1> <data-set2> some-information </data-set2> I want to split the above file into 3... (5 Replies)
Discussion started by: Pratik4891
5 Replies

8. Shell Programming and Scripting

Splitting xml file into several xml files using perl

Hi Everyone, I'm new here and I was checking this old post: /shell-programming-and-scripting/180669-splitting-file-into-several-smaller-files-using-perl.html (cannot paste link because of lack of points) I need to do something like this but understand very little of perl. I also check... (4 Replies)
Discussion started by: mcosta
4 Replies

9. Shell Programming and Scripting

Splitting a single xml file into multiple xml files

Hi, I'm having a xml file with multiple xml header. so i want to split the file into multiple files. Sample.xml consists multiple headers so how can we split these multiple headers into multiple files in unix. eg : <?xml version="1.0" encoding="UTF-8"?> <ml:individual... (3 Replies)
Discussion started by: Narendra921631
3 Replies

10. Shell Programming and Scripting

Issue splitting file based on XML tags

more a-d.txt1 <a-dets> <a-serv> <aserv>mymac14,mymac15:MYAPP:mydom:/web/domain/mydom/config <NMGR>:MYAPP:/web/bea_apps/perf/NMGR/NMGR1034 <a-rep-string> 11.12.10.01=192.10.00.26 10.20.18.10=192.10.00.27 </a-rep-string> </a-serv> <w-serv>... (2 Replies)
Discussion started by: mohtashims
2 Replies
MRTG-FORUM(1)							       mrtg							     MRTG-FORUM(1)

NAME
mrtg-forum - Interactive Help for MRTG users SYNOPSIS
There is a lot of written documentation for mrtg, but nevertheless you may have a problem where you can't find the solution. In this case some Human Help may be necessary. With MRTG there are several ways to get Humans to help you. MAILING-LIST There are three mailing lists for MRTG available. mrtg for discussion among mrtg users. mrtg-announce for announcements regarding new versions of mrtg related software. mrtg-developers for discussion among people who write software in connection with mrtg or who hack mrtg itself. THE RULES Please note that the memebers of the mrtg mailinglist value politeness highly. This means behave in a way you would like others to behave towards you. o No shouting. (NO CAPS) o No rude language o No demands. everybody is on the list out of their own free will. If you do not get an answer to your question, chances are high that you did not give sufficent details about the nature of your problem or that the answer to your problem is in the documentation. o If you do not follow the rules you will be unsubscribed from the list with no further questions asked. o Decisions about your unsubscription from the list will be taken by Alex van den Bogaerdt <alex at ergens.op.het.net> Paul C. Williamson <pwilliamson at MANDTBANK.COM> If you feel that you have been treated unfairly, you may send mail to me and explain the situation. Tobi Oetiker <tobi@oetiker.ch> SUBSCRIBING These lists are managed by a mailing-list management program (listar). It allows you to subscribe to these lists by sending a message with the subject: subscribe to the following address: listname-request@lists.oetiker.ch You will then get a message asking you to confirm your subscription. For posting to the lists use the following address listname@lists.oetiker.ch Note that only people who are subscribed to the list can post. Further information about the usage of the mailing lists is available by sending a message with the subject line help to either one of the request addresses. There is also a webinterface to the lists on http://lists.oetiker.ch List archives are on http://www.mail-archive.com/index.php?hunt=mrtg and http://gmane.org/find.php?list=mrtg NEWSGROUP
For discussion of MRTG or related topics on the Usenet, please send your posts to: news:comp.dcom.net-management Many MRTG users are in this forum and will help you. You can also find an archive of past activity from this Newsgroup on: http://groups.google.com/group/comp.dcom.net-management IRC Channel For discussion of MRTG and related topics on IRC, the `#mrtg' channel on EFNet has been created. More information can be found at http://mrtg.easymac.org MRTG Japan There is a special Mailinglist for MRTG Users in Japan. It carries translations of the traffic from MRTG-ANNOUNCE as well as updates on the Japanese Translation of the MRTG documentation. Go to http://www.mrtg.jp/ for further Information. MRTG Turkiye There is a special Mailinglist for MRTG Users in Turkey. Go to http://groups.yahoo.com/group/mrtg-turkiye/ for more information. AUTHOR
Tobias Oetiker <tobi@oetiker.ch> and many contributors 2.16.2 2008-05-16 MRTG-FORUM(1)
All times are GMT -4. The time now is 02:20 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy