Getting VALUE from Big XML File -- That's All


Login or Register to Reply

 
Thread Tools Search this Thread
# 1  
Old 01-15-2016
Getting VALUE from Big XML File -- That's All

We got data that was supposed to be CSV, but was sent in a huge XML file.

I've downloaded xmlstarlet, but I'm darned if I can get it to operate the "sel" feature to look down a path and get any sort of value. I see pieces of what should be paths, but they seem to have extraneous characters, and I don't know how to use the various <...> fields to make s decent query. For example,
I want to get: <es:mixedModeRadio>false</es:mixedModeRadio> from the below small piece of the XML file: How?

Code:
xmlstarlet sel "/<configData dnPrefix="Undefined">/<xn:SubNetwork id="ONRM_ROOT_MO_R">/<xn:SubNetwork     id="MyTown">/<xn:MeContext id="LL12345">/<xn:VsDataContainer id="LL12345">"

Is there an easier way? Is there some intermediate step I'm missing?

Here's a very tiny part of a very large file:

Code:
<?xml version="1.0" encoding="UTF-8"?>
<bulkCmConfigDataFile xmlns:un="utranNrm.xsd"
    xmlns:es="Edward.15.25.xsd"
    xmlns:xn="genericNrm.xsd" xmlns:gn="geranNrm.xsd" xmlns="configData.xsd">
    <fileHeader fileFormatVersion="32.615 V4.5" vendorName="Edward"/>
    <configData dnPrefix="Undefined">
        <xn:SubNetwork id="ONRM_ROOT_MO_R">
            <xn:SubNetwork id="MyTown">
                <xn:attributes>
                    <xn:userDefinedNetworkType>MY_SERVERS</xn:userDefinedNetworkType>
                    <xn:userLabel>MyTown</xn:userLabel>
                </xn:attributes>
                <xn:MeContext id="LL12345">
                    <xn:VsDataContainer id="LL12345">
                        <xn:attributes>
                            <xn:vsDataType>vsDataMeContext</xn:vsDataType>
                            <xn:vsDataFormatVersion>EdwardSpecificAttributes.15.25</xn:vsDataFormatVersion>
                            <es:vsDataMeContext>
                                <es:userLabel>LL12345</es:userLabel>
                                <es:ipAddress>11.164.0.116</es:ipAddress>
                                <es:neMIMversion>vF.1.107</es:neMIMversion>
                                <es:lostSynchronisation>SYNCHRONISED</es:lostSynchronisation>
                                <es:bcrLastChange>1452424403156</es:bcrLastChange>
                                <es:bctLastChange>1452160614628</es:bctLastChange>
                                <es:multiStandardRbs6k>true</es:multiStandardRbs6k>
                                <es:mixedModeRadio>false</es:mixedModeRadio>
                                <es:mirrorMIBversion>F.1.100.S.1.6</es:mirrorMIBversion>
                                <es:stnNodes></es:stnNodes>
                            </es:vsDataMeContext>
                        </xn:attributes>
                    </xn:VsDataContainer>
                    <xn:ManagedElement id="1">
                        <xn:attributes>
                            <xn:locationName></xn:locationName>
                            <xn:userDefinedState></xn:userDefinedState>
                            <xn:vendorName>Edward</xn:vendorName>
                            <xn:userLabel>LL12345</xn:userLabel>
                            <xn:managedElementType>ERBS</xn:managedElementType>
                            <xn:swVersion>108991/23_R0DX</xn:swVersion>
                            <xn:managedBy>SubNetwork=ONRM_ROOT_MO_R,ManagementNode=ONRM</xn:managedBy>


Last edited by Don Cragun; 01-15-2016 at 05:04 PM.. Reason: Add CODE and ICODE tags.
# 2  
Old 01-16-2016
Not sure I understand your question. Do you need help with xmlstarlet or just need to extract that line?
# 3  
Old 01-16-2016
As a complete newbie to XML, I need help with xmlstarlet. That particular line is just an example of one of the values I need to extract. I don't understand the syntax of how to use xmlstarlet to do that sort of thing. An example would help.

Last edited by gmark99; 01-16-2016 at 09:55 AM..
# 4  
Old 01-19-2016
Option for sel (select) is using xpath.

What is Xpath ? May be you might get some basics from URL:
XPath Tutorial

Code:
bash-2.03$ xml
XMLStarlet Toolkit: Command line utilities for XML
Usage: xml [<options>] <command> [<cmd-options>]
where <command> is one of:
   ed    (or edit)      - Edit/Update XML document(s)
   sel   (or select)    - Select data or query XML document(s) (XPATH, etc)
   tr    (or transform) - Transform XML document(s) using XSLT
   val   (or validate)  - Validate XML document(s) (well-formed/DTD/XSD/RelaxNG)
   fo    (or format)    - Format XML document(s)
   el    (or elements)  - Display element structure of XML document
   c14n  (or canonic)   - XML canonicalization
   ls    (or list)      - List directory as XML
   esc   (or escape)    - Escape special XML characters
   unesc (or unescape)  - Unescape special XML characters
   pyx   (or xmln)      - Convert XML into PYX format (based on ESIS - ISO 8879)
   p2x   (or depyx)     - Convert PYX into XML
<options> are:
   --version            - show version
   --help               - show help
Wherever file name mentioned in command help it is assumed
that URL can be used instead as well.

Type: xml <command> --help <ENTER> for command help

Most of distro has xmllint. That is my fav.
This User Gave Thanks to chakrapani For This Post:
gmark99 (01-19-2016)
# 5  
Old 01-19-2016
One of my biggest concerns is the size of the file we're looking at -- 3.5Gig. I've been told that xmllib2 has problems with files that are around a few hundred lines. Is XMLSTARLET fairly stable with large files?
# 6  
Old 01-20-2016
I have used xmllint with huge files
Code:
xmllint --format hugefile.xml >> hugefile_formatted.xml

to get formatted or use --shell option to get you values using xpath.
# 7  
Old 01-25-2016
Wonderful! Now let me ask one more question: Examples of "XPATH" don't quite correspond to examples I see, so tags like "/first/second/third" from a body of XML that looks like:
Code:
<first>
    <second>
        <third>

... in my case is more like:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<first xmlns:un="Floo.xsd"
    xmlns:es="MoreFloo.xsd"
    xmlns:xn="MoreEvenFloo.xsd" xmlns:gn="geranNrm.xsd" xmlns="configData.xsd">
 <second fileFormatVersion="32.615 V4.5" vendorName="Edward"/>
    <third dnPrefix="Undefined">
        <xn:SubNetwork id="ONRM_ROOT_MO_R">

So my question is how much of these additional strings are used in the path?
Code:
/<first xmlns:un="Floo.xsd"
    xmlns:es="MoreFloo.xsd"
    xmlns:xn="MoreEvenFloo.xsd" xmlns:gn="geranNrm.xsd" xmlns="configData.xsd">/ <second fileFormatVersion="32.615 V4.5" vendorName="Edward"/>/<third dnPrefix="Undefined">/

or... the same minus the "<" and ">" ???

In the meantime, I'll keep looking over the documentation and hope to encounter this.

Thanks!!
Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
Split Big XML file Base on tag pareshkp Shell Programming and Scripting 4 11-10-2016 03:17 PM
Splitting a single xml file into multiple xml files Narendra921631 Shell Programming and Scripting 3 03-03-2016 02:50 PM
Split xml file into multiple xml based on letterID vx04 Shell Programming and Scripting 5 02-05-2016 09:17 AM
Comparing delta values of one xml file in other xml file sharsour Shell Programming and Scripting 2 11-27-2013 05:28 AM
XML: parsing of the Google contacts XML file ripat Shell Programming and Scripting 9 12-25-2012 10:07 AM
Help required in Splitting a xml file into multiple and appending it in another .xml file ganesan kulasek Shell Programming and Scripting 4 07-26-2012 01:22 PM
How to add the multiple lines of xml tags before a particular xml tag in a file mjavalkar Shell Programming and Scripting 0 06-25-2012 04:54 PM
big xml file with nested loop parse unclecameron Shell Programming and Scripting 1 12-21-2010 02:37 PM
Need to Split Big XML into multiple xmls dprakash Shell Programming and Scripting 8 10-28-2010 10:13 PM
How big is too big a config.log file? NeedLotsofHelp UNIX for Dummies Questions & Answers 2 04-04-2010 05:34 AM
How to remove xml namespace from xml file using shell script? Gary1978 Shell Programming and Scripting 10 10-26-2008 09:32 PM
How to parse a XML file using PERL and XML::DOm girigopal Shell Programming and Scripting 0 06-27-2005 06:46 AM
How to view a big file(143M big) chenhao_no1 UNIX for Dummies Questions & Answers 3 07-25-2002 08:45 AM