Sponsored Content
Full Discussion: processing xml with awk
Top Forums Shell Programming and Scripting processing xml with awk Post 302662389 by Corona688 on Tuesday 26th of June 2012 12:56:19 PM
Old 06-26-2012
I once wrote a generic XML scanner which produces output similar to what you want. It produces columns from tags in a generic way without hardcoding tags/attributes. It has a weakness in that it can't handle spaces inside tag attributes.

Getting those two 'env' tags into one can be done with sed.

Code:
$ cat xmlg.awk
BEGIN { RS="<";         FS=">"; ORS="\r\n";

        # Change this to alter how many close-tags in a row are needed
        # before a row of data is printed.
        DEP=1
        SEP="\t"
        }

# Skip weird XML specification lines or blank records
/^\?/ || /^$/   {       next    }

# Handle close tags
/^[/]/  {
        N=D;    while((N>0) && ("/"STACK[N] != $1))     N--;

        if("/"STACK[N] == $1)   D=(N-1);
        POP++;

        if(POP == DEP)
        {
                if(!HEADER++)
                {
                        split(ARG[1], Z, SUBSEP);
                        printf("%s %s", Z[2], Z[3]);
                        for(N=2; N<=ARG_; N++)
                        {
                                split(ARG[N], Z, SUBSEP);
                                printf("%s%s %s", SEP, Z[2], Z[3]);
                        }

                        printf("\n");
                }

                printf("%s", DATA[ARG[1]]);
                for(N=2; N<=ARG_; N++)
                        printf("%s%s", SEP, DATA[ARG[N]]);
                printf("\n");
        }
        next
}

# Handle open tags
{
        gsub(/^[ \r\n\t]*/, "", $2);    # Whitespace isn't data
        gsub(/[ \r\n\t]*$/, "", $2);
        sub(/\/$/, "", $(NF-1));

        # Reset parameters
        POP=0;

        M=split($1, A, " ");
        STACK[++D]=A[1];

        if((!MAX) || (D>MAX)) MAX=D;    # Save max depth

        # Handle parameters
        Q=split(A[2], B, " ");
        for(N=1; N<=Q; N++)
        {
                split(B[N], C, "=");
                gsub(/['"]/,"", C[2]);

                I=D SUBSEP STACK[D] SUBSEP C[1];
                if(!SEEN[I]++)
                        ARG[++ARG_]=I;

                DATA[I]=C[2];
        }

        if($2)
        {
                I=D SUBSEP STACK[D] SUBSEP "CDATA";
                if(!SEEN[I]++)
                        ARG[++ARG_]=I;

                DATA[I]=$2;
        }
}

$ sed 's/env="\([^"]*\)" env="\([^"]*\)"/env="\1\2"/g' 3.xml | awk -f xmlg.awk
rel ver mod name        node env        ins ip  ins ip
123     on      ac1     10.192.0.1      10.192.0.2
123     on      ac2     10.192.0.3      10.192.0.4
123     on      pr      10.192.0.5      10.192.0.6
123     off     ac1     10.192.0.7      10.192.0.6
123     off     ac2     10.192.0.8      10.192.0.6
123     off     pr      10.192.0.9      10.192.0.6

$

This User Gave Thanks to Corona688 For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell Script syntax for XML processing

Hi All, I am new to Shell scripting. I have a log file containing XML Messages.Each XML Message is accompanied with a timestamp.I need to count the the number of messages that get logged in a particular timeinterval.Is there any command/Syntax to achieve this. Any code/example is... (5 Replies)
Discussion started by: vignesh53
5 Replies

2. Shell Programming and Scripting

need help on xml processing

I am trying to divide a xml file(my.xml) like this: <?xml version="1.0" encoding="UTF-8"?> <Proto PName="hmmmmmmm"> <Menu id="A" ver="1"> <P> <P name="AA" Type="X"/> <P name="BB" Type="Y"/> <P name="CC" Type="Z"/> </P> ... (4 Replies)
Discussion started by: demoprog
4 Replies

3. Shell Programming and Scripting

awk processing

Hi all Is there a way in awk to know that you are processing your final line of input if you do no know how many lines were in the input to begin with? Thanks (7 Replies)
Discussion started by: pxy2d1
7 Replies

4. Shell Programming and Scripting

CSV processing to XML

Hi, i am really fresh with shell scripting and programming, i have an issue i am not able to solve to populate data on my server for Cisco IP phones. I have CSV file within the following format: ;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;... (9 Replies)
Discussion started by: angel2008
9 Replies

5. Shell Programming and Scripting

Read content between xml tags with awk, grep, awk or what ever...

Hello, I trying to extract text that is surrounded by xml-tags. I tried this cat tst.xml | egrep "<SERVER>.*</SERVER>" |sed -e "s/<SERVER>\(.*\)<\/SERVER>/\1/"|tr "|" " " which works perfect, if the start-tag and the end-tag are in the same line, e.g.: <tag1>Hello Linux-Users</tag1> ... (5 Replies)
Discussion started by: Sebi0815
5 Replies

6. Programming

help me with perl script xml processing

Hi everyone, I have Xml files in a folder, I need to extract some attribute values form xml files and store in a hash. My xml file look like this. <?xml version="1.0" encoding="UTF-8"?> <Servicelist xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"... (0 Replies)
Discussion started by: pavani reddy
0 Replies

7. Shell Programming and Scripting

Help with XML file processing

I need to get all session_ID 's for product="D-0002" from a XML file: Sample input: <session session_ID="6411206" create_date="2012-04-10-10.22.13.000000"> <marketing_info> <program_id>D4AWFU</program_id> <subchannel_id>abc</subchannel_id> </marketing_info> ... (1 Reply)
Discussion started by: karumudi7
1 Replies

8. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

9. Shell Programming and Scripting

How to add Xml tags to an existing xml using shell or awk?

Hi , I have a below xml: <ns:Body> <ns:result> <Date Month="June" Day="Monday:/> </ns:result> </ns:Body> i have a lookup abc.txtt text file with below details Month June July August Day Monday Tuesday Wednesday I need a output xml with below tags <ns:Body> <ns:result>... (2 Replies)
Discussion started by: Nevergivup
2 Replies

10. UNIX for Beginners Questions & Answers

General Purpose XML Processing

I've been kicking this around for a while now, I might as well post it here. v0.0.9, now properly supporting self-closing tags. v0.0.8, an important quoting fix and a minor change which should handle special <? <!-- etc. tags without seizing up as often. Otherwise the code hasn't changed much.... (6 Replies)
Discussion started by: Corona688
6 Replies
AnyData::Format::XML(3pm)				User Contributed Perl Documentation				 AnyData::Format::XML(3pm)

NAME
AnyData::Format::XML - tiedhash and DBI access to XML SYNOPSIS
# access XML data via a multi-dimensional tied hash # see AnyData.pod for full details # use AnyData; my $table = adTie( 'XML', $file, $mode, $flags ); OR # convert data to and from XML # see AnyData.pod for full details # use AnyData; adConvert( 'XML', $file1, $any_other_format, $file2, $flags ); adConvert( $any_other_format, $file1, 'XML', $file2, $flags ); OR # access the data via DBI and SQL # see DBD::AnyData.pod for full details # use DBI; my $dbh = DBI->connect( 'dbi:AnyData' ); $dbh->func('mytable','XML',$file,$flags,'ad_catalog'); See below for a description of the optional flags that apply to all of these examples. DESCRIPTION
This module allows you to create, search, modify and/or convert XML data and files by treating them as databases without having to actually create separate database files. The data can be accessed via a multi-dimensional tiedhash using AnyData.pm or via DBI and SQL commands using DBD::AnyData.pm. See those modules for complete details of usage. The module is built on top of Michel Rodriguez's excellent XML::Twig which means that the AnyData interfaces can now include information from DTDs, be smarter about inferring data structure, reduce memory consumption on huge files, and provide access to many powerful features of XML::Twig and XML::Parser on which it is based. Importing options allow you to import/access/modify XML of almost any length or complexity. This includes the ability to access different subtrees as separate or joined databases. Exporting and converting options allow you to take data from almost any source (a perl array, any DBI database, etc.) and output it as an XML file. You can control the formating of the resulting XML either by supplying a DTD listing things like nesting of tags and which columns should be output as attributes and/or you can use XML::Twig pretty_print settings to generate half a dozen different levels of compactness or whitespace in how the XML looks. The documentation below outlines the special flags that can be used in either of the interfaces to fine-tune how the XML is treated. The flags listed below define the relationship between tags and attributes in the XML document and columns in the resulting database. In many cases, you can simply accept the defaults and the database will be built automatically. However, you can also fine tune the generation of the database by specifying which tags and attributes you are interested in and their relationship with database columns. USAGE
Prerequisites To use the tied hash interface, you will need AnyData XML::Twig XML::Parser To use the DBI/SQL interface, you will need those, and also DBI DBD::AnyData Required flags ( none ) If no flags are specified, then the module determines the database structure from examining the file or data itself, making use of the DTD if there is one, otherwise scanning the first child of the XML tree for structural information. Optional flags If the default behavior is not sufficient, you may either specify a "record_tag" which will be used to define column names, or you can define an entire tag-to-column mapping. For simple XML, no flags are necessary: <table> <row row_id="1"><name>Joe</name><location>Seattle</location></row> <row row_id="2"><name>Sue</name><location>Portland</location></row> </table> The record_tag will default to the first child, namely "row". The column names will be generated from the attributes of the record tag and all of the tags included under the record tag, so the column names in this example will be "row_id","name","location". If the record_tag is not the first child, you will need to specify it. For example: <db> <table table_id="1"> <row row_id="1"><name>Joe</name><location>Seattle</location></row> <row row_id="2"><name>Sue</name><location>Portland</location></row> </table> <table table_id="2"> <row row_id="1"><name>Bob</name><location>Boise</location></row> <row row_id="2"><name>Bev</name><location>Billings</location></row> </table> </db> In this case you will need to specify "row" as the record_tag since it is not the first child of the tree. The column names will be generated from the attributes of row's parent (if the parent is not the root), from row's attributes and sub tags, i.e. "table_id","row_id","name","location". In some cases you will need to specify an entire tag-to-column mapping. For example, if you want to use a different name for the database column than is used in the XML (especially if the XML tag is not a valid SQL column name). You'd also need to specify a mapping if there are two tags with the same name in different places in the XML tree. The column mapping is a reference to an array of column definitions. A column definition is either a simple name of a tag, or a hash reference with the key containing the full path of the XML tag and the value containing the desired column name alias. For example: col_map => [ 'part_id', 'part_name', 'availability' ]; That will find the first three tags with those names and create the database using the same names for the tags. Or: col_map => [ { '/parts/shop/id' => 'shop_id'}, { '/parts/shop/part/id' => 'part_id'}, { '/parts/shop/part/name' => 'part_name'}, ]; That would find the three tags referenced on the left and create a database with the three column names referenced on the right. When exporting XML, you can specify a DTD to control the output. For example, if you import a table from CSV or from an Array, you can output as XML and specify which of the columns become tags and which become attributes and also specify the nesting of the tags in your DTD. The XML format parser is built on top of Michel Rodriguez's excellent XML::Twig which is itslef based on XML::Parser. Parameters to either of those modules may be passed in the flags for adTie() and the other commands including the "prettyPrint" flag to specify how the output XML is displayed and things like ProtocolEncoding. ProtocolEncoding defaults to 'ISO-8859-1', all other flags keep the defaults of XML::Twig and XML::Parser. See the documentation of those modules for details; CAUTION: Unlike other formats, the XML format does not save changes to the file as they are entered, but only saves the changes when you explicitly request them to be saved with the adExport() command. AUTHOR &; COPYRIGHT copyright 2000, Jeff Zucker <jeff@vpservices.com> all rights reserved perl v5.10.1 2010-01-03 AnyData::Format::XML(3pm)
All times are GMT -4. The time now is 06:01 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy