Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Shell Script syntax for XML processing

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell Script syntax for XML processing
# 1  
Old 02-04-2008
Shell Script syntax for XML processing
Hi All,

I am new to Shell scripting.

I have a log file containing XML Messages.Each XML Message is accompanied with a timestamp.I need to count the the number of messages that get logged in a particular timeinterval.Is there any command/Syntax to achieve this.

Any code/example is welcome.

Thanks
Vignesh.
# 2  
Old 02-04-2008
Quote:
Any code/example is welcome.
Smilie

I thought it should have been the other way.

To parse XML messages, I suggest using XML::Parser

XML::Parser - A perl module for parsing XML documents - search.cpan.org

Quite easy, quick and efficient to use the above module
# 3  
Old 02-04-2008
Shell Scripting Syntax for XML Processing
Hi madhan,


Thanks for ur reply.Actually there is no need to parse my XML.Each XML message is accompanied with a timestamp value(it is not a XML field).I just need to count the no of messages within a particular time interval using shell scripting.


My XML looks like this

2008-01-27 00:05:00 (2008-01-27 00:05:00.055000000Z): message={Data="<?xml version="1.0" encoding="UTF-8"?>
<Envelope>
<Header>
....
....
</Header>
<Body>
....
....
</Body>
</Envelope>"
2008-01-27 00:05:12 (2008-01-27 00:05:12.055000000Z): message={Data="<?xml version="1.0" encoding="UTF-8"?>
<Envelope>
<Header>
....
....
</Header>
<Body>
....
....
</Body>
</Envelope>"

Thanks
Vignesh
# 4  
Old 02-04-2008
Can be done in python (works upto seconds (not milliseconds) and also works across days):

LOG
Code:
2008-01-27 00:05:00 (2008-01-27 00:05:00.055000000Z): message={Data="<?xml version="1.0" encoding="UTF-8"?>
<Envelope>
<Header>
....
....
</Header>
<Body>
....
....
</Body>
</Envelope>
2008-01-27 00:05:12 (2008-01-27 00:05:12.055000000Z): message={Data="<?xml version="1.0" encoding="UTF-8"?>
<Envelope>
<Header>
....
....
</Header>
<Body>
....
....
</Body>
</Envelope>
2008-01-27 00:05:12 (2008-01-27 00:06:10.055000000Z): message={Data="<?xml version="1.0" encoding="UTF-8"?>
<Envelope>
<Header>
....
....
</Header>
<Body>
....
....
</Body>
</Envelope>

Python script:
Code:
import time
import datetime
import re
import sys

#---------------------------------------------------------
# Variables and defs
#---------------------------------------------------------

if len(sys.argv) == 1:
        print "Usage : ", sys.argv[0], "<file_name>", "from_date(in_double_quotes)", "to_date(in_double_quotes)"
        quit()

[file, frm, to] = sys.argv[1:4]
dtfmt = "%Y-%m-%d %H:%M:%S"
patstr = "^[0-9]{4}-[0-9]{2}-[0-9]{2} "

def mk_date(dstr, dpat):
        return time.mktime(time.strptime(dstr, dpat))

fobj = mk_date(frm, dtfmt)
tobj = mk_date(to, dtfmt)
datepat = re.compile(patstr)

#---------------------------------------------------------
# main part
#---------------------------------------------------------
fh = open(file, 'r')
count = 0
for line in fh:
        if (datepat.match(line)):
                dt = line.split('(')[1].split(')')[0][0:19]
                dobj = mk_date(dt, dtfmt)
                if tobj > dobj > fobj:
                        count += 1
print "Total ocurrances between ", frm, " and ", to, ":", count

fh.close()

Usage:
Code:
C:\>lr.py
Usage :  C:\lr.py <file_name> from_date(in_double_quotes) to_date(in_double_quotes)

C:\>lr.py log.xml "2008-01-27 00:04:59" "2008-01-27 00:05:13"
Total ocurrances between  2008-01-27 00:04:59  and  2008-01-27 00:05:13 : 2

HTH
# 5  
Old 02-05-2008
Thanks for your help.
# 6  
Old 02-05-2008
A perl solution:

Code:
#!/usr/bin/perl
use strict;
use warnings;
my ($start,$end) = @ARGV;
unless ($start =~ /^\d{4}-\d\d-\d\d\s+\d\d:\d\d:\d\d$/ && 
        $end   =~ /^\d{4}-\d\d-\d\d\s+\d\d:\d\d:\d\d$/) {
   die qq{Usage: perl path/to/count.pl "start" "end"
start and end = "YYYY-MM-DD HH:MM:SS" including quotes};
}
my $count = 0;
(my $s = $start) =~ tr/- ://d;
(my $e = $end)   =~ tr/- ://d;
open (my $in, 'path/to/input_file') or die "$!"; 
while(<$in>){
    if (/^(\d\d\d\d)-(\d\d)-(\d\d)\s+(\d\d):(\d\d):(\d\d)/) {
        $count++ if ($e ge "$1$2$3$4$5$6" && "$1$2$3$4$5$6" ge $s);
    }
}
close $in;
print "Number of messages found between $start and $end: $count\n";

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

2. Programming

help me with perl script xml processing

Hi everyone, I have Xml files in a folder, I need to extract some attribute values form xml files and store in a hash. My xml file look like this. <?xml version="1.0" encoding="UTF-8"?> <Servicelist xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"... (0 Replies)
Discussion started by: pavani reddy
0 Replies

3. Shell Programming and Scripting

Syntax error calling TCL script from shell script

hello everyone i am beginner on shell scripting .and i am working on my project work on ad hoc network i wrote a batch (.sh) to do a looping and execute a tcl script i wrote before in each iteration ..but i got this problem " syntax error near unexpected token `('... (1 Reply)
Discussion started by: marcoss90
1 Replies

4. Shell Programming and Scripting

File Processing in shell script

i have a file with following type of data abcd : gggggg gggggg ; 1234 gggggg ; 5678 gggggg ; 3434 gggggg ; 6565 gggggg ; 1231 1234 ; vvvv ;Eng=Myfirstname 5678 ; xyzf ;Eng=Mysecondname 3434 ; xyzf ;Eng=Mythirdname 6565 ; xyzf ;Eng=Mysfourthname 1231 ; xyzf ;Eng=Mysfifthname... (7 Replies)
Discussion started by: telangmadhuri
7 Replies

5. Shell Programming and Scripting

Help with shell script for url processing

Hi, My objective is to make a shell script that, when run, you can input multiple links at once. text is then inserted between the http:// part and the following url. example : http://google.be ==> http://sometext.google.be it would be great if it could then open all the created links (wich... (5 Replies)
Discussion started by: tooster
5 Replies

6. Shell Programming and Scripting

2 file processing script in C shell

I want to use an awk for the following scenario but not sure if it will work or not. I have two input file: F1 and F2 F1 02 05 08 F2 00 01 02 03 04 05 06 07 08 09 10 (1 Reply)
Discussion started by: jclanc8
1 Replies

7. Shell Programming and Scripting

Help Syntax Shell-Script

Hi Guys, iīve a question ... but itīs a litte bit tricky: iīve a 3 php-scripts which runīs via cron at night. These script reads an xml-file a writes it in an MySQL-DB. I named them (for example here ) Script1 - Script3. The XML-Files i named xml1 - xml3. Now, iīve build a Batch-file,... (10 Replies)
Discussion started by: jackcracker
10 Replies

8. Shell Programming and Scripting

How to remove xml namespace from xml file using shell script?

I have an xml file: <AutoData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Table1> <Data1 10 </Data1> <Data2 20 </Data2> <Data3 40 </Data3> <Table1> </AutoData> and I have to remove the portion xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" only. I tried using sed... (10 Replies)
Discussion started by: Gary1978
10 Replies

9. Shell Programming and Scripting

need help on xml processing

I am trying to divide a xml file(my.xml) like this: <?xml version="1.0" encoding="UTF-8"?> <Proto PName="hmmmmmmm"> <Menu id="A" ver="1"> <P> <P name="AA" Type="X"/> <P name="BB" Type="Y"/> <P name="CC" Type="Z"/> </P> ... (4 Replies)
Discussion started by: demoprog
4 Replies

10. Shell Programming and Scripting

Shell script not processing if statement properly

Hi I am trying to create a shell script that will look for a contracthead file first and if the contract head file does not exist on day1 exit script. Now on day2 if contracthead exists or not run the script uploading files in order such as contract line then contract contact so the... (2 Replies)
Discussion started by: jonathan184
2 Replies
Login or Register to Ask a Question