Parse Logfile output variable

Ikon · #1 (**permalink**) 12-09-2008

Code:

<SUMMARY filecount_excluded="0" dirbytes_sent="3367893" dirbytes_hashcache="13275664" ..and so on..>
<session numthreads="1" type="avtarbackup" ndispatchers="1" ..and so on..><host numprocs="4" 
speed="900" osuser="root" name="ashsux01" memory="24545" /><build time="11:04:53" msgversion="13-10" 
appname="avtar" ..and so on../><dirstats numfiles="193129" numbytes="1461473417216" numdirs="0" />
</session><errorsummary exitcode="0" errors="0" warnings="0" fatals="0" />
</SUMMARY>

I would like to output the following:

Code:

SUMMARY_filecount="0"
SUMMARY_dirbytes="3367893"
...
SUMMARY_session_numthreads="1"
SUMMARY_session_type="avtarbackup" 
...
SUMMARY_session_build_time="11:04:53" 
SUMMARY_session_build_msgversion="13-10" 
...
SUMMARY_session_dirstats_numfiles="193129" 
SUMMARY_session_dirstats_numbytes="1461473417216" 
...
SUMMARY_errorsummary_exitcode="0" 
SUMMARY_errorsummary_errors="0" 
...

There could be any number of tags within tags and any number of variables.

I tried doing some searching but wasnt sure what exactally to search for.

joeyg · #2 (**permalink**) 12-09-2008

Code:

> cat file103 | sed "s/<\/session/~&/g" | tr "~" "\n"

Which would separate data onto each own line,
then grep for the lines you want to see?

[Sorry I couldn't test, but your sample data did not have the appropriate tags or session markers for me to truly analyze.]

Ikon · #3 (**permalink**) 12-09-2008

Here is an actual snip from the log:

Code:

<SUMMARY filecount_excluded="0" dirbytes_sent="3367893" dirbytes_hashcache="13275664"><session numthreads="1" 
type="avtarbackup" ndispatchers="1"><host numprocs="4" speed="900" osuser="root" name="ashsux01" memory="24545" />
<build time="11:04:53" msgversion="13-10" appname="avtar"/><dirstats numfiles="193129" numbytes="1461473417216" 
numdirs="0" /></session><errorsummary exitcode="0" errors="0" warnings="0" fatals="0" /></SUMMARY>

The problem im running into is <SUMMARY has its own vars then within SUMMARY it has SESSION and within SESSION it has its vars and then DIRSTATS, BUILD.

like this:

Code:

<SUMMARY VARS...>
	<SESSION VARS...>
		<BUILD VARS.../>
		<DIRSTATS VARS.../>
	</SESSION>
	<ERRORSUMMARY VARS...\>
</SUMMARY>

The actual log file has many others within summary and some not called summary they might be called something completely different than SUMMARY or SESSION.

vidyadhar85 · #4 (**permalink**) 12-09-2008

then i think you have to do it with two or more awk
i tried some thing but not getting right hope you work more on it

Code:

awk '/^<SUMMARY/{print $0}' file|awk -F"[=_]" 'BEGIN{RS=" "}$0!="<SUMMARY"{print "SUMMARY_"$1"="$3}'

Code:

awk '/^<session/{print $0}' file|awk -F"[= ]" 'BEGIN{RS=" "}$0!="<session"{print "SUMMARY_session_"$1"="$2}'

Ikon · #5 (**permalink**) 12-09-2008

Those are pretty goot starting points. They are pretty close.

I will be working on it all day tomorrow, when I get a working script i will be sure to share.

vidyadhar85 · #6 (**permalink**) 12-09-2008

Quote:

Originally Posted by Ikon

Those are pretty goot starting points. They are pretty close.

I will be working on it all day tomorrow, when I get a working script i will be sure to share.

will be looking forward for the answer..
regards,
vidya

Ikon · #7 (**permalink**) 12-11-2008

Well I decided to write it in perl...

Im having some problems:

If you compair the log with the output its skipping the first item, ie CUSTOMER_join, ADDRESS_street, TODAY_date.....

This is not the actual log file it will be reading it just a quick on I put together. The actual log if faily large.

Code:

 
# cat testfile.log
<CUSTOMER join="1/1/2008" last="12/20/2008">
<NAME name="John" lname="Smith">
<ADDRESS street="main" number="123" city="Orlando" state="Florida" zip="12345" />
</DATA>
</CUSTOMER>
<TODAY date="12/11/2008" time="12:12:12" />

Code:

# perl readlog.pl testfile.log
CUSTOMER_last: 12/20/2008
NAME_lname: Smith
ADDRESS_number: 123
ADDRESS_city: Orlando
ADDRESS_state: Florida
ADDRESS_zip: 12345
TODAY_time: 12:12:12

Code:

 
# cat readlog.pl
$filename = $ARGV[0];
open FILE, $filename or die $!;
while (<FILE>) {
        push(@fields, defined($1) ? $1:$3)
        while m/([^<>]+)/g;
}
close(FILE);
$head="";
foreach (@fields) {
        if ($_ =~ /^([A-Za-z0-9]+) /) {
                $line = $_;
                ($header,$data) = split(/ /, $line, 2);
                if ( $head == "" ) {
                        $head = $header;
                } else {
                        $head = $head."_".$header;
                }
                @subs = split(/" /,$data);
                for($i = 0; $i < @subs; $i++) {
                        ($str, $strdata) = split (/=/,$subs[$i]);
                        $strdata =~ s/^"//;
                        $strdata =~ s/"$//;
                        $head =~ s/_{2,}/_/;
                        if ($str !~ /\//) {
                                print $head."_".$str.": ".$strdata."\n";
                        }
                }
                if ($data =~ /\/$/) {
                        @h = split(/_/,$head);
                        $max = @h - 1;
                        $head =~ s/$h[$max]// ;
                }
        } else {
                if ($_ =~ /^\//) {
                         @h = split(/_/,$head);
                        $max = @h - 1;
                        $head =~ s/$h[$max]// ;
                }
        }
}

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
logfile	naveeng.81	Shell Programming and Scripting	1	04-07-2008 11:32 AM
Logfile Size exceeded ????	skyineyes	Shell Programming and Scripting	3	01-18-2008 08:51 AM
last month's logfile	deep_kol	Shell Programming and Scripting	2	06-26-2007 02:27 PM
Extracting Logfile Entries	harpdl	Shell Programming and Scripting	2	07-13-2006 01:40 PM
sort a logfile	joerg	Shell Programming and Scripting	7	06-23-2004 01:28 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread: