Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

Shell script to parsing log

Shell Programming and Scripting


Tags
shell script, shell scripting, unix scripting, unix scripting basics

Closed    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 12-05-2008   -   Original Discussion by justbow
justbow's Unix or Linux Image
justbow justbow is offline
Registered User
 
Join Date: Dec 2008
Last Activity: 14 December 2017, 2:50 AM EST
Posts: 90
Thanks: 15
Thanked 1 Time in 1 Post
Unix or Linux Question Shell script to parsing log

Hi I Have log like this :
0 234: [APPLICATION 1] {
3 2: [sequenceNumber] 04 EE
7 14: [timeStamp] '20081114081'
23 1: [trigger] 00
79 10: [msisdn] '38809'
91 15: [imsi] '528111510010159'
143 29: [applicationDescription] 'Streaming/downloading service'
174 3: [custom0] 'MTV'
179 43: [contentUrl] 'rtsp://172.28/MTV2GO-Loop.sdp'
224 1: [contentRate] 05
227 1: [priceType] 01
230 2: [currencyCode] 39 36
234 1: [currencyExponent] FE
: }
237 248: [APPLICATION 1] {
240 2: [sequenceNumber] 04 EF
244 14: [timeStamp] '20081114082116'
260 1: [trigger] 02
316 10: [msisdn] '38809'
328 15: [imsi] '528111510010'
360 1: [paymentMethod] 01
380 29: [applicationDescription] 'Streaming/downloading service'
411 3: [custom0] 'MTV'
416 43: [contentUrl] 'rtsp://172.28/MTV2GO-Loop.sdp'
461 3: [transferredVolume] 19 83 87
466 1: [timeDuration] 5D
469 1: [debitedCost] 08
472 1: [chargingStatus] 01
475 1: [contentRate] 05
478 1: [priceType] 01
481 2: [currencyCode] 39 36
485 1: [currencyExponent] FE
: }
712 236: [APPLICATION 1] {
715 2: [sequenceNumber] 04 F1
719 14: [timeStamp] '20081114082'
735 1: [trigger] 01
791 10: [msisdn] '38997'
803 15: [imsi] '528111500081770'
820 10: [ipAddress] '10.84.8.66'
832 1: [clientProtocol] 01
835 1: [paymentMethod] 01
855 29: [applicationDescription] 'Streaming/downloading service'
886 35: [contentUrl] 'rtsp://172.28/cnn.sdp'
923 3: [transferredVolume] 26 D0 CD
928 2: [timeDuration] 00 A9
932 1: [debitedCost] 0E
935 1: [chargingStatus] 01
938 1: [contentRate] 05
941 1: [priceType] 01
944 2: [currencyCode] 39 36
948 1: [currencyExponent] FE
: }
.....
.....


I Want that log become just only "[trigger] 02"
and then collect into Timestamp,msisdn,timeduration,contentUrl

anybody can help me ??
Sponsored Links
    #2  
Old Unix and Linux 12-05-2008   -   Original Discussion by justbow
justbow's Unix or Linux Image
justbow justbow is offline
Registered User
 
Join Date: Dec 2008
Last Activity: 14 December 2017, 2:50 AM EST
Posts: 90
Thanks: 15
Thanked 1 Time in 1 Post
I've try with this and its not working Linux

for FILE in $filename
do
temp1=`cat $FILE | grep "timeStamp]" | awk '{print$4}' | cut -d"'" -f2`
temp2=`cat $FILE | grep "trigger] 02" | awk '{print$4}'`
temp3=`cat $FILE | grep "msisdn]" | awk '{print$4}' | cut -d"'" -f2`
temp4=`cat $FILE | grep "contentUrl]" | awk '{print$4}' | cut -d"'" -f2`
temp5=`cat $FILE | grep "timeDuration]" | awk '{print$4}'`
done

echo $temp1 $temp2 $temp3 $temp4 $temp5
Sponsored Links
    #3  
Old Unix and Linux 12-05-2008   -   Original Discussion by justbow
Franklin52's Unix or Linux Image
Franklin52 Franklin52 is offline
Registered User
 
Join Date: Feb 2007
Last Activity: 5 December 2017, 5:22 PM EST
Location: The Netherlands
Posts: 7,746
Thanks: 139
Thanked 558 Times in 520 Posts
This should give the desired items of the section [trigger] 2. It prints the 3th and the 4th field, adjust and play around with the code if you only want to print the 4th field.


Code:
awk '
/\[trigger\] 02/{print s;f=1}
f && /\[msisdn\]/{print $3,$4}
f && /\[contentUrl\]/{print $3,$4}
f && /\[timeDuration\]/{print $3,$4;exit}
{s=$3 " " $4}' logfile

Explanation:


Code:
/\[trigger\] 02/{print s;f=1}

if we match [trigger] 2, print the time stamp we've saved in the variable s (1 line above) and assign 1 (true) to the variable f.


Code:
f && /\[msisdn\]/{print $3,$4}
f && /\[contentUrl\]/{print $3,$4}
f && /\[timeDuration\]/{print $3,$4;exit}

If f is true (1) print field 3 and 4 if one of the line match. After printing the time duration exit the program.


Code:
{s=$3 " " $4}' logfile

Store the actual line in the variable s, this is 1 line before the [trigger] line.

Use nawk or /usr/xpg4/bin/awk on Solaris.

Regards
    #4  
Old Unix and Linux 12-05-2008   -   Original Discussion by justbow
justbow's Unix or Linux Image
justbow justbow is offline
Registered User
 
Join Date: Dec 2008
Last Activity: 14 December 2017, 2:50 AM EST
Posts: 90
Thanks: 15
Thanked 1 Time in 1 Post
Yep! it's working..Thanks Franklin52

Quote:
[~/bow/coba]$ cat inputfile.log | awk '
/\[trigger\] 02/{print s;f=1}
f && /\[msisdn\]/{print $3,$4}
f && /\[contentUrl\]/{print $3,$4}
f && /\[timeDuration\]/{print $3,$4;exit}
{s=$3 " " $4}'
Result :
Quote:
[timeStamp] '20081114082116'
[msisdn] '38809'
[contentUrl] 'rtsp://172.28/MTV2GO-Loop.sdp'
[timeDuration] 5D



But the problem is the log bigger.. contain the format like that

[APPLICATION 1] { ----> begin

;;;;;;

} ----> end

it mixed from [trigger] 00, [trigger] 01 , and [trigger] 02
and the information from [trigger] 01 there are always no value of [timeDuration]

Or should i give the example

Any solution ??
Sponsored Links
    #5  
Old Unix and Linux 12-05-2008   -   Original Discussion by justbow
Franklin52's Unix or Linux Image
Franklin52 Franklin52 is offline
Registered User
 
Join Date: Feb 2007
Last Activity: 5 December 2017, 5:22 PM EST
Location: The Netherlands
Posts: 7,746
Thanks: 139
Thanked 558 Times in 520 Posts
The problem is not clear to me, can you give an example?

Regards
Sponsored Links
    #6  
Old Unix and Linux 12-05-2008   -   Original Discussion by justbow
justbow's Unix or Linux Image
justbow justbow is offline
Registered User
 
Join Date: Dec 2008
Last Activity: 14 December 2017, 2:50 AM EST
Posts: 90
Thanks: 15
Thanked 1 Time in 1 Post
Shell script to parsing log

Maybe you can see the log here


and i want that log become just only "[trigger] 02"
and then collect into Timestamp,msisdn,timeduration,contentUrl

the log always begin
[APPLICATION 1] {

and end with }

should i separate per index and then

to make it clearly you can see here

i still want that log become just only "[trigger] 02"
and then collect into Timestamp,msisdn,timeduration,contentUrl

If you see the stream always begin with :
[APPLICATION 1] { ----> begin

;;;;;;

} ----> end

Should i separate into index then,,

Regards, Linux

Last edited by vbe; 12-05-2008 at 01:19 PM.. Reason: xduplicate because of moderation status
Sponsored Links
    #7  
Old Unix and Linux 12-05-2008   -   Original Discussion by justbow
Franklin52's Unix or Linux Image
Franklin52 Franklin52 is offline
Registered User
 
Join Date: Feb 2007
Last Activity: 5 December 2017, 5:22 PM EST
Location: The Netherlands
Posts: 7,746
Thanks: 139
Thanked 558 Times in 520 Posts
Do you want the output of all the sections with "[trigger] 02"?
Post the desired output.
Sponsored Links
Closed

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Parsing XML using shell script zorosinister Shell Programming and Scripting 9 05-01-2012 03:53 AM
XML parsing using shell script LavanyaP Shell Programming and Scripting 2 05-13-2011 05:25 AM
XML parsing in a shell script. dashok.83 Shell Programming and Scripting 1 09-05-2010 12:13 PM
Parsing XML using Shell Script Kage Musha Shell Programming and Scripting 1 01-18-2010 10:33 AM
shell script parsing with sed jjamd64 UNIX for Dummies Questions & Answers 5 12-11-2007 04:51 PM



All times are GMT -4. The time now is 02:49 AM.