Shell script to parsing log


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script to parsing log
# 1  
Old 12-05-2008
Question Shell script to parsing log

Hi I Have log like this :
0 234: [APPLICATION 1] {
3 2: [sequenceNumber] 04 EE
7 14: [timeStamp] '20081114081'
23 1: [trigger] 00
79 10: [msisdn] '38809'
91 15: [imsi] '528111510010159'
143 29: [applicationDescription] 'Streaming/downloading service'
174 3: [custom0] 'MTV'
179 43: [contentUrl] 'rtsp://172.28/MTV2GO-Loop.sdp'
224 1: [contentRate] 05
227 1: [priceType] 01
230 2: [currencyCode] 39 36
234 1: [currencyExponent] FE
: }
237 248: [APPLICATION 1] {
240 2: [sequenceNumber] 04 EF
244 14: [timeStamp] '20081114082116'
260 1: [trigger] 02
316 10: [msisdn] '38809'
328 15: [imsi] '528111510010'
360 1: [paymentMethod] 01
380 29: [applicationDescription] 'Streaming/downloading service'
411 3: [custom0] 'MTV'
416 43: [contentUrl] 'rtsp://172.28/MTV2GO-Loop.sdp'
461 3: [transferredVolume] 19 83 87
466 1: [timeDuration] 5D
469 1: [debitedCost] 08
472 1: [chargingStatus] 01
475 1: [contentRate] 05
478 1: [priceType] 01
481 2: [currencyCode] 39 36
485 1: [currencyExponent] FE
: }
712 236: [APPLICATION 1] {
715 2: [sequenceNumber] 04 F1
719 14: [timeStamp] '20081114082'
735 1: [trigger] 01
791 10: [msisdn] '38997'
803 15: [imsi] '528111500081770'
820 10: [ipAddress] '10.84.8.66'
832 1: [clientProtocol] 01
835 1: [paymentMethod] 01
855 29: [applicationDescription] 'Streaming/downloading service'
886 35: [contentUrl] 'rtsp://172.28/cnn.sdp'
923 3: [transferredVolume] 26 D0 CD
928 2: [timeDuration] 00 A9
932 1: [debitedCost] 0E
935 1: [chargingStatus] 01
938 1: [contentRate] 05
941 1: [priceType] 01
944 2: [currencyCode] 39 36
948 1: [currencyExponent] FE
: }
.....
.....


I Want that log become just only "[trigger] 02"
and then collect into Timestamp,msisdn,timeduration,contentUrl

anybody can help me ??
# 2  
Old 12-05-2008
I've try with this and its not working Smilie

for FILE in $filename
do
temp1=`cat $FILE | grep "timeStamp]" | awk '{print$4}' | cut -d"'" -f2`
temp2=`cat $FILE | grep "trigger] 02" | awk '{print$4}'`
temp3=`cat $FILE | grep "msisdn]" | awk '{print$4}' | cut -d"'" -f2`
temp4=`cat $FILE | grep "contentUrl]" | awk '{print$4}' | cut -d"'" -f2`
temp5=`cat $FILE | grep "timeDuration]" | awk '{print$4}'`
done

echo $temp1 $temp2 $temp3 $temp4 $temp5
# 3  
Old 12-05-2008
This should give the desired items of the section [trigger] 2. It prints the 3th and the 4th field, adjust and play around with the code if you only want to print the 4th field.

Code:
awk '
/\[trigger\] 02/{print s;f=1}
f && /\[msisdn\]/{print $3,$4}
f && /\[contentUrl\]/{print $3,$4}
f && /\[timeDuration\]/{print $3,$4;exit}
{s=$3 " " $4}' logfile

Explanation:

Code:
/\[trigger\] 02/{print s;f=1}

if we match [trigger] 2, print the time stamp we've saved in the variable s (1 line above) and assign 1 (true) to the variable f.

Code:
f && /\[msisdn\]/{print $3,$4}
f && /\[contentUrl\]/{print $3,$4}
f && /\[timeDuration\]/{print $3,$4;exit}

If f is true (1) print field 3 and 4 if one of the line match. After printing the time duration exit the program.

Code:
{s=$3 " " $4}' logfile

Store the actual line in the variable s, this is 1 line before the [trigger] line.

Use nawk or /usr/xpg4/bin/awk on Solaris.

Regards
# 4  
Old 12-05-2008
Yep! it's working..Thanks Franklin52

Quote:
[~/bow/coba]$ cat inputfile.log | awk '
/\[trigger\] 02/{print s;f=1}
f && /\[msisdn\]/{print $3,$4}
f && /\[contentUrl\]/{print $3,$4}
f && /\[timeDuration\]/{print $3,$4;exit}
{s=$3 " " $4}'
Result :
Quote:
[timeStamp] '20081114082116'
[msisdn] '38809'
[contentUrl] 'rtsp://172.28/MTV2GO-Loop.sdp'
[timeDuration] 5D



But the problem is the log bigger.. contain the format like that

[APPLICATION 1] { ----> begin

;;;;;;

} ----> end

it mixed from [trigger] 00, [trigger] 01 , and [trigger] 02
and the information from [trigger] 01 there are always no value of [timeDuration]

Or should i give the example

Any solution ??
# 5  
Old 12-05-2008
The problem is not clear to me, can you give an example?

Regards
# 6  
Old 12-05-2008
Shell script to parsing log

Maybe you can see the log here


and i want that log become just only "[trigger] 02"
and then collect into Timestamp,msisdn,timeduration,contentUrl

the log always begin
[APPLICATION 1] {

and end with }

should i separate per index and then

to make it clearly you can see here

i still want that log become just only "[trigger] 02"
and then collect into Timestamp,msisdn,timeduration,contentUrl

If you see the stream always begin with :
[APPLICATION 1] { ----> begin

;;;;;;

} ----> end

Should i separate into index then,,

Regards, Smilie

Last edited by vbe; 12-05-2008 at 01:19 PM.. Reason: xduplicate because of moderation status
# 7  
Old 12-05-2008
Do you want the output of all the sections with "[trigger] 02"?
Post the desired output.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

parsing using shell script

I have a file parameters.txt which contains 151524 151525 I have another file OID.csv which contains NE Version Object Type ID SDK param name Object OID test1 Start: 4.2 End: 4.2 pan 151524 speed ... (5 Replies)
Discussion started by: LavanyaP
5 Replies

2. Shell Programming and Scripting

XML parsing using shell script

I have a xml file like this <bul:collectionStrategy name="strategy1"> <bul:collectionTemplateGroup name="15min group"/> <bul:collectionTemplateGroup name="hourly group"/> </bul:collectionStrategy> <bul:CollectionTemplateGroup name="hourly group" > ... (2 Replies)
Discussion started by: LavanyaP
2 Replies

3. Shell Programming and Scripting

XML parsing in a shell script.

Below is a XML I have... <?xml version="1.0" encoding="UTF-8" ?> <component xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:XXXXX-www-Install-Manifest manifest.xsd" xmlns="urn:qqqqq-Install-Manifest" name="OM" ... (1 Reply)
Discussion started by: dashok.83
1 Replies

4. Shell Programming and Scripting

Parsing XML using Shell Script

Hello, I'm a starting shell scripter and no Perl knowledge. I've trying to do this for a while: I want to parse an XML file and get certain data out of it and write that data into a CSV file, all this using Shell Scripting (in Bash). Or Perl without any XML Parser/Interpreter (if possible). ... (1 Reply)
Discussion started by: Kage Musha
1 Replies

5. Shell Programming and Scripting

Performance of log parsing shell script very slow

Hello, I am an absolute newbie and whatever I've written in the shell script (below) has all been built with generous help from googling the net and this forum. Please forgive any schoolboy mistakes. Now to the qn, my input file looks like this - 2009:04:03 08:21:41:513,INFO... (7 Replies)
Discussion started by: sowmitr
7 Replies

6. UNIX for Dummies Questions & Answers

shell script parsing with sed

#I'm quite new to scripting and my boss has asked me to solve a simple problem and sadly, I can't figure out how to do it. Any help is appreciated. :confused: #The following is a small shell script and the output that it produces for google.com. #!/bin/sh whois $1 | grep "Name Server"... (5 Replies)
Discussion started by: jjamd64
5 Replies

7. Shell Programming and Scripting

Parsing a line in Shell Script

I have record line somthing like below with first line showing char spacing not real record line 1 | 2 | 3rd Field--------------|-4th field--| This is charcatersapcing of line DF20000000000000000130.7890000000750 I shoudl get two line from above line 1st line should 1 | 2 | 3rd... (3 Replies)
Discussion started by: unishiva
3 Replies

8. Shell Programming and Scripting

Shell script for parsing 300mb log file..

am relatively new to Shell scripting. I have written a script for parsing a big file. The logic is: Apart from lot of other useless stuffs, there are many occurances of <abc> and corresponding </abc> tags. (All of them are properly closed) My requirement is to find a particular tag (say... (3 Replies)
Discussion started by: gurpreet470
3 Replies

9. Shell Programming and Scripting

Parsing a file in Shell Script

Hi, I have a requirement. I have an application which can take a file as inputs. Now the file can contain any number of lines. The tool has to pick up the first uncommented line and begin processing it. For example the file could be like this: #MANI123|MANI1234 #MANI234|MANI247... (4 Replies)
Discussion started by: sendhilmani123
4 Replies

10. Shell Programming and Scripting

shell script argument parsing

how to parse the command line argument to look for '@' sign and the following with '.'. In my shell script one of the argument passed is email address. I want to parse this email address to look for correct format. rmjoe123@hotmail.com has '@' sign and followed by a '.' to be more... (1 Reply)
Discussion started by: rmjoe
1 Replies
Login or Register to Ask a Question