Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Get a part of a String from a log file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Get a part of a String from a log file
# 1  
Old 12-09-2010
Get a part of a String from a log file
Hey there,

I'm searched for one day your forum for a similar solution. Didn't find one, sorry Smilie
Now here is what I'm searching for. I have the following multiple lines from a log (Edit: log name is SystemOut.log - this is not my day. I Apologize):

Code:
000042e2 1_BLA_Yab I logging.LogInfo logInfo 34721747: Excecution Time:16811 ms, id: _AB:9003012a.bdea3d8b.75efe9d1.e3184cd8

I want to read the number 16811 and check if it is bigger then 40000. If so, close an special port on the server. Easy, that's the whole thing, don't expect getting it from you. I'm happy to get the number (will dance in the office).
Edit: Trick is, the log is about 13Mb big and contains old log data. So it will shut down immediately if it find a old 40000. For this reason I choosed tail because it doesn't matter if we miss some large ms numbers. Sorry I don't mentioned this before.

Here is what I have so far:
Code:
tail -f SystemOut.log | perl -pe 's/Excecution Time:.*ms/\e[6;31;43m$&\e[0m/g'

This little Perl code (on AIX 6.2), marks "Excecution Time:16811" yellow and red. this work is done manually atm. So I think you get the plan. It has not to be perl, any other solution will be fine. It's just I'm a little rookie to Perl an a noob for anything else on AIX.

Thank you for some help or advice.
greets nobodyhere
Last edited by nobodyhere; 12-10-2010 at 05:08 AM..
# 2  
Old 12-09-2010
Code:
# a='000042e2 1_BLA_Yab I logging.LogInfo logInfo 34721747: Excecution Time:16811 ms, id: _AB:9003012a.bdea3d8b.75efe9d1.e3184cd8'
# b=${a% ms,*}
# t=${b##*:}
# echo $t
16811

Code:
# echo $a | sed 's/.*cution Time://;s/ ms.*//'
16811

Last edited by ctsgnb; 12-09-2010 at 10:42 AM..
This User Gave Thanks to ctsgnb For This Post:
nobodyhere
# 3  
Old 12-09-2010
Thanks for the fast help ctsgnb!

Started and watched it working,
Code:
echo $a | sed 's/.*cution Time://;s/ ms.*//' SystemOut.log

fine thing. Problem ist, the script is reading the whole log file. Is there a option to get the last lines like tail?

I tried
Code:
tail -f SystemOut.log | echo $a | sed 's/.*cution Time://;s/ ms.*//'

but nothing happened. What did I wrong?
Last edited by nobodyhere; 12-10-2010 at 05:09 AM..
# 4  
Old 12-09-2010
To process all lines in log having Excecution Time:"
Code:
grep "Excecution Time:" SystemOut.log | sed 's/.*Excecution Time:\(.*\) ms.*/\1/'

OR
Code:
grep "Excecution Time:" SystemOut.log | sed 's/.*Excecution Time://;s/ ms.*//'

To process last 100 lines in log having Excecution Time:"
Code:
tail -100 SystemOut.log | grep "Excecution Time:" | sed 's/.*Excecution Time:\(.*\) ms.*/\1/'

OR
Code:
tail -100 SystemOut.log | grep "Excecution Time:" | sed 's/.*Excecution Time://;s/ ms.*//'

Last edited by anurag.singh; 12-10-2010 at 05:31 AM..
This User Gave Thanks to anurag.singh For This Post:
nobodyhere
# 5  
Old 12-09-2010
Code:
# tail -1 <(sed -n 's/.*cution Time://;s/ ms.*//p' logfile)
16811

or , if the <(...) notation doesn't work , you can do :

Code:
sed -n 's/.*cution Time://;s/ ms.*//p' logfile | tail -1

or
Code:
awk -F'[: ]' '/cution Time:/{x=$(NF-5)}END{print x}' logfile

or
Code:
nawk -F'[: ]' '/cution Time:/{x=$(NF-5)}END{print x}' logfile

Last edited by ctsgnb; 12-09-2010 at 11:47 AM..
# 6  
Old 12-10-2010
Thanks Anurag and thanks again ctsgnb!

This will take some time to understand, never worked with shell scripts before. So I better start reading to finish this ...

greets
nobody
Last edited by nobodyhere; 12-10-2010 at 05:30 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to extract part of string from all log files.?

Hi All, Let us say we have 5 log files, extract the data from all log files and save the output in a file. home/log/first.log home/log/second.log home/log/third.log home/log/four.log home/log/five.log I want to extract the following text from the log files and save the output in a file.... (7 Replies)
Discussion started by: ROCK_PLSQL
7 Replies

2. Shell Programming and Scripting

Print particular string in a field of csv file - part 2

Hi, all I need your help and suggestions. I want to print particular strings in a field of a csv file and show them in terminal. Here is an example of the csv file. SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw... (7 Replies)
Discussion started by: refrain
7 Replies

3. Shell Programming and Scripting

Grep a part of file based on string identifiers

consider below file contents cat myOutputFIle.txt 8 CCM-HQE-ResourceHealthCheck: Resource List : No RED/UNKNOWN resource Health entries found ---------------------------------------------------------- 9 CCM-TraderLogin-Status: Number of logins: 0... (4 Replies)
Discussion started by: vivek d r
4 Replies

4. Shell Programming and Scripting

Getting the date part from the log file.

Hi, I have a script that runs to generate a log file which is in this format: Start: Friday, April 29, 2011 18:30 User : Host : Database : I need to write a script that reads this log file (RUN.LOG) and rename the existing log file... (7 Replies)
Discussion started by: ashok@119
7 Replies

5. Shell Programming and Scripting

Get part of the current file name as a string

Hi All, I've been searching through existing post and have been able to find most of what I want but am still stuck... I'm trying to get part of the current file name as a string. The file name is in the form of: basefilename_xxx(...).sh I need the bit xxx(...) and it varies in length. ... (6 Replies)
Discussion started by: imonkey
6 Replies

6. Shell Programming and Scripting

Finding a string in a text file and posting part of the line

What would be the most succinct way of doing this (preferably in 1 line, maybe 2): searching the first 10 characters of every line in a text file for a specific string, and if it was found, print out characters 11-20 of the line on which the string was found. In this case, it's known that there... (13 Replies)
Discussion started by: busdude
13 Replies

7. Shell Programming and Scripting

Tailing last modified part of log file

I have a log file which contains data like this This log file is updated twice a day at 7am and 6pm, I want a script(which i will make run at 7:10am and 6:10pm) which should fetch only the last appended lines since last update.. I mean.. if i execute the script at 7.10am 3/3/2010 it... (4 Replies)
Discussion started by: user__user3110
4 Replies

8. Shell Programming and Scripting

get the part of log between 2 string

Hi Let's say I have a log file with this format: Marker1 End of Marker1 .... Marker1 End of Marker1 .... Marker1 End of Marker1 how to get the last part between End of Marker1 and Marker1? (1 Reply)
Discussion started by: melanie_pfefer
1 Replies

9. UNIX for Dummies Questions & Answers

replace part of single string in a file

hi! i have a file consisting of the following lines: (BTW, = space) . . . 12ME_T1mapping_flip30bshortf 13DCE_whole_brainbshortf 13DCE_3Dbshortf . . . the list of scans starts at 1 and goes on sometimes up to 60 scans. i would like to change only the lines that contain 'whole' to... (2 Replies)
Discussion started by: nixjennings
2 Replies

10. Shell Programming and Scripting

How to extract certain part of log file?

Hi there, I'm having some problem with UNIX scripting (ksh), perhaps somebody can help me out? For example: ------------ Sample content of my log file (text file): -------------------------------------- File1: .... info_1 ... info_2 ... info_3 ... File2: .... info_1 ... info_2 ...... (10 Replies)
Discussion started by: superHonda123
10 Replies
Login or Register to Ask a Question