Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

Script to read log file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Script to read log file
# 1  
Old 03-01-2012
Script to read log file
Hi,

Im looking for a shell script which will search for a particular string in a log file as below scenario

1. I need to run URL http://localhost/client/update?feedid=200 in shell script at(eg)4:00 PM which will not take more than 15 mins to complete.

2. After 15 mins i need to check the log from 4:00 PM to 4:15 PM.

3. Need to check logs file for 2 strings (a) Feed id=200 (b) Item=Product example, if these 2 string matches i need to check for "Error" string.

4. If 3 string matches i will send a mail stating "Failed"

5. If string "Error" is not found in last 15 mins log(only string a & b should match) then i will execute another 6 URLS and atlast will trigger mail stating "Success"

6. Is this above sequence possible in shell script? Please guide me and my script is below - Im really stuck in comparing string and checking last 15 mins log.

7. Log with Error
Code:
[INFO 12-03-01:04:10:25] Starting FeedRunner for feed: 200  Product example Items 
[INFO 12-03-01:04:12:40] Error FeedRunner for feed: 200 Product example Items
[INFO 12-03-01:04:20:25] Stopping FeedRunner for feed : 200  Product example Items Feed time taken 231743

8. Log without Error
Code:
[INFO 12-03-01:04:10:25] Starting FeedRunner for feed: 200  Product example Items 
[INFO 12-03-01:04:20:25] Stopping FeedRunner for feed : 200  Product example Items Feed time taken 2743

9. Script
Code:
#!/bin/sh

START=$(date +"%y-%m-%d:%T")
echo $START
curl "http://localhost/mcfeeds/runfeed?feedid=200"
END=$(date "%y-%m-%d:%T")
echo $END

cat /path/logfile.log | sed -n '/$START/,/$END/ p' > output.txt

grep '200'&'Product example'&'Error' output.txt

Thanks
Paulwintech
Last edited by Paulwintech; 03-02-2012 at 03:03 AM..
# 2  
Old 03-01-2012
I don't think there's any point making 'start' and 'end' dates like you have there. It'll only take a fraction of a second to get the feed, and those dates aren't useful in comparing anything in the log file.

What's your system? What's your shell? On Linux, GNU awk has features allowing you to get epoch time from timestamps, which is very handy for comparing how long ago things happened...
# 3  
Old 03-02-2012
Hi Corona688,

Thanks you!!! Please let me know any example to compare and to search in log file.

Regards
Paulwintech

---------- Post updated at 11:05 AM ---------- Previous update was at 10:43 AM ----------

Hi,

Is my script right now? Please help me if im wrong

Code:
#!/bin/bash

SUBJECT="SET-EMAIL-SUBJECT"
EMAIL="admin@somewhere.com"

START=$(date +"%y-%m-%d:%T")
echo $START
curl "http://localhost/mcfeeds/runfeed?feedid=200"
END=$(date "%y-%m-%d:%T")
echo $END

cat /path/logfile.log | sed -n '/$START/,/$END/ p' > output.txt

if [egrep -i '200|Product example|Error' output.txt]
then 
/bin/mail -s "$SUBJECT" "$EMAIL" < "Feed Failed"
else
curl "URL1"
curl "URL1"
curl "URL1"
curl "URL1"
curl "URL1"
curl "URL1"
/bin/mail -s "$SUBJECT" "$EMAIL" < "Feed Success"
fi

Last edited by Paulwintech; 03-05-2012 at 01:19 AM..
# 4  
Old 03-02-2012
Please tell us more about the logfile /path/logfile.log.
Does the program append to the log every time it is run or is the log created from scratch?
If the log is appended you could consider adding your own entries to the log containing the values of your $START and $END variables such that you can find them with "sed".

Please reply to the above request: What Operating System and version do you have and what Shell are you using?

There are multiple logic and syntax errors in your most recent script.
Last edited by methyl; 03-02-2012 at 08:07 AM..
# 5  
Old 03-02-2012
Quote:
Originally Posted by Paulwintech
Hi Corona688,

Thanks you!!! Please let me know any example to compare and to search in log file.

Regards
Paulwintech
I asked you what your system was.

This was not an off-hand question. Different systems have very different ways of working with dates. Any answer I give you right now has good odds of being useless to you, since it's for the wrong system.

Please answer my questions so I can help you.
# 6  
Old 03-05-2012
Hi Corona688/methyl,

Operating System and version
CentOS release 5.5 (Final)

Shell are you using?
#!/bin/bash

Does the program append to the log every time it is run or is the log created from scratch?
--Yes it updates the log every time it runs

Thanks
Paulwintech
# 7  
Old 03-09-2012
Quote:
Does the program append to the log every time it is run or is the log created from scratch?
--Yes it updates the log every time it runs
Best approach is to make your own entries to the log from shell. One before the run and one after the run. Stick to the syntax of the logfile itself for the left hand side of the logfile entry, but include your own unique start and end string to your special lines in order to make it possible to use your original "sed" syntax.
I would put your unique search string in the text portion of the message and use only alphanumeric characters. You just need to include the date but in numeric yyyymmddhhmmss format and something to distinguish start from end.
For example
Code:
[INFO 12-03-01:04:10:24] S20120301041024
[INFO 12-03-01:04:10:25] Starting FeedRunner for feed: 200  Product example Items 
[INFO 12-03-01:04:12:40] Error FeedRunner for feed: 200 Product example Items
[INFO 12-03-01:04:20:25] Stopping FeedRunner for feed : 200  Product example Items Feed time taken 231743
[INFO 12-03-01:04:10:26] E20120301041026

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to read a log file and run 2nd script if the dates match

# cat /tmp/checkdate.log SQL*Plus: Release 11.2.0.1.0 Production on Mon Sep 17 22:49:00 2012 Copyright (c) 1982, 2009, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production FIRST_TIME NEXT_TIME... (1 Reply)
Discussion started by: SarwalR
1 Replies

2. Shell Programming and Scripting

HELP: Shell Script to read a Log file line by line and extract Info based on KEYWORDS matching

I have a LOG file which looks like this Import started at: Mon Jul 23 02:13:01 EDT 2012 Initialization completed in 2.146 seconds. -------------------------------------------------------------------------------- -- Import summary for Import item: PolicyInformation... (8 Replies)
Discussion started by: biztank
8 Replies

3. Shell Programming and Scripting

script to constantly read the last 500 new logs in a log file

Hello, I would like to write a bash script that would monitor a log file for a certain number of logs, let's say 500 logs and when it reaches that number to write the last log to another file. For example, I want to watch the /var/adm/messages and everytime, there is 500 new logs that are... (1 Reply)
Discussion started by: Pouchie1
1 Replies

4. Shell Programming and Scripting

Read from Log file in Ksh

I have a log file like.. IMPORT from /dataserver/ftp/bits/mdr/mdr_data_discon.dat OF DEL ..... Number of rows read = 1376 Number of rows skipped = 0 Number of rows inserted = 1374 Number of rows updated = 0 Number of rows rejected = 2 Number of rows... (4 Replies)
Discussion started by: ramse8pc
4 Replies

5. Shell Programming and Scripting

Unix script help to read log file

Hi I have a big log file :08,936 DEBUG HttpConnectionManager.getConnection: config = 11:39:08,936 DEBUG Getting free connection, 11:39:08,989 DEBUG Freeing connection, hostConfig=HostConfiguration 11:39:08,989 DEBUG Notifying no-one, there are no waiting threads 11:39:09,046... (4 Replies)
Discussion started by: javaholics
4 Replies

6. Shell Programming and Scripting

Help w/ script to read file and parse log message

Hi, I am working on the script to parsing the specific message like "aaaa" in multiple log files like N1-***,N2-***,N3-***... The script is to find the list of lof files which contains the message "aaaa" and export the list into excel filE. Can anyone give help? Thanks (2 Replies)
Discussion started by: shyork2001
2 Replies

7. Solaris

Read zipped log file

If we have a big zipped log file, how can we look for a specific string in this zipped log file without unzipping it? Thanks, (2 Replies)
Discussion started by: Pouchie1
2 Replies

8. Shell Programming and Scripting

shell script to read a line in gps receiver log file and append that line to new file

Hi, I have gps receiver log..its giving readings .like below Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GPSD,R=1 $GPGSV,3,1,11,08,16,328,40,11,36,127,00,28,33,283,39,20,11,165,00*71... (3 Replies)
Discussion started by: gudivada213
3 Replies

9. Shell Programming and Scripting

Shell script to read multiple log files

Hi all, I have to generate some report from shell script .We have stacktrace log file which generate hourly basis. So now my q is that how this shell script will read all stacktrace log file for particlular day and parse accordingly desire output. Any help or suggestion as i am newbie with... (1 Reply)
Discussion started by: esungoe
1 Replies

10. Shell Programming and Scripting

How to read a specific value from a Log file?

Hi, I have a .log file in which it has many values. But i need some specific values. How it can be done using Shell Script. Please explain in detail. Thankx in advance. Sathish D V. (8 Replies)
Discussion started by: cooolthud
8 Replies
Login or Register to Ask a Question