Finding errors in log file only in last 10 minutes


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Finding errors in log file only in last 10 minutes
# 1  
Old 09-12-2011
Finding errors in log file only in last 10 minutes

Hi there,

I have a log file that I need to check every 10 minutes to find if a specific error exists but only in that 10 minute period. The reason is that the log is quite large, and will frequently contain these errors, so I only want alerting if it in the last 10 minutes - I don't want continually to be alerted for the rest of the day once this error happens.

I'm ok with all the alerting aspects, and setting up a cronjob to run every 10 mins, and mail me if the alert is triggered, the problem I am having is just identifying if this error exists in the last 10 minutes of the log file.

I've had a look through the forum, and can see similar posts, but none of them seem to work for me. I think I need to use awk to check if the datetime in the log is greater than 10 minutes ago, this is my sticking point.

I know I can get the time of 10 minutes ago by doing this...

Code:
date=`date --date='10 minutes ago' "+%Y-%m-%d %H:%M:%S"`

and my logfile with it's errors looks like this (The timestamps are exact, but I've amended the actual error though...)

Code:
12-09-2011 11:58:43 ERROR: Application_name - some other error
12-09-2011 11:58:43 ERROR: Application_name - some other error
12-09-2011 12:03:03 ERROR: Application_name - error I'm looking for
12-09-2011 12:03:03 ERROR: Application_name - some other error
12-09-2011 12:05:16 ERROR: Application_name - some other error
12-09-2011 12:14:22 ERROR: Application_name - error I'm looking for
12-09-2011 12:14:22 ERROR: Application_name - some other error
12-09-2011 12:15:08 ERROR: Application_name - some other error
12-09-2011 12:15:08 ERROR: Application_name - some other error

I'm ok with things like awk and grep to find the error, it's just coming up with an awk command to find the error *in the last 10 minutes* I'm struggling with. I thought it was just a matter of getting the current time - 10 mins into a variable, then comparin that with awk but the things I've tried either return everything or nothing.

I'm reletively new to shell scripting, and this is my first post (so apologies if I've done anything stupid!)

Thanks in advance,
Paul B
# 2  
Old 09-12-2011
Try something like this;
Code:
tac INPUTFILE | awk -v fin=120400 '{
  t=$2
  gsub(/:/, "", t)
  if (t < fin) exit 
  if ($0 ~ /error I'"'"'m looking for/) print
}'

# 3  
Old 09-12-2011
Hi,

This is good, and does do what I'm after, thanks very much Smilie

However, I was just wondering, how would I change it to print *all* the errors that have happened in the last 10 minutes? The above (if I understand it correctly) will just return the most recent line which is true. (This is fine though, as it will work for my alerting - it'd just be also handy if I knew how many times this error has occurred in the last 10 mins).

Thanks again,
Paul B

---------- Post updated at 01:43 PM ---------- Previous update was at 01:35 PM ----------

Sorry, I think I'm being stupid, that is already what this does, I just appeared to run it on a log file that only had one error in the last 10 minutes...

Sorry!

Thanks again for the help.

Paul B
# 4  
Old 09-12-2011
Ok. Deleted.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Copy last 30 minutes' contents from a log file

Hi Guys, I am writing a bash script to capture the last 30 minutes's contents from log file to a new file. This job is a scheduled job and will run every 30 minutes. The log file is db2diag.log in DB2. I am having difficulties copying the last 30 minutes's contents. Can someone please help me.... (4 Replies)
Discussion started by: naveed
4 Replies

2. Shell Programming and Scripting

Check file creation Time minutes and if file older then 5 minutes execute some stuff

Hello all, Info: System RedHat 7.5 I need to create a script that based on the creation time, if the file is older then 5 minutes then execute some stuff, if not exit. I thought to get the creation time and minutes like this. CreationTime=$(stat -c %y /tmp/test.log | awk -F" " '{ print... (3 Replies)
Discussion started by: charli1
3 Replies

3. Shell Programming and Scripting

Grep a log file for the last 5 minutes of contents every 5 minutes

Hi all, System Ubuntu 16.04.3 LTS i have the following log INFO 2019-02-07 15:13:31,099 module.py:700] default: "POST /join/8550614e-3e94-4fa5-9ab2-135eefa69c1b HTTP/1.0" 500 2042 INFO 2019-02-07 15:13:31,569 module.py:700] default: "POST /join/6cb9c452-dcb1-45f3-bcca-e33f5d450105... (15 Replies)
Discussion started by: charli1
15 Replies

4. Shell Programming and Scripting

Finding latest file in dir but getting syntax errors

I believe there are couple of syntax issues in my script, couldn't find them :( can someone help me with fixing it to make it work. cd /abcde/ #get the latest filename excluding subdirs filename=`ls -ltr | grep ^- | tail -1 | awk '{print $8}'` #get system date and file timestamp and... (3 Replies)
Discussion started by: simpltyansh
3 Replies

5. Shell Programming and Scripting

Need help in getting the Last 30 minutes logs from the Log File

I have a log file with the below contents : log_file_updated.txt : Jul 5 03:33:06 rsyslogd: was Jul 5 03:33:09 adcsdb1 rhsmd: This system is registered. Sep 2 02:45:48 adcsdb1 UDSAgent: 2015-07-05 04:24:48.959 INFO Worker_Thread_4032813936 Accepted connection from host <unknown>... (3 Replies)
Discussion started by: rahul2662
3 Replies

6. Shell Programming and Scripting

Script to search for a pattern in 30 minutes from a log file

Hello All, I have to write a script which will search for diffrent patterns like "Struck" "Out of Memory" , etc from a log file in Linux box's. Now I will be executing a cron job to find out the results by executing the script once in every 30 minutes. suppose time is 14-04-29:05:31:09 So I... (3 Replies)
Discussion started by: Shubhasis Mathr
3 Replies

7. Shell Programming and Scripting

Log File Creations for every 60 minutes

Hi All, Below script will make a copy of the existing log file with the then timestamp details. I am looking to create a copy of the existing log file for every 60 minutes and when the file limit reaches to 5, the 6th copy should overwrite the first backedup file which means all the time it... (3 Replies)
Discussion started by: Upendra Bestha
3 Replies

8. Shell Programming and Scripting

Grepping the last 30 minutes of a log file...

I need to know if anyone can assist me on how to grab the last (we'll just say "x" minutes) of a log file. How do you tell the grep command without specifying an exact window of time? (So relative instead of absolute.) Thanks, Jon (2 Replies)
Discussion started by: jtelep
2 Replies

9. Shell Programming and Scripting

get last 5 minutes' log from log file?

Hi all, I have tried to figure out a way to automatically get the last 5 minutes' log content from log file, at first, my thoughts like this, sed -n "/ $(date +\%R -d "-5 min")/,$"p syslog > newfile, but quickly I found it did not work, say I have a syslog file as following, Jul 19... (5 Replies)
Discussion started by: fedora
5 Replies

10. Shell Programming and Scripting

finding files only minutes old?

I have files that are being created in a directory constantly. I want to be able write a script that would check this directory every 15 minutes and copy only the files that were newer than the files that were copied the previous 15 minutes. On linux, I found this to be as easy as running the... (2 Replies)
Discussion started by: cstovall
2 Replies
Login or Register to Ask a Question