Grep a log file for the last 5 minutes of contents every 5 minutes


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grep a log file for the last 5 minutes of contents every 5 minutes
# 8  
Old 02-08-2019
It doesn't work neither,
any other ideas?
# 9  
Old 02-08-2019
It works for me with your log file. Try for example to increase the time interval from 5 minutes to 500

--- Post updated at 15:05 ---

comment out all lines leave only this in the script
Code:
awk -F, -v d="$(date --date="-5000 minutes" +"%Y-%m-%d %H:%M:%S,%3N")" \
'$0 ~ "HTTP/1.0\" 500 2042$" {if ( d < substr($1, 10)) error++} END {print error}' file

If not working, try to change the filter or index substr 10

Code:
awk -F, -v d="$(date --date="-5000 minutes" +"%Y-%m-%d %H:%M:%S,%3N")" \
'$0 ~ "500 2042$" {if ( d < substr($1, 10)) print}' file

# 10  
Old 02-08-2019
Code:
#!/bin/sh
check=$(awk -F, -v d="$(date --date="-5 minutes" +"%Y-%m-%d %H:%M:%S,%3N")" \
'$0 ~ "HTTP/1.0\" 500 2042$" {if ( d < substr($1, 10)) error++} END {print error}' /tmp/test.log)

if [ $check -gt 5 ]
then
   echo "Bad News, Need to send Alert"
else
   echo "Exit, everything is ok"
fi

So i have the following error now,


Code:
sh -x test.sh
+ date --date=-5 minutes +%Y-%m-%d %H:%M:%S,%3N
+ awk -F, -v d=2019-02-08 15:12:29,459 $0 ~ "HTTP/1.0\" 500 2042$" {if ( d < substr($1, 10)) error++} END {print error} /tmp/test.log
+ check=
+ [ -gt 5 ]
test.sh: 5: [: -gt: unexpected operator
+ echo Exit, everything is ok
Exit, everything is ok

Any ideas?
Thank you.
# 11  
Old 02-08-2019
I again made a mistake, did not take nanoseconds into account. Remove them from the template
Code:
date --date="-5 minutes" +"%Y-%m-%d %H:%M:%S,%3N"

Together with a comma. They are not needed

--- Post updated at 16:08 ---

Code:
awk -F, -v d="$(date --date="-500minutes" +"%Y-%m-%d %H:%M:%S")" \
        '$0 ~ "HTTP/1.0\" 500 2042$" {if ( d < substr($1, 10)) print}' file

# 12  
Old 02-08-2019
Still having the same issue
Code:
test.sh: 5: [: -gt: unexpected operator
+ echo Exit, everything is ok

How do i fix this?
Thanks
# 13  
Old 02-08-2019
The check= indicates something went wrong with your awk command as it delivered nothing on its stdout to be assigned to the variable. So

- why the "double quotes in double quotes" for the date command?
- why the extra precision in the date format?
- why the comma field separator?
- why the complicated matching?How about
Code:
check=$(awk -v d="$(date --date='-5 minutes' +'%Y-%m-%d %H:%M:%S')" '/HTTP\/1.0" 500 2042$/ {if ( d < $2 " " $3) error++} END {print error}' /tmp/test.log)


Last edited by RudiC; 02-09-2019 at 04:24 AM..
This User Gave Thanks to RudiC For This Post:
# 14  
Old 02-08-2019
And, while we're at it, why not have awk process the entrie stuff for us? Like


Code:
awk -v d="$(date --date='-5 minutes' +'%Y-%m-%d %H:%M:%S')" '
/HTTP\/1.0" 500 2042$/  {if ( d < $2 " " $3) error++
                        }
END                     {if (error >= 5) print "/bin/mail -s \"" error " occurrences ...\" myemail@mydomain.com"
                           else          print "echo \"Exit, everything is ok\""
                        }
'  /tmp/test.log

You'll see the mail or echo happy output depending on your relevant log file error count. If happy with what you see, pipe the output through sh so the resp. command will be executed.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Copy last 30 minutes' contents from a log file

Hi Guys, I am writing a bash script to capture the last 30 minutes's contents from log file to a new file. This job is a scheduled job and will run every 30 minutes. The log file is db2diag.log in DB2. I am having difficulties copying the last 30 minutes's contents. Can someone please help me.... (4 Replies)
Discussion started by: naveed
4 Replies

2. Shell Programming and Scripting

Check file creation Time minutes and if file older then 5 minutes execute some stuff

Hello all, Info: System RedHat 7.5 I need to create a script that based on the creation time, if the file is older then 5 minutes then execute some stuff, if not exit. I thought to get the creation time and minutes like this. CreationTime=$(stat -c %y /tmp/test.log | awk -F" " '{ print... (3 Replies)
Discussion started by: charli1
3 Replies

3. UNIX for Beginners Questions & Answers

How to convert days hours minutes seconds to minutes?

Hi, please help with below time conversion to minutes. one column values: 2 minutes 16 seconds 420 msec 43 seconds 750 msec 0 days 3 hours 29 minutes 58 seconds 480 msec 11 seconds 150 msec I need output in minutes(total elapsed time in minutes) (2 Replies)
Discussion started by: ramu.badugula
2 Replies

4. Shell Programming and Scripting

Grep last 2 minutes log only

I have newbie, which i use for checking last one hours log file, but i want to check 2 minutes log and discard old log, only match current time with last 2 minutes. Ex log. 2018-07-03 20:09:17 2018-07-03 20:05:17 2018-07-03 20:05:18 2018-07-03 20:05:20 2018-07-03 20:06:22 2018-07-03... (5 Replies)
Discussion started by: ooilinlove
5 Replies

5. Shell Programming and Scripting

Need help in getting the Last 30 minutes logs from the Log File

I have a log file with the below contents : log_file_updated.txt : Jul 5 03:33:06 rsyslogd: was Jul 5 03:33:09 adcsdb1 rhsmd: This system is registered. Sep 2 02:45:48 adcsdb1 UDSAgent: 2015-07-05 04:24:48.959 INFO Worker_Thread_4032813936 Accepted connection from host <unknown>... (3 Replies)
Discussion started by: rahul2662
3 Replies

6. Shell Programming and Scripting

Grep last 30 minutes log only

I have below command, which i use for checking last two hours log file, but i want to check 30 minutes log and discard old log, only match current time with last 30 minutes. Command i am using. This below attach log file format is for this week, but sometime it got change in time of log, so i need... (6 Replies)
Discussion started by: learnbash
6 Replies

7. Shell Programming and Scripting

Log File Creations for every 60 minutes

Hi All, Below script will make a copy of the existing log file with the then timestamp details. I am looking to create a copy of the existing log file for every 60 minutes and when the file limit reaches to 5, the 6th copy should overwrite the first backedup file which means all the time it... (3 Replies)
Discussion started by: Upendra Bestha
3 Replies

8. Shell Programming and Scripting

Script to grep for a string in log files generated in last 15 minutes.

Dear Guru's I've a requirment to grep for a string in series of log files that are getting generated almost every minute. I'm looking to schedule a script every 15 mountes,in order to check if the error string has been generated in any of the log files generated in last 15 minutes. Please... (3 Replies)
Discussion started by: rajivatnova
3 Replies

9. Shell Programming and Scripting

Grepping the last 30 minutes of a log file...

I need to know if anyone can assist me on how to grab the last (we'll just say "x" minutes) of a log file. How do you tell the grep command without specifying an exact window of time? (So relative instead of absolute.) Thanks, Jon (2 Replies)
Discussion started by: jtelep
2 Replies

10. Shell Programming and Scripting

Convert minutes to hours, minutes, seconds

How would you convert lets say a 1000 minutes to hours, minutes, seconds (1 Reply)
Discussion started by: Vozx
1 Replies
Login or Register to Ask a Question