Grep a log file for the last 5 minutes of contents every 5 minutes Post: 303030331

Sponsored Content

Top Forums Shell Programming and Scripting Grep a log file for the last 5 minutes of contents every 5 minutes Post 303030331 by charli1 on Friday 8th of February 2019 06:09:47 AM

02-08-2019

Registered User

Grep a log file for the last 5 minutes of contents every 5 minutes

Hi all,
System Ubuntu 16.04.3 LTS
i have the following log

Code:

INFO     2019-02-07 15:13:31,099 module.py:700] default: "POST /join/8550614e-3e94-4fa5-9ab2-135eefa69c1b HTTP/1.0" 500 2042
INFO     2019-02-07 15:13:31,569 module.py:700] default: "POST /join/6cb9c452-dcb1-45f3-bcca-e33f5d450105 HTTP/1.0" 500 2042
INFO     2019-02-07 15:14:25,379 module.py:700] default: "POST /join/f30c767d-fcd2-44c2-83a9-d927fv10f7a8 HTTP/1.0" 500 2042
INFO     2019-02-07 15:14:46,531 module.py:700] default: "POST /join/57e7930c-99ea-419a-b807-fad6327cd6e9 HTTP/1.0" 500 2042
INFO     2019-02-07 15:15:06,966 module.py:700] default: "POST /join/57e7930c-99ea-419a-b807-fad6447cd6e9 HTTP/1.0" 500 2042
INFO     2019-02-07 15:15:06,966 module.py:700] default: "POST /join/57e793rfc-98ea-418a-b817-fad6347cd6e9 HTTP/1.0" 500 2042

i need to create a monitor for this log and grep only last 5 minutes of this log HTTP/1.0" 500 2042, and make an action if the match count is higher then 5 in last 5 minutes.

I tyied the following script but for some reason i can't get it working

Code:

#!/bin/bash
#
check=$(awk -v d1="$(date --date="-5 min" "+%Y-%m-%d %H:%M:%S,%3N")" -v d2="$(date "+%Y-%m-%d %H:%M:%S,%3N")" '$0 > d1 && $0 < d2 || $0 ~ d2' /var/log/messages | grep -i "HTTP/1.0" 500 2042")
count=$(awk -v d1="$(date --date="-5 min" "+%Y-%m-%d %H:%M:%S,%3N")" -v d2="$(date "+%Y-%m-%d %H:%M:%S,%3N")" '$0 > d1 && $0 < d2 || $0 ~ d2' /var/log/messages | grep -ci "HTTP/1.0" 500 2042") 
if [ $count -gt 5 ] 
then
   echo $CHECK | /bin/mail -s "$count occurrences of the error message has been found in the last 5 minutes" myemail@mydomain.com 
else
   echo "Exit, everything is ok" > /dev/null
fi

The error i gues is with grep because there are the " in the middle.
Could somebody help me on this what i'm i missing here?
Thank you in advance.

Last edited by charli1; 02-08-2019 at 08:06 AM..

charli1

View Public Profile for charli1

Find all posts by charli1

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert minutes to hours, minutes, seconds

How would you convert lets say a 1000 minutes to hours, minutes, seconds

2. Shell Programming and Scripting

Grepping the last 30 minutes of a log file...

I need to know if anyone can assist me on how to grab the last (we'll just say "x" minutes) of a log file. How do you tell the grep command without specifying an exact window of time? (So relative instead of absolute.) Thanks, Jon

3. Shell Programming and Scripting

Script to grep for a string in log files generated in last 15 minutes.

Dear Guru's I've a requirment to grep for a string in series of log files that are getting generated almost every minute. I'm looking to schedule a script every 15 mountes,in order to check if the error string has been generated in any of the log files generated in last 15 minutes. Please...

4. Shell Programming and Scripting

Log File Creations for every 60 minutes

Hi All, Below script will make a copy of the existing log file with the then timestamp details. I am looking to create a copy of the existing log file for every 60 minutes and when the file limit reaches to 5, the 6th copy should overwrite the first backedup file which means all the time it...

5. Shell Programming and Scripting

Grep last 30 minutes log only

I have below command, which i use for checking last two hours log file, but i want to check 30 minutes log and discard old log, only match current time with last 30 minutes. Command i am using. This below attach log file format is for this week, but sometime it got change in time of log, so i need...

6. Shell Programming and Scripting

Need help in getting the Last 30 minutes logs from the Log File

I have a log file with the below contents : log_file_updated.txt : Jul 5 03:33:06 rsyslogd: was Jul 5 03:33:09 adcsdb1 rhsmd: This system is registered. Sep 2 02:45:48 adcsdb1 UDSAgent: 2015-07-05 04:24:48.959 INFO Worker_Thread_4032813936 Accepted connection from host <unknown>...

7. Shell Programming and Scripting

Grep last 2 minutes log only

I have newbie, which i use for checking last one hours log file, but i want to check 2 minutes log and discard old log, only match current time with last 2 minutes. Ex log. 2018-07-03 20:09:17 2018-07-03 20:05:17 2018-07-03 20:05:18 2018-07-03 20:05:20 2018-07-03 20:06:22 2018-07-03...

8. UNIX for Beginners Questions & Answers

How to convert days hours minutes seconds to minutes?

Hi, please help with below time conversion to minutes. one column values: 2 minutes 16 seconds 420 msec 43 seconds 750 msec 0 days 3 hours 29 minutes 58 seconds 480 msec 11 seconds 150 msec I need output in minutes(total elapsed time in minutes)

9. Shell Programming and Scripting

Check file creation Time minutes and if file older then 5 minutes execute some stuff

Hello all, Info: System RedHat 7.5 I need to create a script that based on the creation time, if the file is older then 5 minutes then execute some stuff, if not exit. I thought to get the creation time and minutes like this. CreationTime=$(stat -c %y /tmp/test.log | awk -F" " '{ print...

10. UNIX for Beginners Questions & Answers

Copy last 30 minutes' contents from a log file

Hi Guys, I am writing a bash script to capture the last 30 minutes's contents from log file to a new file. This job is a scheduled job and will run every 30 minutes. The log file is db2diag.log in DB2. I am having difficulties copying the last 30 minutes's contents. Can someone please help me....

LEARN ABOUT MOJAVE

curlopt_postfieldsize_large

CURLOPT_POSTFIELDSIZE_LARGE(3)				     curl_easy_setopt options				    CURLOPT_POSTFIELDSIZE_LARGE(3)

NAME

       CURLOPT_POSTFIELDSIZE_LARGE - size of POST data pointed to

SYNOPSIS

       #include <curl/curl.h>

       CURLcode curl_easy_setopt(CURL *handle, CURLOPT_POSTFIELDSIZE_LARGE,
				 curl_off_t size);

DESCRIPTION

       If  you	want to post data to the server without having libcurl do a strlen() to measure the data size, this option must be used. When this
       option is used you can post fully binary data, which otherwise is likely to fail. If this size is set to -1, the library will use  strlen()
       to get the size.

DEFAULT

       -1

PROTOCOLS

       HTTP(S)

EXAMPLE

       CURL *curl = curl_easy_init();
       if(curl) {
	 const char *data = large_chunk;
	 curl_off_t length_of_data; /* set somehow */

	 curl_easy_setopt(curl, CURLOPT_URL, "http://example.com");

	 /* size of the POST data */
	 curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE_LARGE, length_of_data);

	 curl_easy_setopt(curl, CURLOPT_POSTFIELDS, data);

	 curl_easy_perform(curl);
       }

AVAILABILITY

       Along with HTTP

RETURN VALUE

       Returns CURLE_OK if HTTP is supported, and CURLE_UNKNOWN_OPTION if not.

SEE ALSO

       CURLOPT_POSTFIELDS(3), CURLOPT_COPYPOSTFIELDS(3), CURLOPT_POSTFIELDSIZE(3),

libcurl 7.54.0							 February 03, 2016				    CURLOPT_POSTFIELDSIZE_LARGE(3)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert minutes to hours, minutes, seconds

Discussion started by: Vozx

2. Shell Programming and Scripting

Grepping the last 30 minutes of a log file...

Discussion started by: jtelep

3. Shell Programming and Scripting

Script to grep for a string in log files generated in last 15 minutes.

Discussion started by: rajivatnova

4. Shell Programming and Scripting

Log File Creations for every 60 minutes

Discussion started by: Upendra Bestha