Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Script to grep if 404 error got generated in the log files in last 5 minutes Post 303046108 by Pranav_31 on Friday 24th of April 2020 01:20:18 AM
Old 04-24-2020
Script to grep if 404 error got generated in the log files in last 5 minutes

I have a log file which are linked with prefix current date and suffix with IP_address.log 2020_04_22.shared_http_server.192.168.XX.XX.log

I want to monitor these logs at an interval of every 5 minutes. I am looking to schedule a script at an interval of every 5 minutes,in order to grep if 404 error got generated in the log files in last 5 minutes and send the lines of the logs which contains 404 error in different file.

I have written a script to get this done and i am using cron to run this script for every 5 min.Whenever the script runs, it reads the whole log file and throws all the lines of the logs which contains 404 error in different file.But, i want those lines of the logs containing 404 error message to be sent in different file which got generated in last 5 minutes.

Code:
find /home/pranav -type f -name "2020_04_22.shared_http_server.192.168.XX.XX.log" |while read file
  do
    RESULT=$(egrep "[^0](404)" $file)
      if [[ ! -z $RESULT ]]
         then
            echo "Error(s) in $file on $HOSTNAME at "$(date)": $RESULT">> log_result.txt
     fi
  done

This script is able to grep the lines which contains 404 error message but i want those lines of the logs containing 404 error message to be sent in different file which got generated in last 5 minutes.

Could somebody help me on this what i'm i missing here?
Thank you in advance.

Last edited by rbatte1; 04-30-2020 at 04:26 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to Grep column 3 from csv file generated yesterday

Hello, Can any one please assist how to scirpt it: Every day a new log file is create and I want to process only the one generated yesterday and get the data of column 3 and 6. For example today's date is 24 then I want to get the data of log file created on 23rd. Log Files in... (7 Replies)
Discussion started by: sureshcisco
7 Replies

2. Shell Programming and Scripting

Retrieve logs generated in last 10 mins from a log file using 'grep' command

HI All, I have a log file where the logs will be in the format as given below: 2011-05-25 02:32:51 INFO PROCESS STARTING 2011-05-25 02:32:52 INFO PROCESS STARTED . . . I want to retrieve only the logs which are less than 5 mins older than current time using grep... (3 Replies)
Discussion started by: rvhg16
3 Replies

3. Shell Programming and Scripting

Script to grep for a string in log files generated in last 15 minutes.

Dear Guru's I've a requirment to grep for a string in series of log files that are getting generated almost every minute. I'm looking to schedule a script every 15 mountes,in order to check if the error string has been generated in any of the log files generated in last 15 minutes. Please... (3 Replies)
Discussion started by: rajivatnova
3 Replies

4. Shell Programming and Scripting

Move all .log except those generated in the last 5 minutes

RHEL 5.8 In the directory /u03/pkms/app_logs I have several hundreds of log files as shown below. $ pwd /u03/pkms/app_logs $ ls -alrt *.log | tail -50 -rw-r----- 1 oracle dba 9439232 May 4 13:57 mvtpcem_1_722892404_94157.log -rw-r----- 1 oracle dba 9227264 May 4 13:57... (8 Replies)
Discussion started by: kraljic
8 Replies

5. Shell Programming and Scripting

Grep last 30 minutes log only

I have below command, which i use for checking last two hours log file, but i want to check 30 minutes log and discard old log, only match current time with last 30 minutes. Command i am using. This below attach log file format is for this week, but sometime it got change in time of log, so i need... (6 Replies)
Discussion started by: learnbash
6 Replies

6. Shell Programming and Scripting

How to capture hostnames or ip address in the log files generated?

Team, I have prepared a script which I run from my remote machine(Jump-box)server Below is the script FILEDATE=`date +%F` LOGFILE=/home/abc/scripts/xyz.$FILEDATE.log find /home/abc/scripts/xyz.*.log -type f -mtime -3 -delete touch $LOGFILE exec 1>$LOGFILE 2>&1 #healthcheck batch runs... (1 Reply)
Discussion started by: whizkidash
1 Replies

7. Web Development

How to log http 404 error to a separate log file?

Apache Web Server: how to log http 404 error to a separate log file and i do not want to log in access.log please advice. (2 Replies)
Discussion started by: raghur77
2 Replies

8. Shell Programming and Scripting

Grep last 2 minutes log only

I have newbie, which i use for checking last one hours log file, but i want to check 2 minutes log and discard old log, only match current time with last 2 minutes. Ex log. 2018-07-03 20:09:17 2018-07-03 20:05:17 2018-07-03 20:05:18 2018-07-03 20:05:20 2018-07-03 20:06:22 2018-07-03... (5 Replies)
Discussion started by: ooilinlove
5 Replies

9. Shell Programming and Scripting

Grep a log file for the last 5 minutes of contents every 5 minutes

Hi all, System Ubuntu 16.04.3 LTS i have the following log INFO 2019-02-07 15:13:31,099 module.py:700] default: "POST /join/8550614e-3e94-4fa5-9ab2-135eefa69c1b HTTP/1.0" 500 2042 INFO 2019-02-07 15:13:31,569 module.py:700] default: "POST /join/6cb9c452-dcb1-45f3-bcca-e33f5d450105... (15 Replies)
Discussion started by: charli1
15 Replies

10. UNIX for Beginners Questions & Answers

Error Code Generated by Script

Hi, I have written a script with several variables derived from here documents. However, when I run the following code, the exit status becomes 1: BMC_ACEs="Rich" read -r -d '' BMC_ACL <<EOF ### ### ACL Rack01-BMCMgmt_IN ### $BMC_ACEs EOF If I then type echo $?, it displays a 1. ... (4 Replies)
Discussion started by: richmaxw
4 Replies
NEWSYSLOG(8)						    BSD System Manager's Manual 					      NEWSYSLOG(8)

NAME
newsyslog -- maintain system log files to manageable sizes SYNOPSIS
newsyslog [-nrsvF] [-f config_file] [file ...] DESCRIPTION
newsyslog is a program that should be scheduled to run periodically by cron(8). When it is executed it archives log files if necessary. If a log file is determined to require archiving, newsyslog rearranges the files so that ``logfile'' is empty, ``logfile.0'' has the last period's logs in it, ``logfile.1'' has the next to last period's logs in it and so on, up to a user-specified number of archived logs. Optionally the archived logs can be compressed to save space. A log can be archived for three reasons: 1. It is larger than the configured size (in kilobytes). 2. A configured number of hours have elapsed since the log was last archived. 3. The configured time for rotation of the log occurred within the last 60 minutes. The granularity of newsyslog is dependent on how often it is scheduled to run by cron(8). It is recommended that newsyslog be run once hourly. When starting up, newsyslog reads in a configuration file to determine which logs may potentially be archived. By default, this configura- tion file is /etc/newsyslog.conf. Each line of the file contains information about a particular log file that should be handled by newsyslog. Each line has six mandatory fields and three optional fields, with whitespace separating each field. Blank lines or lines begin- ning with ``#'' are ignored. The fields of the configuration file are as follows: logfile_name Name of the system log file to be archived. owner:group This optional field specifies the owner and group for the archive file. The ``:'' is essential, even if the owner or group field is left blank. The field may be numeric, or a name which is present in /etc/passwd or /etc/group. For backward compatibility, ``.'' is usable in lieu of ``:'', however use of this feature is discouraged. mode Specify the mode of the log file and archives. ngen Specify the number of archive files to be kept besides the log file itself. size When the size of the log file reaches size kilobytes, the log file will be trimmed as described above. If this field is replaced by an asterisk ('*'), then the size of the log file is not taken into account when determining when to trim the log file. when The when field can consist of an interval, a specific time, or both. If the when field is an asterisk ('*') log rotation will depend only on the contents of the size field. Otherwise, the when field consists of an optional interval in hours, optionally followed by an '@'-sign and a time in a restricted ISO 8601 format or by an '$'-sign and a time specification for logfile rotation at a fixed time once per day, per week or per month. If a time is specified, the log file will only be trimmed if newsyslog is run within one hour of the specified time. If an interval is specified, the log file will be trimmed if that many hours have passed since the last rotation. When both a time and an interval are specified, the log will be trimmed if either condition is met. There is no provision for specification of a timezone. There is little point in specifying an explicit minutes or seconds component in the current implementation, since the only comparison is `within the hour'. ISO 8601 restricted time format The lead-in character for a restricted ISO 8601 time is an '@'-sign. The particular format of the time in restricted ISO 8601 is: [[[[[cc]yy]mm]dd][T[hh[mm[ss]]]]]. Optional date fields default to the appropriate component of the current date; optional time fields default to midnight; hence if today is January 22, 1999, the following date specifications are all equivalent: '19990122T000000' '990122T000000' '0122T000000' '22T000000' 'T000000' 'T0000' 'T00' '22T' 'T' '' Day, week and month time format The lead-in character for day, week and month specification is a '$'-sign. The particular format of day, week and month specifica- tion is: [Dhh], [Ww[Dhh]] and [Mdd[Dhh]] respectively. Optional time fields default to midnight. The ranges for day and hour speci- fications are: hh hours, range 0 ... 23 w day of week, range 0 ... 6, 0 = Sunday dd day of month, range 1 ... 31, or the letter L or l to specify the last day of the month. Some examples: $D0 rotate every night at midnight $D23 rotate every day at 23:00 hr $W0D23 rotate every week on Sunday at 23:00 hr $W5D16 rotate every week on Friday at 16:00 hr $MLD0 rotate at the last day of every month at midnight $M5D6 rotate on every 5th day of month at 6:00 hr flags This field specifies any special processing that is required. These flags are parsed in a case insensitive manner. Individual flags and their meanings: - This flag means nothing - it is used as a spacer when no flags are set. b The file is a binary file or is not in syslogd(8) format: the ASCII message which newsyslog inserts to indicate that the logs have been trimmed should not be included. c Create an empty log file if none currently exists. n No signal should be sent when the log is trimmed. p The first historical log file (i.e. the historical log file with the suffix ``.0'') should not be compressed. j Archived log files should be compressed with bzip2(1) to save space. z Archived log files should be compressed with gzip(1) to save space. path_to_pid_file This optional field specifies the file name to read to find the daemon process id. If this field is missing, it defaults to the /var/run/syslogd.pid file. A signal of type sigtype is sent to the process id contained in this path_to_pid_file file. This field must start with '/' in order to be recognized properly. sigtype This optional field specifies the type of signal to be sent to the daemon process. This may be a numeric or symbolic value. By default a SIGHUP (hang-up) will be sent. OPTIONS
The following options can be used with newsyslog: -f config_file Use config_file instead of /etc/newsyslog.conf as the configuration file. -n Do not trim the logs, but print out what would be done if this option were not specified: -n implies -v. -r Remove the restriction that newsyslog must be running as root. When running as a regular user, newsyslog will not be able to send a HUP signal to syslogd(8), so this option should be used only when debugging or trimming user generated logs. -s Do not signal daemon processes. -v Run in verbose mode. In this mode each action that is taken will be printed. -F Force trimming of the logs, even if the trim conditions have not been met. This option is useful for diagnosing system problems by providing you with fresh logs. If additional command line arguments are given, newsyslog will only examine log files that match those arguments; otherwise, it will examine all files listed in the configuration file. FILES
/etc/newsyslog.conf newsyslog configuration file. SEE ALSO
bzip2(1), gzip(1), syslog(3), syslogd(8) BSD
December 21, 2007 BSD
All times are GMT -4. The time now is 06:13 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy