Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Script to grep if 404 error got generated in the log files in last 5 minutes Post 303046108 by Pranav_31 on Friday 24th of April 2020 01:20:18 AM
Old 04-24-2020
Script to grep if 404 error got generated in the log files in last 5 minutes

I have a log file which are linked with prefix current date and suffix with IP_address.log 2020_04_22.shared_http_server.192.168.XX.XX.log

I want to monitor these logs at an interval of every 5 minutes. I am looking to schedule a script at an interval of every 5 minutes,in order to grep if 404 error got generated in the log files in last 5 minutes and send the lines of the logs which contains 404 error in different file.

I have written a script to get this done and i am using cron to run this script for every 5 min.Whenever the script runs, it reads the whole log file and throws all the lines of the logs which contains 404 error in different file.But, i want those lines of the logs containing 404 error message to be sent in different file which got generated in last 5 minutes.

Code:
find /home/pranav -type f -name "2020_04_22.shared_http_server.192.168.XX.XX.log" |while read file
  do
    RESULT=$(egrep "[^0](404)" $file)
      if [[ ! -z $RESULT ]]
         then
            echo "Error(s) in $file on $HOSTNAME at "$(date)": $RESULT">> log_result.txt
     fi
  done

This script is able to grep the lines which contains 404 error message but i want those lines of the logs containing 404 error message to be sent in different file which got generated in last 5 minutes.

Could somebody help me on this what i'm i missing here?
Thank you in advance.

Last edited by rbatte1; 04-30-2020 at 04:26 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to Grep column 3 from csv file generated yesterday

Hello, Can any one please assist how to scirpt it: Every day a new log file is create and I want to process only the one generated yesterday and get the data of column 3 and 6. For example today's date is 24 then I want to get the data of log file created on 23rd. Log Files in... (7 Replies)
Discussion started by: sureshcisco
7 Replies

2. Shell Programming and Scripting

Retrieve logs generated in last 10 mins from a log file using 'grep' command

HI All, I have a log file where the logs will be in the format as given below: 2011-05-25 02:32:51 INFO PROCESS STARTING 2011-05-25 02:32:52 INFO PROCESS STARTED . . . I want to retrieve only the logs which are less than 5 mins older than current time using grep... (3 Replies)
Discussion started by: rvhg16
3 Replies

3. Shell Programming and Scripting

Script to grep for a string in log files generated in last 15 minutes.

Dear Guru's I've a requirment to grep for a string in series of log files that are getting generated almost every minute. I'm looking to schedule a script every 15 mountes,in order to check if the error string has been generated in any of the log files generated in last 15 minutes. Please... (3 Replies)
Discussion started by: rajivatnova
3 Replies

4. Shell Programming and Scripting

Move all .log except those generated in the last 5 minutes

RHEL 5.8 In the directory /u03/pkms/app_logs I have several hundreds of log files as shown below. $ pwd /u03/pkms/app_logs $ ls -alrt *.log | tail -50 -rw-r----- 1 oracle dba 9439232 May 4 13:57 mvtpcem_1_722892404_94157.log -rw-r----- 1 oracle dba 9227264 May 4 13:57... (8 Replies)
Discussion started by: kraljic
8 Replies

5. Shell Programming and Scripting

Grep last 30 minutes log only

I have below command, which i use for checking last two hours log file, but i want to check 30 minutes log and discard old log, only match current time with last 30 minutes. Command i am using. This below attach log file format is for this week, but sometime it got change in time of log, so i need... (6 Replies)
Discussion started by: learnbash
6 Replies

6. Shell Programming and Scripting

How to capture hostnames or ip address in the log files generated?

Team, I have prepared a script which I run from my remote machine(Jump-box)server Below is the script FILEDATE=`date +%F` LOGFILE=/home/abc/scripts/xyz.$FILEDATE.log find /home/abc/scripts/xyz.*.log -type f -mtime -3 -delete touch $LOGFILE exec 1>$LOGFILE 2>&1 #healthcheck batch runs... (1 Reply)
Discussion started by: whizkidash
1 Replies

7. Web Development

How to log http 404 error to a separate log file?

Apache Web Server: how to log http 404 error to a separate log file and i do not want to log in access.log please advice. (2 Replies)
Discussion started by: raghur77
2 Replies

8. Shell Programming and Scripting

Grep last 2 minutes log only

I have newbie, which i use for checking last one hours log file, but i want to check 2 minutes log and discard old log, only match current time with last 2 minutes. Ex log. 2018-07-03 20:09:17 2018-07-03 20:05:17 2018-07-03 20:05:18 2018-07-03 20:05:20 2018-07-03 20:06:22 2018-07-03... (5 Replies)
Discussion started by: ooilinlove
5 Replies

9. Shell Programming and Scripting

Grep a log file for the last 5 minutes of contents every 5 minutes

Hi all, System Ubuntu 16.04.3 LTS i have the following log INFO 2019-02-07 15:13:31,099 module.py:700] default: "POST /join/8550614e-3e94-4fa5-9ab2-135eefa69c1b HTTP/1.0" 500 2042 INFO 2019-02-07 15:13:31,569 module.py:700] default: "POST /join/6cb9c452-dcb1-45f3-bcca-e33f5d450105... (15 Replies)
Discussion started by: charli1
15 Replies

10. UNIX for Beginners Questions & Answers

Error Code Generated by Script

Hi, I have written a script with several variables derived from here documents. However, when I run the following code, the exit status becomes 1: BMC_ACEs="Rich" read -r -d '' BMC_ACL <<EOF ### ### ACL Rack01-BMCMgmt_IN ### $BMC_ACEs EOF If I then type echo $?, it displays a 1. ... (4 Replies)
Discussion started by: richmaxw
4 Replies
syslogd(1M)															       syslogd(1M)

NAME
syslogd - log system messages SYNOPSIS
configfile] markinterval] logfile] DESCRIPTION
The command reads and logs messages into a set of files described by the configuration file Options recognizes the following options: Allows all messages except consecutive duplicate messages without reordering them. Turn on debugging. Prevent the kernel from directly printing its messages on the system console. In this case, is responsible for routing all kernel messages to their proper destination. Use configfile instead of Wait markinterval minutes between mark messages, instead of 20 minutes. Don't listen to socket. Use logfile instead of Don't suppress duplicate messages. While logging the messages coming from remote system, IP address will be logged instead of the hostname. Add priority and facility encoded code at the second field of the message line. Refer to syslog(3C) manpage for these priority and facility encoding codes. creates the file if possible, containing a single line with its process ID. This can be used to kill or reconfigure To kill send it a terminate signal: To make re-read its configuration file, send it a signal: collects messages from the UNIX domain socket an Internet domain socket specified in the named pipe and from the kernel log device By default, local programs calling send log messages to the UNIX domain socket (see syslog(3C)). If UNIX domain sockets are not configured on the system, they write to the named pipe instead. If INET domain sockets are not configured, does not receive messages forwarded from other hosts, nor does it forward messages (see below). Each message is one line. A message can contain a priority code and facility code as the second field of the line. Priorities and Facili- ties are defined in the header file When is invoked using script, user can update the required options in file. By default contains option. Before starting the command, the script recreates after putting the contents into the file By default, is overwritten by the contents of If you want to retain the contents of the previous file, configure in You can set the parameter to the number of lines (in thousands) to be retained from the previous file. For example, to retain 20,000 lines from the previous file along with the contents of the previous in the present put in By default is set to 0. configures itself when it starts up and whenever it receives a hangup signal. Lines in the configuration file consist of a to determine the message priorities to which the line applies and an The action field is separated from the selector by one or more tabs. Selectors are semicolon separated lists of priority specifiers. Each priority has a indicating the subsystem that generated the message, a dot, and a indicating the severity of the message. Symbolic names can be used. An asterisk selects all facilities. All messages of the specified level or higher (greater severity) are selected. More than one facility can be selected, using commas to separate them. For example: selects all facilities at the level and the and facilities at the level. The known facilities and levels recognized by are those listed in syslog(3C) converted to lowercase without the leading The additional facility has a message at priority sent to it every 20 minutes (this can be changed with the flag). The facility is not enabled by a facility field containing an asterisk. The level can be used to disable a particular facility. For example, selects all messages except messages. The second part of each line describes where the message is to be logged if this line is selected. There are four forms: o A file name (beginning with a leading slash). The file is opened in append mode. If the file does not exist, it is created. o A host name preceded by an character. Selected messages are forwarded to the on the named host. o A comma-separated list of users. Selected messages are written to those users' terminals if they are logged in. o An asterisk. Selected messages are written to the terminals of all logged-in users. Blank lines and lines beginning with a character are ignored. For example, the configuration file: logs all kernel messages and 20 minute marks onto the system console, all mail system messages to and all messages at and above, except mail messages, to the file Messages at and above are logged to the console and to the users and if they are logged in. messages are writ- ten to all logged-in users' terminals, and forwarded to the host Only a superuser can invoke Notes logs messages into a set of files. Once the size of a log file reaches 2 GB, stops logging to that file. You can configure the maximum size of log files by setting the variable in The values of can be any positive integer greater than 2, representing the maximum size of the file in GB. When uses the limit imposed by the file system on file size. logs messages in a locale-independent fashion as a stream of bytes and will replace each newline character in the message with a blank space except for the last newline character. Applications using the services of can log messages in different locales. However, be careful when configuring so that messages from different locales do not get logged to the same log file. WARNINGS
A configuration file selector selects all messages at the specified level The configuration lines: cause the logfile to get copies of all messages at level and above. Kernel panic messages are not sent to All HP-UX kernel messages are treated as if they had the priority level. If is invoked with the option and terminates abnormally, kernel messages will not appear on the system console. In that case, reinvoke without the option to enable the kernel to send its messages to the system console. does not support logging to Therefore, if a named pipe is specified in the configuration file, the behavior of is undefined, and may lose messages if blocked or terminated on a does not support long user and group names on the current release, HP-UX 11i V3. logs messages in a locale-independent fashion. o assumes that ASCII control characters do not form intermediate bytes of the characters of a multibyte locale. o truncates the last character when the maximum length of the message is reached, even though it is a valid multibyte character. AUTHOR
was developed by the University of California, Berkeley. FILES
The kernel log device The named pipe on which reads log messages The UNIX domain socket on which reads log messages Configuration file Configuration file for maximum log size Process ID SEE ALSO
logger(1), syslog(3C). syslogd(1M)
All times are GMT -4. The time now is 08:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy