monitor log entries and send e-mail


 
Thread Tools Search this Thread
Homework and Emergencies Homework & Coursework Questions monitor log entries and send e-mail
# 8  
Old 06-01-2010
Hi, since $EMAILMESSAGE is a variable and not a file you can not use input redirection like that. Also it is typically good practice to quote variable references..
You could try:
Code:
echo "$EMAILMESSAGE" | /bin/mailx -s "$SUBJECT" "$RECIPIENT"

or better yet:
Code:
printf "%s\n" "$EMAILMESSAGE" | /bin/mailx -s "$SUBJECT" "$RECIPIENT"

Which would protect against awkward characters in the email message body.

Is $RECIPIENT set to a valid email address?

Last edited by Scrutinizer; 06-01-2010 at 09:37 AM..
This User Gave Thanks to Scrutinizer For This Post:
# 9  
Old 06-01-2010
Bug

Quote:
Originally Posted by Scrutinizer
Hi, since $EMAILMESSAGE is a variable and not a file you can not use input redirection like that. Also it is typically good practice to quote variable references..
You could try:
Code:
echo "$EMAILMESSAGE" | /bin/mailx -s "$SUBJECT" "$RECIPIENT"

or better yet:
Code:
printf "%s\n" "$EMAILMESSAGE" | /bin/mailx -s "$SUBJECT" "$RECIPIENT"

Which would protect against awkward characters in the email message body.

Is $RECIPIENT set to a valid email address?
Hey Scrutinizer,
You seem to live up to your display id.I have made the changes accordingly. And yes the e-mail id is set to a valid id. I recieved the mail with all the the messages that contain error or warning.
I will put up the updated code separately.
SmilieSmilieSmilieSmilieSmilie
Could you also give me a hint on how i could do the tasks of script. The main task is to scan the log file for new entries. I am not able to figure out how to scan the log file for the first time and then when the script is run the 2nd time, it needs to scan only new entries and ignore previous entries.
My approach was to create a tmp_log file of the log that is being scanned and then read from there. But i then realized if a different log file is scanned every time then my approach would fail.
What is your opinion? Could you please give a logic and possible commands to use.
I might consider using wc for counting line numbers and then compare the logfile given asn argument with tmp_log that the script creates. then find the difference of the lines and print that in a second tmp_log file and do operations on that.

Again, Thanks for you valuable feebacks.

---------- Post updated at 11:08 PM ---------- Previous update was at 10:52 PM ----------

New improvements in the script are:
This script now searches for string mentioned searched using egrep.
Stores output of egrep in a variable and is passed to the email message variable.
It successfully sends an email with the entries of the specified log file that matched the string pattern in egrep
The code is below. I have tried to highlight all the recent updates that were made:
Code:
#!/bin/sh -x
#
#
#
PRINTF=/usr/bin/printf
EGREP=/usr/bin/egrep
TAIL=/usr/bin/tail
COPY=/usr/bin/cp
CAT=/usr/bin/cat
LOG_FILE="$1"
LOG_PATH="/var/log/$1"
#To check if log file to check has been given as argument when running the script
if [ $# -ne 0 ] ; then
           $PRINTF "%s: Is the name of the log file you will be scanning. \n" $1
           $PRINTF $LOG_FILE"\n"
           $PRINTF $LOG_PATH"\n"

                if [ -f $LOG_PATH ];       #if logfile exits
                then
                        echo "File $LOG_FILE exists \n"
                        $COPY $LOG_PATH ./  #create copy of original log file in current directory
                        $CAT -n $LOG_FILE > ./tmp_log #Store a copy of original log file in a temporary log file with number index

                        # script to check for string to search
                        SEARCH=`$EGREP -i 'error|warning' $1` #search for strings error or warning in any form
#                       SCAN_LOG=`$TAIL $1|[${SEARCH}]`  #scanning logfile for string pattern

                        # script to send simple email
                        SUBJECT="Error/Warning messages in logfile" # email subject
                        RECIPIENT="validmailid@domain.extension" # Email To ?
#                       EMAILMESSAGE="$SCAN_LOG" # Email text/message
                        EMAILMESSAGE="$SEARCH" # Email text/message
                        echo "mail not sent yet"
                        $PRINTF "%s\n" "$EMAILMESSAGE" | /bin/mailx -s "$SUBJECT" "$RECIPIENT"  #send email with subject message to specified mail id
                                echo "mail sent"
                else
                        echo "File $LOG_FILE does not exists \n"
                fi

        else
           $PRINTF "%s: Need the name of the log that needs to be checked\n" $0
#       EXIT (1)
fi

The output of the script now is as below:
Code:
> sh logmon1 log.04
log.04: Is the name of the log file you will be scanning.
log.04
/../../../../log.04
File log.04 exists

mail not sent yet
mail sent
>


Last edited by vin8465; 06-01-2010 at 10:13 AM.. Reason: Added output of script
# 10  
Old 06-01-2010
Quote:
Originally Posted by vin8465
Code:
                        echo "mail not sent yet" # not sure, but this line probably needs to be placed in the else part ?
                        $PRINTF "%s\n" "$EMAILMESSAGE" | /bin/mailx -s "$SUBJECT" "$RECIPIENT"  #send email with subject message to specified mail id
                                echo "mail sent"
                else
                        echo "File $LOG_FILE does not exists \n"
                fi

This User Gave Thanks to pseudocoder For This Post:
# 11  
Old 06-02-2010
MySQL check for which line of code is being executed

Hi pseudocoder,
you are right. But my main intention of putting the echo there was to check what line in the script has been executed so far. Also I need to figure out how nt to send a mail if nothing is found from the searched string pattern.

I still am trying to figure out how the script can be modified to monitor and report only new entries when it is run for the second time with the same log file as an argument.

---------- Post updated at 08:44 AM ---------- Previous update was at 08:42 AM ----------

Smiliethanks for pointing that out. I will remember to take that out as it looks a bit misleading.

---------- Post updated at 01:36 PM ---------- Previous update was at 08:44 AM ----------

I have reformatted the code to suit the needs of the script task.
I have tried to highlight all the pieces of code that has been shifted, added or changed in blue.
All improvements required are highlighted in red. I have updated certain parts of the code and highlighted it using orange
Code:
#!/bin/sh -x
#
#
#############
#
#THE PURPOSE OF THIS SCRIPT 'logmon'?
#THE SCRIPT IS INTENDEED TO MONITOR LOG FILES AND REPORT ANY ENTRIES THAT CONTAIN THE PATTERN "ERROR" OR "WARNING" IN ANY FORMAT
#THE SCRIPT SENDS AN EMAIL TO THE ID MENTIONED IN THE RECIPIENT VARIABLE IF THE PATTERN IS FOUND. NOTE:- THE SCRIPT IS INTENDED 
#TO CHECK A LOG FILE COMPLETELY FOR THE FIRST TIME THE LOG FILE IS BEING SCANNED AND THEN CHECKS ONLY THE NEWLY ADDED LOG ENTRIES
#HENCEFORTH.
#
#############
#
#WHAT DOES THE SCRIPT REQUIRE TO EXECUTE?
#THIS SCRIPT TAKES AN ARGUMENT WHICH IS THE ABSOLUTE PATH WITH THE NAME OF THE FILE THAT YOU WISH TO SCAN.
#
############
#WHAT DOES MY SCRIPT DO SO FAR?
#IT CHECKS IF AN ARGUMENT IS PASSED WHEN THE SCRIPT IS BEING EXECUTED. IT EXITS OUT WITH AN ERROR MESSAGE
#IF THE ARGUMENT IS NOT GIVEN. WHEN THE SCRIPT IS GIVEN THE ABSOLUTE PATH OF THE FILE TO SCAN FOR STRING PATTERNS, IT FIRST GIVES
#INFORMATION OF WHICH FILE IS GOING TO BE SCANNED.
#FINALLY IT DISPLAYS A MESSAGE AFTER SENDING THE EMAIL TO THE ID MENTIONED IN THE RECIPIENT. 
#IF THE LOG FILE BEING SEARCHED IS NOT LOCATED IN THE PATH DISPLAYED EARLIER,IT DISPLAYS A FILE NOT FOUND MESSAGE AND EXITS THE SCRIPT
#THE FILE NOW CREATES A TEMPORARY FILE tmp_log WHICH BASICALLY IS A COPY OF THE LOG FILE BEING SCANNED. ANOTHER FILE diff_result IS CREATED WHICH
#CONTAINS UPDATED ENTRIES OF THE CURRENTLY SCANNED LOG FILE
#
############
#
#WHAT NEEDS TO BE DONE?
#LOGIC FOR SEARCHING ONLY UPDATED ENTRIES TO THE CURRENTLY SCANNED LOG FILE IS ALMOST DONE BUT REQUIRES TUNING.
#FIND HOW MANY TIMES A PARTICULAR ENTRY IS REPEATED TO GIVE LOGISTICS OF WHICH ERROR OCCURS HOW MANY TIMES.
#NOTE:- TIME STAMP IS TO BE IGNORED FOR THIS SCRIPT WHEN COMPARING DUPLICATE ENTRIES.
#
############

PRINTF=/usr/bin/printf
EGREP=/usr/bin/egrep
TAIL=/usr/bin/tail 
COPY=/usr/bin/cp
CAT=/usr/bin/cat
DIFF=/usr/bin/diff
LOG_FILE_PATH="$1"                 # should contain the absolute path of the file to be scanned
SEARCH=`$EGREP -i 'error|warning' $1`         #search for strings error or warning in any form
RECIPIENT="validmailid"     # Email To ?


#To check if log file to be scanned has been given as argument when running the script
if [ $# -ne 0 ] ; then
           $PRINTF "%s is the log file that will be read \n" $1

        if [ -f $1 ];       #check if log file exists
        then
               $PRINTF "File exists \n"


            # script to check for string to search
#            SCAN_LOG=`$TAIL $1|[${SEARCH}]`  #scanning logfile for string pattern

            # Here am trying to grab all the updated entries in the log file with the match pattern.
            # So far I am not able to figure out how to do so. I have commented it as it only grabs the last
            # few lines that were updated in the log file.

            # script to send simple email
            SUBJECT="Error/Warning messages in logfile" # email subject
#            EMAILMESSAGE="$SCAN_LOG" # Email text/message
            EMAILMESSAGE="$SEARCH" # Email text/message
                if [ -z "$SEARCH" ];  #Check if any value has been added to variable that will later be passed to EMAILMESSAGE variable. This is basically to avoid sending an e-mail if nothing was found in the log matching the search pattern
                then
                $PRINTF "There is no message to send \n"
                else
                $PRINTF "%s\n" "$EMAILMESSAGE" | /bin/mailx -s "$SUBJECT" "$RECIPIENT"  #send email with subject message to specified mail id 
                $PRINTF "Mail sent to %s" $RECIPIENT
                fi
                $DIFF -b $1 ./tmp_log > ./diff_result
                $COPY $1 ./tmp_log # To keep track of what is updated when this log file is scanned again
#Here am using diff to compare the entries updated in the current log which is being scanned with the tmp_log that contains the entries of log file before the script was run 
#through the log for the second time. I am still trying to perfect it. It basically outputs the result of diff in diff_result file and then I need to do a pattern match on that file
#to e-mail only newly updated entries of the log file.
        else
               $PRINTF "File %s  does not exist \n" $1
            $PRINTF "Please place log file to be scanned in the above mentioned path \n" 
        fi

else
$PRINTF "Script %s: needs the absolute path of the log that needs to be checked\n" $0
fi

I get the following error if the search pattern doesnt match. At least that that is what the problem according to me. The output of an unsuccessful run is as below:
Code:
sh logmon /var/log/syslog.0
/var/log/syslog.0 is the log file that will be read
File exists
logmon: test: argument expected

The output of a successful run is as below:
Code:
sh logmon /<dir>/<dir>/<dir>/log.01
/<dir>/<dir>/<dir>/log.01 is the log file that will be read
File exists
Mail sent to <validmailid>

I am not sure that the following code works as it should.
Code:
if [ -z $SEARCH ];
                then
                $PRINTF "There is no message to send \n"
                else
                $PRINTF "%s\n" "$EMAILMESSAGE" | /bin/mailx -s "$SUBJECT" "$RECIPIENT"  #send email with subject message to specified mail id 
                $PRINTF "Mail sent to %s" $RECIPIENT
                fi



---------- Post updated at 01:55 PM ---------- Previous update was at 01:36 PM ----------

Found the problem with the if statement. Changes are in blue
Code:
 if [ -z "$SEARCH" ];
                                then
                                $PRINTF "There is no message to send \n"
                                else
                                $PRINTF "%s\n" "$EMAILMESSAGE" | /bin/mailx -s "$SUBJECT" "$RECIPIENT"  #send email with subject message to specified mail id
                                $PRINTF "Mail sent to %s" $RECIPIENT
                                fi

The output now if the log file does not contain the string pattern is
Code:
sh logmon /var/log/syslog.0
/var/log/syslog.0 is the log file that will be read
File exists
There is no message to send



---------- Post updated at 06:05 PM ---------- Previous update was at 01:55 PM ----------

The script now basically works fine. Certain advanced requirements of the script task that i have not implemented are:
  1. If same script is renamed to logcheck then the reports the script generates should be displayed in the command line.
  2. Error messages that are repeated are to be counted and only one instance of error message is to be reported followed by the comment "The above message appeared n times"
Note: Time stamp is being ignored when making the above improvements. i.e a log entry having identical time stamps will not be considered when grouping messages.

The code is as below. I have reformatted the entire code, so have not highlighted parts of the code.
Code:
#!/bin/sh
#set -x
#
#
#############
#
#THE PURPOSE OF THIS SCRIPT 'logmon'?
#THE SCRIPT IS INTENDEED TO MONITOR LOG FILES AND REPORT ANY ENTRIES THAT CONTAIN THE PATTERN "ERROR" OR "WARNING" IN ANY FORMAT
#THE SCRIPT SENDS AN EMAIL TO THE ID MENTIONED IN THE RECIPIENT VARIABLE IF THE PATTERN IS FOUND.
# NOTE:- THE SCRIPT IS INTENDED TO CHECK A LOG FILE COMPLETELY WHEN IT IS SCANNED FOR THE FIRST TIME AND THEN CHECKS ONLY THE NEWLY ADDED LOG ENTRIES
#HENCEFORTH.
#
#############
#
#WHAT DOES THE SCRIPT REQUIRE TO EXECUTE?
#THIS SCRIPT TAKES AN ARGUMENT WHICH IS THE ABSOLUTE PATH WITH THE NAME OF THE FILE THAT YOU WISH TO SCAN.
#
############
#
#WHAT DOES MY SCRIPT DO SO FAR?
#IT CHECKS IF AN ARGUMENT IS PASSED WHEN THE SCRIPT IS BEING EXECUTED.
#IT EXITS OUT WITH AN ERROR MESSAGE IF THE ARGUMENT IS NOT GIVEN.
#ON PROPER EXECUTION, IT FIRST GIVES INFORMATION OF WHICH FILE IS GOING TO BE SCANNED.
#FINALLY IT DISPLAYS A MESSAGE  OF WHETHER AN E-MAIL WAS SENT OR NOT
#IF THE LOG FILE BEING SEARCHED IS NOT LOCATED IN THE PATH DISPLAYED EARLIER,IT DISPLAYS A FILE NOT FOUND MESSAGE AND EXITS THE SCRIPT
#NOTE: THE SCRIPT CHECKS WHETHER THE CURRENTLY SCANNED LOG FILE HAS BEEN PREVIOUSLY SCANNED OR NOT. IF NOT SCANNED A TMP_LOG FILE IS CREATED
#AND CONTENTS OF THE LOG FILE IS DUMPED INTO TMP_LOG. IF SCANNED PREVIOUSLY, THEN ALL UPDATED ENTRIES ARE PLACED IN A TEMPORARY FILE AND ONLY
#THOSE ENTRIES ARE CHECKED & REPORTED FOR THE SEARCH PATTERN
#
############
#
PRINTF=/usr/bin/printf
EGREP=/usr/bin/egrep
TAIL=/usr/bin/tail
COPY=/usr/bin/cp
CAT=/usr/bin/cat
DIFF=/usr/bin/diff
REMOVE=/usr/bin/rm
#LOG_FILE_PATH="$1"                             # should contain the absolute path of the file to be scanned
SEARCH=`$EGREP -i 'error|warning' $1`           #search for strings error or warning in any form
RECIPIENT="validmailid@domain.extension"        # Email To ?
SUBJECT="Error/Warning messages in logfile"     # email subject


if [ $# -ne 0 ] ;    #To check if log file to be scanned has been given as argument when running the script
then
           $PRINTF "%s is the log file that will be read \n" $1

                if [ -f $1 ];       #check if log file to be scanned exists
                then
                        $PRINTF "File exists \n"

                        if [ -f ./tmp_log ];     #To check if current log file was scanned for the first time
                        then

                                $DIFF -b $1 ./tmp_log > ./diff_result #Resource that contains only updated entries of the current log file
                                SEARCH_UPDATES_ONLY=`$EGREP -i 'error|warning' ./diff_result`
                                $REMOVE ./diff_result
                                EMAILMESSAGE="$SEARCH_UPDATES_ONLY" # Email text/message Contains log entries from diff_result
                                if [ -z "$SEARCH_UPDATES_ONLY" ];
                                then
                                        $PRINTF "There is no message to send \n"

                                else
                                        $PRINTF "%s\n" "$EMAILMESSAGE" | /bin/mailx -s "$SUBJECT" "$RECIPIENT"  #sends email of updated log entries that match the pattern
                                        $PRINTF "Mail sent to %s" $RECIPIENT
                                fi
                        else
                                EMAILMESSAGE="$SEARCH" # Email text/message #Contains log entries from actual log file
                                if [ -z "$SEARCH" ];
                                then
                                        $PRINTF "There is no message to send \n"
                                else
                                        $PRINTF "%s\n" "$EMAILMESSAGE" | /bin/mailx -s "$SUBJECT" "$RECIPIENT"  #send email with all log entries  that matche the search pattern
                                        $PRINTF "Mail sent to %s" $RECIPIENT
                                fi
                        fi
                        $COPY $1 ./tmp_log # To keep track of what is updated when this log file is scanned again
                else
                        $PRINTF "File %s  does not exist \n" $1
                        $PRINTF "Please place log file to be scanned in the above mentioned path \n"
                fi

else
        $PRINTF "Script %s: needs the absolute path of the log that needs to be checked\n" $0
fi

The output of a successful run will be as below depending on the situation:
Code:
 sh logmon /<dir>/<dir>/<dir>/log.01
/<dir>/<dir>/<dir>/log.01 is the log file that will be read
File exists
There is no message to send

Code:
 sh logmon /<dir>/<dir>/<dir>/log.01
/<dir>/<dir>/<dir>/log.01 is the log file that will be read
File exists
Mail sent to validmailid@domain.extension>

Could someone please help me with the advanced parts. I need urgent help to finish this script off today itself.

Last edited by vin8465; 06-02-2010 at 09:25 PM.. Reason: Updated code to match code format
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Client was not authenticated to send anonymous mail during MAIL FROM (in reply to MAIL FROM comm

I am having trouble getting mail to work on a red hat server. At first I was getting this message. Diagnostic-Code: X-Postfix; delivery temporarily suspended: connect to :25: Connection refused Then added the port to my firewall. Then I temporarily turned off selinux. I then copied this file... (1 Reply)
Discussion started by: cokedude
1 Replies

2. Shell Programming and Scripting

Monitor a file and send mail

I want to monitor the maillog file in real time and send a mail when a certain grep condition is met. Every time the grep condition is met a mail will be sent. I wish to ignore all grep conditions 30 mins after each mail and thereafter continue monitoring. For example: Condition is met, mail... (1 Reply)
Discussion started by: proactiveaditya
1 Replies

3. Shell Programming and Scripting

Search the string in the active log and send mail

Hello, I wanted to search specific string in the acitve log file and send an email if the search string found in the log. Log file is written by application all the time. So, script has to search if any new log entry has the specific string for example " sample exception" and send an email. (1 Reply)
Discussion started by: balareddy
1 Replies

4. Shell Programming and Scripting

Monitor file if match then send mail

Hi I want to monitor a file even if the file rotate. When a text occurs I want to send a mail. Something like this but it's not working correctly: tail -F mylog.log | grep 'MatchMe' | while read line do echo $(date +"%Y-%m-%d %H:%M:%S") MatchMe occurs | mail -s "MatchMe"... (1 Reply)
Discussion started by: chitech
1 Replies

5. Shell Programming and Scripting

Basic script for monitor send mail service

Hi All Need help Can any one share a basic script that is used for monitor sendmail service whether online, offline.etc in solaris Thanks in advance Zimmy (5 Replies)
Discussion started by: zimmyyash
5 Replies

6. Shell Programming and Scripting

Looking for shell script to monitor CPU utilization and send mail once exceed 75%

Dear Group, I'm look for shell script to Monitor CPU usage and send mail once it exceed 75% I'm running Suse10.4. (3 Replies)
Discussion started by: clfever
3 Replies

7. Shell Programming and Scripting

How to monitor log file for a Error and generate the e-mail ( Please help)

This is my log file and this is live log. Any abnormal error other than following I need to generate the email. Log path : /DER/app/admin/ABC/bdump/erg.log Current log# 2 seq# 103046 mem# 0: /ora2/oradata/ABC/redo02a.log Current log# 2 seq# 103046 mem# 1:... (7 Replies)
Discussion started by: hishanms
7 Replies

8. Shell Programming and Scripting

Monitor log file for a Error and generate the e-mail.

This is my log file and this is live log. Any abnormal error other than following I need to generate the email. Log path : /DER/app/admin/ABC/bdump/erg.log Current log# 2 seq# 103046 mem# 0: /ora2/oradata/ABC/redo02a.log Current log# 2 seq# 103046 mem# 1:... (1 Reply)
Discussion started by: hishanms
1 Replies

9. UNIX for Advanced & Expert Users

need to configure mail setting to send mail to outlook mail server

i have sun machines having solaris 9 & 10 OS . Now i need to send mail from the machines to my outlook account . I have the ip adress of OUTLOOK mail server. Now what are the setting i need to do in solaris machines so that i can use mailx or sendmail. actually i am trying to automate the high... (2 Replies)
Discussion started by: amitranjansahu
2 Replies

10. Emergency UNIX and Linux Support

monitor log entries and send e-mail

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! I am stuck and worried. My assignment was due a day ago and I was too busy completing other assignments due during the same time. I worry that not completing this assignment... (1 Reply)
Discussion started by: vin8465
1 Replies
Login or Register to Ask a Question