Log search and mail it if the log is updated before 24 hours from the current time


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Log search and mail it if the log is updated before 24 hours from the current time
# 1  
Old 07-09-2013
RedHat Log search and mail it if the log is updated before 24 hours from the current time

Hi ,

We have around 22 logs , each has different entries. I have to automate this using shell script. The ideas which am sharing is given below

1) We use only TAIL -100 <location and name of the log> Command to check the logs.

2) We want to check whether the log was updated before 24 hours or current time stamp, if found it should send a mail the log looks good.

3) We want to check whether there is an ERROR message found on the log (only ERROR, not error,Error) , if found it should send a mail like Error message was found or else exit without sending a mail.

Suggestion needed is how to gather all 22 logs in one single mail , if it has updated before 24 hours. and how to gather the ERROR message if it is presented in the log and mail us seperately with the log name . Could any1 help me with this thread and idea?
# 2  
Old 07-09-2013
post examples of your log files, expected output and what you expect to have in the email ... it would also help if you post what you have done with your script so far ...
# 3  
Old 07-10-2013
Well, this is what I would do:
Quote:
1) We use only TAIL -100 <location and name of the log> Command to check the logs.
Create a plain text file (name it mylogs, for example) with the absolute path to each log, 1 per line, i.e.,
Code:
/var/log/mail.log
/var/log/some.log
.
.
.
/var/log/some_other.log

Then in my script I'd do this (hasn't been tested, but you can tweak it a little in order to suit your needs):
Code:
#!/bin/bash
mailcontents=mailbody
while read line
do
    last_mod_time=$(stat -c '%Y' $line) # this line checks the log's last modification time and converts it to Unix's epoch
    last_24_hours=$(date +%s -d "24 hours ago") # this line returns the epoch for the current timestamp minus 24 hours

    if [ $last_mod_time -lt $last_24_hours ]; then
        echo "Log $line has NOT been updated in the last 24 hours" >> $mailcontents
        echo "============" >> $mailcontents
    else
        echo "Log $line was updated during the last 24 hours" >> $mailcontents
        echo "============" >> $mailcontents
    fi
    
    tail -100 $line > checklog

    error=$(cat checklog | grep ERROR | wc -l)
    if [ $error -gt 0 ]; then
        echo "ERROR found in log $line" >> $mailcontents
    fi

done < mylogs

if [ -s $mailcontents ]; then
    mail -s "Log errors and last modification times" youremail@yourdomain.com < $mailcontents
fi

rm $mailcontents checklog

Hope it helps Smilie.
# 4  
Old 07-10-2013
Super Dude... Will try your idea and let you know the result asap.. Thanks a lot for the valuable info SmilieSmilie Smilie
# 5  
Old 07-11-2013
Dude,

as per your idea , i have created a text document with a name of LOgcheck.txt by adding the logs

My doubts are
1) whether i have to add "cd /pathname of the log" or only "/pathname of the name " in the text document.

2) what will this line " tail -100 $line > checklog will do?
3) what will this line "rm $mailcontents checklog" will do?

Am really sorry to bother you if these doubts sound so silly , am new to scripting Smilie
# 6  
Old 07-11-2013
No worries. I'm more than glad to be able to help.
Let me refer to the script itself to answer your questions:

Quote:
Dude,

as per your idea , i have created a text document with a name of LOgcheck.txt...
Note that you don't need the .txt extension. You can leave it there if you want, though. Just a thing here, I assume you meant "Logcheck.txt" - without the uppercase "O". This kind of typo can cause your script to run with issues or not run at all.

Quote:
1) whether i have to add "cd /pathname of the log" or only "/pathname of the name " in the text document.
You don't need to add the cd command - just the absolute path to the log. The loop
Code:
while read line
do
[...bunch of commands here :) ...]
done < Logcheck.txt

does this:
It reads the Logcheck.txt file (that's what the "<" does - it tells the while loop to use the file as input) line by line, one at a time, and during every run it assigns the value of each line to the line variable. Picture this scenario. Let's say the Logcheck.txt file looks exactly as I mentioned in my first post. During the first loop, the line variable (referenced in the rest of the loop as $line) will take the value /var/log/mail.log, and that value will be used until the loop ends and starts again, this time with the line variable taking the value of the 2nd line in your Logcheck.txt file.
Quote:
2) what will this line " tail -100 $line > checklog will do?
That line saves the last 100 lines of each log (refer to the first answer - $line here will be replaced by the absolute path to each script during each loop) to the checklog file. (Note that I didn't use the extension here).
So during the first loop this will actually be:
Code:
tail -100 /var/log/mail.log > checklog

"checklog" is an auxiliary file where we're redirecting the output of each log and that's where we'll search for errors later.
Quote:
3) what will this line "rm $mailcontents checklog" will do?

Am really sorry to bother you if these doubts sound so silly , am new to scripting Smilie
That line will delete your auxiliary files mailbody (see the top of the script) and checklog once the loop has ended and you don't need them anymore. You can comment out that line if you want to keep those files.
Let me know if you need further help Smilie.


---------- Post updated at 09:38 PM ---------- Previous update was at 09:09 PM ----------

Updated version, tested Smilie. Works like a charm.
1) I created the file named mylogs.
2) Updated script:
Code:
#!/bin/bash
mailcontents=mailbody
while read line
do
    if [ ! -f $line ]; then
        echo "The file $line doesn't exist. Continuing with the next file..." >> $mailcontents
        echo "============" >> $mailcontents 
        continue
    else
        last_mod_time=$(stat -c '%Y' $line) # this line checks the log's last modification time and converts it to Unix's epoch
        last_24_hours=$(date +%s -d "24 hours ago") # this line returns the epoch for the current timestamp minus 24 hours

        if [ $last_mod_time -lt $last_24_hours ]; then
            echo "Log $line has NOT been updated in the last 24 hours" >> $mailcontents
        else
            echo "Log $line was updated during the last 24 hours" >> $mailcontents
        fi
    
        tail -100 $line > checklog

        error=$(grep ERROR checklog | wc -l) # We look for the lines containing the word "ERROR" in the checklog file.
                                             # Then we redirect the output to the wc -l command that will count the number
                                             # of lines where the word ERROR appears.
    
        if [ $error -gt 0 ]; then # If this condition is satisfied, that means the word ERROR appeared at least once in
                                  # the log that's being examined in the current loop.
            echo "ERROR found in log $line" >> $mailcontents
        else
            echo "No errors found in $line" >> $mailcontents
        fi
    fi
echo "============" >> $mailcontents
done < mylogs

if [ -s $mailcontents ]; then
    mail -s "Log errors and last modification times - $(date +'%A %B %d, %Y')" myemail@gmail.com < $mailcontents
fi

rm $mailcontents checklog # Delete auxiliary files when we're done.

The results:
Image

---------- Post updated at 10:21 PM ---------- Previous update was at 09:38 PM ----------

Added file test.log with the word ERROR in 2 lines, and included it in the mylogs file (where the list of the logs is stored):
Image
The results in the email: (note that the email includes the lines with the errors). You need to add the following line
Code:
echo -e "\t" $(grep ERROR checklog) >> $mailcontents

right beneath this one:
Code:
echo "ERROR found in log $line" >> $mailcontents

Image

Last edited by gacanepa; 07-11-2013 at 09:44 PM..
# 7  
Old 07-12-2013
Am so happy for the above explanations dude,, Thanks a lot for spending your valuable time for answering this question and doubts... Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Log File updated time

hi can any one please help on below .im new to shell scrpting i need to write a shell script which will check particular log file is presented or not in specific location ,if yes how long it was not modified/not rolling ?if its not modified/log is not rolling script will have to send mail (9 Replies)
Discussion started by: 4Learning
9 Replies

2. Shell Programming and Scripting

Search the string in the active log and send mail

Hello, I wanted to search specific string in the acitve log file and send an email if the search string found in the log. Log file is written by application all the time. So, script has to search if any new log entry has the specific string for example " sample exception" and send an email. (1 Reply)
Discussion started by: balareddy
1 Replies

3. Shell Programming and Scripting

Search for logs traced between specific date and time from log file

HI, I want to search for a logs which are trace between specific date and time from logs file. My logs are generated like this :- Tue Jun 18 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process. Thu Jun 20 05:00:02 EEST 2013 | file_check.sh| Message:script has files to... (5 Replies)
Discussion started by: ketanraut
5 Replies

4. Shell Programming and Scripting

awk : Search for text between two time frame (12 hours)

I have created the script to grep the errors from weblogic logs files and redirecting output to file.txt ...From file.txt I'm using awk command to collect the past 20 mins output...The script running from cron every 15 mins... The script working well... Now the challenges, I'm trying to use... (27 Replies)
Discussion started by: zenkarthi
27 Replies

5. Homework & Coursework Questions

Sort current logged in users by log in time (supposedly to be very easy but I'm missing something)

1. The problem statement, all variables and given/known data: Show all users who are currently logged in, sorted from earliest to latest log in time. The log in time includes the month, day, and time. 2. Relevant commands, code, scripts, algorithms: finger, who, sort, pipe, head, tail, ... (8 Replies)
Discussion started by: vtmd
8 Replies

6. UNIX for Dummies Questions & Answers

Adding hours and minutes to current date (Only to date not to time)

Hi, I want to add some hours and minutes to the current date. For example, if the current date is "July 16, 2012 15:20", i want to add 5 hours 30 minutes to "July 16, 2012 00:00" not to "July 16, 2012 15:20". Please help. Thanks! (4 Replies)
Discussion started by: manojgarg
4 Replies

7. Shell Programming and Scripting

How can view log messages between two time frame from /var/log/message or any type of log files

How can view log messages between two time frame from /var/log/message or any type of log files. when logfiles are very big and especially many messages with in few minutes, I would like to display log messages between 5 minute interval. Could you pls give me the command? (1 Reply)
Discussion started by: johnveslin
1 Replies

8. UNIX for Dummies Questions & Answers

Execute crontab for every 4 hours and begin from current time

I want to add a crontab entry which should execute for every 4 hours and that 4 hours calculation should begin from the current time. Normally if I set the crontab entry like this, 00 */4 30 05 * root date >>/tmp/cronout The above will execute the date command for every 4 hours like... (7 Replies)
Discussion started by: Ganeshwari
7 Replies

9. Solaris

files updated in last 10 hours should be moved

Hi, I would like to move all files that are updated in last 10 hrs. to some temporary folder. Please help. (3 Replies)
Discussion started by: sanjay1979
3 Replies

10. Shell Programming and Scripting

shell script not getting current error messages with time from alert.log

Hi All, I need to get current error messages with time from alert.log.Below is my shell script but it's not working to meet this objective. could anyone pls share on the above issue for resolution: #################################################################### ## ckalertlog.sh ##... (2 Replies)
Discussion started by: a1_win
2 Replies
Login or Register to Ask a Question