Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

log monitoring

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting log monitoring
# 1  
Old 07-08-2010
log monitoring
Hi All;

I have a log file(dynamic) which i need to monitor; the format of the log

file is as follows

Code:
#Session ID    STATUS

The status can be one of the three /starting ;/loading ;/success

Example
Code:
#Session ID STATUS
ABC   /started.html
XYZ   /started.html
ABC   /loading.html
ABC   /success.html
XYZ   /loading.html
MNO  /started.html
XYZ   /loading.html
XYZ    /success.html
XYZ    /started.html
XYZ    /loading.html
MNO  /loading.html
MNO  /success.html
XYZ    /success.htm


The criteria to monitor is that NO Session ID is to have more than ONE

LOADING status


I need to run a script to monitor this every growing file.


I have written a script;I request you all to provide some inputs on the code
as which is the easiest way of running it every 90 seconds

My script does a while loop for entries (all session IDS) now the question is

that the file is quite big I dont want to check the entries.Can you help me

on Varibalising the file line count with each run and then the file needs to

process only the lines which have been added.

the code is a follows

Code:
while true
do
MAIL_RECEPIENT="root"
LOGDIR="/tmp_test"
EXTENSION=`date +%Y%m%d-%s`
awk '{print $1}' $1|sort -u|while read IN;                                    

           do

grep "$IN" $1; done|grep -i -A2 "l
oading" |uniq -d > $LOGDIR/log.$EXTENSION

if [ -s $LOGDIR/log.$EXTENSION ]
then
cat $LOGDIR/log.$EXTENSION|while read SD
do
sid=`echo $SD|awk '{print $1}'`
status=`echo $SD|awk '{print $2}'`
echo "Session ID $sid has an issue.Please check"|mailx -s "Session ID $sid 

having issue"  $MAIL_RECIEPIENT
echo "Session ID $sid has an issue.Please check"
done
fi
sleep 90
done

Last edited by pludi; 07-08-2010 at 02:12 AM.. Reason: code tags, please...
# 2  
Old 07-08-2010
You could do a `wc -l` and use the output with tail to get the updated lines.

In bash
Code:
$ cat buf
1
2
3
$ wc -l buf
       3 buf
$ n=$(wc -l buf| awk '{ print $1}')
$ cat buf
1
2
3
4
5
6
$ tail +$(($n+1)) buf
4
5
6

# 3  
Old 07-08-2010
If the file is being appended for every run and only the appended lines need to be processed. Suppose run1 is the file of the first run and run2 is the file of second run which has same content as run1 and some extra appended lines. The appended lines can be brought into a separate file and this file can be processed.

The command and ouput is :
Code:
bash-3.00$ cat run1
Hi
Bye
bash-3.00$ cat run2
Hi
Bye
See
You
bash-3.00$ tail -n $((`cat run2 | wc -l` - `cat run1 | wc -l`)) run2
See
You
bash-3.00$

This command gives the extra lines from run2. Hope this helps.

Thanks.
Last edited by pludi; 07-08-2010 at 04:10 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Monitoring script for Log file

Hi, Iam new to unix , plz help me to write below script. I need to write a script for Monitoring log file when any error occurs it has to send a mail to specified users and it should be always pick latest error not the existing one and the script should be able to send mail all errors (more... (1 Reply)
Discussion started by: vij05
1 Replies

2. Shell Programming and Scripting

Monitoring script for a log file

Hi, I need to get a script working to monitor a log file and throw an alert via mailx as soon as a particular error is encountered. I do not want repeatative email notifications of same error so simply cat logfile and grepping the error would not work. Here is what i planned but it seems... (2 Replies)
Discussion started by: roshan.171188
2 Replies

3. Shell Programming and Scripting

Monitoring log for FATAL in last 15mins only

Hi all, I need to develop the following script. Could anyone please provide me some inputs. 1) Monitor the system log file (sys.log) for any lines in the parameter file that will contain all the different phrases of FATAL errors that the process has to scan for in the sys.log. If any FATAL... (7 Replies)
Discussion started by: stunnerz_84
7 Replies

4. Shell Programming and Scripting

Online log monitoring script

#!/bin/bash tail /oracle/app/admin/ABC/bdump/alert_ABC.log >> tempoutput& Error=`egrep 'error|warn|critical|fail|ORA-1683' tempoutput` echo "$Error" |mailx -s "ABC Error " ABCD@domain.lk cat /dev/null > tempoutput I wrote this script and put in to cronjob every 5 min. so every 5... (4 Replies)
Discussion started by: hishanms
4 Replies

5. Shell Programming and Scripting

Monitoring a users log in time?

how do i start with this guys? Sample run: $ LOGTIME it2015678 <enter> User it2015678 is CRUZ Tommy H And has logged on to the system for: 8 hours 12 minutes from the time this script was run. (1 Reply)
Discussion started by: skypigeon
1 Replies

6. Shell Programming and Scripting

Automate Log Monitoring Process

I am a new member of this forum and am also new to unix shell scripting. I joined the forum to seek for help to achieve my task as this forum helps people. here's what i do manually on daily basis 1)Loginto different unix box 2)Ftp the log files (morethan 50 each dir) to windows 3)use text pad... (3 Replies)
Discussion started by: sharugan
3 Replies

7. Shell Programming and Scripting

Monitoring log file

Hi, I ned to monitor the tomcat log file called "catalina.out" for "Out of memory" error. the script should monitor this file and send us the mail as soon as it finds the string "Out of memory" in the file. can ypu suggest me which is the best way to do this? (4 Replies)
Discussion started by: shivanete
4 Replies

8. Windows & DOS: Issues & Discussions

Log monitoring in windows

Hi, I'd like to know if there is a way to monitor a log file conitnuously for one or more strings and if found, send an alarm. It should also take care not to inlcude the old log file entries. Thanks. (2 Replies)
Discussion started by: er_ashu
2 Replies

9. Shell Programming and Scripting

Log Monitoring through Perl

Hi, I am new to perl. I want to write a perl script to monitor logs. Where i want to monitor exceptions logged or any kind of error strings. I have a dir(On Solaris) with multiple log file which keeps rolling to .gz file after some time in that same dir. These logs files size keeps on... (0 Replies)
Discussion started by: solitare123
0 Replies

10. UNIX for Dummies Questions & Answers

log monitoring

Hi there, I have an application runnig on HP_UX which logs critical mesages to a log file. What I would like to do is tail the log file and report on new messages. Easy....I here you say. The log file is continuing to be written to and the check scritp will be executed from cron. I was... (2 Replies)
Discussion started by: nhatch
2 Replies
Login or Register to Ask a Question