I am stuck and worried. My assignment was due a day ago and I was too busy completing other assignments due during the same time. I worry that not completing this assignment will fail me. Need urgent help in completing the script asap. I dont want to sound pushy to get the answer but just wanted to explain the situation.
1. The problem statement, all variables and given/known data:
I have mentioned a detailed requirement to be very clear. The points with '*' are the tasks that the script is
expected to do. All other information provided is for clear understanding of the script functionality.
The basic form of the script will simply send a copy of all lines that contain either "error" or "warning"
to the user specified in the script as RECIPIENT. Note that RECIPIENT is a shell variable that will contain
an e-mail address.
The basic script will be able to look at a log file periodically and report on any new lines that contain
either "error" or "warning". The search for this text is to be "case insensitive". This means that the script
needs to accept a line containing either "error", "Error", "ERROR" or the word error spelt in any other
combination of upper and lower case letters as being functionally equivalent.
The basic script, named "logmon" (for log monitor) will need to do the following:
* be able to run in an environment that does not include a path. (i.e. the environment provided by
cron)
* accept the name of a log file as the sole command line argument
* search the specified log file for lines that have been added since the previous search.
* construct an e-mail reporting any lines that contain either "error" or "warning" in any combination
of upper and lower case letters
* send the e-mail to the address specified in the RECIPIENT shell variable which is defined within the
script
Optional requirements for now
Include only putting a single copy of a message in an e-mail, but reporting
how many times the message is repeated. Also modifying the script so that the same script will either
report via e-mail, or directly to a terminal, depending on how it is called. The final option will
be a system that reports errors directly to the root console.
(Note that this will need to be implemented in a way that sends the message to your account and does not actually send a message to root for testing purpose.)
Please add a comment or comments in your script that state which, if any, of the optional requirements have been implemented.
* modify the script so that, instead of listing multiple lines with the same message, it lists one instance
of each duplicated line (Note that for the purposes of this requirement, two lines that share everything
except the timestamp are considered to be identical lines.)
* follows the single instance of a duplicated line with a line that reports how many times the previous
line repeats
* add code that will report the above to the command line when it is called with the name "logcheck"
(How could you have one script that has two different names?)
Note that any data that needs to be passed from one time when the script runs to the next time cannot be passed in a temporary file. It will
need to be stored in a secure location.
Certain commands that I thought might be helpful are diff, wc, tail, grep, awk, and sort. Can anyone think of a way to do the above. Any suggestions
are most welcome.
Thanks in Advance for all the help.
2. Relevant commands, code, scripts, algorithms:
Algorithm of the program that I have come up with
Step1: Enter Logname(name of log file) as argument when running script
Step2: Check if argument is given, if not then exit with error else check for filename
Step3: If filename given as argument does not exists, exit with error else check for search string pattern in log file
Step4: if string pattern found then copy log entry to a variable and temp file
Step5: Check for similar log entries ignoring date stamp
Step6: if similar log entry found, increment counter
Step7: if currently running script name is logmon then send email else send message to current terminal
3. The attempts at a solution (include all code and scripts):
Code:
#!/bin/sh -x
#
#
#
PRINTF=/usr/bin/printf
GREP=/usr/bin/grep
TAIL=/usr/bin/tail
#To if log file to check has been given as argument when running the script
if [ $# -eq 0 ] ; then
$PRINTF "%s: Need the name of the log that needs to be checked\n" $0
$PRINTF $0 "\tis the name of the script you are running" #Just to display what is the script name
$PRINTF $1 "\tis the name of the log file you want to scan" #Just to display what is the argument
exit 1
fi
# script to send simple email
SUBJECT="Enter Subject Here" # email subject
RECIPIENT="yourmailid@domain.com" # Email To ?
EMAILMESSAGE="My test script" # Email text/message
/bin/mailx -s "$SUBJECT" "$RECIPIENT" < $EMAILMESSAGE # send an email using /bin/mail
# script to check for string to search
SEARCH='grep -i -E -q 'error|warning'' #search for strings error or warning in any form
SCAN_LOG='tail $1|[${SEACH}]' #scanning logfile for string pattern
#Keep track of similar entries
MESSAGE_COUNTER=0
MESSAGE="$LOG_ENTRY"
4. Complete Name of School (University), City (State), Country, Name of Professor, and Course Number (Link to Course):
RMIT University, Melbourne,Australia,Don,http://www.rmit.edu.au/courses/039989
Note: Without school/professor/course information, you will be banned if you post here! You must complete the entire template (not just parts of it).
---------- Post updated at 10:37 AM ---------- Previous update was at 10:11 AM ----------
The log file structure looks like below
Code:
Apr 26 03:18:54 goanna.cs.rmit.edu.au SRS Proxy[17590]: [ID 663108 daemon.error] Could not upload bulk message
Apr 26 03:21:25 goanna.cs.rmit.edu.au last message repeated 3 times
Apr 26 03:31:41 goanna.cs.rmit.edu.au puppetd[25496]: [ID 702911 daemon.notice] (//Node[server_base]/pca/File[/opt/isunix/output]) Dependency file[/var/tmp/patchdiag.xref] has 1 failures
Apr 26 03:31:41 goanna.cs.rmit.edu.au puppetd[25496]: [ID 702911 daemon.warning] (//Node[server_base]/pca/File[/opt/isunix/output]) Skipping because of failed dependencies
Apr 26 03:31:41 goanna.cs.rmit.edu.au puppetd[25496]: [ID 702911 daemon.notice] Finished catalog run in 134.93 seconds
Apr 26 03:40:55 goanna.cs.rmit.edu.au SRS Proxy[17590]: [ID 663108 daemon.error] Could not upload bulk message
Apr 26 03:44:40 goanna.cs.rmit.edu.au last message repeated 3 times
Apr 26 03:45:56 goanna.cs.rmit.edu.au SRS Proxy[17590]: [ID 388765 daemon.error] Could not send message from queue: /var/SUNWsrspx/SRSQueueStore/store/queues/REGISTRATION/registration
Apr 26 03:49:11 goanna.cs.rmit.edu.au SRS Proxy[17590]: [ID 663108 daemon.error] Could not upload bulk message
Apr 26 03:52:52 goanna.cs.rmit.edu.au SRS Proxy[17590]: [ID 601329 daemon.warning] QueueStore reclaimed 1 entry, 6144 bytes
Apr 26 04:05:59 goanna.cs.rmit.edu.au puppetd[25496]: [ID 702911 daemon.warning] (//Node[server_base]/pca/Exec[pca-generate-missingrs]) Skipping because of failed dependencies
Apr 26 04:05:59 goanna.cs.rmit.edu.au puppetd[25496]: [ID 702911 daemon.notice] (//Node[server_base]/pca/File[/opt/isunix/output]) Dependency file[/var/tmp/patchdiag.xref] has 1 failures
Apr 26 04:05:59 goanna.cs.rmit.edu.au puppetd[25496]: [ID 702911 daemon.warning] (//Node[server_base]/pca/File[/opt/isunix/output]) Skipping because of failed dependencies
Apr 26 04:06:00 goanna.cs.rmit.edu.au puppetd[25496]: [ID 702911 daemon.notice] Finished catalog run in 135.37 seconds
Apr 26 04:06:41 goanna.cs.rmit.edu.au SRS Proxy[17590]: [ID 663108 daemon.error] Could not upload bulk message
Apr 26 04:07:56 goanna.cs.rmit.edu.au SRS Proxy[17590]: [ID 388765 daemon.error] Could not send message from queue: /var/SUNWsrspx/SRSQueueStore/store/queues/REGISTRATION/registration
Apr 26 04:11:11 goanna.cs.rmit.edu.au SRS Proxy[17590]: [ID 663108 daemon.error] Could not upload bulk message
Apr 26 04:22:56 goanna.cs.rmit.edu.au SRS Proxy[17590]: [ID 663108 daemon.error] Could not upload bulk message
Apr 26 04:25:26 goanna.cs.rmit.edu.au last message repeated 2 times
---------- Post updated at 05:11 PM ---------- Previous update was at 10:37 AM ----------
If someone could help me out in rectifying the problem!!!
Code:
#!/bin/sh -x
#
#
#
PRINTF=/usr/bin/printf
GREP=/usr/bin/grep
TAIL=/usr/bin/tail
LOG_FILE="$1"
#To check if log file to check has been given as argument when running the script
if [ $# -ne 0 ] ; then
$PRINTF "%s: Is the name of the log file you will be scanning. \n" $1
$PRINTF $LOG_FILE"\n"
if [ -f $LOG_FILE ]; #if logfile exits
then
echo "File $LOG_FILE exists \n"
# script to check for string to search
SEARCH='grep -i -E -q 'error|warning'' #search for strings error or warning in any form
SCAN_LOG='tail $1|[${SEARCH}]' #scanning logfile for string pattern
# script to send simple email
SUBJECT="Error/Warning messages in logfile" # email subject
RECIPIENT="emailid@domain.extension" # Email To ?
EMAILMESSAGE="SCAN_LOG" # Email text/message
/bin/mailx -s "$SUBJECT" "$RECIPIENT" < $EMAILMESSAGE # send an email using /bin/mail
else
echo "File $LOG_FILE does not exists \n"
fi
else
$PRINTF "%s: Need the name of the log that needs to be checked\n" $0
# EXIT (1)
fi
The output that I got for the above code :
Code:
> sh logmon1 log-out
log-out: Is the name of the log file you will be scanning.
log-out
File log-out exists
logmon1: warning: not found
logmon1: SCAN_LOG: cannot open
Last edited by vin8465; 06-01-2010 at 04:33 AM..
Reason: Script Requirements highlighted
If I execute this statement on the command line, I get an error:
Code:
$ SEARCH='grep -i -E -q 'error|warning''
warning: command not found
You probably meant to use backticks (`) instead of quotes (')
This is sometimes difficult to see, that is why I prefer using $( ... ) instead of `...`
Also if you use grep you have give it something to grep (file or stdin)..
If you reference a variable you have to use a $-sign .
Code:
EMAILMESSAGE="$SCAN_LOG"
This User Gave Thanks to Scrutinizer For This Post:
If I execute this statement on the command line, I get an error:
Code:
$ SEARCH='grep -i -E -q 'error|warning''
warning: command not found
You probably meant to use backticks (`) instead of quotes (')
This is sometimes difficult to see, that is why I prefer using $( ... ) instead of `...`
Also if you use grep you have give it something to grep (file or stdin)..
If you reference a variable you have to use a $-sign .
Code:
EMAILMESSAGE="$SCAN_LOG"
Thanks for pointing that out.
I also noticed that none of the commands like GREP that i defined have $ in front of them. I have corrected them. I will update the code and post it up again. Code someone please help me out with this.. I am finding it very confusing to do the script. Its the first time that i am writing such a script. I have hardly 3 months of experience and that too only by completing labs sheets of my uni.
---------- Post updated at 09:01 PM ---------- Previous update was at 08:15 PM ----------
I have made some improvements from the previous code and rectified the wrongs notified, thanks to
The code now execute without any error but gets stuck and i have to suspend the script manually. I am not able to figure out what the problem is and how am i supposed the implement the logic. Please see code below. I have tried to highlight all new lines of code that i have added:
Code:
#!/bin/sh -x
#
#
#
PRINTF=/usr/bin/printf
GREP=/usr/bin/grep
TAIL=/usr/bin/tail
COPY=/usr/bin/cp
CAT=/usr/bin/cat
LOG_FILE="$1"
LOG_PATH="/var/log/$1"
#To check if log file to check has been given as argument when running the script
if [ $# -ne 0 ] ; then
$PRINTF "%s: Is the name of the log file you will be scanning. \n" $1
$PRINTF $LOG_FILE"\n"
$PRINTF $LOG_PATH"\n"
if [ -f $LOG_PATH ]; #if logfile exits
then
echo "File $LOG_FILE exists \n"
$COPY $LOG_PATH ./
$CAT -n $LOG_FILE > ./tmp_log
# $CAT tmp_log | less
#SEARCH=`GREP -i -E -q 'error|warning' $LOG_PATH` #search for strings error or warning in any form
# script to check for string to search
SEARCH=`$GREP -i 'error|warning'` #search for strings error or warning in any form
# SCAN_LOG=`$TAIL $1|[${SEARCH}]` #scanning logfile for string pattern
# script to send simple email
SUBJECT="Error/Warning messages in logfile" # email subject
RECIPIENT="emailid@domain.extension" # Email To ?
# EMAILMESSAGE="$SCAN_LOG" # Email text/message
EMAILMESSAGE="$SEARCH" # Email text/message
/bin/mailx -s "$SUBJECT" "$RECIPIENT" < $EMAILMESSAGE # send an email using /bin/mail
echo "mail sent"
else
echo "File $LOG_FILE does not exists \n"
fi
else
$PRINTF "%s: Need the name of the log that needs to be checked\n" $0
# EXIT (1)
fi
"logmon1" 48 lines, 1420 characters
The output is this:
Code:
> sh logmon1 log.03
log.03: Is the name of the log file you will be scanning.
log.03
/../../log.03
File log.03 exists
^C
>
Just to add. I also do not receive any e-mail message. Some one please help me here with the logic also. The requirements of the script is highlighted in blue in the original post.
Thanks in advance!!!
I found another problem:
The below command when given in command line gives output if the word error is there in the file
Code:
grep - i 'error' <filename>
Apr 26 03:40:55 goanna.cs.rmit.edu.au SRS Proxy[17590]: [ID 663108 daemon.error] Could not upload bulk message
Apr 26 03:45:56 goanna.cs.rmit.edu.au SRS Proxy[17590]: [ID 388765 daemon.error] Could not send message from queue: /var/SUNWsrspx/SRSQueueStore/store/queues/REGISTRATION/registration
Apr 26 03:49:11 goanna.cs.rmit.edu.au SRS Proxy[17590]: [ID 663108 daemon.error] Could not upload bulk message
But this gives no output even if the string are present in the file that is being searched
Code:
grep -i 'error|warning' <filename>
>
Last edited by vin8465; 06-01-2010 at 08:28 AM..
Reason: Wanted to show how the program doesnt exit without manually exiting it "^C"
So silly of me. But do you think the logic is alright. Any inputs on how should i go about completing the script functionality. I am learning all the commands as am doing the script, but i am already a day over the submission date and dont want to spend too much time trying to analyse what could be done.
I believe that self learning will be better for me in the long run and i intend to do the advanced course of the subject but i do need to submit this script asap so as to pass the subject. So am requesting the experts of the forum to help me out here.
---------- Post updated at 09:42 PM ---------- Previous update was at 09:35 PM ----------
Quote:
Originally Posted by Scrutinizer
Hi Vin8465,
if you use grep you have give it something to grep (file or stdin)..
In this segment you do not specify the input file, so grep is waiting for input.
Code:
SEARCH=`$GREP -i 'error|warning'`
I changed the line into
Code:
SEARCH=`$GREP -i 'error|warning' $1`
but the output seems to be the same.
Code:
> sh logmon1 log.03
log.03: Is the name of the log file you will be scanning.
log.03
/../../log.03
File log.03 exists
^C
>
I am not able to figure out why the script just keeps waiting. Is there a way to check what the script is waiting for?
I too was trying that out. But you found it out before i could. There are no errors now and the script also seems to execute to completion.
Although everything seems to run to perfection, i dont receive any email. Is the command
Code:
SEARCH=`$EGREP -i 'error|warning' $1`
not storing the output to SEARCH?
When i run the script with the grep changed to egrep, i get the output in my terminal.
How should i redirect the output to SEARCH and then put it in the EMAILMESSAGE variable?
I am having trouble getting mail to work on a red hat server. At first I was getting this message.
Diagnostic-Code: X-Postfix; delivery temporarily suspended: connect to :25: Connection refused
Then added the port to my firewall. Then I temporarily turned off selinux. I then copied this file... (1 Reply)
I want to monitor the maillog file in real time and send a mail when a certain grep condition is met. Every time the grep condition is met a mail will be sent. I wish to ignore all grep conditions 30 mins after each mail and thereafter continue monitoring.
For example:
Condition is met, mail... (1 Reply)
Hello,
I wanted to search specific string in the acitve log file and send an email if the search string found in the log.
Log file is written by application all the time. So, script has to search if any new log entry has the specific string for example " sample exception" and send an email. (1 Reply)
Hi
I want to monitor a file even if the file rotate. When a text occurs I want to send a mail.
Something like this but it's not working correctly:
tail -F mylog.log | grep 'MatchMe' |
while read line
do
echo $(date +"%Y-%m-%d %H:%M:%S") MatchMe occurs | mail -s "MatchMe"... (1 Reply)
Hi All
Need help
Can any one share a basic script that is used for monitor sendmail service whether online, offline.etc in solaris
Thanks in advance
Zimmy (5 Replies)
This is my log file and this is live log.
Any abnormal error other than following
I need to generate the email.
Log path : /DER/app/admin/ABC/bdump/erg.log
Current log# 2 seq# 103046 mem# 0: /ora2/oradata/ABC/redo02a.log
Current log# 2 seq# 103046 mem# 1:... (7 Replies)
This is my log file and this is live log.
Any abnormal error other than following
I need to generate the email.
Log path : /DER/app/admin/ABC/bdump/erg.log
Current log# 2 seq# 103046 mem# 0: /ora2/oradata/ABC/redo02a.log
Current log# 2 seq# 103046 mem# 1:... (1 Reply)
i have sun machines having solaris 9 & 10 OS . Now i need to send mail from the machines to my outlook account . I have the ip adress of OUTLOOK mail server. Now what are the setting i need to do in solaris machines so that i can use mailx or sendmail.
actually i am trying to automate the high... (2 Replies)
Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted!
I am stuck and worried. My assignment was due a day ago and I was too busy completing other assignments due during the same time. I worry that not completing this assignment... (1 Reply)
06-01-2010
monitor log entries and send e-mail
