Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Intelligent scaning of log files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Intelligent scaning of log files
# 1  
Old 05-27-2012
Intelligent scaning of log files
Dear experts,
I have a problem and I am not clear on how to attack this. Let me define the problem as simply as possible.

1)There are several log files in a directory
2) Script should open each log file and scan for errors (grep for certain strings - say - error1, error2, error3)
3) If any errors are found - email few lines above and below of that error
4) then at the end of the log file, append some unique string, say , @EMAILED-ERRORS@
5) Next day, we will start from the unique string @EMAILED-ERRORS@ for any new errors in the log files
The problem is how to reach to the last occurrence of @EMAILED-ERRORS@ in a file? Please do not tell me to use "tac" as it is not on the AIX server.
Can this be easily done with SHELL scripts?

Any ideas will be appreciated.
Thanks in advance
-new SW
# 2  
Old 05-27-2012
Your requirement seems a bit strange to me, but here's most of what you need. Tested the tidbits on AIX 6.1 w/minimal Linux RPMs installed on it.

Code:
MailFile="/tmp/mail.tmp"
cp /dev/null ${MailFile}
for file in `find /tmp -type f -name 'foo*' -exec ls {} \;`; do
    [[ `cat ${file} | sed -n 'H; /^@EMAILED-ERRORS@/h; ${g;p;}' | egrep  -c "error1|error2|error3"` -eq 0 ]] && continue
    # Otherwise, we have a match.
    for line in `cat ${file} | sed -n 'H; /^@EMAILED-ERRORS@/h; ${g;p;}' | sed -n '/error[123]/{=;x;}`; do
        [[ $((${line}-5)) -lt 0 ]] && start=0 || start=$((${line}-5))
        [[ $((${line}+5)) -gt `wc -l $file|awk '{print $1}'` ]] && end=`wc -l $file | awk '{print $1}'` || end=$((${line}+5))
        echo "ERROR: Found error on file: ${file} line: ${line}!" >> ${MailFile}
        sed '${start},${end}!d' ${file} >> ${MailFile}
    done
    echo "@EMAILED-ERRORS@" >> ${file}
done
cat ${MailFile} | mail -s "Errors" joe@gmail.com

Should do it for you, not the cleanest, but it'll work.

Edit:
Code:
sed -n 'H; /^@EMAILED-ERRORS@/h; ${g;p;}'

- Print from the last occurrence of our regex to EOF
Code:
sed '${start},${end}!d'

= 'Delete' everything that is not from line # $start to line # $end
Last edited by Scrutinizer; 05-27-2012 at 05:32 AM.. Reason: Clarity on SED statements. mod: code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[Solved] intelligent splitting?

Hi, I want to split a file into multiple ones, with a new file for every line in the old file. Typically it is in this format 0.25 20 35.7143 0.5 31 55.3571 0.85 3 5.35714 1.3 2 3.57143 I can make new files by using split or other simple awk commands. But sometimes, the file is like... (7 Replies)
Discussion started by: jamie_123
7 Replies

2. Shell Programming and Scripting

Need a script for intelligent diff

Hi, I have 2 files which represent data in a Sybase table and I need to run a diff on them, and based on the first column (which is the primary key) in each file, create 3 files, one for inserts, one for deletes and one for updates Example: old.txt contains server1,a,b,c server2,d,e,f... (4 Replies)
Discussion started by: MARKPARE
4 Replies

3. Cybersecurity

how to hide os type from scaning of nmap or nessus

my os is freebsd 7.1 just open sshd and hide the sshd banner nessus still report correctly the os type how to hide os type from scaning of nmap or nessus? Thanx so much. my ipfw rule: ipfwcmd="ipfw -q add allow tcp" localip="192.168.1.254" $ipfwcmd from any to $localip 22 in setup... (2 Replies)
Discussion started by: overdose
2 Replies

4. UNIX for Dummies Questions & Answers

A more intelligent SDIFF

Hi all I have two files which are essentially the same. However the way an exponent is written is different (i.e. in 1 file, a particular number might be written as 1.43230000E+02 whereas in another it might be 1.4323E2). If I use SDIFF then the program will merely check the ASCII characters... (1 Reply)
Discussion started by: robbiegregg
1 Replies

5. Shell Programming and Scripting

more intelligent way of uninstalling a RPM

Hi all, I'm writing an uninstaller for a bespoke piece of software that we deploy to our Linux terminals. One of the packages we install is the JDK (Java Development Kit). Now over the years we have quite a number of different versions installed with different package names. In my uninstaller... (0 Replies)
Discussion started by: _Spare_Ribs_
0 Replies
Login or Register to Ask a Question