Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Searching for a string in a log file with little movement

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Searching for a string in a log file with little movement
# 1  
Old 03-27-2010
Searching for a string in a log file with little movement
I have a script which tails a log file and if it finds certain strings in the data tailed it sends an email, basically like this:

Code:
tail -f logfile > tmp.file &
sleep 10
kill $!
STRING=$(grep -c "string" tmp.file)

[ $STRING -gt 2 ] && echo $STRING | mailx -s "Warning.." admin@123.com

When the string is found I sleep the script for 15mins in order to suppress duplicate alerts.. but sometimes nothing is written to the error log in hours so tail -f will find the same string again when the script reruns.

Anyone have a way around this, basically i want to search for the string only if there is activity in the error log.
# 2  
Old 03-27-2010
Why not monitor the latest time stamp on the logfile, it will tell you when it's last been modyfied, if it differs from last check, then there are some lines that has been written to it, tail them out and search for your string.
ie:
Code:
 ~> ls -l /var/log/messages
-rw-r----- 1 root adm 520646 27 mar 21:41 /var/log/messages
~> date
sat mar 27 21:52:07 CET 2010
~> /etc/init.d/sysklogd restart
Restarting system log daemon....
~> ls -l /var/log/messages
-rw-r----- 1 root adm 520733 27 mar 21:52 /var/log/messages

You can even use some "wc -l" on the file to make sure it's only the latest written lines you're tail'ing from it.
# 3  
Old 03-27-2010
empty tmp.file first.

Code:
rm -f tmp.file

# 4  
Old 03-27-2010
This problem seems to be due to the way tail -f works, it writes the last ten lines of the log file to the tmp.file even if there isn't any movement in the log file. I wonder is it possible to turn off that behavior?

Anyway I am going try get round this by using the file attributes of the error log, thanks Redhead.

Hi rdcwayx, i already null the tmp.file before the script exists.
# 5  
Old 03-27-2010
what about the -s (sleep interval) option of tail ?
# 6  
Old 03-27-2010
Hi Frans, how would you use the -s argument in this case?
# 7  
Old 03-27-2010
Hi moxy,

i'm not sure if its what exactly you want but try it, then tell us if it works or not, i couldnt test my code:

Code:
occ=`grep -c "string" logfile`

num_of_occ=`awk -v pattern="string" -v last="$occ" 
'$0~pattern{i++} i==last{print NR;exit}' logfile`

sleep 900

occ_=`grep -c "string" logfile`

num_of_last_occ=`awk -v pattern="string" -v lastocc="$occ_2" 
'$0~pattern{i++} i==lastocc{print NR;exit}' logfile`

if [ $num_of_occ -lt $num_of_last_occ ]; then mailx -s "WARNING!" admin@123.com

Actually only "grep -c" could be enough too Smilie

Code:
occ=`grep -c "string" logfile`

sleep 900

occ_=`grep -c "string" logfile`

if [ occ -lt occ_ ]; then mailx -s "WARNING!" admin@123.com

Last edited by EAGL€; 03-27-2010 at 07:43 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Searching a string in a particular file name

Hello, I have a file name like FIRST_DPF_DAILY_CUST_0826152322.txt i need to extract the string after the third "_" underscore upto timestamp ends i.e CUST_0826152322 can anyone help me with the code Thank you! Regards Srikanth Sagi (3 Replies)
Discussion started by: srikanth_sagi
3 Replies

2. Shell Programming and Scripting

Searching a string stored in other file

I need to design a script which can do the following: I have two files abc.txt and constant.hmtl abc.txt contains some 5 string that I need to see if they exist in contants.html. Constants.hmtl is a very large file around 800 lines. I want to search all the strings present in file... (8 Replies)
Discussion started by: hemasid
8 Replies

3. Shell Programming and Scripting

Searching a string in a file using perl

Hi I would like to read a file using perl and search for a string (last entry). Then read that into an array and do further grep File content for ex: comp=a,value=30,runtime=12,type=lic comp=d,value=15,runtime=2,type=lic comp=a,value=90,runtime=43,type=lic... (1 Reply)
Discussion started by: vivek_damodaran
1 Replies

4. Shell Programming and Scripting

Searching for a specific string in a file

Hi I am trying to search for a certain set of patterns within a file, and then perform other commands based on output. testfile contents: password requisite pam_cracklib.so lcredit=-1 ucredit=-1 ocredit=-1 script: D="dcredit=-1" if then echo $D exists else echo $D doesnt... (8 Replies)
Discussion started by: bludhemn
8 Replies

5. Shell Programming and Scripting

searching each file for a string

Hi Guys... I want to search for each file that contains a particular string. e.g find . -print | xargs grep -i string_name Now my issue is the files that I search in are gzipped. Will I be able to find the string, using the above commands, even if the files are gzipped? Please... (2 Replies)
Discussion started by: Phuti
2 Replies

6. Shell Programming and Scripting

Searching a string in a file

Hi, I am new to unix shell scripting. I have a requirement. Could anyone help me writing the script for the same? Here goes the requirement: I have a config file let's say temp.config. Here is the data in the config file temp.config : ------------- name=victor age=42 state=texas... (5 Replies)
Discussion started by: badrimohanty
5 Replies

7. UNIX for Dummies Questions & Answers

searching for a string in a file

I need to search for a specific string in a file and if this string exist I need to replace it with something else. I am not sure how I could do this, using an if statement. (2 Replies)
Discussion started by: ROOZ
2 Replies

8. Shell Programming and Scripting

Extracting a string from one file and searching the same string in other files

Hi, Need to extract a string from one file and search the same in other files. Ex: I have file1 of hundred lines with no delimiters not even space. I have 3 more files. I should get 1 to 10 characters say substring from each line of file1 and search that string in rest of the files and get... (1 Reply)
Discussion started by: mohancrr
1 Replies

9. Shell Programming and Scripting

Complicated string searching in a file

Hi folks, Following a part of opmn.xml file: <process-type id="OC4J_RiGHTv_PLATOR81" module-id="OC4J"> <environment> <variable id="LD_LIBRARY_PATH" value="/home/ias/v10.1.2/lib" append="true"/> <variable id="SHLIB_PATH"... (5 Replies)
Discussion started by: nir_s
5 Replies

10. UNIX for Dummies Questions & Answers

searching for a string though file system

Is there a way to search an entire file system for the occurance of a string..... other than grep. I have a large directory structure and I'm not certain that grep <string> */*/*/*... is all that effective - especially as I can't be sure of the number of levels to go down without heaps of... (3 Replies)
Discussion started by: peter.herlihy
3 Replies
Login or Register to Ask a Question