Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
Search Forums:



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

Reply    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 02-09-2012
Registered User
 

Join Date: Sep 2009
Location: Kolkata,India
Posts: 256
Thanks: 17
Thanked 0 Times in 0 Posts
Question Scan a file in realtime and execute certain commands on encountering 5 consecutive identical lines

Mysql log has something like below:

Quote:
1649560 Query select * from actions where uniqueid like 'ernie-1328697839.1233158-%-dequeue' and class='queue' and server='ernie' and time<='1328698322' and action='dequeue' limit 1
1649560 Query select * from actions where uniqueid like 'ernie-1328697839.1233158-%-dequeue' and class='queue' and server='ernie' and time<='1328698322' and action='dequeue' limit 1
1649560 Query select * from actions where uniqueid like 'ernie-1328697839.1233158-%-dequeue' and class='queue' and server='ernie' and time<='1328698322' and action='dequeue' limit 1
1649560 Query select * from actions where uniqueid like 'ernie-1328697839.1233158-%-dequeue' and class='queue' and server='ernie' and time<='1328698322' and action='dequeue' limit 1
1649560 Query select * from actions where uniqueid like 'ernie-1328697839.1233158-%-dequeue' and class='queue' and server='ernie' and time<='1328698322' and action='dequeue' limit 1
1649560 Query select * from actions where uniqueid like 'ernie-1328697839.1233158-%-dequeue' and class='queue' and server='ernie' and time<='1328698322' and action='dequeue' limit 1
1649560 Query select * from actions where uniqueid like 'ernie-1328697839.1233158-%-dequeue' and class='queue' and server='ernie' and time<='1328698322' and action='dequeue' limit 1
1649560 Query select * from actions where uniqueid like 'ernie-1328697839.1233158-%-dequeue' and class='queue' and server='ernie' and time<='1328698322' and action='dequeue' limit 1
I need a bash shell script that will do the following:

1) The script will scan the mysql.log file constantly in real time (something like tail -F mysql.log)
2) If it encounters 5 consecutive identical lines then it would invoke some commands (say echo "yes")

Please help.

Last edited by proactiveaditya; 02-09-2012 at 07:53 AM..
Sponsored Links
    #2  
Old 02-09-2012
methyl methyl is offline Forum Staff  
Moderator
 

Join Date: Mar 2008
Posts: 5,652
Thanks: 205
Thanked 560 Times in 539 Posts
This appears to be a continuation of:
http://www.unix.com/shell-programmin...ignP=302596963

What does "tail -F" do on your Operating System? Not valid on mine. What Operating System and version do you have?

In general it is a major programming exercise to "tail -f" a file and carry out any further processing in a pipeline. There have been posts on this board which suggest using named pipes.
Sponsored Links
    #3  
Old 02-09-2012
balajesuri's Avatar
#! /bin/bash
 

Join Date: Apr 2009
Location: India
Posts: 1,019
Thanks: 9
Thanked 285 Times in 277 Posts
tail -F is same as tail -f --retry


Code:
man tail
       --retry
              keep trying to open a file even if it is inaccessible when tail starts or if it becomes inaccessible later; useful when  following  by  name,  i.e., with --follow=name

Available on GNU/Linux
    #4  
Old 02-09-2012
Scrutinizer's Avatar
mother ate her
 

Join Date: Nov 2008
Location: Amsterdam
Posts: 5,371
Thanks: 80
Thanked 1,107 Times in 1,011 Posts
If you are using GNU/Linux checkout fflush() in gawk:

Code:
tail -F infile | awk 'function ff(){ if(A[p]>4) {print "yes - " p; fflush() ; A[p]=0 } } {ff(); A[$0]++} $0!=p{p=$0} END{ff()}'

*edit* Actually, this should suffice, since the END section will never happen:

Code:
tail -f infile | awk 'A[p]>4{print "yes - " p; fflush(); A[p]=0} $0!=p{p=$0} {A[p]++}'


Last edited by Scrutinizer; 02-09-2012 at 01:30 PM..
Sponsored Links
    #5  
Old 02-09-2012
Registered User
 

Join Date: Sep 2009
Location: Kolkata,India
Posts: 256
Thanks: 17
Thanked 0 Times in 0 Posts
What about:
Quote:
tail -F mysql.log|<THE_BELOW_AWK_SCRIPT>
Quote:
awk 'BEGIN {l="";c=0}
$0 != l { l=$0; c=0 }
l=$0 {c++; if (c>=5) {system("echo Yes")}}'
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Removing consecutive lines in a file deneuve01 Shell Programming and Scripting 11 10-06-2011 02:31 PM
finding and removing 2 identical consecutive words in a text cocostaec Shell Programming and Scripting 11 05-01-2011 10:53 PM
how to delete two consecutive lines from the file aoussenko Shell Programming and Scripting 5 04-12-2011 02:34 PM
Find time difference between two consecutive lines in same file. vilibit Shell Programming and Scripting 6 03-14-2011 06:16 AM
Cutting n consecutive lines from a file... Vishnu UNIX for Dummies Questions & Answers 2 10-18-2002 10:49 AM



All times are GMT -4. The time now is 04:55 AM.