Count of matched pattern occurences by minute and date in a log file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Count of matched pattern occurences by minute and date in a log file
# 1  
Old 02-11-2012
Count of matched pattern occurences by minute and date in a log file

Anyone knows how to use AWK to achieve the following

Sun Feb 12 00:41:01-00:41:59 Success:2 Fail:2
Sun Feb 12 00:42:01-00:42:59 Success:1 Fail:2
Sun Feb 12 01:20:01-01:20:59 Success:1 Fail:2
Mon Feb 13 22:41:01-22:41:59 Success:1 Fail:1

log file:

[Sun Feb 12 00:41:17 2012] Success
[Sun Feb 12 00:41:19 2012] Success
[Sun Feb 12 00:41:50 2012] Fail
[Sun Feb 12 00:41:57 2012] Fail
[Sun Feb 12 00:42:03 2012] Fail
[Sun Feb 12 00:42:17 2012] Success
[Sun Feb 12 00:42:18 2012] Fail
[Sun Feb 12 01:20:10 2012] Fail
[Sun Feb 12 01:20:18 2012] Success
[Sun Feb 12 01:20:34 2012] Fail
[Mon Feb 13 22:41:13 2012] Success
[Mon Feb 13 22:41:18 2012] Fail
# 2  
Old 02-11-2012
Have a go with this:


Code:
awk '
    {
        split( $4, a, ":" )
        base = sprintf( "%s %s %s %s:%s", substr( $1, 2 ), $2, $3, a[1], a[2] );
        if( !seen[base]++ )
        {
            order[++oidx] = base;
            time[base] = sprintf( "%s:%s", a[1], a[2] );
        }
        count[base " " $NF]++;
    }

    END {
        for( i = 1; i <= oidx; i++ )
            printf( "%s:00-%s:59 Success: %d  Fail: %d\n", order[i], time[order[i]], count[order[i] " Success"], count[order[i] " Fail"] );
    }' input-file


This preserves the input order which I assumed was desired.
This User Gave Thanks to agama For This Post:
# 3  
Old 02-11-2012
thanks mate!
# 4  
Old 02-12-2012
Nice! What if i want it be calculated by hour instead of minute? which part i need to change?
# 5  
Old 02-12-2012
Since the log entries are already in de right order we could do without arrays:

Code:
awk -F'[][ \t:]*' '
      function pr(){
        print p,"Success:"s,"Fail:"f
      }
      {
         n=$2" "$3" "$4" "$5":"$6":00-"$5":"$6":59"
      }
      n!=p{
         pr()
         s=f=x
         p=n
      }
      $NF=="Success"{
         s++
      }
      $NF=="Fail"{
         f++
      }
      END{
         pr()
      }' infile

For Hours use:
Code:
n=$2" "$3" "$4" "$5":00-"$5":59"


Last edited by Scrutinizer; 02-12-2012 at 06:21 AM..
# 6  
Old 02-12-2012
Your doesn't seem accurate though...

the output seems weird too.

Sat Feb 11 14:23:00-14:23:59 Success:4 Fail:2
Sat Feb 11 13:23:00-13:23:59 Success: Fail:
Sat Feb 11 14:23:00-14:23:59 Success:3 Fail:3
Sat Feb 11 13:23:00-13:23:59 Success: Fail:
Sat Feb 11 14:23:00-14:23:59 Success:1 Fail:2
Sat Feb 11 14:24:00-14:24:59 Success:2 Fail:2
Sat Feb 11 13:24:00-13:24:59 Success: Fail:
Sat Feb 11 14:24:00-14:24:59 Success:1 Fail:2
Sat Feb 11 13:24:00-13:24:59 Success: Fail:
Sat Feb 11 14:24:00-14:24:59 Success:6 Fail:2
Sat Feb 11 13:24:00-13:24:59 Success: Fail:
Sat Feb 11 14:24:00-14:24:59 Success:7 Fail:2
Sat Feb 11 13:24:00-13:24:59 Success: Fail:

---------- Post updated at 01:54 AM ---------- Previous update was at 01:51 AM ----------

Agama.. mind sharing which part i need to change in order to calculated by hour?
# 7  
Old 02-12-2012
@timmywong: It will only work if the log file is in chronological order, which I thought might be the case based on your sample. What OS is this?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Count occurences of a character in a file by sorting results

Hello, I try to sort results of occurences in an array by using awk but I can't find the right command. that's why I'm asking your help ! :) Please see below the command that I run: awk '{ for ( i=1; i<=length; i++ ) arr++ }END{ for ( i in arr ) { print i, arr } }' dictionnary.txt ... (3 Replies)
Discussion started by: destin45
3 Replies

2. Shell Programming and Scripting

ksh : need to get the 4 th line above and 2 nd below the matched pattern in the log file

I have a log file as given below 012/01/21 10:29:02 (111111) Processing Job '23_369468343464564' 2012/01/21 10:29:02 (111111) Making Job '23_369468343464564.0'... 2012/01/21 10:29:04 (111111) Jobnumber '23_369468343464564' was successful 2012/01/21 10:29:04 ... (12 Replies)
Discussion started by: rpm120
12 Replies

3. Shell Programming and Scripting

How to use sed to search a particular pattern in a file backward after a pattern is matched.?

Hi, I have two files file1.txt and file2.txt. Please see the attachments. In file2.txt (which actually is a diff output between two versions of file1.txt.), I extract the pattern corresponding to 1172c1172. Now ,In file1.txt I have to search for this pattern 1172c1172 and if found, I have to... (9 Replies)
Discussion started by: saurabh kumar
9 Replies

4. UNIX for Dummies Questions & Answers

Count pattern occurences

hi, I have a text..and i need to find a pattern in the text and count to the no of times the pattern occured. i have used grep command ..but the problem is , it shows the occurrences of the pattern but doesn't count no of times the pattern occuries. (5 Replies)
Discussion started by: nvnni
5 Replies

5. Shell Programming and Scripting

Count of matched pattern occurences by time in a log file

We have a log file, the format is similar to this: 08/04/2011 05:03:08 Connection Success 08/04/2011 05:13:18 Connection Success 08/04/2011 05:23:28 Connection Fail 08/04/2011 05:33:38 Connection Success 08/04/2011 06:14:18 Connection Success 08/04/2011 06:24:28 Connection Fail 08/04/2011... (6 Replies)
Discussion started by: clu
6 Replies

6. Shell Programming and Scripting

How to count the number of occurences of this pattern?

Hi all, I have a pattern like this in a file: 123 4 56 789 234 5 67 789 121 3 56 789 222 4 65 789 321 6 90 100 478 8 40 789 243 7 80 789 How can I count the number of occurences of '789' (4th column) in this set...? Thanks for all your help! K (7 Replies)
Discussion started by: kripssmart
7 Replies

7. UNIX for Dummies Questions & Answers

How to count the occurences of a specific word in a file in bash shell

Hello, I want to count the occurences of a specific word in a .txt file in bash shell. Can somebody help me pleaze?? Thanks!!! (2 Replies)
Discussion started by: mskart
2 Replies

8. UNIX for Dummies Questions & Answers

Count of matched pattern occurance

In a file a pattern is occured many times randomly. Even it may appear more then once in the same line too. How i can get the number of times that pattern appeared in the file? let the file name is abc.txt and the pattern is "xyz". I used the following code: grep -ic "xyz" abc.txt but it is... (3 Replies)
Discussion started by: palash2k
3 Replies

9. HP-UX

count occurences of specific character in the file

For counting the occurences of specific character in the file I am issuing the command grep -o 'character' filename | wc -w It works in other shells but not in HP-UX as there is no option -o for grep. What do I do now? (9 Replies)
Discussion started by: superprogrammer
9 Replies

10. Shell Programming and Scripting

grep and count no of occurences in every line of a file

Hi Folks, I have a input file of the below format. ~~~OLKIT~OLKIT~1~~TBD~BEST PAGER & WIRELESS~4899 COMMON MARKET PLACE~~~DUBLIN~KS~43016~I~Y~DIRECT~D~~0 BPGRWRLS~~~OLKIT~OLKIT~1~~TBD~BEST PAGER & WIRELESS~4899 COMMON MARKET PLACE~~~DUBLIN~KS~43016~I~Y~DIRECT~D~~0... (12 Replies)
Discussion started by: srikanthgr1
12 Replies
Login or Register to Ask a Question