Count number of errors within logs for last 6 months


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Count number of errors within logs for last 6 months
# 1  
Old 02-27-2012
Count number of errors within logs for last 6 months

I have directory /test/logs which has multiple logs:

audit.log
audit.log.1
audit.log.2
audit.log.3
audit.log.4
audit.log.5

audit.log is current log file and audit.log.X are archive log files. I need to search within these log files and count word "error-5" logged within last 6 months (180 days). Any entries of "error-5" older than 6 months should be ignored. These log files have following date format: 2011 May 09 04:04:38:289.

Any ideas?

Thank you,
djanu
# 2  
Old 02-27-2012
If you have GNU date this should work:
Code:
awk -v D=$(date -d " -180 days" +%Y%m%d) '
BEGIN {
    split("Jan,Feb,Mar,Apr,May,Jun,Jul,Aug,Sep,Oct,Nov,Dec", T, ",")
    for(i=1;i<13;i++) M[T[i]]=sprintf("%02d", i);
}
($1M[$2]$3)+0 < D { F++}
END { print "Found " F " errors" } ' /test/audit.log*

# 3  
Old 02-27-2012
Run with the below perl script (d180.pl)
Code:
cat audit.log* | ./d180.pl


Code:
#! /usr/bin/perl

use Time::Local;

%mon2num = qw(
  Jan 0  Feb 1  Mar 2  Apr 3  May 4  Jun 5
  Jul 6  Aug 7  Sep 8  Oct 9  Nov 10 Dec 11
);

$now=time();
$day180=86400*180;

sub lessthan180 {
        local $rc = 0;
        ($yr, $mth, $day, $hms) = split(/ /);
        ($hr, $min, $sec) = split(/:/, $hms);

        # strip all leading zero
        $day =~ s/^0*//;
        $hr  =~ s/^0*//;
        $min =~ s/^0*//;
        $sec =~ s/^0*//;

        $diff = $now - timelocal($sec, $min, $hr, $day, $mon2num{$mth}, $yr);
        if ( $diff < $day180 ) {
                $rc = 1;
        }
        return $rc;
}


while ( <STDIN> ) {
        chomp();
        if ( lessthan180($_) == 1 && /error-5/ ) {
                print $_, "\n";
        }
}

# 4  
Old 02-28-2012
Thank you! Perl script works. I have one more question:

1. What if I had a file errorcodes.txt with a list of multiple error codes:

error-5
error-4
error-3
error-800

Can Perl script count how many matches of each error code it finds within 180 days. Something like:

error-5 8
error-4 0
error-3 29
error-800 2
# 5  
Old 02-28-2012
Here it is in awk:

Code:
awk -v D=$(date -d " -180 days" +%Y%m%d) ' 
BEGIN {
    split("Jan,Feb,Mar,Apr,May,Jun,Jul,Aug,Sep,Oct,Nov,Dec", T, ",")
    for(i=1;i<13;i++) M[T[i]]=sprintf("%02d", i);
}
NR==FNR{A[$0]=0;next}
($1M[$2]$3)+0 < D {for(s in A) if($0 ~ s) A[s]++}
END{for(s in A)print s" "A[s]}' errorcodes.txt /test/audit.log*

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to grep logs for Errors

Hi Guys, I want to write a script which can grep the logs (server.log) from a file for Error String and output to a other file. Problems: How to know about the errors only between the current restart and not in previous as server.log has earlier restarts also? thanks for the help! Much... (5 Replies)
Discussion started by: ankur328
5 Replies

2. Shell Programming and Scripting

Naming number to months

Hi, I have many files of monthly means from model simulation such as 1,2,3,4.....12, corresponding to jan, feb, mar,....., dec. e.g. avg_1.nc, avg_2.nc, ............., avg_12.nc. i want to write a shell script such that it should conveet number into months and year such as..... (4 Replies)
Discussion started by: mahesh shinde
4 Replies

3. Shell Programming and Scripting

How to grep logs for errors and receive specific additional lines?

Hi there, I have a script that I've used to find errors in my Minecraft Server logs. But I'd like to refine that script to be more useful. Here is the script: grep -n "SEVERE" /minecraft/server.log | awk -F":" '{print $1-2 "," $1+10 "p"}' | xargs -t -i sed -n {} /minecraft/server.log >>... (15 Replies)
Discussion started by: nbsparks
15 Replies

4. UNIX for Dummies Questions & Answers

Scan logs for errors in the last hour only.

Hi there. Is there a way to scan a specific log file for errors that occurred in the last hour (time when script is run - 60 minutes)? I have a script that will change to a directory where the log files are kept and will then grep the files for defined strings, but I need to make sure that... (2 Replies)
Discussion started by: jimbojames
2 Replies

5. Shell Programming and Scripting

Pattern count on rotating logs for the past 1 Hr

Hi All, I have a requirement to write a shell script to search the logs in past 1 hour and extract some pattern from it and count it cumulatively to a file. The problem which I'm facing here is - logs rotates on size basis, say if size of log reaches 5 MB then new log will be generated and... (7 Replies)
Discussion started by: Gem_In_I
7 Replies

6. Shell Programming and Scripting

Command to Count the files which is Older than 3 months?

Hi Gurus the count of files in a particular Directory... ls -lrth | grep -c ^- can any one share the command to Count the files which is Older than 3 months So please help me out in this Thanks in Advance (2 Replies)
Discussion started by: SeenuGuddu
2 Replies

7. Shell Programming and Scripting

line count with if /else - syntax errors

this is the script: ps -ef|grep "x_jobstat 10 v001" > jobstatv001.txt ps -ef |grep "x_jobserver 10 v001" >> jobstatv001.txt #Sample text - each line preceded by 4 spaces # root 133064 102986 0 08:49:28 pts/6 0:00 grep x_jobstat 10 v001 # root 137550 1 0 Nov 08 - 0:28... (6 Replies)
Discussion started by: kwalkner
6 Replies

8. Shell Programming and Scripting

how to add the number of row and count number of rows

Hi experts a have a very large file and I need to add two columns: the first one numbering the incidence of records and the another with the total count The input file: 21 2341 A 21 2341 A 21 2341 A 21 2341 C 21 2341 C 21 2341 C 21 2341 C 21 4567 A 21 4567 A 21 4567 C ... (6 Replies)
Discussion started by: juelillo
6 Replies

9. Shell Programming and Scripting

To find out the logs count

Hi, I am using the below command to find out the mail logs which will grep the repeated message ids: less /var/log/messages |awk '{print +$6}'| sort | uniq -c | sort -nr OUTPUT: 506 1246382279 404 1246373467 303 1246383457 303 1246382268 300 1246379705 202... (7 Replies)
Discussion started by: gsiva
7 Replies
Login or Register to Ask a Question