Log File


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Log File
# 1  
Old 12-01-2009
Log File

Hi All,

Having a log file with thousounds of lines as:

Code:
01-JAN-2009 15:06:06 15.64.3.22
01-JAN-2009 15:06:07 15.64.3.22

etc...

How can i output a much smaller filtered list:

Grab the ips i need (do the avg in a time range of 30 min for example). And then output the Timestamp and Ip Address in a list.

Thanks in advance.
# 2  
Old 12-01-2009
a begin that will extract the lines containing the desired IP :
Code:
#!/bin/bash
LOGFILE="path_to_your_log_file"
OUTPUTFILE="path_to_your_output_file"
read -p "Give the IP : " IP
grep $IP $LOGFILE # > $OUTPUTFILE
# remove the comment above to record the output in a file

# 3  
Old 12-01-2009
And if i desire more than one?
# 4  
Old 12-01-2009
Create some variables of known information to save on typing and use grep

Code:
my_date='01-JAN-2009'
my_ip='15\.64\.3\.22' #use "\" to escape the "." for grep
time1='15:[0-2].:..'   #I'm setting a time range of 15:00:00 - 15:30:00 with these two
time2='15:30:00'       #time variables, you can adjust as needed with regular expressions

grep -E "${my_date} ${time1} ${my_ip}|${my_date} ${time2} ${my_ip}" file.log


Last edited by rwuerth; 12-01-2009 at 10:38 AM..
# 5  
Old 12-01-2009
Now in this time range i would like to know how i could do the avg on this time range, or for a whole file, do it in a way that it goes throuhg the whole file.....going from 10-10 minutes, counting the duplicates and making an avg, putting it in a new file.
# 6  
Old 12-01-2009
I'd suggest searching this forum for "time math" and looking at the the results for information on that. Many posts on the topic, and it's quite involved, moreso than I can get into here.
# 7  
Old 12-02-2009
So far i have this solution, but it isnt working...it should grab the ip counts, doing an average in 10 minutes...and outputting a result...

Code:
#!/usr/bin/perl

use strict;
use warnings;

my (%k, $t1, %sum, %count, $block) = ( 15.64.3.22=> 1, 10.64.2.23 => 1);

sub report {
  print join (",", ++$block,
    map { $_, $sum{$_} || 0, $count{$_} || 0,
      $count{$_} ? $sum{$_} / $count{$_} : "" } keys %k), "\n";
}

while (<>) {
  chomp;
  my ($date ,$timestamp, $ip) = split (/ /);
  next unless $timestamp;
  my ($h, $m, $s) = split (/:/, $timestamp);
  my $t = 3600*$h + 60*$m + $s; 

  if (! defined $t1 || $t > $t1) {
    report if defined $t1;
    $t1 = $t + 10;
    %sum = %count = ();
  }
  if ($k{$event}) {
    ++$count{$event};
    $sum{$event} += $value;
  }
}

report if %count;

Thanks in advance
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Web Development

How to log http 404 error to a separate log file?

Apache Web Server: how to log http 404 error to a separate log file and i do not want to log in access.log please advice. (2 Replies)
Discussion started by: raghur77
2 Replies

2. Shell Programming and Scripting

Check log file size every 10 minute. Alert if log not update

How to check log size every 10min. by script (can use crontab) if log size not change with alert "Log not update" Base run on SunOS 5.8 Generic_Virtual sun4u sparc SUNW,SPARC-Enterprise logFiles="log1.log log2.log" logLocation="/usr/home/test/log/" Out put. Tue Jan 31... (3 Replies)
Discussion started by: ooilinlove
3 Replies

3. HP-UX

Script to monitor /var/opt/resmon/log/event.log file

AM in need of some plugin/script that can monitor HP-UX file "/var/opt/resmon/log/event.log" . Have written a scrip in sh shell that is working fine for syslog.log and mail.log as having standard format, have interrogated that to Nagios and is working as I required . But same script failed to... (3 Replies)
Discussion started by: Shirishlnx
3 Replies

4. BSD

Kerberos log file does not log when ticket is destroyed

Hi, in the log file there is line when the ticket is issued but when an user destroys the ticket there is no record. Does someone have an idea? (0 Replies)
Discussion started by: gaspar
0 Replies

5. Shell Programming and Scripting

Perl's buffered I/O is causing me to miss latest log file entries in log colorizer. How to fix?

I've been finding myself using a log file colorizer written in perl to reformat and colorize the output from many different programs. Mainly, however, I use it to make the output from "tail -f" commands more readable. The base perl script I use is based on "colorlogs.pl" available from the... (1 Reply)
Discussion started by: rcsteiner
1 Replies

6. Shell Programming and Scripting

Capture all error message in Log file and send the Log file by email

Hi I have a requirement to write a script to capture all errors in a Logfile and send the file in email. If there is any error occurred the subject of email will be ERROR , If there are no error occurred the subject of email will be SUCCESS. So I created a Log file and put the Appropriate... (2 Replies)
Discussion started by: dgmm
2 Replies

7. Shell Programming and Scripting

Delete log file entries based on the Date/Timestamp within log file

If a log file is in the following format 28-Jul-10 ::: Log message 28-Jul-10 ::: Log message 29-Jul-10 ::: Log message 30-Jul-10 ::: Log message 31-Jul-10 ::: Log message 31-Jul-10 ::: Log message 1-Aug-10 ::: Log message 1-Aug-10 ::: Log message 2-Aug-10 ::: Log message 2-Aug-10 :::... (3 Replies)
Discussion started by: vikram3.r
3 Replies

8. Shell Programming and Scripting

Rename a log file to old, then create a new log file.

Hi all, I have about 15 directories all with exactly the same structure. I have a file in each of them called log.txt. This file sits in /home/ftp/*usernamehere*/ftptransfer/log/ Username here is the only change in each of the 15 directories. I want to create a SIMPLE shell script that... (5 Replies)
Discussion started by: mokachoka
5 Replies

9. Shell Programming and Scripting

Rename log file according to log file age

Hi Experts, i am new in shell programming. i am having some problem of checking the log file age and rename it. for example: when i run the shell script, the script will check how old is the log file, if the log is already 10 days old(variable). then the log file will rename. hope to... (3 Replies)
Discussion started by: ewawong
3 Replies

10. Shell Programming and Scripting

alert_oss.log oracle log file in hp-unix

Hi I have go this alert_oss.log that is basically capturing all the oracle errorlogs.Now the problem is that it is one huge file and to see log of some particular date i tried cat alert_oss.log | grep 'Mar 25 10:44:45 2007' > alert_25.txt is not giving me the required output. pls suggest ... (3 Replies)
Discussion started by: Assassin
3 Replies
Login or Register to Ask a Question