Visit Our UNIX and Linux User Community


Grepping log file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grepping log file
# 8  
Old 10-20-2009
man cron and crontab
# 9  
Old 10-20-2009
Np; if you only want the IP:

Code:
sed -ne "s/.*HOST=\([^)]*\)).*/\1/p" infile
10.60.4.2

# 10  
Old 10-20-2009
How can i apply crono or cronotab to my data? I think periods of 1 hour are the best to analze, dont you agree?

I should count the number of ips per hour......so the count i have already made :

Code:
#!/usr/bin/perl -w
use strict;
my %counts = ();


my $file = "access.log";
open (FH, "< $file") or die "Can't open $file for read: $!";
my @lines;
while (<FH>) {
    push (@lines, $_);
}
for (@lines) {
 $counts{$_}++;
 }
 foreach my $keys (keys %counts) {
 print "$keys = $counts{$keys}\n";
}



close FH or die "Cannot close $file: $!";



---------- Post updated at 04:50 AM ---------- Previous update was at 04:23 AM ----------

Cron and cronotab schedule a job to be done according to the local time. But my case is different, so, based on the data i have - log file - how can i collect the date/time and split them into chunks of 1 hour....of course each time is associated to a ip address. Any ideias?
# 11  
Old 10-20-2009
So you want to count how often each IP shows up per hour?
# 12  
Old 10-20-2009
Yes, that is exactly what i want. In the specific hour a chunk of information, the time, date and its mapping to the ip address. Should the files be split into seperate files having for example the title (month, day, hour), i am not sure of the best solution.

---------- Post updated at 06:58 AM ---------- Previous update was at 06:17 AM ----------

Maybe a Array of Hashes could be the best solution ?
# 13  
Old 10-21-2009
Here a point to start at with awk without splitting it up on hours, just taking the whole input file:
Code:
$> cat infile
16-OCT-2009 09:11:47 10.65.4.24
16-OCT-2009 09:11:47 10.3.4.11
16-OCT-2009 10:11:47 10.3.4.11
16-OCT-2009 10:11:47 10.65.4.24
16-OCT-2009 10:11:47 10.3.4.11
16-OCT-2009 11:11:47 10.65.4.24
16-OCT-2009 11:11:47 10.65.4.24
16-OCT-2009 11:11:47 10.3.4.11
$> awk 'NR==FNR{a[$3]+=1; next} END{for(x in a){print x,"->",a[x]}}' infile
10.65.4.24 -> 4
10.3.4.11 -> 4

# 14  
Old 10-21-2009
Thanks for the feedback, this is a brilliant way of treating the data, how could i seperate this log into smaller chunks, lets say seperate files based on hours or days, and f.e. release an alert if the number of counts achieves a certain limit, throwing a message f.e. >= 100 -> "red alert"

---------- Post updated at 05:18 AM ---------- Previous update was at 04:58 AM ----------

I was thinking as having the final output of the file as:

The output of all the times-> corresponding ip of that chunk, for example (between 1 and 2 am) and in the final this counting report.

Previous Thread | Next Thread
Test Your Knowledge in Computers #457
Difficulty: Easy
NTP was created to synchronize all participating computers to within a few milliseconds of Coordinated Universal Time (UTC).
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ksh Script, Reading A File, Grepping A File Contents In Another File

So I'm stumped. First... APOLOGIES... my work is offline in an office that has zero internet connectivity, as required by our client. If need be, I could print out my script attempts and retype them here. But on the off chance... here goes. I have a text file (file_source) of terms, each line... (3 Replies)
Discussion started by: Brusimm
3 Replies

2. Shell Programming and Scripting

Grepping only dates from a log file

Hi All, I have a log file where every line contains a date and some other data, i want to grep only the date from every line to a different file. Please help how to get this. Thanks in advance !! (25 Replies)
Discussion started by: nanz143
25 Replies

3. Shell Programming and Scripting

Grepping file and returning passed variable if the value does not exist in file at all.

I have a list of fields that I want to check a file for, returning that field if it not found at all in the file. Is there a way to do a grep -lc and return the passed variable too rather then just the count? I am doing some crappy work-around now but I was not sure how to regrep this for :0 so... (3 Replies)
Discussion started by: personalt
3 Replies

4. UNIX for Dummies Questions & Answers

grepping log files

I have a log file and I have two unique strings which represent the start and end of the text I want to obtain. How can I get all the text inbetween this start string and the end string? Thanks (2 Replies)
Discussion started by: chrisjones
2 Replies

5. Shell Programming and Scripting

Grepping string from out file

Guys .. Need to pull this highlighted strings irrespective of line numbers & should be echoed . But these strings are from Outfile from different dir. In which way this can be grepped ?? Need an idea http-timeout 120 seconds persistent-timeout 180 seconds host-rewriting on ... (7 Replies)
Discussion started by: raghunsi
7 Replies

6. Shell Programming and Scripting

Grepping the last 30 minutes of a log file...

I need to know if anyone can assist me on how to grab the last (we'll just say "x" minutes) of a log file. How do you tell the grep command without specifying an exact window of time? (So relative instead of absolute.) Thanks, Jon (2 Replies)
Discussion started by: jtelep
2 Replies

7. Shell Programming and Scripting

Grepping Errors in a file

Hey All, I have to grep for an error from a file and get the results of errror in a different file...... But there should be no duplicate entries. Can anyone help me in giving a shell script for this This is file which contains pattern error which I am supposed to grep and put this in a... (4 Replies)
Discussion started by: achararun
4 Replies

8. Shell Programming and Scripting

Loop and grepping into a file

I wrote this script for: 1. Get the Web log for today 2. Give me a list of all the IP addresses that have accessed the web server today 3. Remove a list of known IPs listed in a file (line by line) 4. Mail the final file to selected recipients. I am unable to do part 3. In the script... (3 Replies)
Discussion started by: skotapal
3 Replies

9. UNIX for Dummies Questions & Answers

grepping the first 3 characters from a file

give this a try and let me know if it works grep '^' filename rachael (2 Replies)
Discussion started by: rachael
2 Replies

10. UNIX for Dummies Questions & Answers

grepping the first 3 characters from a file

Hi I was wondering if it's possible to use a command to get the first 3 characters of a line in a text file, I tried grep but it returns the whole line but I am only interested in the first 3 characters. Is this possible with grep or I need any other command? Also is it possible deleting from... (2 Replies)
Discussion started by: g-e-n-o
2 Replies

Featured Tech Videos