Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Grepping log file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grepping log file
# 8  
Old 10-20-2009
man cron and crontab
# 9  
Old 10-20-2009
Np; if you only want the IP:

Code:
sed -ne "s/.*HOST=\([^)]*\)).*/\1/p" infile
10.60.4.2

# 10  
Old 10-20-2009
How can i apply crono or cronotab to my data? I think periods of 1 hour are the best to analze, dont you agree?

I should count the number of ips per hour......so the count i have already made :

Code:
#!/usr/bin/perl -w
use strict;
my %counts = ();


my $file = "access.log";
open (FH, "< $file") or die "Can't open $file for read: $!";
my @lines;
while (<FH>) {
    push (@lines, $_);
}
for (@lines) {
 $counts{$_}++;
 }
 foreach my $keys (keys %counts) {
 print "$keys = $counts{$keys}\n";
}



close FH or die "Cannot close $file: $!";



---------- Post updated at 04:50 AM ---------- Previous update was at 04:23 AM ----------

Cron and cronotab schedule a job to be done according to the local time. But my case is different, so, based on the data i have - log file - how can i collect the date/time and split them into chunks of 1 hour....of course each time is associated to a ip address. Any ideias?
# 11  
Old 10-20-2009
So you want to count how often each IP shows up per hour?
# 12  
Old 10-20-2009
Yes, that is exactly what i want. In the specific hour a chunk of information, the time, date and its mapping to the ip address. Should the files be split into seperate files having for example the title (month, day, hour), i am not sure of the best solution.

---------- Post updated at 06:58 AM ---------- Previous update was at 06:17 AM ----------

Maybe a Array of Hashes could be the best solution ?
# 13  
Old 10-21-2009
Here a point to start at with awk without splitting it up on hours, just taking the whole input file:
Code:
$> cat infile
16-OCT-2009 09:11:47 10.65.4.24
16-OCT-2009 09:11:47 10.3.4.11
16-OCT-2009 10:11:47 10.3.4.11
16-OCT-2009 10:11:47 10.65.4.24
16-OCT-2009 10:11:47 10.3.4.11
16-OCT-2009 11:11:47 10.65.4.24
16-OCT-2009 11:11:47 10.65.4.24
16-OCT-2009 11:11:47 10.3.4.11
$> awk 'NR==FNR{a[$3]+=1; next} END{for(x in a){print x,"->",a[x]}}' infile
10.65.4.24 -> 4
10.3.4.11 -> 4

# 14  
Old 10-21-2009
Thanks for the feedback, this is a brilliant way of treating the data, how could i seperate this log into smaller chunks, lets say seperate files based on hours or days, and f.e. release an alert if the number of counts achieves a certain limit, throwing a message f.e. >= 100 -> "red alert"

---------- Post updated at 05:18 AM ---------- Previous update was at 04:58 AM ----------

I was thinking as having the final output of the file as:

The output of all the times-> corresponding ip of that chunk, for example (between 1 and 2 am) and in the final this counting report.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ksh Script, Reading A File, Grepping A File Contents In Another File

So I'm stumped. First... APOLOGIES... my work is offline in an office that has zero internet connectivity, as required by our client. If need be, I could print out my script attempts and retype them here. But on the off chance... here goes. I have a text file (file_source) of terms, each line... (3 Replies)
Discussion started by: Brusimm
3 Replies

2. Shell Programming and Scripting

Grepping only dates from a log file

Hi All, I have a log file where every line contains a date and some other data, i want to grep only the date from every line to a different file. Please help how to get this. Thanks in advance !! (25 Replies)
Discussion started by: nanz143
25 Replies

3. Shell Programming and Scripting

Grepping file and returning passed variable if the value does not exist in file at all.

I have a list of fields that I want to check a file for, returning that field if it not found at all in the file. Is there a way to do a grep -lc and return the passed variable too rather then just the count? I am doing some crappy work-around now but I was not sure how to regrep this for :0 so... (3 Replies)
Discussion started by: personalt
3 Replies

4. UNIX for Dummies Questions & Answers

grepping log files

I have a log file and I have two unique strings which represent the start and end of the text I want to obtain. How can I get all the text inbetween this start string and the end string? Thanks (2 Replies)
Discussion started by: chrisjones
2 Replies

5. Shell Programming and Scripting

Grepping string from out file

Guys .. Need to pull this highlighted strings irrespective of line numbers & should be echoed . But these strings are from Outfile from different dir. In which way this can be grepped ?? Need an idea http-timeout 120 seconds persistent-timeout 180 seconds host-rewriting on ... (7 Replies)
Discussion started by: raghunsi
7 Replies

6. Shell Programming and Scripting

Grepping the last 30 minutes of a log file...

I need to know if anyone can assist me on how to grab the last (we'll just say "x" minutes) of a log file. How do you tell the grep command without specifying an exact window of time? (So relative instead of absolute.) Thanks, Jon (2 Replies)
Discussion started by: jtelep
2 Replies

7. Shell Programming and Scripting

Grepping Errors in a file

Hey All, I have to grep for an error from a file and get the results of errror in a different file...... But there should be no duplicate entries. Can anyone help me in giving a shell script for this This is file which contains pattern error which I am supposed to grep and put this in a... (4 Replies)
Discussion started by: achararun
4 Replies

8. Shell Programming and Scripting

Loop and grepping into a file

I wrote this script for: 1. Get the Web log for today 2. Give me a list of all the IP addresses that have accessed the web server today 3. Remove a list of known IPs listed in a file (line by line) 4. Mail the final file to selected recipients. I am unable to do part 3. In the script... (3 Replies)
Discussion started by: skotapal
3 Replies

9. UNIX for Dummies Questions & Answers

grepping the first 3 characters from a file

give this a try and let me know if it works grep '^' filename rachael (2 Replies)
Discussion started by: rachael
2 Replies

10. UNIX for Dummies Questions & Answers

grepping the first 3 characters from a file

Hi I was wondering if it's possible to use a command to get the first 3 characters of a line in a text file, I tried grep but it returns the whole line but I am only interested in the first 3 characters. Is this possible with grep or I need any other command? Also is it possible deleting from... (2 Replies)
Discussion started by: g-e-n-o
2 Replies
Login or Register to Ask a Question