Visit Our UNIX and Linux User Community


perl script to filter logfile


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting perl script to filter logfile
# 1  
Old 08-30-2002
perl script to filter logfile

i was wondering if anyone can help me, how could i write in perl a a script that would look through a log file and print onscreen the contents of the log file excluding lines that contain '192.168.1.' and entries that contain directory paths that arent in the directory /usr/local/httpd/htdocs/ i have no expierience in perl. what i want to do is this, i want to be able to run this script to view one log file, the access_log in /var/log/httpd/ at first i would do this with 'grep -v 192.168.1. access_log that works to get rid of my internal network hits, but im tired of havintg to sift through all the people with port scanners hits on my server, most of the traffic on the site is just people running port scanners on my ip block. so if anyone can help i would apreaciate it alot, also i am thinking that this will start me off learning perl. if more info is needed i will gladly provide, thanks alot!
# 2  
Old 08-30-2002
Re: perl script to filter logfile

Quote:
Originally posted by norsk hedensk
i was wondering if anyone can help me, how could i write in perl a a script that would look through a log file and print onscreen the contents of the log file excluding lines that contain '192.168.1.' and entries that contain directory paths that arent in the directory /usr/local/httpd/htdocs/ i have no expierience in perl. what i want to do is this, i want to be able to run this script to view one log file, the access_log in /var/log/httpd/ at first i would do this with 'grep -v 192.168.1. access_log that works to get rid of my internal network hits, but im tired of havintg to sift through all the people with port scanners hits on my server, most of the traffic on the site is just people running port scanners on my ip block. so if anyone can help i would apreaciate it alot, also i am thinking that this will start me off learning perl. if more info is needed i will gladly provide, thanks alot!
Actually, you can do this with a piped grep command if I am following you correctly.

How about this?

Code:
grep -v "192.168.1" access_log | grep -v "/usr/local/httpd/htdocs/" | less

# 3  
Old 08-30-2002
that would work but i want to only see the entries that pertain to files that exist on the server, what i want to do is filter out log file entries that contain paths to files that dont exist, actually now that i think about i i think that i could do what you suggested except i will exclude the error number 404 to see only succesful hits. thanks for you help! haha i know i would not have thought of that if you didnt suggest piping grep, thank you!
# 4  
Old 08-30-2002
that worked great, that saved me ALOT of time!!! what it ends up is
grep -v 192.168.1. access_log | grep -v 404 | more (or less!)

thanks again for the help
# 5  
Old 08-30-2002
Quote:
Originally posted by norsk hedensk
that worked great, that saved me ALOT of time!!! what it ends up is
grep -v 192.168.1. access_log | grep -v 404 | more (or less!)

thanks again for the help
Not a problem. Always glad to help.

*Assumes manly muscle pose, once again, co-workers look puzzled * Smilie

Previous Thread | Next Thread
Test Your Knowledge in Computers #37
Difficulty: Easy
The hard disk space dedicated for swapping out processes on a memory-constrained Unix system is called the 'swap channel'.
True or False?

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Logfile monitoring with logfile replacement

Bonjour, I've wrote a script to monitor a logfile in realtime. It is working almost perfeclty except for two things. The script use the following technique : tail -fn0 $logfile | \ while read line ; do ... some stuff done First one, I'd like a way to end the monitoring script if a... (3 Replies)
Discussion started by: Warluck
3 Replies

2. Shell Programming and Scripting

Perl script to filter the string

Hi folks, I have a log file with the lines in the below format. Jul 1 23:00:51 10.212.3.251 SS: %SYS-7-CLI_SCHEDULE: some error occured I want to split the line based on the " %SYS-7-CLI_SCHEDULE: " value. The criteria is the should store the word that starts with % i.e., ... (1 Reply)
Discussion started by: scriptscript
1 Replies

3. UNIX for Dummies Questions & Answers

perl filter unique and concatenate

Hi experts, I have some input like below, TEST A function W TEST A function X TEST B function Y TEST C function Z TEST C function ZY i would like to have below output, TEST A function W&X TEST B function Y TEST C function Z&ZY Please kindly help on this, i am cracking my head... (2 Replies)
Discussion started by: mingfatty
2 Replies

4. Shell Programming and Scripting

Unable to create logfile with local time stamp using perl

Hello All, Morning, I am facing problem with my code while creating a log with name as current time stamp using perl. Here is the code. #!/usr/bin/perl my $time=localtime; my ($day,$month,$date,$tm,$year)=split(/ /,$time); my $stamp=$year."_".$month."_".$date; my... (4 Replies)
Discussion started by: krsnadasa
4 Replies

5. Shell Programming and Scripting

Logfile monitor script

Hi, I'm trying to write a logfile monitor script that reads the logfile and then emails out once there is an error with SQL in. Here is my attempt below which does not work. I'm not much of a scripter as you can probably see but some pointers in the right direction would be much appreciated. ... (3 Replies)
Discussion started by: elmesy
3 Replies

6. Shell Programming and Scripting

Logfile rotation script.

I'm trying to find or create a Perl script that: Checks for and creates these files: notes notes.1 notes.2 notes.3 notes.4 The first represents the current log file and the others are older versions. Each time the script runs it would check for the existence of notes.3 and, if it exists,... (3 Replies)
Discussion started by: HardyV2
3 Replies

7. Shell Programming and Scripting

Attaching a logfile to the Script

Hello guys. I've recently written a basic utilities script just for home use. and i want to attach a logfile to it that will record all the commands that where executed in that script. Then just so i can add the d%b%y% and make each logfile unique and i can look back in each logfile to see what i... (9 Replies)
Discussion started by: matt02
9 Replies

8. Shell Programming and Scripting

Script to manipulate logfile text

Hi guys, I was wandering if a Shell guru could give me some advice on tackling a problem. I have used a mixture of grep, cut and awk to get data from a log file in the following format: 14/11/08 10:39: Checking currenly : Enabled 14/11/08 10:39: Records allocated : 221... (11 Replies)
Discussion started by: rosspaddock
11 Replies

Featured Tech Videos