03-20-2011
How to find ip addresses in logfiles?
Hi guys,
I need to check a few log files as below to find out whether certain ip addresses is present on these log files.
type8code0: ls -alt
-rw-r--r-- 1 root other 796219588 Mar 20 02:25 logfile
drwxr-xr-x 2 root root 1536 Mar 20 02:00 .
-rw-r--r-- 1 root other 1854093343 Mar 20 02:00 logfile.hour02
-rw-r--r-- 1 root other 366729263 Mar 20 01:00 logfile.hour01
-rw-r--r-- 1 root other 9001399293 Mar 20 00:47 logfile.20.Z
-rw-r--r-- 1 root other 8267721901 Mar 19 00:45 logfile.19.Z
-rw-r--r-- 1 root other 7498682761 Mar 18 00:39 logfile.18.Z
-rw-r--r-- 1 root other 6196926607 Mar 17 00:31 logfile.17.Z
-rw-r--r-- 1 root other 4794493570 Mar 16 00:23 logfile.16.Z
I've saved the list of ip addresses in “iplist” file.
cat iplist
10.10.10.10
10.10.10.11
10.10.10.12
10.10.10.13
10.10.10.14
What is the best command to do this?
This is what I do now, but it takes sometime. Hopefully there is easy way to do this.
grep 10.10.10.10 logfile > output_logfile_10.10.10.10
grep 10.10.10.11 logfile > output_ logfile_10.10.10.11
and so on
zcat logfile.16.Z | grep 10.10.10.10 > output_logfile.16.Z_10.10.10.10
zcat logfile.16.Z | grep 10.10.10.11 > output_logfile.16.Z_10.10.10.11
and so on
Thanks
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
My server has only has access logs turned on. How do I turn on the other standard logs (i.e. I'd like to see the referring urls).
Thanks in advance. (3 Replies)
Discussion started by: pingdom
3 Replies
2. UNIX for Dummies Questions & Answers
I support an app that outputs alert and audit messages to one log file (vendor says they can't be separated). The script that I have written takes a copy (mv cmd) of the file to do the separation and reformatting. I have a problem that I loose records (messages are being written constantly, upto 3+... (5 Replies)
Discussion started by: nhatch
5 Replies
3. IP Networking
Arright, here's what I'm trying to do. I want to dig up currently active IP addresses on my subnet, and my present strategy is to ping every address until I find active ones, then ping them more often to verify their status. Next, I want to find the names of the computers associated with those... (1 Reply)
Discussion started by: sladuuch
1 Replies
4. Shell Programming and Scripting
Hi All
There are some cron jobs ,which runs 24 hrs. Log files are generated when one job fails.
So I need the log files to be emailed to my personal e-mail id. So that I can see the log files at my home If there is any error.
How can I implement this in Unix shell programming.
Thanks... (4 Replies)
Discussion started by: deep_kol
4 Replies
5. Shell Programming and Scripting
Hi,
I have a lot of logfiles like fooYYYYMM.log (foo200301.log, foo200810.log) with lines like
YYYY-MM-DD TIMESTAMP,text1,text2,text3...
but I need (for postprocessing) the form fooYYYYMMDD.log (so foo200402.log becomes foo20040201.log, foo20040202.log...)
with unmodified content of lines.
... (1 Reply)
Discussion started by: clzupp
1 Replies
6. Shell Programming and Scripting
Hi All,
I have a peculiar problem. I will call a script from another script.
Script abc.ksh is called by ABC.ksh as
ABC.ksh abc.ksh
in abc.ksh I will create and redirect all the statements to log file.
ABC.ksh will also has a log file. I want all the logs generated in file abc in ABC... (5 Replies)
Discussion started by: javeed7
5 Replies
7. Red Hat
Hi,
I need to logrotate logs in directories in /var/log/httpd/. There are 4 directories in /var/log/httpd/... these directories are /var/log/httpd/access/
/var/log/httpd/debug/
/var/log/httpd/error/
/var/log/httpd/required/
Each of the access, required, error and debug directories have around... (1 Reply)
Discussion started by: renuka
1 Replies
8. Shell Programming and Scripting
Hi,
I have a file having following content.
<sip:9376507346@97.208.31.7:51088
<sip:9907472291@97.208.31.7:51208
<sip:8103742422@97.208.31.7:51024
<sip:9579892841@97.208.31.7:51080
<sip:9370904222@97.208.31.7:51104
<sip:9327665215@97.208.31.7:51104
<sip:9098364262@97.208.31.7:51024... (2 Replies)
Discussion started by: SunilB2011
2 Replies
9. UNIX for Advanced & Expert Users
Hi,
I have a web server running on Debian 6.0.4 in a computer outside my university, but the web URL is blocked by my university, the security group of the university said because it was scanning computers inside university.
I could not find any applications in my web server are doing... (3 Replies)
Discussion started by: hce
3 Replies
10. UNIX for Beginners Questions & Answers
Hi,
I have a file with a list of bunch of IP addresses from different VLAN's . I am trying to find the list the number of each vlan occurence in the output
Here is how my file looks like
1.1.1.1
1.1.1.2
1.1.1.3
1.1.2.1
1.1.2.2
1.1.3.1
1.1.3.2
1.1.3.3
1.1.3.4
So what I am trying... (2 Replies)
Discussion started by: new2prog
2 Replies
LEARN ABOUT DEBIAN
mergelogs
MERGELOGS(1) General Commands Manual MERGELOGS(1)
NAME
mergelogs - merge and consolidate web server logs
SYNOPSIS
mergelogs -p penlog [-c] [-d] [-j jitter] [-t seconds] server1:logfile1 [server2:logfile2 ...]
EXAMPLES
mergelogs -p pen.log 10.0.0.1:access_log.1 10.0.0.2:access_log.2
mergelogs -p pen.log 10.0.18.6:access_log-10.0.18.6 10.0.18.8:access_log-10.0.18.8
DESCRIPTION
When pen is used to load balance web servers, the web server log file lists all accesses as coming from the host running pen. This makes it
more difficult to analyze the log file.
To solve this, pen creates its own log file, which contains the real client address, the time of the access, the target server address and
the first few bytes of the requests.
Mergelogs reads pen's log file and the log files of all load balanced web servers, compares each entry and creates a combined log file that
looks as if the web server cluster were a single physical server. Client addresses are replaced with the real client addresses.
In the event that no matching client address can be found in the pen log, the server address is used instead. This should never happen, and
is meant as a debugging tool. A large number of these indicates that the server system date needs to be set, or that the jitter value is
too small.
You probably don't want to use this program. Penlog is a much more elegant and functional solution.
OPTIONS
-c Do not cache pen log entries. The use of this option is not recommended, as it will make mergelogs search the entire pen log for
every line in the web server logs.
-d Debugging (repeat for more).
-p penlog
Log file from pen.
-j jitter
Jitter in seconds (default 600). This is the maximum variation in time stamps in the pen and web server log files. A smaller value
will result in a smaller pen log cache and faster processing, at the risk of missed entries.
-t seconds
The difference in seconds between the time on the pen server and UTC. For example, this is 7200 (two hours) in Finland.
server:logfile
Web server address and name of log file.
AUTHOR
Copyright (C) 2001-2003 Ulric Eriksson, <ulric@siag.nu>.
SEE ALSO
pen(1), webresolve(1), penlog(1), penlogd(1)
LOCAL MERGELOGS(1)