UNIX for Dummies Questions & Answers

Hello,

I need to extract IP info from few large files into a single file with IP info only. I guess I can use grep, uniq and redirection but I not sure how. Is there a way to do this with a single command? Your help is greatly appreciated.

Welcome to the forums.

Please post an sample of the log file you wish to extract data from (change IP's and any sensitive information first), and an example of the intended output. Without this, it's impossible to assist. The output of uname -a and echo $SHELL is always a boon too....

Thanks,
ZB

here is an example:

if this is the log file:


192.168.1.100 - - [08/Dec/2005:18:16:07 -0500] "GET /images/welcome.png HTTP/1.1"
200 3032
192.168.1.101 - - [08/Dec/2005:18:16:07 -0500] "GET /images/google.gif HTTP/1.1" 2
00 1680
192.168.1.120 - - [08/Dec/2005:18:16:07 -0500] "GET /images/jeeves.gif HTTP/1.1" 2
00 1976
192.168.1.1 - - [08/Dec/2005:18:16:07 -0500] "GET /images/dogpile.gif HTTP/1.1"
200 985
192.168.100.1 - - [08/Dec/2005:18:16:07 -0500] "GET /images/yahoo.gif HTTP/1.1" 20
0 718
192.168.1.1 - - [08/Dec/2005:18:16:07 -0500] "GET /images/lycos.gif HTTP/1.1" 20
0 1348
192.168.10.1 - - [08/Dec/2005:18:16:07 -0500] "GET /images/av.jpg HTTP/1.1" 200 1
632
192.168.1.11 - - [08/Dec/2005:18:16:07 -0500] "GET /images/valid.gif HTTP/1.1" 20
0 2328
192.168.1.1 - - [08/Dec/2005:18:16:08 -0500] "GET /images/easy.gif HTTP/1.1" 200
1518
192.168.1.1 - - [08/Dec/2005:18:32:18 -0500] "GET / HTTP/1.1" 200 3643
23.34.54.121 - - [18/Dec/2005:08:24:47 -0500] "GET /car/ext-1.jpg HTTP/1.1" 200 4
3611
23.34.54.122 - - [18/Dec/2005:08:25:10 -0500] "GET /car/int-2.jpg HTTP/1.1" 200 4
7963
23.34.54.110 - - [18/Dec/2005:08:25:12 -0500] "GET /car/int-1.jpg HTTP/1.1" 200 5
7079
23.34.54.111 - - [18/Dec/2005:08:31:41 -0500]

[Mon Oct 28 22:02:10 2005] [error] [client 192.168.1.1] File does not exist: /ho
me/httpd/di0de/index.php
[Tue Oct 29 01:23:18 2005] [error] [client 200.81.81.70] client sent HTTP/1.1 re
quest without hostname (see RFC2616 section 14.23): /
[Tue Oct 29 10:27:07 2005] [error] [client 216.39.100.132] File does not exist:
/home/httpd/di0de/scripts/..%5c..%5cwinnt/system32/cmd.exe
[Tue Oct 29 16:30:16 2005] [error] [client 192.168.1.1] File does not exist: /ho
me/httpd/di0de/010102.html

I would like to extract only the IP info into a new file such as:

192.168.1.1
192.168.1.100
192.168.1.11
23.34.54.111
...

syslog1, syslog2 and syslog3

I can use grep and uniq but I am not sure how to extract the IP info only and redirect it to a new file.

di0de

Code:

grep "GET" syslog* | cut -d"-" -f 1

I used the pattern "GET" in syslogs
If you want to narrow down the lines further use egrep.

This is an awk script to find IP addresses...

Code:

awk '
BEGIN{
    r = "[0-9][0-9]?[0-9]?"
    r = r "\\." r "\\." r "\\." r
}
match ($0, r){
    a[substr($0, RSTART, RLENGTH)] = 1
}
END{
    for (x in a)
        print x
}
' syslog1 syslog2 syslog3 > newlog

how do I implement and execute this script? thanks for your help.