Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

To find the IP adress in the log file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting To find the IP adress in the log file
# 1  
Old 03-07-2009
To find the IP adress in the log file
Hi,

I need to find out the repeated IP address from the apache log file from my box. I did try to come out with the script, but I could not grep out the repeated Ip Address from the error_logs and need to redirect to a file. Can you guys please help me out of this problem.

Thanks in Advance.
# 2  
Old 03-07-2009
show us what you've tried and where your problems are...
# 3  
Old 03-07-2009
the apache log file should have the IP address in Column one, so use awk or cut to get column 1, then sort the results through unique. The below will generate a list of IP address along with the number of times each was encountered sorted with the greatest number at the bottom.

cat access_log | awk '{print $1}' | sort | uniq -c | sort -n
# 4  
Old 03-07-2009
Hi ldapswandog,

Thank you for your help... The command workout a lot. This command list all the IP address in the log file. But even more specific, how can i pull out the Ip Address who's entry has been repeatedly occurred in the log file. Exactly is, if an IP 192.168.1.20 if repeatly occured in my log file, The command should grep out the ips. I hope it make scene..
# 5  
Old 03-07-2009
I use such a system to ban ip's that have made too many unsuccessful login attempts in a certain period of time. Imagine you have an access file like this one (extract):
Code:
Apr 26 15:56:53 monserveur sshd[30750]: Invalid user zoe from 89.110.150.203
Apr 26 16:00:10 monserveur sshd[30986]: Invalid user zachary from 89.110.150.203
Apr 26 20:18:15 monserveur sshd[5159]: Invalid user johnbe from 210.243.170.181
Apr 26 20:18:15 monserveur sshd[5159]: Invalid user allanz from 210.243.170.181
Apr 26 20:22:06 monserveur sshd[5341]: Invalid user frederik78 from 210.243.170.181
Apr 26 20:22:06 monserveur sshd[5341]: Invalid user xgridagent from 210.243.170.181
Apr 26 20:22:16 monserveur sshd[5349]: Invalid user xgridcontroller from 210.243.170.181
Apr 26 20:23:43 monserveur sshd[5419]: Invalid user zzz from 210.243.170.181
Apr 26 20:23:43 monserveur sshd[5419]: Invalid user zzz from 210.243.170.181
Apr 28 02:58:04 monserveur sshd[20403]: Invalid user xfs from 72.93.200.84
Apr 28 02:58:04 monserveur sshd[20403]: Invalid user xfs from 72.93.200.84
Apr 28 02:58:10 monserveur sshd[20409]: Invalid user zephyr from 72.93.200.84
Apr 28 03:02:18 monserveur sshd[20669]: Invalid user yellow from 72.93.200.84
Apr 28 03:02:39 monserveur sshd[20691]: Invalid user xxx from 72.93.200.84
Apr 28 03:03:22 monserveur sshd[20735]: Invalid user year from 72.93.200.84
Apr 28 14:16:32 monserveur sshd[6556]: Invalid user Zmeu from 88.191.46.60
Apr 28 14:17:14 monserveur sshd[6611]: Invalid user za from 88.191.46.60

The following code will extract all ip's that have made more than 2 unsuccessful attempts in one minute. You first need to build a awk array indexing on [date time ip]: Apr 28 20:18 123.123.123.123
Code:
awk -F'[ :]' '{_[$1 $2 $3 $4 $13]++} _[$1 $2 $3 $4 $13]>2 {print $13}' access.log

Code:
210.243.170.181
72.93.200.84

Hope this will put you on track.
# 6  
Old 03-07-2009
Thank you for your reply . I have applied the command, but I am getting an error as

# awk -F'[ :]' '{_[$1 $2 $3 $4 $13]++} _[$1 $2 $3 $4 $13]>2 {print $10}' access.log

awk: cmd. line:2: fatal: cannot open file `access.log' for reading (No such file or directory)
# 7  
Old 03-07-2009
Based on the file contect that @ripat provided using the cmdline script I provided the command will list how many times and IP address was found, which is what you requested.

cat test11.txt | awk '{print $NF}' | sort | uniq -c | sort -n
2 88.191.46.60
2 89.110.150.203
6 72.93.200.84
7 210.243.170.181

If you have a large access_log that has mostly single IP address access the remove the single entries from the output by using grep to remove them. the below remove any that appeared 2 or less times

cat test11.txt | awk '{print $NF}' | sort | uniq -c | sort -n | grep "[3-9] "

6 72.93.200.84
7 210.243.170.181
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk script to find repeated IP adress from trace file (.tr)

+ 8.00747 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Enqueue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 0 protocol 17 offset (bytes) 0 flags length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 >... (11 Replies)
Discussion started by: Nipa
11 Replies

2. IP Networking

adress traffic to tap0

Hello, I have a problem of routing traffic on two virtual interfaces I have created on my machine (CentOs6) By using tunctl I created two virtual interfaces tap1 and tap2 let s imagine I gave them two different address tap1: 10.1.1.1 net 255.255.255.0 tap2: 10.1.2.1 net 255.255.255.0 ... (0 Replies)
Discussion started by: nicandro
0 Replies

3. Shell Programming and Scripting

sort file with email adress

Hi All, I have a file which is "|" pipe delimited. The file has 3 fields. the last field contains email ids which has different host names. I want to sort the file based on host name, which is in 3rd filed and needs to create a file for each host. For example, if out of 1000 records, 10... (6 Replies)
Discussion started by: ace_friends22
6 Replies

4. Shell Programming and Scripting

Trying to get an IP adress from a file

This is probably a real n00b question but i`m not able to figure it out. I have a folder of configuration files that contain IP-adresses. The line i`m interested in looks like this: IP_ADDRESS="123.123.123.1123" Some have muliple ip adresses, so the line will look like : ... (5 Replies)
Discussion started by: DaneV
5 Replies

5. IP Networking

Changing the Ip adress permanently

well i'm trying to change the ip adress on an old alphaserver runing tru64 4.0F using the ifconfig hme0 IP_ADDRESS mask MASK broadcast BROADCAST and when i check it using ifconfig -a it shows the new ip and all is well but when the server is rebooted it reverts back to the old ip (3 Replies)
Discussion started by: randUSR()
3 Replies

6. Shell Programming and Scripting

find log file between two dates

Dear All, Please can you help me to crack this query? If the log files for the task above all had a naming convention of myoutput_YearMonthDay.log (i.e. myoutput_20060215) How would you find only those log files created between the 10th and the 20th of each month going back the last 365 days.... (1 Reply)
Discussion started by: justin_mca
1 Replies

7. HP-UX

Change IP Adress

I want change my IP address and hostname in my machine by use the console. Can any one tell me how can I execute that by command ? Thanks & Regards (1 Reply)
Discussion started by: magasem
1 Replies

8. AIX

Capture IP Adress

hello I need for a script to capture the ip address from the connected user. I have 5 logical partitions. With "who", i have the ip adress, but only for 2 servers. Do you know another command to know the ip address of connected clients ? thank you (14 Replies)
Discussion started by: pascalbout
14 Replies

9. Solaris

IP-Adress

Hello together how can I find a ipadress from a login into remote system console? Thanks a lot Urs (1 Reply)
Discussion started by: MuellerUrs
1 Replies

10. UNIX for Dummies Questions & Answers

MAC-Adress

Hello I need to show my MAC-Adress on a Unix System, is there someone that know how? (2 Replies)
Discussion started by: nkochr
2 Replies
Login or Register to Ask a Question