Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

[Solved] Isolating & Counting IP from log file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting [Solved] Isolating & Counting IP from log file
# 1  
Old 02-15-2014
[Solved] Isolating & Counting IP from log file
Dear Community,
today my website was under attack for several hours. 2 specific IPs make a tons of "get requests" to a specific page and apache server goes up and down. Now the problem is solved because I put in firewall blacklist these IPs, but I took a lot of time to analyze the apache log to discover what's wrong.

What I would like to do now is make a script to read the apache log (named access.log) and count the same IP who are makeng multiple request. So in other words this is part of the log:
Code:
37.59.55.202 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register HTTP/1.1" 200 12413 "-" "...." "-"
37.59.55.202 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register HTTP/1.1" 200 12413 "-" "...." "-"
37.59.55.202 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register HTTP/1.1" 200 3302 "-" "....." "-"
37.59.55.202 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register HTTP/1.1" 200 12413 "-" "...." "-"
50.7.167.19 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register HTTP/1.1" 200 12413 "-" "...." "-"
50.7.167.19 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register  HTTP/1.1" 200 12413 "-" "....." "-"
50.7.167.19 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register  HTTP/1.1" 200 12413 "-" "...." "-"

So I would like to count the same IPs and get an output something like:
37.59.55.202 = 4
50.7.167.19 = 3
etc...etc...

So I can recognize in a fast way which is the IP/s to block.
Could someone address me to make a bash script to do that?

Many Thanks
Lucas
# 2  
Old 02-15-2014
Try:
Code:
awk '{c[$1]++}END{for (i in c) print i,c[i]}' access_log

This User Gave Thanks to bartus11 For This Post:
Lord Spectre
# 3  
Old 02-15-2014
AAhh very good, thanks! Smilie
But is there a way to filter specific count to show only occurencies greater then "X"?
This because otherwise I'll get tons of output.
# 4  
Old 02-15-2014
Code:
awk '{c[$1]++}END{for (i in c) if (c[i]>100) print i,c[i]}' access_log

This User Gave Thanks to bartus11 For This Post:
Lord Spectre
# 5  
Old 02-15-2014
Thanks again, and if I can ask, is it possibile to ordering from bigger to lowest?
Is not properly necessary, but will help me a lot!! Smilie
# 6  
Old 02-15-2014
Code:
awk '{c[$1]++}END{for (i in c) if (c[i]>100) print i,c[i]}' access_log | sort -nrk2

This User Gave Thanks to bartus11 For This Post:
Lord Spectre
# 7  
Old 02-15-2014
Unfortunately the sort doesn't work Smilie
But not problem, you helped me a lot!! Smilie

EDIT: My mistake... Works perfect!!!
You're the master! Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need Script to ZIP/SAVE & then DELETE Log file & send a mail conformation for any error

ENVIROMENT Linux: RHEL 6.4 Log Path: /usr/iplanet/servers/https-company/logs Log Format: user.log.03-15-2015 I have log4j log rotation enabled rotating files on a daily basis. The rotated logs are NOT compressed & are taking up too much space. I need a script that will run daily that... (1 Reply)
Discussion started by: admin_job_admin
1 Replies

2. Shell Programming and Scripting

[Solved] Error in script while counting processes

Hi all, Below is a script I'm writing and giving me error: #!/usr/bin/sh if ; then echo "Success!" else echo "Failure!" fi Normally if I do ps -ef|grep dw.sap|wc -l it gives me output of 18. So my script checks if it's greater than 17 it echoes success else failure ... (5 Replies)
Discussion started by: frum
5 Replies

3. Shell Programming and Scripting

[Solved] Counting The Number of Lines Between Values with Multiple Variables

Hey everyone, I have a bunch of lines with values in field 4 that I am interested in. If these values are between 1 and 3 I want it to count all these values to all be counted together and then have the computer print out LOW and the number of lines with those values in between 1 and 3,... (2 Replies)
Discussion started by: VagabondGold
2 Replies

4. Red Hat

Need Script to ZIP/SAVE & then DELETE Log file & DELETE ZIPS older than 12 months

ENVIROMENT Linux: Fedora Core release 1 (Yarrow) iPlanet: iPlanet-WebServer-Enterprise/6.0SP1 Log Path: /usr/iplanet/servers/https-company/logs I have iPlanet log rotation enabled rotating files on a daily basis. The rotated logs are NOT compressed & are taking up too much space. I... (7 Replies)
Discussion started by: zachs
7 Replies

5. Shell Programming and Scripting

[Solved] BASH - chaining TEST and COMMAND with && and II

Can you explain what this line of script is doing. What I have understood is : -- variable C is the name of a software which is either not installed, so it must be installed or allready installed and then should be update if newer version found -- branch B="$B $C" is to install the software --... (4 Replies)
Discussion started by: jcdole
4 Replies

6. UNIX for Dummies Questions & Answers

[solved] Where & what bash env file, Mac OS?

Hi! I wanted to simplify my bash prompt, so I edited my etc/bashrc file. I thought this was the file that would override any other env files. When I opened it, I saw that the way it was setup was not what my prompt looked like, although I forget exactly what was there. But i edited it the way I... (1 Reply)
Discussion started by: sudon't
1 Replies

7. Shell Programming and Scripting

[Solved] Counting specific characters within each field

Hello, I have a file like following: ALB_13554 1 1 1 ALB_13554 1 2 1 ALB_18544 2 0 2 ALB_18544 1 0 1 This is a sample of my file, my real file has 441845 number of fields. What I want to do is to calculate the number of 1 and 2 in each column using AWK, so, the output file looks like... (5 Replies)
Discussion started by: Homa
5 Replies

8. Shell Programming and Scripting

Help with awk array syntax & counting script

..... (3 Replies)
Discussion started by: elbee11
3 Replies

9. UNIX for Dummies Questions & Answers

syntax for counting & printing record count

Hi I have a complex script which outputs a text file for loading into a db. I now need to enhance this script do that I can issue an ‘lp' command to show the count of the number of records in this file. Can anybody give me the necessary syntax ? (2 Replies)
Discussion started by: malts18
2 Replies

10. IP Networking

Unix Scripts & Counting TCP Connections

Here's a question I received on a test recently. I'm new to Linux/Unix so if this is easy, don't kill me. What scripting or tools could you use to count and sort the number of connections from each internal host? I'd appreciate any feedback and resources. "The Cisco PIX firewall provides... (5 Replies)
Discussion started by: daveohr
5 Replies
Login or Register to Ask a Question