Sponsored Content
Top Forums Shell Programming and Scripting [Solved] Isolating & Counting IP from log file Post 302888599 by Lord Spectre on Saturday 15th of February 2014 04:37:42 PM
Old 02-15-2014
[Solved] Isolating & Counting IP from log file

Dear Community,
today my website was under attack for several hours. 2 specific IPs make a tons of "get requests" to a specific page and apache server goes up and down. Now the problem is solved because I put in firewall blacklist these IPs, but I took a lot of time to analyze the apache log to discover what's wrong.

What I would like to do now is make a script to read the apache log (named access.log) and count the same IP who are makeng multiple request. So in other words this is part of the log:
Code:
37.59.55.202 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register HTTP/1.1" 200 12413 "-" "...." "-"
37.59.55.202 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register HTTP/1.1" 200 12413 "-" "...." "-"
37.59.55.202 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register HTTP/1.1" 200 3302 "-" "....." "-"
37.59.55.202 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register HTTP/1.1" 200 12413 "-" "...." "-"
50.7.167.19 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register HTTP/1.1" 200 12413 "-" "...." "-"
50.7.167.19 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register  HTTP/1.1" 200 12413 "-" "....." "-"
50.7.167.19 - - [15/Feb/2014:09:12:10 +0100] "GET /index.php?do=register  HTTP/1.1" 200 12413 "-" "...." "-"

So I would like to count the same IPs and get an output something like:
37.59.55.202 = 4
50.7.167.19 = 3
etc...etc...

So I can recognize in a fast way which is the IP/s to block.
Could someone address me to make a bash script to do that?

Many Thanks
Lucas
 

10 More Discussions You Might Find Interesting

1. IP Networking

Unix Scripts & Counting TCP Connections

Here's a question I received on a test recently. I'm new to Linux/Unix so if this is easy, don't kill me. What scripting or tools could you use to count and sort the number of connections from each internal host? I'd appreciate any feedback and resources. "The Cisco PIX firewall provides... (5 Replies)
Discussion started by: daveohr
5 Replies

2. UNIX for Dummies Questions & Answers

syntax for counting & printing record count

Hi I have a complex script which outputs a text file for loading into a db. I now need to enhance this script do that I can issue an ‘lp' command to show the count of the number of records in this file. Can anybody give me the necessary syntax ? (2 Replies)
Discussion started by: malts18
2 Replies

3. Shell Programming and Scripting

Help with awk array syntax & counting script

..... (3 Replies)
Discussion started by: elbee11
3 Replies

4. Shell Programming and Scripting

[Solved] Counting specific characters within each field

Hello, I have a file like following: ALB_13554 1 1 1 ALB_13554 1 2 1 ALB_18544 2 0 2 ALB_18544 1 0 1 This is a sample of my file, my real file has 441845 number of fields. What I want to do is to calculate the number of 1 and 2 in each column using AWK, so, the output file looks like... (5 Replies)
Discussion started by: Homa
5 Replies

5. UNIX for Dummies Questions & Answers

[solved] Where & what bash env file, Mac OS?

Hi! I wanted to simplify my bash prompt, so I edited my etc/bashrc file. I thought this was the file that would override any other env files. When I opened it, I saw that the way it was setup was not what my prompt looked like, although I forget exactly what was there. But i edited it the way I... (1 Reply)
Discussion started by: sudon't
1 Replies

6. Shell Programming and Scripting

[Solved] BASH - chaining TEST and COMMAND with && and II

Can you explain what this line of script is doing. What I have understood is : -- variable C is the name of a software which is either not installed, so it must be installed or allready installed and then should be update if newer version found -- branch B="$B $C" is to install the software --... (4 Replies)
Discussion started by: jcdole
4 Replies

7. Red Hat

Need Script to ZIP/SAVE & then DELETE Log file & DELETE ZIPS older than 12 months

ENVIROMENT Linux: Fedora Core release 1 (Yarrow) iPlanet: iPlanet-WebServer-Enterprise/6.0SP1 Log Path: /usr/iplanet/servers/https-company/logs I have iPlanet log rotation enabled rotating files on a daily basis. The rotated logs are NOT compressed & are taking up too much space. I... (7 Replies)
Discussion started by: zachs
7 Replies

8. Shell Programming and Scripting

[Solved] Counting The Number of Lines Between Values with Multiple Variables

Hey everyone, I have a bunch of lines with values in field 4 that I am interested in. If these values are between 1 and 3 I want it to count all these values to all be counted together and then have the computer print out LOW and the number of lines with those values in between 1 and 3,... (2 Replies)
Discussion started by: VagabondGold
2 Replies

9. Shell Programming and Scripting

[Solved] Error in script while counting processes

Hi all, Below is a script I'm writing and giving me error: #!/usr/bin/sh if ; then echo "Success!" else echo "Failure!" fi Normally if I do ps -ef|grep dw.sap|wc -l it gives me output of 18. So my script checks if it's greater than 17 it echoes success else failure ... (5 Replies)
Discussion started by: frum
5 Replies

10. Shell Programming and Scripting

Need Script to ZIP/SAVE & then DELETE Log file & send a mail conformation for any error

ENVIROMENT Linux: RHEL 6.4 Log Path: /usr/iplanet/servers/https-company/logs Log Format: user.log.03-15-2015 I have log4j log rotation enabled rotating files on a daily basis. The rotated logs are NOT compressed & are taking up too much space. I need a script that will run daily that... (1 Reply)
Discussion started by: admin_job_admin
1 Replies
HTTP::Date(3pm) 					User Contributed Perl Documentation					   HTTP::Date(3pm)

NAME
HTTP::Date - date conversion routines SYNOPSIS
use HTTP::Date; $string = time2str($time); # Format as GMT ASCII time $time = str2time($string); # convert ASCII date to machine time DESCRIPTION
This module provides functions that deal the date formats used by the HTTP protocol (and then some more). Only the first two functions, time2str() and str2time(), are exported by default. time2str( [$time] ) The time2str() function converts a machine time (seconds since epoch) to a string. If the function is called without an argument or with an undefined argument, it will use the current time. The string returned is in the format preferred for the HTTP protocol. This is a fixed length subset of the format defined by RFC 1123, represented in Universal Time (GMT). An example of a time stamp in this format is: Sun, 06 Nov 1994 08:49:37 GMT str2time( $str [, $zone] ) The str2time() function converts a string to machine time. It returns "undef" if the format of $str is unrecognized, otherwise whatever the "Time::Local" functions can make out of the parsed time. Dates before the system's epoch may not work on all operating systems. The time formats recognized are the same as for parse_date(). The function also takes an optional second argument that specifies the default time zone to use when converting the date. This parameter is ignored if the zone is found in the date string itself. If this parameter is missing, and the date string format does not contain any zone specification, then the local time zone is assumed. If the zone is not ""GMT"" or numerical (like ""-0800"" or "+0100"), then the "Time::Zone" module must be installed in order to get the date recognized. parse_date( $str ) This function will try to parse a date string, and then return it as a list of numerical values followed by a (possible undefined) time zone specifier; ($year, $month, $day, $hour, $min, $sec, $tz). The $year will be the full 4-digit year, and $month numbers start with 1 (for January). In scalar context the numbers are interpolated in a string of the "YYYY-MM-DD hh:mm:ss TZ"-format and returned. If the date is unrecognized, then the empty list is returned ("undef" in scalar context). The function is able to parse the following formats: "Wed, 09 Feb 1994 22:23:32 GMT" -- HTTP format "Thu Feb 3 17:03:55 GMT 1994" -- ctime(3) format "Thu Feb 3 00:00:00 1994", -- ANSI C asctime() format "Tuesday, 08-Feb-94 14:15:29 GMT" -- old rfc850 HTTP format "Tuesday, 08-Feb-1994 14:15:29 GMT" -- broken rfc850 HTTP format "03/Feb/1994:17:03:55 -0700" -- common logfile format "09 Feb 1994 22:23:32 GMT" -- HTTP format (no weekday) "08-Feb-94 14:15:29 GMT" -- rfc850 format (no weekday) "08-Feb-1994 14:15:29 GMT" -- broken rfc850 format (no weekday) "1994-02-03 14:15:29 -0100" -- ISO 8601 format "1994-02-03 14:15:29" -- zone is optional "1994-02-03" -- only date "1994-02-03T14:15:29" -- Use T as separator "19940203T141529Z" -- ISO 8601 compact format "19940203" -- only date "08-Feb-94" -- old rfc850 HTTP format (no weekday, no time) "08-Feb-1994" -- broken rfc850 HTTP format (no weekday, no time) "09 Feb 1994" -- proposed new HTTP format (no weekday, no time) "03/Feb/1994" -- common logfile format (no time, no offset) "Feb 3 1994" -- Unix 'ls -l' format "Feb 3 17:03" -- Unix 'ls -l' format "11-15-96 03:52PM" -- Windows 'dir' format The parser ignores leading and trailing whitespace. It also allow the seconds to be missing and the month to be numerical in most formats. If the year is missing, then we assume that the date is the first matching date before current month. If the year is given with only 2 digits, then parse_date() will select the century that makes the year closest to the current date. time2iso( [$time] ) Same as time2str(), but returns a "YYYY-MM-DD hh:mm:ss"-formatted string representing time in the local time zone. time2isoz( [$time] ) Same as time2str(), but returns a "YYYY-MM-DD hh:mm:ssZ"-formatted string representing Universal Time. SEE ALSO
"time" in perlfunc, Time::Zone COPYRIGHT
Copyright 1995-1999, Gisle Aas This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.14.2 2012-03-30 HTTP::Date(3pm)
All times are GMT -4. The time now is 10:41 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy