I noticed a few w00tw00ts in our Apache2 logfile the other day, so I thought I would write a quick post on blocking them with iptables. Feel free to improve upon any of my scripts or ideas in this thread.

First of all, what is a w00tw00t and where might we find one?

Well, a w00tw00t is an signature left by a web vulnerability scanner called DFind that has the signature below and you can find them in your Apache logfiles, for example:

neo@forum:# grep "GET /w00tw00t.at.ISC.SANS.DFind:)" /website/logs/apache2/access.log
88.80.222.117 - - [25/Nov/2009:08:38:36 +0000] "GET /w00tw00t.at.ISC.SANS.DFind:) HTTP/1.1" 400 226 "-" "-"

wget http://www.novirusthanks.org/dfind-logs/ip-list;mv ip-list w00tw00t_list

grep w00tw00t /website/logs/apache2/access.log | awk 'BEGIN { FS = " " } ; { print $1 }' >> w00tw00t_list

awk '{
if ($0 in stored_lines)
   x=1
else
   print
   stored_lines[$0]=1
}' w00tw00t_list > w00t_new

mv w00t_new w00tw00t_list

while read ip
do
iptables -A INPUT -s "$ip"/24 -j DROP
done < w00tw00t_list

while read ip
do
iptables -A INPUT -s "$ip" -j DROP
done < w00tw00t_list

iptables -L -n

It was such an inviting possibility for some evening fun so let's have a crack at it!

(NOT tested)

#/bin/sh
#good (?) working dir
cd /tmp
#Get a fresh list? do rm w00tw00t_list first
#I lynx...
[ -r w00tw00t_list ] || lynx -dump http://www.novirusthanks.org/dfind-logs/ip-list > w00tw00t_list
#append unique entries from weblog
grep w00tw00t /website/logs/apache2/access.log | cut -d" " -f1 |sort -u >> w00tw00t_list
#apply ip rules
while read ip; do iptables -A INPUT -s "$ip"/24 -j DROP; done < w00tw00t_list
#List resulting block list
iptables -L -n

Nice write-up but it's a non-standard and maintenance-prone "solution". Maybe people not like you (;-p) should choose a combination of iptables rate limiting, webserver "BrowserMatch" and mod_security filtering instead?..

Great! Describe your implementation "step-by-step" in a detailed write-up and avoid hand-waving and we'll have a look

PS (Edit): mod_security can be a very big performance killer on a very busy web server.... intercepting every URL and trying to match each one against a long list of rules can kill performance.




---------- Post updated at 23:23 ---------- Previous update was at 23:00 ----------

I like it, especially using cut and sort versus awk. I always use wget, so I should look into using lynx from time-to-time!

Those disappointed by the lack of details handouts sure could call it RTF(ine)M or accuse me of handwaving, NP, but anyone with basic GNU/Linux admin skills (as in knowing how to read the documentation) should be able to cobble up the parts themselves.


Sure performance-wise you'll want to filter like "DynamicOnly", not log what you don't need and group regular expressions, but "very big performance killer"? Naw, I'd call that unsubstantiated if presented without cold hard numbers...


BTW, about the script, having a separate chain instead of putting everything in INPUT allows you to route traffic in a more fine-grained way. The script then essentially could be compressed to a oneliner something like:

iptables -F BLOCKCHAIN || iptables -N BLOCKCHAIN; ( curl -s http://www.novirusthanks.org/dfind-logs/ip-list | grep -v '#"; awk '/w00tw00t/ {print $2}' /var/log/httpd/*access* ) | sort -u | xargs -iX iptables -A BLOCKCHAIN -s 'X' -j DROP

Our experience is everything contributes to performance and applying something to the front end of the web server will definitely effect performance.

When you discount performance off-hand, I can only assume you do not operate a web server with thousands of concurrent users and millions of PVs a month.

Everything effects performance. Everything. Web operators talk performance. It is one of our favorite topics!

I think you may be arguing for the sake of argument. Just a simple Google search yields the article, 4 reasons not to use mod_security, concluding,
So, my impression is that you don't operate a web server with millions of PVs a month and thousands of concurrent users at peak, because even off loading tiny gif and jpg icons, which seems trival and small, can significantly reduce Apache2 workers and CPU load, etc.

Computing is all about performance optimization.

Having said that, we are considering mod_security for emergencies and temporary stop gaps until we can put a better performing solution in place in certain scenarios. It is certainly possible the performance hit will be small; but from what I have read about mod_security, and experiences here, it will certainly have
an impact on performance.

---------- Post updated at 21:04 ---------- Previous update was at 20:56 ----------

Speaking of mod_security performance quotes, I think this quote from Securing Apache Web Server with mod_security in the Linux Gazette sums it up nicely:



---------- Post updated at 21:16 ---------- Previous update was at 21:04 ----------

I like parts of this quote from Basics of mod_security:

.

Regarding the second statement, that is really relative to overall performance of the server. It is very easy for big servers will smallish loads to say "security over performance".

Editorial Comments:

If security was always preferable to performance, then F1 race cars would be built with heavier material

There is no shortage of self-proclaimed security experts in the world who ignore performance, in my experience in IT security most of my career.

To Followup.......

When you are searching your logfiles for w00tw00ts, be careful not to mistakenly identify legitimate requests from friendly hosts, for example, requests for posts with w00tw00t in the URL who might be reading a post you have on the topic

I updated my example to reflect this:

grep "GET /w00tw00t.at.ISC.SANS.DFind:)" /website/logs/apache2/access.log