The Bot Hunter: An Event Processing Challenge Post: 302225305

Sponsored Content

Special Forums UNIX and Linux Applications Virtualization and Cloud Computing The Bot Hunter: An Event Processing Challenge Post 302225305 by Linux Bot on Friday 15th of August 2008 05:40:08 AM

08-15-2008

Registered User

The Bot Hunter: An Event Processing Challenge

Tim Bass
08-15-2008 02:35 AM
Recently we penned The Attack of the Spiders from the Clouds where we mentioned how cloud computing infrastructures can be used to stage malicous or accidential network attacks.

Today I challenge our CEP/ESP/EP vendors (or SIs) to create the following solution to detect and block rogue bots on Apache web sites.** I will install and test each submitted solution on The UNIX Forums and post the results here.

Here are some basic requirements:

Your solution must run on Linux and be installable and configurable remotely with SSH or HTTP.* There will be no physical access to the server. No exceptions.
Preferrably, the configuration can be done with a Web-Based Interface (WBI) - a browser.
Your solution will listen to continuous updates to the Apache2 access log, exact location configurable in your solution, and identify robots ( bots), also known as spiders, from the log.
Your solution will provide a confidence metric, key indicator (KI), for each bot detected, from 0 to 10, where 10 indicates “absolutely a bot,” 0 is “absolutely not a bot.”
Your solution will update the IP address of each bot and KI you identify in a file/table called, for example, ./bot_scorecard.txt where each line is an IP address of a bot, followed by a semicolon (or other delimiter of your choice) and the confidence factor, for example,* 10.0.0.1;10 means that 10.0.0.1 is a bot, 100% sure.
Your solution must compare bots detected to a file/table called, for example, ./bots_allowed.txt and ./bots_denied.txt that are in the format IP address/mask, for example 10.0.0.1/24, or 10.0.0.1/32.
If the KI “confidence factor” of the IP address of your detected bot is higher than the tunable “is a bot” KI, then your solution should update the tables/files and then call iptables and block the bot.
It should send an email to one or more email addresses with a message, for example:* “New Bot Detected - Confidence 8″ with IP address, etc. in the message.* Another example would be an email, “Bot Blocked” - with details, etc.
You cannot automatically block any traffic that is not a bot.* Blocking one “non-bot” results in failure, no exceptions.

These are some basic requirements; I don’t want to restrict your thinking or solution, so be creative!* Feel free to ask any questions in the comment section of this thread.

Remember, sometimes you may have to manage the state of IP addresses for days, or hours, before you can accurately deterimine if it is a bot based on behavior alone.** So, you will need to work with both long and short time windows.* Latency is not important. Detection accurate is importance.

Anyone care to submit a solution for testing?

Source...

Linux Bot

View Public Profile for Linux Bot

Find all posts by Linux Bot

LEARN ABOUT DEBIAN

poe::component::irc::projects

POE::Component::IRC::Projects(3pm)			User Contributed Perl Documentation			POE::Component::IRC::Projects(3pm)

NAME

       POE::Component::IRC::Projects - A listing of projects that use POE::Component::IRC

SYNOPSIS

	perldoc POE::Component::IRC::Projects

DESCRIPTION

       POE::Component::IRC::Projects strives to document projects that are using POE::Component::IRC>. Projects can include subclasses, bot
       frameworks, bots, etc. The only stipulation for inclusion is that the project utilises POE::Component::IRC>.

       Inclusion to ( or inversely, exclusion from ) this list does not imply any sort of endorsement ( or disapproval ) of the said project.

BOT FRAMEWORKS ( CPAN )
       An alphabetically ordered list of bot frameworks, that are available on CPAN.

       Amethyst
	   Amethyst is a bot core capable of handling parsing and routing of messages between connections and brains. Amethyst can handle an
	   arbitrary number of connections of arbitrary types (given an appropriate module in Amethyst::Connection::*), routing these messages
	   fairly arbitrarily through multiple processing cores (brains, live in Amethyst::Brain::*), and responding to these messages on other
	   arbitrary connections.

       Bot::BasicBot
	   Basic bot system designed to make it easy to do simple bots, optionally forking longer processes (like searches) concurrently in the
	   background.

       Bot::Pluggable
	   This is a very small (but important) part of a pluggable IRC bot framework. It provides the developer with a simply framework for
	   writing Bot components as Perl modules.

       Bot::Infobot
	   Bot::BasicBot::Pluggable based replacement for the venerable infobot.

       IRC::Bot
	   A complete bot, similar to eggdrop using POE::Component::IRC. Allows access to all channel user management modes. Provides !seen
	   functions, a complete help system, logging, dcc chat interface, and it runs as a daemon process. IRC::Bot utilizes Cache::FileCache for
	   seen functions, and for session handling.

       ThreatNet::Bot::AmmoBot
	   ThreatNet::Bot::AmmoBot is the basic foot soldier of the ThreatNet bot ecosystem, fetching ammunition and bringing it to the channel.
	   It connects to a single ThreatNet channel, and then tails one or more files scanning for threat messages while following the basic
	   channel rules.

EXTENSIONS ( CPAN )
       POE::Component::IRC::Plugin::RSS::Headlines
	   A POE::Component::IRC plugin that provides RSS headline retrieval.

       POE::Component::IRC::Plugin::URI::Find
	   A POE::Component::IRC plugin that finds URIs in channel traffic.

       POE::Component::IRC::Plugin::POE::Knee
	   A POE::Component::IRC plugin that runs Acme::POE::Knee races.

       POE::Component::IRC::Plugin::Blowfish
	   A POE::Component::IRC plugin that provides blowfish encryption.

       POE::Component::IRC::Plugin::YouTube::MovieFindStore
	   A plugin for finding, resolving .FLV, and optionally storing YouTube URIs.

       POE::Component::IRC::Object
	   A slightly simpler OO interface to PoCoIRC

       POE::Component::IRC::Onjoin
	   This module implements a class that provides moved message and onjoin services as an IRC bot. Based on the configuration parameters
	   passed to it via its constructor it will connect to a channel on a server and immediately send everyone on that channel a message
	   privately. It will also send the same message to the channel itself publically at the specified interval. All users joining the channel
	   thereafter will also receive the message.

PROXIES 
/ BOUNCERS ( CPAN )
       App::Bondage
	   A featureful easy-to-use IRC bouncer.

perl v5.14.2							    2011-12-07					POE::Component::IRC::Projects(3pm)