Tim Bass
10-11-2008 06:15 AM
This is a continuation of
The Genesis of Complex Event Processing: Asymmetric Capabilities and
CEP, Event Noise and Asymmetric Event Processing where I have been discussing the motivation behind CEP and adaptive analytics in cyberspace.
Around the same time that Professor Luckham and his team was working on CEP applications in network management and security management, I was leading efforts to build network and security management control centers for the
United States Air Force.* In the beginning, dating back to 1994, my Internet-related work was for
Air Combat Command (ACC), working out of ACC headquarters at
Langley Air Force Base.
In 1997, I lead a technical team that developed countermeasures against an actual distributed Internet-based attack on the Langley AFB SMTP email infrastructure.* This attack was documented in a technical paper,
E-Mail Bombs and Countermeasures: Cyber Attacks on Availability and Brand Integrity, IEEE Network Magazine, Vol. 12, No. 2, pp. 10-17, March/April 1998.* In addition, this attackand countermeasures I designed was featured in Popular Science Magazine in an 1998 article,
War.Com and other news channels.* I also published a number of related papers on this topic.
Our team used a rule-based approach for countermeasures against massive email bombs attacks on the Langley Air Force Base email infrastructure.** We called this rule-based system,
BombShelter. and it was written in
PERL.* I developed both the original software architecture and the original working prototype for BombShelter (in two days) and then we turned the software over to our team who used the rule-based approach for daily attack countermeasures.
I watched for days, and then weeks, as my team designed rules, and the attackers wrote new attacks that circumvented the rules.* Some folks in the Pentagon used to say that I “lead the effort to fight the first war in cyberspace”.** It might have have been the first cyberwar, I am not sure, but it was certainly the first publicly documented cyberwar.* There is no doubt about this.
Without getting into all the historical footnotes and significance of this cyberwar that was fought with experts and rule-based systems, I would like to jump to an important conclusion.
Rule-based systems are useful, but have limited functionality and scaleability in most complex event processing applications.
Rule-based systems are human resource intensive because rule-based systems cannot learn and adapt on their own, humans learn and then write new rules.* This is how rule-based systems work.
This is the motivation behind why I spend a lot of time to search for new, more efficient and adaptive methods as alternatives to rule-based systems.** After extensive research, I published a series of papers on the future of intrusion detection in the Internet.*
Intrusion Detection Systems & Multisensor Data Fusion - Creating Cyberspace Situational Awareness [1], helped lead an evolution in Internet security, particularly in the area of network-based intrusion detection systems (IDS).
In my published research work, motivated by limitations with rule-based approaches, I used the same mature functional model that is used to process missile attacks, control global air traffic, and other complex event processing applications in physical space; but I applied these concepts to cyberspace.
Around the same time, Professor Luckham and others were working on similar problems, all related to real-time detection and response to threats in cyberspace.* They were also funded by the US government.
Sidebar: Stream processing of transaction- based systems (databases), another area of interest, was focused on a totally different problem, which was the low latency processing of straight-thru processing in databased-oriented systems.** These stream processing systems were, and remain however,* rule-based systems.* The problems we were trying to solve in cyberspace, however, cannot be efficiently and pragmatically solved by rule-based systems alone.* Only relatively simple scenarios can be efficiently detected by rule-based stream processing systems.
The vast majority of complex event processing classes of problems require rules plus advanced algorithms that can learn and adapt in real-time.*** I know this, not from reading papers or taking university classes on rule-bases systems, but from working on some very challenging operational problems in real-time.*** This is why I remain interested in complex event processing and why I continue to elaborate on why rule-based systems have limitations.
Source...
