Tim Bass
10-01-2008 10:22 PM
In The Genesis of Complex Event Processing: Asymmetric Capabilities I introduced the abstract concept of “asymmetric processing capabilities” to describe the foundations of complex event processing.** If you take a few moments to review the first CEP projects from Stanford University, you will see that the application of CEP was toward* solving myriad asymmetric event processing problems in distributed networks.*** These applications included challenging problems such as:

• Network Level Monitoring and Management,
  
• Cyber Security: Network Intrusion Detection,
  
• Enterprise Monitoring and Management,
• Modeling and Simulation of Collaborative Business Processes,
• Business Policy Monitoring, and
• Analysis and Debugging of Distributed Systems.

In each of the CEP application examples above, the amount of event information available to software developers can be staggering; however, despite all the available information, the capability to sense-and-respond to threats and opportunities is crude, at best.

Folks who work in network and security management, for example, are bombarded with event information.* However, this deluge of event information is, for the most part, “noise” that is difficult to understand.** In network management one of the most difficult things to accomplish is to find the root cause of an outage or performance problem.** This is why researchers at Stanford were funded to focused on research topics such as (above), the Analysis and Debugging of Distributed Systems.

These are the classes of asymmetric event processing problems that define complex event processing, or CEP.** Processing events by mediating events, routing events, or running a rule-set against events and making a processing decision are all perfectly valid event processing applications.** However, the core reason to have “complex event processing” is to solve event processing problems where there exists a significant asymmetry between the deluge of “event noise”* (Professor Luckham called this phenomena the “event cloud”) and detecting business-relevant, actionable complex events in an climate of uncertainty and noise.

In my next post on this topic I will briefly the review motivation behind my 1999 ACM paper, Intrusion Detection Systems and Multisensor Data Fusion, where we were working on solving complex distributed security challenges based on real-world experiences with the problems of asymmetric processing capabiilities.* I will discuss why we evolved from an early rule-based expert system model to a more advanced inference model that was not dependent solely on rule-based thinking. * I will also explain why other researchers and developers experienced in complex event detection applications have come to the same conclusion.



Source...