Tim Bass
09-29-2008 10:31 AM
More often than not, folks working in the field of complex event processing do not truly understand CEP.** We often see the same folks try to position and mischaracterize CEP as business process orchestration, business process management, event-driven architecture or even an evolution of service-oriented architecture.*** Well-intended, this mischaracterization of CEP is often for sales and marketing purposes.* However, sometimes the mischaracterization of CEP is from a lack of understanding of what CEP was designed to accomplish.* These mischaracterizations have very little to do with the original intent of complex event processing.
Originally, researchers in CEP were not trying to solve a problem of streaming data or streaming events.** Often we read this mischaracterization by folks in the database/streaming domain, as they were focused on the low latency processing of streaming events.** A natural extension of this research has been stream processing software (often called “engines”) that process streaming data with continuous queries, for example market data feeds for algo-trading or best market order execution.* This mischaracterization is partly responsible for why we see many order processing applications in market data stream processing mislabled as “complex event processing” applications.
The genesis of complex event processing was not the stream processing need for “feeds and speed” but the processing capability to solve what can be characterized as the “problem of asymmetric capabilties”.** The term “asymmetric” has been used in the military domain. For example we often hear the term “
asymmetric warfare.”* However, in general the concept of “asymmetrical processing capablities” is the true genesis for CEP and related processing concepts and domains. * It is this genesis that distinguishes CEP from EDA, SOA, SOR, and so many other technology oriented concepts.
In order to illustrate what I mean by “asymmetrical processing capablities” we will take the example of the evolution of rocketry.*** In the early days, scientists learned how to make rockets, I assume with gunpowder and similar chemical compounds to launch rockets.** Over many years the application of rocketry advanced much faster than the ability to understand the situations created in the sky.*** In other words, folks could fill the skies with rockets long before they had the capability to track and identify (or sense and respond to)* the rockets in real time.
Therefore, the concept of “asymmetrical processing capablities” is the situation where there is a capability, such as “launch a rocket, sense-and-respond,” that is asymmetric in nature.*** In other words, the capability to detect multiple rocket launches creates an asymmetric situation where it is easy to launch rockets, but hard to detect and defend against those launches.
The same concept can be applied to everyday air travel.** If we could only fly airplanes, but did not have the capability to track the planes, understand situations in airspace, and then respond to changing situations, air travel would be quite difficult.** Lucky for us, the global traveller, there is symmetry in the capabilities to build and fly aircraft and the capabilities to detect, track and follow the evolving situations in the sky.
The genesis of CEP was to solve the problem of asymmetry in cyberspace, or if you prefer, distributed data networks.** The folks who identified, early on,* the problems associated with asymmetry in cyberspace were folks working the the field of network and security management.*** This is because there has been, and is currently, a great asymmetry between the capablities to “launch a process or transaction” in cyberspace and the capabilties to detect and track what is going on in the same domain.
In my next post on this topic, we will go into some details of this asymmetry and review the first CEP projects from Stanford University in the context of asymmetric processing capabilities in cyberspace.
Source...
