Matt Rothera
Fri, 15 Feb 2008 08:54:44 -0500
In recent posts, much has been made of various language and technology choices for CEP (SQL versus an Imperative EPL, such as Progress Apama’s Monitor Script). As also was pointed out, the specific constructs of the CEP language have benefits for certain use cases. For example, the SQL variants of CEP focus on real-time event filtering . Use cases such as these put more emphasis on the “event streaming and filtering” piece of the CEP, but less on the “act” portion. There is typically little or no ability to perform complex actions in response to the events, outside of building these pieces of CEP applications using conventional languages.


So what are the drawbacks of the SQL based approach? Imperative based EPL provides additional value beyond the initial event detection, and provide the ability to solve more complex problems. For example, for certain applications, the actions themselves may incur the need to spawn new rules on the event streams. There may be a need to call out to an external system, loop through the results, and conditionally spawn new rules, each having additional cascading actions which require similar treatment. The actions themselves become new instances of event detection, and must operate in the context of the original initiating event.


Is a filtering-centric CEP approach potentially a “dead-end road”? The great promise of an event-driven enterprise is the ability to begin to automate business processes in response to potentially complex events. For this to become a reality, the CEP engine must be able to not only detect, filter, and analyze a high volume of events, but also be able to orchestrate complex actions within milliseconds from initial detection. Some of those actions may themselves involve complex correlation, as responses are coordinated between multiple systems to achieve the end result.


With an imperative EPL, not only can you perform the front-end event pattern detection, but you can also create arbitrarily complex actions that run in real-time. A CEP platform must strive to not only make the event detection performant, reliable, and easy from a development point of view, but also provide facilities to build the corresponding real-time actions (which could be arbitrarily complex). As was mentioned in previous posts, lower level facilities such as looping and conditional constructs are important, but also higher level facilities and constructs would be important. For example, the ability to model any arbitrary FSM (Finite State Machine) to handle the complex action portion of CEP is important, especially when higher level, visual modeling tools are available to help developers build these types of applications.




A related blog post entitled “CEP as the EDA Application Server”, the author refers to a CEP engine as a potential choice for an implementing an event-driven application. If this be the case, the CEP engine must have the characteristics described above. In the event where certain tasks are offloaded to other components such as a BPEL engine within an ESB environment, the CEP engine still must be able to correlate one or more responses in the context of the originating event, and conditionally launch additional actions based on the results. It seems an imperative EPL would be an absolute requirement to meet the needs of this use case.


At Progress Apama, we strive to create the complete complex event processing platform, which involves providing facilities to not only detect patterns in one or more high volume event streams, but also to respond with complex actions in milliseconds. One classic event driven application that demonstrates this is Algorithmic Trading, where the complex action is orchestrating a trade. However, other types of event driven applications are clearly emerging, such as:

• Operations Center Monitoring and Control: Monitor a vast gas pipeline, energy grid, or water treatment plant and provide reactive control to events in real-time.
  
• Communications Enabled Business Process: Based on complex events within the enterprise, setup voice or chat sessions between key parties to ensure that the issue is resolved quickly.
  
• Battlefield Operations: Monitor a vast number of objects (tanks, personnel, equipment), and take directed actions and orders within milliseconds based on the exact nature of the battlefield conditions.

These are just some of the examples. There are many others.


The point in calling them out is to suggest that we must not become so transfixed by the ability to tap into event streams that we presume that a real event-driven application ends there. Arguably more important – is what you can do with the information.


Source...