Louis Lovas
Wed, 28 Nov 2007 17:11:13 -0500
CEP Design Patterns

<!-- /* Font Definitions */ @font-face {font-family:Verdana; panose-1:2 11 6 4 3 5 4 4 2 4; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:536871559 0 0 0 415 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:&quot;&quot;; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:&quot;Times New Roman&quot;; mso-fareast-font-family:&quot;Times New Roman&quot;;}a:link, span.MsoHyperlink {color:blue; text-decoration:underline; text-underline:single;}a:visited, span.MsoHyperlinkFollowed {color:purple; text-decoration:underline; text-underline:single;}span.SpellE {mso-style-name:&quot;&quot;; mso-spl-e:yes;}span.GramE {mso-style-name:&quot;&quot;; mso-gram-e:yes;}@page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;}div.Section1 {page:Section1;}-->Much has been written on CEPdesign patterns. You can find two very good whitepapers on the subject on theComplex Event Processing website. Thesewhitepapers and various blogs very clearly andsuccinctly describe a number of use-cases that show how CEP engines with theircomplex pattern detection and temporal awareness can be used effectively toderive real business value from streaming data sources. However, in readingthese documents and contrasting them to my own experiences with Apama CEP applications they miss a very real andfundamental design pattern. That of instance life cycle management. I seeComplex Event Processing as much more than pattern detection but as the needfor pattern detection and the expression of business logic executing within amanaged life cycle context.

What I mean by the instance lifecycle is the need to invoke multiple application instances upon receipt of specificcontrol events or the detection ofevent patterns. The execution context and the application instance that runs within it define a unit of work. The life timeof instances is driven directly by the application's semantics. Instances mighthave a long or short life span. There are three basic traits of the instancelife cycle; creation, modification and deletion. Creation is simply the need toestablish a new object instance and context for its execution. Modification isthe need to support dynamic changes to the operational state (i.e. runtimeparameters). Lastly, is support for terminating a running instance – eitherabruptly or gracefully. The instancelife cycle can be managed by various means - other parts of an application orfrom a user's dashboard. Interfaces to create new,modify existing or delete instances, given adequate security privileges shouldbe present. The runtime environment ofthe CEP engine should provide a means to establish and manage these applicationinstances.


To be a bit more definitive,instances can represent many differing aspects of an application. In the Algo Trading world 'instances' generally refer to runningtrading strategies. The basic template of a strategy is to take a set of inputparameters during creation. Forexample, an Arbitrage strategy might take two instrument names and a BollingerBand range as input. Modifying the running strategy might be the ability toincrease or decrease the Band range. Deleting the strategy would cause it toeither complete or cancel any outstanding orders its managing and thenterminate. Given this example, executing multiple concurrent instances of theArbitrage strategy should be a straight forward task and the CEP engine shouldprovide the corresponding language semantics to make this a simple task.

Instances are not limited to thistop-level construct of a strategy. They could represent many things in a CEPengine. Using another finance example, a CEP application designed for bestexecution of client orders would represent each client or parent order as an instance. Each subsequent child order alsorepresents a managed instance with its own life cycle. For a CEP application monitoring amanufacturing environment, instances might represent parts moving through aproduction process from raw material to finished goods. A managed instance could represent a mortgageapplication as it flows through qualification in a BAM application within abank.

In some respects I'm notidentifying a revelation or an idea all that new. Commercial applications ofall sorts incorporate the notion of instance life cycle management. It's aclassic design pattern that is provided in some form in both developmentlanguages and application frameworks (i.e. AppServers).The point I'm attempting to identify is that one should be wary of gettingenamored with the uniqueness of CEP. It's not just about design patterns for filtering and enrichment butalso incorporating classic and traditional constructs for application designand implementation. In the traditionallanguages such as java and C++ the instance design pattern can take numerousforms. It can be as simple as the newoperator. Thus an instance can be represented as a new object instance of aclass. An instance could take the form a thread via CreateThread or in the old-schoolUnix mentality, an instance can be a forked process. Granted all of these are not equivalent,there are pros and cons in using them for life cycle management. Furthermore, frameworks such as J2EEapplication servers provide their own instance management schemes to make theapplication development task of instance management easier. CEP engines thatare implemented using an embedded model, specifically those that run within thecontext of a host language (i.e. java) and/or infrastructure (i.e. AppServer) can leverage these inherent lifecycle instancemanagement capabilities.

Other languages have managed toleverage the best of these options to create safer more scalable instancemanagement. Apama's MonitorScriptevent processing language (EPL) supports an application threading model via a spawn operator. Each instance is referred to as an mThread. Fromthis core capability in the EPL, the life cycle design pattern is built. Thespawn operator allows Apama CEP applications to bebuilt and designed to leverage the best of thread-based concurrent programming(simpler coding model, code isolation and encapsulation, etc.) without theworry of all the complex uses that surround the use of system threads(thrashing, non-deterministic scheduling, poor portability, poor scalability). mThreads in MonitorScriptguarantee predictable behavior, are fully deterministic, fully portable andmassively scalable. This capability isnot adjunct to the Apamaplatform. It's a core capability inherent in the language and heavily leveragedby many of the supporting development and runtime tools.

CEP platforms that narrowly definethe paradigm of event stream processing as a language for filtering andenrichment leave many very important aspects of application managementundefined and as such challenging to implement. The uniqueness and the benefitsof CEP are clearly evident in the many documented designpatterns. However, CEP cannot ignorethe mature aspects of traditional application design such as instance lifecycle management.







Source...