Louis Lovas
Thu, 25 Oct 2007 19:23:55 -0500
In mycontinuing research into CEP vendor's use of the SQL as the language of choicefor CEP applications I am reminded of a past industry initiative – but only in reverse! Not so long ago (relatively speaking of course) a new technologyplatform was being promoted by a number vendors - both large and small. Thatnew technology was Application Servers. The core message and supportingtechnology was a backlash to the prior de facto standards (i.eclient/server, host-based, etc.). The message from App Server vendors was separate your user interface from yourbusiness logic from your data. This message resonated well in the industry formany reasons. As the dust settled on the AppServerlandscape a number of leading vendors (commercial and opensource)emerged – still with that same message. The value of separating UI's frombusiness logic from data was clear then and still is today. User Interfaces are like the fashion andelectronics industry. Everyone wants the coolest (UI) gadgets available(witness Adobe Flex and Microsoft WPF). Business logic is central tooperational effectiveness of the Enterprise. As for the data, once it was de-coupled it meant that business logic componentscould break out of their silos and access data from all parts of anorganization. SOA has now superseded AppServers asthe leading edge technology for commercial applications – but that separation message still rings true.
Now a newparadigm enters the arena – Complex Event Processing. Numerous vendors are paradingtheir platform in front of prospects, customers, analysts and the market atlarge. The dust is far from settled buta couple of paradigms are starting to emerge. One builds on the archetype SQLdatabase syntax which I'll refer to as an Event Query Language (EQL). The other model is one that builds uponclassic development languages, those used for building complete applications.I'll refer to that as an Event Programming Language (EPL).
What I'vebegun to see from the CEP EQL (SQL-based, remember from my definition) vendorsis an attempt to convince the industry that SQL (with its streaming extensions)is a language for building CEP applications. As such they are violating the separationrules (of business logic and data) that I outline above. Furthermore, themashing together of business logic and data makes building different iterationsof such applications and evolving those iterations over time tremendouslycumbersome. Business logic is ever evolving and must be easily adaptable to changingbusiness climates, competitive pressures and regulatory agencies. The semantics of this logic should be easilyarticulated using an appropriate metaphor. An EPL by definition provides thesyntactic wealth of expression for this purpose. A Rules-style metaphor is alsoa viable alternative. Apama'sMonitorscript, a mature EPL and the Apama Scenario Modeler which provides that Rules stylemetaphor are well suited for the purpose of building complete CEPapplications.
Not to betoo inflammatory, SQL is well suited for filtering or enriching data, whetherthat is from traditional relational databases or from streaming data sources(via the streaming extensions). However, it's no more suitable for the semanticexpression of business logic for CEP than it is (or ever was) for traditionalcommercial applications (in any deployed form; host-based, client/server, AppServer or SOA).
Looking ata few EQL examples you'll being to see the pattern that I'm referring to.First, a simple, easy to understand example:
SELECT symbol, VWAP(price)FROM Ticker [RANGE 15 minutes]
From thissimple SQL statement it's easy to see basic filtering and enrichment of the rawTicker data. The result set is temporallyorganized into 15 minute buckets and grouped by symbol with a calculated value –VWAP. However, once you move beyond simple enrichment into complex conditiondetection the language becomes horribly unwieldy. Furthermore, once you add theneed to manage state (see When all you have is a hammer everything looks like a nail)you've moved beyond unwieldy to undecipherable or more likely – impossible toimplement.
Here's oneexample published by an EQL vendor:
CREATE VIEW vwap_stream (vwap_price) AS RStream(SELECTsymbol, VWAP(price) FROM ticker [RANGE 15 minutes]);
CREATE VIEW vwap_outside_price(vwap_outside_count) AS SELECTCOUNT(*) AS price_outside_vwap FROM ticker, vwap_stream [range 15 minutes] WHERE price - vwap_price > 0.02*price AND symbol ="MSFT";

<span style="font-size: 10pt; font-family: "Trebuchet MS"; color: navy;">CREATE VIEW trade_cond_stream (matching_row_count)AS SELECT COUNT(*) FROM ticker [RANGE 2 minute]RECOGNIZE ONE ROW PER MATCH PATTERN [S T] DEFINE S AS |price - PREV(price)|=1.05*PREV(price) AND symbol = "IBM") OR (price