Tim Bass
Mon, 07 Jul 2008 19:20:21 +0000
Courtesy of
Complex Event Processing at Stanford
Complex event processing (CEP) is a new technology. It can be applied to extracting and analyzing information from any kind of distributed message-based system. It is developed from the Rapide concepts of (1) causal event modeling, (2) event patterns and pattern matching, and (3) event pattern maps and constraints. Complex event processing can be applied to a wide variety of Enterprise monitoring and management problems, from low level network management to high level enterprise intelligence gathering.
[h2]Applications of Complex Event Processing:[/h2]
[h2]Presentations:[/h2]
[h2]Publications:[/h2]
- Complex Event Processing in Distributed Systems. David C. Luckham and Brian Frasca, Stanford University Technical Report CSL-TR-98-754, March 1998, 28 pages.Abstract: Complex event processing is a new technology for extracting information from distributed message-based systems. This technology allows users of a system to specify the information that is of interest to them. It can be low level network processing data or high level enterprise management intelligence, depending upon the role and viewpoint of individual users. And it can be changed from moment to moment while the target system is in operation. This paper presents an overview of Complex Event Processing applied to a particular example of a distributed message-based system, a fabrication process management system. The concepts of causal event histories, event patterns, event filtering, and event aggregation are introduced and their application to the process management system is illustrated by simple examples. This paper gives the reader an overview of Complex Event Processing concepts and illustrates how they can be applied using the Rapide toolset to one specific kind of system.
*
- Event Mining with Event Processing Networks. Louis Perrochon and Walter Mann and Stephane Kasriel and David C. Luckham, The Third Pacific-Asia Conference on Knowledge Discovery and Data Mining. April 26-28, 1999. Beijing, China, 5 pages.Abstract: Event Mining discovers and delivers information and knowledge in a real-time stream of data, or events. We show that the process of delivering knowledge by searching patterns in data and subsequent abstraction of found patterns can be applied in real-time to a complex, asynchronous system. Our event processing engine consists of a network of event processing agents (EPAs) running in parallel that interact using a dedicated event processing infrastructure. The agents can be configured at run-time using a formal pattern language. The underlying infrastructure (1) provides an abstract communication mechanism and thus allows dynamic reconfiguration of the communication topology between agents at run-time and (2) provides transparent, location-independent access to all data. These features allow dynamic allocation of EPAs to different threads and processes on different machines at run time.
*
- eJava - Extending Java with Causality. Alexandre Santoro and Walter Mann and Neel Madhav and David Luckham, Proceedings of the 10th International Conference on Software Engineering and Knowledge Engineering, June 1998, 10 pages.Abstract: Programming languages like Java provide designers with a variety of classes that simplify the process of program development. Some of these classes allow one to easily build multithreaded programs. Though useful, especially in the creation of reactive systems, multithreaded programs present challenging problems such as race conditions and synchronization issues. Validating these programs against a specification is not trivial since Java does not clearly indicate thread interaction. These problems can be solved by modifying Java so that it produces computations, collections of events with both causal and temporal ordering relations defined for them. Specifically, the causal ordering is ideal for identifying thread interaction. This paper presents eJava, an extension to Java that is both event based and causally aware, and shows how it simplifies the process of understanding and debugging multithreaded programs.
*
- Event-Based Execution Architectures for Dynamic Software Systems. James Vera, Louis Perrochon, David C. Luckham.
Proceedings of the First Working IFIP Conf. on Software Architecture. 1999. San Antonio, Texas.Abstract: Distributed systems’ runtime behavior can be difficult to understand. Concurrent, distributed activity make notions of global state difficult to grasp. We focus on the runtime structure of a system, its execution architecture, and propose representing its evolution as a partially ordered set of predefined architectural event types. This representation allows a system’s topology to be visualized, analyzed and con-strained. The use of a predefined event types allows the execution architectures of different systems to be readily compared.
*
- Using Context-Based Correlation in Network Operations and Management. Louis Perrochon (work in progress, mail author for newest version)Abstract: Network operation consists to a large degree of reaction to activities happening in the network. Better knowledge of the network at any time allows more appropriate reactions. On the example of intrusion detection, we show how context-based correlation of such activities can provide a more detailed view of the network in shorter time. We first present how we model context and then describe the architecture of the Stanford University CEP context-based correlator. Correlation is specified as event patterns in a declarative language that allows us to specify what needs to be detected, instead of specifying how it should be detected. CEP introduces the concept of causal context to intrusion detection. The correlator is able to process events on-line, as they are generated and it can be reconfigured at dynamically. We then show how it increases detection rate, reduce false alarms, and detect large-scale attack patterns at an early stage.
