|
|
Sponsored Content
Special Forums
News, Links, Events and Announcements
Complex Event Processing RSS News
On Data Mining and Event Processing
Post 302271278 by Linux Bot on Wednesday 24th of December 2008 03:50:07 PM
12-24-2008
On Data Mining and Event Processing
2008-12-24T19:04:00.005+02:00
Today I have travelled to Beer Sheva, the capital of the Negev, which is the south part of Israel, that consists mostly of desert. I have visited Ben-Gurion University, met some old friends, and gave a talk in a well-attended seminar on "the next generation of event processing". I have travelled by train (2 hours and 15 minutes each direction), and since my last visit there five years ago or so, they have built a train station from which there is a bridge which goes to the campus, very convenient. Since I am not a frequent train rider in Israel, I discovered that in both ends of the line, there are no signs saying which trains go on which track, and this is assumed to be common knowledge... Although they do notify when a train entered the station where it is going and from which track, but still they have a point to improve.
Since some of the people attended my talk were data mining people they have wondered about the relationships of event processing and data mining, since I've heard this question before I thought the answer will be of interest to more people.
In the "pattern detection" function of event processing, there is a detection in run-time of patterns that have been predetermined, thus, the system knows what patterns it is looking for, and the functionality is to ensure correct and efficient detection of the predetermined patterns.
Data mining is about looking at historical data to find various things. One of the things that can be found are patterns that have some meaning and we know that if this pattern occurs again then it requires some reaction. The classic example is "fraud detection", in which, based on mining past data, there is a determination that a certain pattern of action indicates suspicion of fraud. In this case the data mining determines the pattern, and the event processing system finds in run-time that this pattern occurs. Note, that not all patterns can be mined from past data, for example- if the pattern is looking for violation of regulations, then the pattern stands for the regulation, but this is not mined, but determined by some regulator and is given explicitly.
So - typically data mining is done off-line and event processing is performed on-line, but again, this is not true for all cases, there are examples in which event processing and mining are mixing in run-time. An example is: there is a traffic model, according to which the traffic light policies are set, but there is also constant monitoring of the traffic, and when the monitored traffic is deviating significantly then the traffic model has to be changed and the policies should be set according to the new traffic model, this is kind of mix between event processing and mining, since the actual mining process is triggered by events, and the patterns may change dynamically as the result of this mining process.
More - Later.
Source...
Today I have travelled to Beer Sheva, the capital of the Negev, which is the south part of Israel, that consists mostly of desert. I have visited Ben-Gurion University, met some old friends, and gave a talk in a well-attended seminar on "the next generation of event processing". I have travelled by train (2 hours and 15 minutes each direction), and since my last visit there five years ago or so, they have built a train station from which there is a bridge which goes to the campus, very convenient. Since I am not a frequent train rider in Israel, I discovered that in both ends of the line, there are no signs saying which trains go on which track, and this is assumed to be common knowledge... Although they do notify when a train entered the station where it is going and from which track, but still they have a point to improve.Since some of the people attended my talk were data mining people they have wondered about the relationships of event processing and data mining, since I've heard this question before I thought the answer will be of interest to more people.
In the "pattern detection" function of event processing, there is a detection in run-time of patterns that have been predetermined, thus, the system knows what patterns it is looking for, and the functionality is to ensure correct and efficient detection of the predetermined patterns.
Data mining is about looking at historical data to find various things. One of the things that can be found are patterns that have some meaning and we know that if this pattern occurs again then it requires some reaction. The classic example is "fraud detection", in which, based on mining past data, there is a determination that a certain pattern of action indicates suspicion of fraud. In this case the data mining determines the pattern, and the event processing system finds in run-time that this pattern occurs. Note, that not all patterns can be mined from past data, for example- if the pattern is looking for violation of regulations, then the pattern stands for the regulation, but this is not mined, but determined by some regulator and is given explicitly.
So - typically data mining is done off-line and event processing is performed on-line, but again, this is not true for all cases, there are examples in which event processing and mining are mixing in run-time. An example is: there is a traffic model, according to which the traffic light policies are set, but there is also constant monitoring of the traffic, and when the monitored traffic is deviating significantly then the traffic model has to be changed and the policies should be set according to the new traffic model, this is kind of mix between event processing and mining, since the actual mining process is triggered by events, and the patterns may change dynamically as the result of this mining process.
More - Later.
Source...
|
|
We Also Found This Discussion For You
1. Shell Programming and Scripting
I'm auditing UID consistency across our hosts, and have created the following datafile, consisting of four fields. I would like to get a count of the combination of the last two fields. ie: I would like to find out how many instances there are of "root 0" and how many of "uucp 5", for every line... (3 Replies)
Discussion started by: akbar
3 Replies
LEARN ABOUT OPENSOLARIS
dat_evd_post_se
dat_evd_post_se(3DAT) Direct Access Transport Library Functions dat_evd_post_se(3DAT) NAME
dat_evd_post_se - post Software event to the Event Dispatcher event queue SYNOPSIS
cc [ flag... ] file... -ldat [ library... ] #include <dat/udat.h> DAT_RETURN dat_evd_post_se( IN DAT_EVD_HANDLE evd_handle, IN const DAT_EVENT *event ) PARAMETERS
evd_handle Handle for an instance of the Event Dispatcher event A pointer to a Consumer created Software Event. DESCRIPTION
The dat_evd_post_se() function posts Software events to the Event Dispatcher event queue. This is analogous to event arrival on the Event Dispatcher software Event Stream. The event that the Consumer provides adheres to the event format as defined in <dat.h>. The first element in the event provides the type of the event (DAT_EVENT_TYPE_SOFTWARE); the rest provide the event-type-specific parameters. These parame- ters are opaque to a Provider. Allocation and release of the memory referenced by the event pointer in a software event are the Consumer's responsibility. There is no ordering between events from different Event Streams. All the synchronization issues between multiple Consumer contexts trying to post events to an Event Dispatcher instance simultaneously are left to a Consumer. If the event queue is full, the operation is completed unsuccessfully and returns DAT_QUEUE_FULL. The event is not queued. The queue over- flow condition does takes place and, therefore, the asynchronous Event Dispatcher is not effected. RETURN VALUES
DAT_SUCCESS The operation was successful. DAT_INVALID_HANDLE The evd_handle parameter is invalid. DAT_INVALID_PARAMETER The event parameter is invalid. DAT_QUEUE_FULL The Event Dispatcher queue is full. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Interface Stability |Standard: uDAPL, 1.1, 1.2 | +-----------------------------+-----------------------------+ |MT-Level |Unsafe | +-----------------------------+-----------------------------+ SEE ALSO
libdat(3LIB), attributes(5) SunOS 5.11 16 Jul 2004 dat_evd_post_se(3DAT)