|
|
04-22-2010
Registered User
26,240,
27
Peas and Carrots
Louis Lovas
02-22-2010 07:16 AM
Inthe words of the auspicious Forrest Gump some things go together like peas and carrots.Truer words were never spoken. Some things just do go together well,sometimes by design, often by accident. I don't think anyone actually planned milk andcookies or popcorn at the movies but nonetheless these things are madefor each other. When it comes to technology the sameharmonious relationships exist.
In the recent Aite report on High Performance Databases (HPDB), themarket for specialized databasesis surveyed along with a handful of vendors in this space. Thisis a cottage industry where the big database vendors don'tplay. It's hard to imagine in this day and age wheredatabase technology is so standardized and mature and a multitude of choice aboundsfrom commercial products to open source that any other databasetechnology and a gang of vendors would have a chance. Yet it is happening and it's thriving.
I believe it has to do with a synergistic relationship to event processing. If CEP is the"peas" thenHPDB's are the "carrots".These two technologies share two fundamental precepts:
I. Extreme Performance,Speeds and Feeds
These HPDB's which are often referred to as Tick databases,are found in the same playground as event processingtechnologies. In the Capital Markets industry they connect to the samemarket sources, consume the same data feeds. Both technologies aredesigned to leverage modern multi-core hardware to consume theever-increasing firehose of data. By the same token, once that data isstored on disk, database query performance is equally important. The massive amount of data collected and is only as good as thedatabase's ability to query it efficiently thus creating another(historical) firehose of data which an event processing engine would bethe consummate consumer.
II. Temporal Awareness, when is the data
Time is a basic principle in event processing technology, applicationstypically have as a premise to analyze data-in-motionwithin a window of time. HPDB's design center is to store and query time seriesdata. Some of the database vendors even bring time to a higher levelbusiness function. They understand the notion of a business Calendar,knowing business hours,business week, holidays, trading hours, etc. Imagine thesimplicity of a query where you want 'Business hours Mon-Fri for the month of February' andthe database itself would know the third Monday was Presidents Day,skipping over that, thus preventing analytic calculations fromskewing erroneously.
Leveraging the Synergy
These two fundamental shared principles provide the basis for a uniqueset of business cases that are only realized by leveraging EventProcessing platforms and High PerformanceDatabases
What ifyou could test new trading algorithms against the last 6 months or 1 -2 years of historical market data but run that test in a matter ofminutes? What if you could be assured that the temporal conditions ofthe strategies (i.e. timed limit orders) behaved correctly anddeterministically matching the movement of time in completesynchronicity with the historical data? These are just a fewof the characteristics that define the harmony between event processingand high performance (Tick) databases.
Querying historical data in-flightto obtain volume curves, moving averages, the latest VWAP and otheranalytics calculations are possible with these high performancedatabases. Leading edge trading algorithms are blending a historicalcontext with the live market and even News. The winners will be thosethat can build these complex algo's and maintain ultra low-latency.Managing positions, order limits andexposure is necessary, doing it in real-time to manage market risk is amandate. In addition to market data, these high performancedatabases can store pre and post trade activity to complementevent-based trading systems and become the basis for trade reportingsystems.
In the Trading LifeCycle,Event Processing and High Performance databases are partnertechnologies harmoniously bound together to form a union where thewhole is greater than the sum of the parts. They are the peas and carrots that together create a host of real-world use-cases that would not be possible as individual technologies.
Myself along with my colleague Dan Hubsher we are doing a 3-part Webinar series entitled "Concept to Profit".The focus is on event processing in the trade lifecycle, but weinclude cases that touch upon high performance databases. You can still register for part 2: Building Trading Strategies in the Apama WorkBench where I will focus on the tools for strategy development aimed at the IT developer.
Once again thanks for reading, you can follow me at twitter, here.
Louie
Source...
02-22-2010 07:16 AM
Inthe words of the auspicious Forrest Gump some things go together like peas and carrots.Truer words were never spoken. Some things just do go together well,sometimes by design, often by accident. I don't think anyone actually planned milk andcookies or popcorn at the movies but nonetheless these things are madefor each other. When it comes to technology the sameharmonious relationships exist. In the recent Aite report on High Performance Databases (HPDB), themarket for specialized databasesis surveyed along with a handful of vendors in this space. Thisis a cottage industry where the big database vendors don'tplay. It's hard to imagine in this day and age wheredatabase technology is so standardized and mature and a multitude of choice aboundsfrom commercial products to open source that any other databasetechnology and a gang of vendors would have a chance. Yet it is happening and it's thriving.
I believe it has to do with a synergistic relationship to event processing. If CEP is the"peas" thenHPDB's are the "carrots".These two technologies share two fundamental precepts:
- A focus on Extreme Performance
- Temporal Awareness
These HPDB's which are often referred to as Tick databases,are found in the same playground as event processingtechnologies. In the Capital Markets industry they connect to the samemarket sources, consume the same data feeds. Both technologies aredesigned to leverage modern multi-core hardware to consume theever-increasing firehose of data. By the same token, once that data isstored on disk, database query performance is equally important. The massive amount of data collected and is only as good as thedatabase's ability to query it efficiently thus creating another(historical) firehose of data which an event processing engine would bethe consummate consumer.
II. Temporal Awareness, when is the data
Time is a basic principle in event processing technology, applicationstypically have as a premise to analyze data-in-motionwithin a window of time. HPDB's design center is to store and query time seriesdata. Some of the database vendors even bring time to a higher levelbusiness function. They understand the notion of a business Calendar,knowing business hours,business week, holidays, trading hours, etc. Imagine thesimplicity of a query where you want 'Business hours Mon-Fri for the month of February' andthe database itself would know the third Monday was Presidents Day,skipping over that, thus preventing analytic calculations fromskewing erroneously.
Leveraging the Synergy
These two fundamental shared principles provide the basis for a uniqueset of business cases that are only realized by leveraging EventProcessing platforms and High PerformanceDatabases
- Back testing algorithms across massive volumes of historical data compressing time
- Blending live and historical data in real-time
- Pre-Trade Risk Management
In the Trading LifeCycle,Event Processing and High Performance databases are partnertechnologies harmoniously bound together to form a union where thewhole is greater than the sum of the parts. They are the peas and carrots that together create a host of real-world use-cases that would not be possible as individual technologies.
Myself along with my colleague Dan Hubsher we are doing a 3-part Webinar series entitled "Concept to Profit".The focus is on event processing in the trade lifecycle, but weinclude cases that touch upon high performance databases. You can still register for part 2: Building Trading Strategies in the Apama WorkBench where I will focus on the tools for strategy development aimed at the IT developer.
Once again thanks for reading, you can follow me at twitter, here.
Louie
Source...