Clustered Databases Versus Virtualization for CEP Applications

 
Thread Tools Search this Thread
Special Forums UNIX and Linux Applications High Performance Computing Clustered Databases Versus Virtualization for CEP Applications
# 1  
Old 11-19-2007
Clustered Databases Versus Virtualization for CEP Applications

Tim Bass
Sat, 17 Nov 2007 04:11:25 +0000
In my earlier*post, A Model For Distributed Event*Processing, I promised to address grid computing, distributed object caching and virtualization, and how these technologies relate to complex event processing.***Some of my readers might forget my earlier roots in networking if I continue to talk about higher level abstractions!* So, in*this follow-up post I will*discuss virtualization*relative to database clustering.
In typical*clustered database environments there are quite a*few major performance constraints.* These constraints limit our capability to architect and design solutions for distributed, complex, cooperative*event processing*problems and scenarios.* Socket-based interprocess communications*(IPCs)*within database clusters*create a performance*limitation contrained by low bandwidth, high latency, and processing*overhead.
In addition, the communications performance between the application layer and the database layer can be limited by both TCP and operating system overhead.* To make matter worse, hardware input-output constraints limits scalability for connecting database servers to disk storage.** These are standard distributed computing constraints.
The*physical*architecture to address scalability in emerging distributed CEP solutions require a*low-latency network communications infrastructure (sometimes called a fabric).* This simple means that event processing agents (EPAs)*require virtualization technologies such as Remote Direct Memory Access (RDMA).**CEP agents (often called CEP engines) should*have*the*capability to*write data directly to the memory spaces of a CEP agent*fabric (sometimes called an event processing network, EPN).** This is similar to*the concept of shared memory*as*an*IPC in UNIX-based systems applied to distributed computing, so all “old hat” UNIX*systems engineers will easily grok these concepts.
RDMA virtualization helps improve performance by bypassing operating-system and TCP overhead resulting in*significantly*higher bandwidth and lower latency in the EPF (Event Processing Fabric - I just*minted a new three letter acronym, sorry!).* This, in turn, improves the*communication speed*between event processing agents in an event processing network (EPN), or EPF (depending on your taste in acronyms).
Scheduling tasks such as a distributed semaphore checking and lock management can*also operate*more efficiently and with higher performance.****Distributed tables*scans, decision tree searches, rule-engine evaluations, Bayesian and neural analytics can all be performed in parallel, *dramatically improving*both performance and*scalability of distributed event processing applications.
In addition, by adopting transparent protocols with existing socket APIs, the CEP architect can*bypass both operating-system and TCP protocol overhead.***In other words, communications*infrastructures for CEP*that optimize networking, interprocess communications, and storage, provide architects*with the underlying tools to*build better solutions to computational complex*problems.
Many of the communications*constraints of earlier distributed architectures for solving complex problems, such as blackboard architectures,* can be mitigated with advances in virtualization.* So, in a nutshell,*virtualization technologies,*are*one of*the most important underlying capabilities required for distributed, high performance CEP applications, in my opinion.
The article, Virtualization hot; ITIL, IPv6 not,**appears to indicate that*some of the top IT managers at Interop*New York might agree with me.**
Unfortunately for a few software vendors, virtualization threatens to dilute their market share for ESB and message bus sale.* (OBTW,*SOA*is DOA.)** “Old hat” UNIX system programmers will recall how the UNIX IPC called “message queues” lost favor to sockets, pipes and shared memory.** A similar trend is happening in the virtualization world with*RDMA as a distributed*shared memory technology versus message-based communications technologies.* I*will opine more on this topic later.



Source...
Login or Register to Ask a Question

Previous Thread | Next Thread

5 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Start Services in Clustered Environment

Hello Experts, I have a requirement to start and stop weblogic services in a clustered environment. First i need to start weblogic server and once the server is in Running mode i need to do SSH to other server and there i need to start Node Manager and Managed server, After these two are in... (1 Reply)
Discussion started by: beginner786
1 Replies

2. Filesystems, Disks and Memory

Clustered filesystem which one to pick

Hi all, I'm a bit new to advanced filesystem types. I've just only learned that if you wish to share a single fibre channel extent with many servers you need to use a clustered filesystem to prevent data corruption. looking through a list of clustered file systems I saw gfs2 which I thought... (1 Reply)
Discussion started by: jokken
1 Replies

3. Red Hat

RHEL 5.5: how to remove a clustered VG?

Hi all, I have a 2 node rhel 5.5 cluster (2 server and 1 quorum disk). I created 2 cluster resources using Luci web console, they are 2 Volume Groups. I want to remove that cluster and shutdown node 2, but I don't want to loose data on Volume Groups clustered. How can I remove that... (0 Replies)
Discussion started by: peppeunz
0 Replies

4. UNIX for Dummies Questions & Answers

netstat command between clustered hosts

I have 2 clustered hosts, is it possible for me to issue a netstat command against 1 host from the other ? (4 Replies)
Discussion started by: murphyboy
4 Replies

5. UNIX for Dummies Questions & Answers

CTRL+H versus ^? versus BACKSPACE

Hi Gurus! I recently got my shell account (HP UX v11) created by our sysadmin and am having problem deleting with the backspace key. After doing some reading, I believe I need to enter a custom "STTY..." statement in my profile. Can someone please help me with the correct "STTY" sequence... (3 Replies)
Discussion started by: alan
3 Replies
Login or Register to Ask a Question