Complex Event Processing: not an architecture issue
In his 2002 book “
The Power Of Events” David Luckham, Professor Emeritus of Electrical Engineering at Stanford University and one of the founders of Rational Software, outlines a set of tools and techniques – referred to as Complex Event Processing (CEP) - for ‘analyzing and controlling the complex series of interrelated events that drive modern information systems’. CEP, he claims, is ‘emerging technology’ that can be ‘applied to a broad spectrum of information system challenges’. Whilst clearly an academic tome, Luckham’s book had a commercial impact, not least at Gartner which introduced yet another entry into the TLA lexicon that permeates our industry: Event-Driven Architecture (EDA). Tibco’s recently announced
BusinessEvents product and Apama’s (
recently acquired by Progress Software) Event Stream Processing technology are also heavily influenced by CEP.
I am in no position to question the academic merits of CEP and, having read the book, I remain convinced that there are a set of real business problems, such as fraud detection in the telecoms market and financial instrument trading in capital markets, which demand the processing and correlation of a complex set of events. However, I think it's time for some level-setting.
Businesses of all shapes and sizes respond to events. Indeed, all business is event-driven. An inbound call to a customer contact centre, the receipt of a new purchase order, the scanning of a supermarket bar code… these are all business events which must be responded to. Similarly, technology capable of ‘analyzing and controlling’ events is hardly new. What about loosely-coupled integration enabled by event-based middleware? What about systems management and monitoring? Like much in our industry, event-based processing acts on a continuum, with a call centre application, an overnight order processing batch run, monthly reporting on supermarket loyalty cards purchases and real-time analysis of telephone call patterns at various points on the scale.
As I have already said, I don’t dispute the need for CEP. Rather, my beef is with the proposition that there is the need for a new "architectural approach" (aka EDA) required to deal with it. To my mind, CEP as defined by Luckham is largely a "point" issue, which can be dealt with via a specific application of technology; rather than something which has to be dealt with in a systemic way by adding new enterprise infrastructure or re-thinking application or infrastructure architecture. My assertion is that CEP is real: but that it does not generalise from a "point" problem to an infrastructure/architecture issue. Capturing and managing business-driven and technology-driven events using IT is absolutely a wider issue - but as Gartner itself points out this is hardly new. And it's certainly not CEP (at least as Luckham defines it). An interesting data point which supports the "CEP as application" view: Tibco’s BusinessEvents offering has evolved over a number of years through its work with a handful of customers with very specific requirements.
The positioning of EDA as complementary to SOA (which started with Gartner, but which has now become that most dangerous of things - "received wisdom") just muddies the water further. Even if you believe that EDA exists as a useful concept, the issues dealt with by SOA and EDA are orthogonal. SOA, at its heart, is a way of thinking about the packaging and presentation of IT assets, which enhances flexibility and hides complexity in messy, distributed, federated environments. It has got nothing to do with how processes behind service interfaces are invoked. The argument that "SOA is about invocation; EDA is about notification" is too technology-focused, too development-focused, and ultimately misleading.
