1 Overview of Oracle CEP

1.1 Introduction to Oracle Complex Event Processing

Oracle CEP (formally known as the WebLogic Event Server) is a Java server for the development and deployment of high-performance event driven applications. It is a lightweight Java application container based on Equinox OSGi, with shared services, including the Oracle CEP Service Engine, which provides a rich, declarative environment based on Oracle Continuous Query Language (Oracle CQL) - a query language based on SQL with added constructs that support streaming data - to improve the efficiency and effectiveness of managing business operations. Oracle CEP supports ultra-high throughput and microsecond latency using JRockit Real Time and provides Oracle CEP Visualizer and Oracle CEP IDE for Eclipse developer tooling for a complete real time end-to-end Java Event-Driven Architecture (EDA) development platform.

Oracle CEP has the capability of deploying user Java code (POJOs) which contain the business logic. Running the business logic within Oracle CEP provides a highly tuned framework for time and event driven applications.

1.2 Conceptual Overview of Oracle CEP

Figure 1-1 provides a high level view of an event-driven system.

Figure 1-1 Example Event -Driven System

Description of "Figure 1-1 Example Event -Driven System"

An event-driven system is generally comprised of several event sources, the real-time event-driven applications, and event sinks. Oracle CEP server and the Oracle CEP applications you deploy to it comprises the event-driven applications. The event sources generate streams of ordinary event data. The Oracle CEP applications listen to the event streams, process these events, and generate notable events. Event sinks receive the notable events.

Event sources, event-driven applications, and event sinks are de-coupled from each other; one can add or remove any of these components without causing changes to the other components. This is a key attribute of event-driven architectures.

Event-driven applications are rule-driven. In Oracle CEP, rules are expressed as queries using the Oracle Continuous Query Language (Oracle CQL). These queries are persisted to a data store and are used for processing the inbound stream of events and generating the outbound stream of events. Queries typically perform filtering and aggregation functions to discover and extract notable events from the inbound event streams. As a result, the number of outbound events is generally much lower than that of the inbound events.

Oracle CEP is a middleware for the development of event-driven applications. An Oracle CEP application is essentially an event-driven application.

Next, consider the application itself, which is hosted by the Oracle CEP server, a light-weight container as shown in Figure 1-2.

Figure 1-2 Oracle CEP Application

Description of "Figure 1-2 Oracle CEP Application"

An Oracle CEP application is typically composed of the following main component types:

• Adapters interface directly to the inbound and outbound stream and relation sources and sinks. Adapters understand the inbound and outbound protocol, and are responsible for converting the event data into a normalized form that can be queried by a processor. Adapters forward the normalized event data into channels or outbound stream and relation sinks.

• Channels are event processing endpoints. Among other things, streams are responsible for queuing event data until the event processing agent can act upon it.

• Processors (or event processing agents) consume normalized event data from a channel, process it using queries, and may generate new events to an output channel.

• Beans register to listen to the output channel, and are triggered by the insertion of a new event into the output channel. This user code is generally a plain-old-Java-object (POJO). The user application makes use of a set of external services, such as JMS, Web services, and file writers, to forward the generated events to external event sinks.

• Event Beans register to listen to the output channel, and is triggered by the insertion of a new event into the output channel. This user code uses the Oracle CEP event bean API so that the bean can be managed by Oracle CEP.

1.3 Event Processing Networks

Adapters, channels, processors, and business logic POJOs can be connected arbitrarily to each other, forming event processing networks (EPN). Examples of topologies of EPNs are:

• Adapter - Channel - Business Logic POJO

  Scenario: no processing is needed; only adaptation from proprietary protocol to some normalized model.

• Adapter - Channel - Processor - Channel - Business Logic POJO

  Scenario: straight through processing to user code.

• Adapter - Channel - Processor - Channel - Business Logic POJO - Channel - Processor - Channel - Business Logic POJO

  Scenario: two layers of event processing; the first processor creates causality between events and the second processor aggregates events into complex (notable) events.

EPNs have two important attributes:

• Event processing networks can be used to create a hierarchy of processing agents, and thus achieve very complex processing of events. Each layer of the EPN aggregates events of its layer into complex events that become simple events in the layer above it.

• Event processing networks improve integrability, that is, the quality of having separately developed components work correctly together. For example, one can add user code and reference to external services at several places in the network.

1.4 Use Cases

The use cases for Oracle CEP span a variety of businesses and applications. Just a few of these diverse use cases include:

• Financial: Algorithmic Trading

  Automate stock trading based on market movement. Sample query: if, within any 20 second window, StockB rises by more than 2% and StockA does not, then automatically buy StockA.

  For an example, see Section 3.10, "Signal Generation Example".

• Transportation: Security and Fraud Detection

  Discover fraudulent activity by detecting patterns among events. Sample query: if a single ID card is used twice in less than 5 seconds to gain access to a city's subway system, alert security for piggybacking.

• Energy and Telecommunications: Alarm Correlation

  Reduce false positive alarms. Sample query: When 15 alarms are received within any 5 second window, but less than 5 similar alarms detected within 30 seconds, then do nothing.

• Health Care: Patient Monitoring

  Monitor the vital signs of a patient and perform some task if a particular event happens. Sample query: When a change in medication is followed by a rise in blood pressure within 20% of maximum allowable for this patient within any 10 second window, alert nearest nurse.

1.5 Summary of Oracle CEP Features

The following list summarizes the main features of Oracle CEP:

• New in 11g Release 1 (11.1.1): Oracle Continuous Query Language (Oracle CQL) is a query language based on SQL with added constructs that support streaming data. Using Oracle CQL, you can express queries on data streams to perform complex event processing (CEP) using Oracle CEP. Oracle CQL is scalable and comprehensive. It provides a wide range of operators (including extensive window operators), functions (including built-in, Colt, and java.lang.Math functions), and statements. Oracle CQL supersedes EPL.

• New in 11g Release 1 (11.1.1): Support for Coherence clustering and distributed cache.

• New in 11g Release 1 (11.1.1): Oracle CEP Visualizer query constructor, query plan generator, and support for Coherence clustering and distributed cache.

• Event Caching—Applications can optionally publish or consume events to and from a cache to increase the availability of the events and increase the performance of their applications.

• Event Record and Playback—The event repository feature of Oracle CEP allows you to record events flowing through an EPN and store them so you can later play back the events.

• Built-in HTTP Publish-Subscribe Adapters—The three built-in HTTP publish-subscribe adapters allow an application to easily to publish (locally and remotely) and subscribe to an HTTP publish-subscribe server channel.

• Built-in JMS Adapters—The two JMS adapters (inbound and outbound) allow you to send and receive messages to and from a JMS queue, respectively, from your application without writing any Java code

• Oracle CEP Visualizer—A Web 2.0 application that consumes data Oracle CEP, displays it in a useful and intuitive way to system administrators and operators, and, for specified tasks, accepts data that is then passed back to Oracle CEP so as to change it configuration

• Multi-server domains (sometimes referred to as clustering)—Oracle CEP now allows multiple servers to be logically connected together for the purposes of management, and physically connected using a shared User Datagram Protocol (UDP) multicast address and port.

• An application server that supports deployment of Plain Old Java applications (POJOs), or Spring applications, for handling large volumes of streaming data with low latency requirements.

• Oracle CEP applications are developed and deployed as event driven applications, that is, a set of custom Spring tags is used to define the event processing network in the EPN assembly file, which extends the standard Spring context file, of your application.

• The application server contains a set of real time services that include a complex event processor (CEP), adapters, and streams. The server is highly tuned for high message throughput and low latency and deterministic behavior.

• The complex event processor is a high performance, continuous query engine for processing high volumes of streaming data. It has full support for filtering, correlation, and aggregation of streaming data from one or more streams.

• The Event Processing Language (EPL), an SQL-like language that allows event data from streams to be declaratively filtered, correlated, aggregated, and merged, with the ability to insert results into other streams for further downstream processing. You define the EPL rules either in an XML file that configures the complex event processor or programmatically using APIs. Oracle CQL supersedes EPL.

• An Adapter SDK that provides all the tools you need to create adapters that listen to incoming data feeds.

• A set of product samples that show both a simple Hello World scenario to get you started and more complex foreign exchange and algorithmic trading scenarios to showcase additional features of Oracle CEP.

• A load generator utility that simulates a data feed, useful for testing your application without needing to connect to a live data feed.

• A monitoring service that includes pre-built instrumentation for measuring throughput and latency at the component level.

• A static and dynamic configuration framework. Static configuration is performed using XML files; dynamic configuration is performed by accessing configuration and runtime MBeans using JMX and with the command-line utility wlevs.Admin.

• Oracle CEP is built on the Oracle microServices Architecture (mSA) which uses an OSGi-based framework to manage services provided by modules or feature sets. Oracle mSA provides the following services:

• Jetty, an HTTP container for running servlets.

• javax.sql.DataSource implementation and thin JDBC drivers for accessing a relational database.

• Logging and debugging.

• Authentication and authorization security.

1.6 Supported Platforms

For detailed information on the platforms that Oracle CEP supports, see: http://www.oracle.com/technology/software/products/ias/files/oracle%20fusion%20middleware%2011gR1%20(11.1.1.x)%20certification%20matrix.xls.

You can find the installation program appropriate for your platform here: http://www.oracle.com/technology/software/products/middleware/htdocs/111110_fmw.html. For more information, see Section 2.2, "Installation Overview".

1.7 Oracle CEP IDE for Eclipse

Oracle CEP IDE for Eclipse is targeted specifically to programmers that want to develop Oracle CEP applications as Figure 1-3 shows.

Figure 1-3 Oracle CEP IDE for Eclipse

Description of "Figure 1-3 Oracle CEP IDE for Eclipse"

The Oracle CEP IDE for Eclipse is a set of plugins for the Eclipse IDE designed to help develop, deploy, and debug Oracle CEP applications.

The key features of Oracle CEP IDE for Eclipse are:

• Project creation wizards and templates to quickly get started building event driven applications.

• Advanced editors for source files including Java and XML files common to Oracle CEP applications.

• Integrated server management to seamlessly start, stop, and deploy to Oracle CEP server instances all from within the IDE.

• Integrated debugging.

• Event Processing Network (EPN) visual design views for orienting and navigating in event processing applications.

• Integrated support for the Oracle CEP Visualizer so you can use the Oracle CEP Visualizer from within the IDE (see Section 1.8, "Oracle CEP Visualizer").

For details, see:

• Oracle CEP Developer's Guide for Eclipse

• http://www.oracle.com/technology/products/event-driven-architecture/cep-ide/11/index.html

1.8 Oracle CEP Visualizer

Oracle provides an advanced run-time administration console called the Oracle CEP Visualizer as Figure 1-4 shows.

Figure 1-4 Oracle CEP Visualizer

Description of "Figure 1-4 Oracle CEP Visualizer"

Using Oracle CEP Visualizer, you can manage, tune, and monitor Oracle CEP server domains and the Oracle CEP applications you deploy to them all from a browser. Oracle CEP Visualizer provides a variety of sophisticated run-time administration tools, including support for Oracle CQL and EPL rule maintenance and creation.

For details, see Oracle CEP Visualizer User's Guide

1.9 New Features in Oracle CEP 11g Release 1 (11.1.1)

This version of Oracle CEP includes the following new features:

• Section 1.9.1, "High Availability and Scalability"

• Section 1.9.2, "Record and Playback of Events"

• Section 1.9.3, "Oracle CEP IDE for Eclipse"

• Section 1.9.4, "Visualizer Administration Console"

• Section 1.9.5, "Oracle Continuous Query Language (Oracle CQL)"

• Section 1.9.6, "Web Services Support"

• Section 1.9.7, "JMS Adapter"

• Section 1.9.8, "Samples"

• Section 1.9.9, "Domain Directory Structure Changes"

1.10 Next Steps

• Install Oracle CEP 11g Release 1 (11.1.1).

  See Chapter 2, "Installing Oracle CEP."

• Run the samples from their respective domains.

  See:

  • Section 3.1, "Overview of the Samples Provided in the Distribution Kit"

  • Section 3.6.1, "Running the HelloWorld Example from the helloworld Domain"

  • Section 3.9.1, "Running the Foreign Exchange Example"

  • Section 3.10.1, "Running the Signal Generation Example"

  • Section 3.11.1, "Running the Event Record/Playback Example"

  • Section 3.7.1, "Running the CQL Example"

• Understand how the sample applications have been programmed by viewing the source and configuration files and then building them from their respective source directories.

  See:

  • Section 3.6.2, "Building and Deploying the HelloWorld Example from the Source Directory"

  • Section 3.9.2, "Building and Deploying the Foreign Exchange Example from the Source Directory"

  • Section 3.10.2, "Building and Deploying the Signal Generation Example from the Source Directory"

• Create your own Oracle CEP domain.

  See:

  • "Creating an Oracle CEP Standalone-Server Domain" in the Oracle CEP Administrator's Guide

  • "Creating an Oracle CEP Multi-Server Domain" in the Oracle CEP Administrator's Guide

• Create a new Oracle CEP application and deploy it to your new domain.

  See "Overview of Creating Oracle CEP Applications" in the Oracle CEP Developer's Guide for Eclipse.

• Pose questions and find solutions using the Oracle CEP forum: http://forums.oracle.com/forums/forum.jspa?forumID=820