Download Order Management Systems

Complex Event Processing
Prof. Richard Holowczak
© 2007-2014 Richard Holowczak All rights reserved.
Streams of Data
•
•
•
•
•
•
•
•
RFID scans
Transportation / Traffic scans
Web site visits
Server logs
Financial / Market data
Manufacturing Processes
Environmental monitoring / Remote sensing
Gaming events
© 2007-2014 Richard Holowczak All rights reserved.
2
Challenges of Streaming Data
• Data may arrive and demand action at a rate
faster than a traditional database can respond
• Require flexibility / generalizability in coding
approaches
– Compare custom C++/Java vs. a query language like
SQL
© 2007-2014 Richard Holowczak All rights reserved.
3
DBMS vs. Stream Processing
1
2
Event
Insert
3
Query
Data
DBMS
Database
4
Query
Result
Action
5
1
2
Query
5
Insert
Event
Data
CEP
Database
3
Query
Result
CEP
4
Action
4
© 2007-2014 Richard Holowczak All rights reserved.
Event Processing Vendors
• Generic Systems
– Oracle CEP
www.oracle.com/us/products/middleware/soa/eventprocessing
– Amazon Kinesis
aws.amazon.com/kinesis
– SQL Stream
www.sqlstream.com
• Financial Services - Oriented
– Sybase Aleri
www.aleri.com
– TIBCO StreamBase www.streambase.com
– Apama
www.softwareag.com/corporate/products/bigdata/apama/
© 2007-2014 Richard Holowczak All rights reserved.
5
Stream / Event Processing
• Data arrives in a stream, e.g.
– Weather data
– RFID scans
– Market Data
• Rules put in place to match patterns in one or
more streams
• When a rule is “satisfied” trigger and Event or Alert
– Possibly triggering some further actions
© 2007-2014 Richard Holowczak All rights reserved.
6
Complex Event Processing –
Example from Apama
Source: Algorithmic Trading: The use of algorithms in automated trading. by John Bates. Dr. Dobb's Portal. March 9, 2007
http://www.ddj.com/hpc-high-performance-computing/197801615
© 2007-2014 Richard Holowczak All rights reserved.
7
CEP – Apama Example Trading Rule
Source: Algorithmic Trading: The use of algorithms in automated trading. by John Bates. Dr. Dobb's Portal. March 9, 2007
8
© 2007-2014 Richard Holowczak All rights reserved.
http://www.ddj.com/hpc-high-performance-computing/197801615
Oracle CEP Example
© 2007-2014 Richard Holowczak All rights reserved.
9
Oracle CEP Applications
• Adapters – connect inbound and outbound data streams
(can also be relational databases)
• Channels – act as queues accepting data and passing it
along to processors
• Processors (event processing agents) – applies standing
queries to channels and produces output events / signals
• Beans / Event Beans – Java Beans apps that are triggered
by events – carry out the work when an event occurs
From: http://www.oracle.com/technetwork/middleware/complex-event-processing/overview/oracle-47.pdf
© 2007-2014 Richard Holowczak All rights reserved.
10
Oracle CEP Development Tools
© 2007-2014 Richard Holowczak All rights reserved.
11
Oracle Continuous Query Language (CQL)
• Two main data objects to operate on: Streams and
Relations
• Stream and Relational (SQL) operators
– Oracle CEP CQL Language Reference
http://docs.oracle.com/cd/E23943_01/apirefs.1111/e12048/toc.htm
© 2007-2014 Richard Holowczak All rights reserved.
12