Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Complex Event Processing Prof. Richard Holowczak © 2007-2014 Richard Holowczak All rights reserved. Streams of Data • • • • • • • • RFID scans Transportation / Traffic scans Web site visits Server logs Financial / Market data Manufacturing Processes Environmental monitoring / Remote sensing Gaming events © 2007-2014 Richard Holowczak All rights reserved. 2 Challenges of Streaming Data • Data may arrive and demand action at a rate faster than a traditional database can respond • Require flexibility / generalizability in coding approaches – Compare custom C++/Java vs. a query language like SQL © 2007-2014 Richard Holowczak All rights reserved. 3 DBMS vs. Stream Processing 1 2 Event Insert 3 Query Data DBMS Database 4 Query Result Action 5 1 2 Query 5 Insert Event Data CEP Database 3 Query Result CEP 4 Action 4 © 2007-2014 Richard Holowczak All rights reserved. Event Processing Vendors • Generic Systems – Oracle CEP www.oracle.com/us/products/middleware/soa/eventprocessing – Amazon Kinesis aws.amazon.com/kinesis – SQL Stream www.sqlstream.com • Financial Services - Oriented – Sybase Aleri www.aleri.com – TIBCO StreamBase www.streambase.com – Apama www.softwareag.com/corporate/products/bigdata/apama/ © 2007-2014 Richard Holowczak All rights reserved. 5 Stream / Event Processing • Data arrives in a stream, e.g. – Weather data – RFID scans – Market Data • Rules put in place to match patterns in one or more streams • When a rule is “satisfied” trigger and Event or Alert – Possibly triggering some further actions © 2007-2014 Richard Holowczak All rights reserved. 6 Complex Event Processing – Example from Apama Source: Algorithmic Trading: The use of algorithms in automated trading. by John Bates. Dr. Dobb's Portal. March 9, 2007 http://www.ddj.com/hpc-high-performance-computing/197801615 © 2007-2014 Richard Holowczak All rights reserved. 7 CEP – Apama Example Trading Rule Source: Algorithmic Trading: The use of algorithms in automated trading. by John Bates. Dr. Dobb's Portal. March 9, 2007 8 © 2007-2014 Richard Holowczak All rights reserved. http://www.ddj.com/hpc-high-performance-computing/197801615 Oracle CEP Example © 2007-2014 Richard Holowczak All rights reserved. 9 Oracle CEP Applications • Adapters – connect inbound and outbound data streams (can also be relational databases) • Channels – act as queues accepting data and passing it along to processors • Processors (event processing agents) – applies standing queries to channels and produces output events / signals • Beans / Event Beans – Java Beans apps that are triggered by events – carry out the work when an event occurs From: http://www.oracle.com/technetwork/middleware/complex-event-processing/overview/oracle-47.pdf © 2007-2014 Richard Holowczak All rights reserved. 10 Oracle CEP Development Tools © 2007-2014 Richard Holowczak All rights reserved. 11 Oracle Continuous Query Language (CQL) • Two main data objects to operate on: Streams and Relations • Stream and Relational (SQL) operators – Oracle CEP CQL Language Reference http://docs.oracle.com/cd/E23943_01/apirefs.1111/e12048/toc.htm © 2007-2014 Richard Holowczak All rights reserved. 12