Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Event Processing in Operational Information Systems: Two Case Studies and BAM/EDA Implications Karsten Schwan, Brian Cooper, Greg Eisenhauer Georgia Institute of Technology Center for Experimental Research in Computer Systems (CERCS) NSF Industry University Co-operative Research Center I. Delta Air Lines Operational Information Systems (OIS) – Internal View High event rates for simple/mediated events Complex events processed/produced by business logic I. Delta Air Lines OIS – External View I. Continuous Event Processing in Delta’s OIS – Complex systems and large event volumes • TPF, DTMI, TIBCO, Tuxedo, Web Services; Mainframes, Clusters, End Systems – event services across multiple system `silos’ » interoperability APIs » event filtering, replication, morphing » JIT XML and event conversion – for outsources services » runtime trust management vs. security? – data tapping – for legacy systems (hardware support?) – deep packet inspection/event morphing (system/network support?) I. Integrated BAM: Continuously Managed Event Flows – Complex system interactions and 24/7 operation: • high reliability and availability: with stateful operation – continuous monitoring and repair » abnormal behavior (e.g., timeout behavior) detection, with human intervention after thresholds exceeded – `poison messages’ and poison message sequences » avoid recovery and/or bound recovery time • online performance management – utility-based event scheduling/routing » ability to distinguish service levels – link to immediate business needs » e.g., revenue management – performance isolation vs. optimization » e.g., isolation from recovery traffic – NOTES: highly distributed event processing; most events carry business data (additional BAM events); BASE, not ACID, for most events; multi-model event processing, not SQL; STATEful processing II. Worldspan: Need for QoS in Business Monitoring Utility Obtained from Worldspan’s Flight Search Engine SLA-driven operation and online event scheduling: • QoS in Business Monitoring for differentiated services 24/7 operation and stateful services: • Management must include incremental updates of service state Huge event volumes Summary Event-based Systems for the Enterprise Domain: • GT Focus: Adaptive/Autonomic Distributed Information Flows • IBM, Tata (iFlow: utility-based, autonomic management of distributed information flows; performance isolation in web-based event flows; online monitoring and management with Eclipse) • HP (automated application deployment; QMon: QoS in business activity monitoring) • Worldspan (`power udpates’: non-intrusive dynamic state updates; utility-based activity monitoring) • Delta, Raytheon (performance isolation/robustness; utility-driven failure management; monitoring web-based infrastructures) • Cisco, Intel (network-level services for event-based systems) • NSF, DARPA, DOE (continual queries; ECho/IQ-ECho:publish/subscribe event system, with resource-aware operation; EV(ent)Path: dynamic overlay creation and management, with runtime event scheduing; event flows and mobility) EDA/BAM Implications • Multiple event/processing models – Monitoring events, Business events, ... • Interoperability – Differently structured event data, eventually should include unstructured data • Complex, domain-specific event processing – Importance of state • state recovery/expiration – Distributed data and processing • Security/performance/reliability implications – Importance of online management • integrated into business event processing • driven by end user utility • strong QoS/real-time constraints • Overlap/conflicts with AC (ICAC) (many companies involved!) – Terminology: • CBEs (events), touchpoints, symptoms/symptom databases, SLAs, SLOs, ... – Technology: • non-intrusive instrumentation, ... Georgia Tech Information Flow Research To construct the interactive information grids of the future and to create the intellectual capital that can advance these technologies and fuel future advances. Enterprise Computing Embedded Systems Scientific .Grid Remote access to the Information Grid Brian Cooper Ling Liu Calton Pu Kishore Ramachandran Karsten Schwan Continual Queries ECho/IQ-ECho Fusion Channels IFlow/EVPath Information anytime, anywhere Timeliness! Robustness! Quality! Security and Trust! Additional Insights Enterprise Systems • Utility-based mapping and configuration in: – shared execution environments High Performance Computing • Large-data events in: – simulation monitoring: e.g., remote data visualization – GT Smartpointer application Pervasive Systems • Online path management in: – situation monitoring and assessment • Location-aware operation in: – mobile end user systems Research Agenda for Event-based Systems • I. Stateful Event Services: – – – – • II. Resource- and Needs-Awareness: – – – • Diverse metrics: bandwidth, power, trust, ... Changing end user needs, application behaviors Performance monitoring/understanding: integrate across user and system levels III. Runtime Management: – – – – • Dynamic service and code deployment (DCG, dynamic compilation) Runtime code modification and adaptation, dynamic data conversion Dynamic state saving and updates (e.g., power updates) Dynamic overlays, … Utility-driven operation New reliability and availability methods `Vertical’ integration: user/system/network levels Multi-dimensional optimization vs. performance robustness IV. Open Infrastructures: – – – App-level (e.g., `inside’ JMS) or `instrumented networks’ `Black box’ operating systems vs. dynamic extension and VM technologies `Closed’ networks vs. application-level services `in’ network devices • e.g., Cisco’s AONS, Intel’s IXP network processors Event Processing in EScience – SmartPointer Example SmartPointer: Dataintensive scientific collaboration Dynamic composition of userspecified services. SOAP Gateway Web-enabled client IQ SmartPipe Remote client IQ IQ EXMD IQ SmartPipe Server IQ SmartPipe Morph Service IQ 2D control SmartPipe Desktop client IQ SmartPipe Ipaq