Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Instruments and Sensors as Grid Services Donald F. (Rick) McMullen1 Kenneth Chiu2 John C. Huffman1 Kia Huffman1 Randall Bramley1 1Indiana 2SUNY University Binghamton Motivations • Instruments and sensors are not well integrated into grids • Data is acquired, processed and stored before it “hits the grid”. • Need methodology for interacting with instruments and sensors in real time from grid applications • Some abstraction of sensor and instrument functionality is needed to make grid applications that use them more robust and flexible. 2 Goals • Integrate instruments and sensors (e.g. real-time data sources) into a Grid computing environment with Grid services interfaces. • Abstract instrument capabilities and functions to reduce data acquisition and analysis applications’ dependence on specialized knowledge about particular instruments. • Move production of metadata as close to instruments as possible and facilitate the automatic production of metadata. • Develop a standard, reusable methodology for “grid enabling” instruments. • Collaborate with scientists in academia and industry in a broad range of disciplines who either develop instruments or whose work depends on the details of using them. 3 Increasing bandwidth per sensor Electron Microscope X-Ray Crystallography Ground motion sensor array Traffic sensors Radio Telescope Wireless ‘mote’ sensor network Increasing number of sensors Increasing real-time application • Simple 2-D analysis of instrument taxonomy • At lease five dimensions identified in more detailed analysis • Project must address enough points (classes) to assure breadth of applicability 4 Initial Applications • High brilliance X-ray crystallography – Large instrument application – Deeply integrated into bio and medical discovery research methodology – Mature analysis software and large user community • Robotic telescopes – Small numbers of sensors: CCD, environmental; some control aspects: filters, aiming, dome. – Global coordination needed for scheduling – Aggregation of disparate sensors into a “composite” instrument • Small sensors – Minimal memory and CPU – Wireless connectivity – Developing parallel project to use ad hoc/swarm networks for data collection for real-time simulation and prediction – Updating of data flows in response to sensor/network reconfiguration. 5 CIMA implementation targets Large scientific instruments Embedded sensors and controllers very large systems, few elements very small systems, many elements PC104 industrial controller board Synchrotron beamline (APS/ALS) MICA Mote wireless 6 sensor/controller board MMSF Automated Telescope • Typical remote access automated observatory • System has 33 distinct “sensors”, 12 controllers – Open/close of roof based on a Polaris transparency monitor and rain detector – simple grid of wires detecting rain drops – Telescope direction and dome control – Filter selection and telescope focus – Liquid nitrogen fills of the CCD dewar jars …. 7 MMSF Observatory Features • Instruments producing data without units – Temperature, humidity cutoffs determined empirically as resistances, not degrees or % • Hierarchical and co-located instruments – Single platform holds three instruments, so orienting one changes orientation of all • Updates to equipment occurs frequently • Data transferred via 28k modem line – middleware needs to work locally, directly between instruments and sensors 8 Observatory Data Architecture • Control Data – object control – instrument control • Object Data (i.e., object of scientific interest) – Full spectrum from raw unitless data to derived data artifacts • Instrument package – – – – – system package data (multiple attributes output) system sensor data (single attribute output) nonsystem sensor data (weather data from NOAA) calibration data access protocols 9 X-Ray Crystallography Proxy Box Instrument Services WAN Instrument Manager LAN Portal LAN Non-grid service Grid service Persistent Data Archive Non-persistent 10 LTER University of Wisconsin Campus Sensors Buoy Env. Agent Oracle Datalogger ORB Config. Agent QA Agent Config. Event (CIMA) Other Locations Trout Lake Station Config. Agent QA Agent 2 ARTS Connection 4 3 Web Server Config. Agent 3 Env. Event (CIMA) Web Browser Config. Agent Web Browser 1 JDBC/ODBC Connection Other Connection • • • Automatic updating of flows Provenance SOAP/ARTS (Antelope Real-Time System) 11 Further applications currently being explored Electron Microscopy -U. Queensland EM group – regional scale, multiple instruments - KISTI 2nd largest EM (after Osaka) Robotic telescopes - Bradford robotic telescope, Oxenhope observatory, Faulkes telescope project Environmental monitoring - Water use - contaminants Industrial monitoring and control, e.g. Train axles - Ore trains – Km long, derailment is very expensive to fix - Temperature sensors on axles monitor bearing status, anticipate wheel failure 12 Architecture 13 Common Instrument Middleware Architecture (CIMA) Elements • Schema for instrument functionality (and ontology for schema attributes); • Data model for representing instrument metrics and calibration; • A small, high performance, embeddable Web Services stack, initially in Java, including Proteus support for multi-protocol, multimodal transport; • Service implementation for accessing the instrument’s functionality and metrics via the Proteus-mediated interface; – Ability to dynamically insert new protocols into running instances • OGSA and WS-RF compliant functions to register with a location service, authenticate users, provide access control to instrument controls and data, send and receive events, and co-schedule the instrument into a Grid computing and storage context. 14 Overview Data Pipeline Instrument Sensor 2 Acquisition Component Analysis Component Curation Component Controller Acquisition Code Analysis Code Curation Code Instrument Presentation Instrument Access Instrument Access Instrument Access Sensor 1 Physical Network Transport Instrument Access Device-Dependent Virtualization Module Remote Access Device-Independent Application Module Shared Implementation GUI Scientist 15 Instrument Access Instrument Presentation Instrument-Specific CIMA Instrument-Independent CIMA Existing Instrument Code Service Implementation Controller OGSI SOAP Binary Format Instrument Hardware Proteus TCP/IP Other Network Protocols 16 Example CIMA minimal knowledge bootstrap procedure CIMA instrument Application Globus init, user authentication, and instrument lookup Application parses description for ports to read for calibration and voltage Proteus/SOAP calls “send description” “RDF description” Service Implementation (SI) returns description of itself and instrument “read calibration port” “calibration” SI returns stored calibration “read thermocouple port” “thermocouple voltage” Application reads thermocouple voltage then computes and displays a temperature SI calls controller function to read and return voltage 17 Services • Instrument Instrument – get – set Sensor • Sensor – get – set Channel Sink Channel Source Channel Source • ChannelSource – get – set – register • ChannelSink Sensor Sensor Channel Source Channel Source – receive – get – set 18 Parcels • Wish to unify our data models, etc. • Toolkit must be application-independent as much as possible. • Attributes – Type (string) – Globally unique ID (string) – Encoding • CDATA, Binary, ASCII, Base64 – Location • Inline, URL, Other • Parcel Sets – Special data used for connectivity information. 19 Technologies 20 Technologies • Web services – XML, SOAP, WSDL, binary XML • Grid services – OGSA/OGSI, WS-RF, DAIS • • • • Axis C++ gSOAP Proteus (SC 2002) XBS (HPC 2004) Schema-specific parsing 21 Proteus Motivation • Web Services for scientific computing – SOAP performs well as a lingua franca • But suffers from performance problems for scientific data – Solution: establish initial communication with SOAP, and then switch to a faster protocol. • Grid intermediaries 22 Proteus Overview • Provides multiprotocol RMI system to applications – Can wrap existing protocol implementations with dynamic invocation • Facilitates use of SOAP as common language – Switch to faster protocols if supported by both sides. • Mediates between protocol providers and applications – Applications use Proteus client API – Providers use Proteus provider API • Allows a new provider to be added (at run-time) without changing application • Generic serialization/deserialization allows marshalling code to be reused for multiple protocols 23 Multiprotocol Process 1 Process 2 Client Client Proteus Proteus ProviderA ProviderA ProviderB ProviderB Protocol A Network Protocol B 24 Proteus APIs Application Client API Proteus Provider API Protocol A Provider Protocol B Provider 25 Schema-Specific Parsing • XML processing stages (conceptual) – Well-formedness • Lexical and syntactic, defined by core XML specification. – Validity • Conformance to a schema, mainly structural – Application 26 Merging Stages Application Application Application Validation Validation Validation Wellformedness Wellformedness Wellformedness Fully articulated Implicitly validated Schema-specific parsing User written Merged 27 Compiler-Based Approach C (fast) XML Schema IR RELAX NG Java C (low-pow) • Front-end parses schema into intermediate representation. • Back-end generates code from intermediate representation. • Intermediate representation is a generalized automata. 28 Summary • Create standards for accessing broad spectrum instruments and sensors. • Incompatible components should still have some base level of interoperability. 29