Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Web and Grid Services Slides taken from a variety of sources: GT4 tutorial, by Borja Sotomayor http://gdp.globus.org/gt4-tutorial/ International Summer school on Grid Computing 2007m by Malcolm Atkinson http://www.iceageeu.org/issgc07/index.cfm HP introduction to WSRF by Sanjay Dahiya http://foss.in/slides/lb2004/wsrf-intro.ppt USC Viterbi School of Engineering • • • • • Service Oriented Architecture Web Services WS-RF Globus implementation of WS-RF OGSA-DAI USC Viterbi School of Engineering Service Oriented Architectures: Three Components Registries Register an available service Send name & description Service Consumers Services USC Viterbi School of Engineering Three Components Registries Request a service Send a description Service Consumers Services USC Viterbi School of Engineering Three Components Registries Set (possibly empty) of matching services Service Consumers Services USC Viterbi School of Engineering Three Components Registries Service Consumers Request service operation Services USC Viterbi School of Engineering Three Components Registries Service Consumers Return result or Error Services USC Viterbi School of Engineering Composed behaviour • Services are themselves consumers – They may compose and wrap other services • The registry is itself a consumer • A federation of registries may deal with registry services reliability & performance • Observer services may report on quality of services and help with diagnostics • Agreements between services may be set up – Service-Level Agreements – Permitting sustained interaction USC Viterbi School of Engineering Web Services USC Viterbi School of Engineering Web Services • • • • Enable communication between machines Provide any number of functionality Can be found and invoked Self-describing—tell you how they can be invoked USC Viterbi School of Engineering Web Services • Web Services are platform-independent and languageindependent, since they use standard XML languages • Most Web Services use HTTP for transmitting messages (such as the service request and response) – Good for going through firewalls • Issues: – Overhead: XML is not as efficient for data transmission as using a proprietary binary code. What you win in portability, you lose in efficiency. – Limited in functionality, no explicit state management or lifecycle management USC Viterbi School of Engineering Typical Service Invocation USC Viterbi School of Engineering The Web Services architecture USC Viterbi School of Engineering A typical web service invocation • Whenever a client needs to communicate with a service, it calls a client stub • The client stub will turn this 'local invocation' into a proper SOAP request • The SOAP request is sent over a network using the HTTP protocol. The server receives the SOAP requests and hands it to the server stub which invokes the service implementation USC Viterbi School of Engineering Web Server • Web service: software that exposes a set of operations • SOAP engine: knows how to handle SOAP requests and responses-Apache Axis • Application server: provides a space for applications that multiple clients can access—a container--Jakarta Tomcat server • Http server: knows how to handle HTTP messages Sometimes a container is described as: SOAP engine + application server + HTTP server USC Viterbi School of Engineering Limitations of Web Services • No explicit state management or lifecycle management • Web services are usually stateless • Many services do not need state USC Viterbi School of Engineering Adding state to services • Need to allow clients to access appropriate state USC Viterbi School of Engineering “Stateless” vs. “Stateful” Services FileTransfer Service move move (A to B) Client • Without state, how does client: – – – – Determine what happened (success/failure)? Find out how many files completed? Receive updates when interesting events arise? Terminate a request? • Few useful services are truly “stateless”, but WS interfaces alone do not provide built-in support for state USC Viterbi School of Engineering Open Grid Services Architecture USC Viterbi School of Engineering Before OGSA • Grid services before OGSA – – – – Resource management (Globus GRAM) Resource discovery (Globus MDS) Data Management (Globus GridFTP, RLS) Security • All had – Different mechanism for access – Different mechanism for discovery – Different mechanisms for management USC Viterbi School of Engineering OGSA • Brings “order” to distributed services • Promotes “open” standards: defined in GGF (now OGF), OASIS • Enables Virtualization – Encapsulation behind a common interface of diverse implementations • Allows the composition of lower-level services to form more sophisticated services • Defines common behaviors that all services must have: – – – – Naming Lifetime management State management Notification USC Viterbi School of Engineering USC Viterbi School of Engineering WSRF USC Viterbi School of Engineering FileTransferService (without WSRF) FileTransfer Service move move (A to B) : transferID Client whatHappen state tellMeWhen cancel • Developer reinvents wheel for each new service – Custom management and identification of state: transferID – Custom operations to inspect state synchronously (whatHappen) and asynchronously (tellMeWhen) – Custom lifetime operation (cancel) USC Viterbi School of Engineering WSRF in a Nutshell • Service • State representation Service EPR EPR EPR – Resource – Resource Property GetRP GetMultRPs Resource SetRP QueryRPs RPs Subscribe SetTermTime Destroy • State identification – Endpoint Reference • State Interfaces – GetRP, QueryRPs, GetMultipleRPs, SetRP • Lifetime Interfaces – SetTerminationTime – ImmediateDestruction • Notification Interfaces – Subscribe – Notify • ServiceGroups USC Viterbi School of Engineering FileTransferService (w/ WSRF) FileTransferService createResource Transfer getRP RPs queryRPs createResource (A to B) : EPR Client destroy • Developer specifies custom method to createResource and leaves the rest to WSRF standards: – State exposed as Resource + Resource Properties and identified by Endpoint Reference (EPR) – State inspected by standard interfaces (GetRP, QueryRPs) – Lifetime management by standard interfaces (Destroy) USC Viterbi School of Engineering Grid Infrastructure: Open Standards Applications of the framework (Compute, network, storage provisioning, job reservation & submission, data management, application service QoS, …) WS-Agreement (Agreement negotiation) WS Distributed Management (Lifecycle, monitoring, …) WS-Resource Framework & WS-Notification* (Resource identity, lifetime, inspection, subscription, …) Web services (WSDL, SOAP, WS-Security, WS-ReliableMessaging, …) USC Viterbi School of Engineering WS-Resource Properties • Each resource has a Resource Properties document. • Resource Properties document is referred in service portType in WSDL. • Defined in XML schema. • Each element in the Resource Properties document is a Resource Property (RP). • Resource properties can be queried using multiple query dialects. • Independent of back-end implementation. USC Viterbi School of Engineering Accessing Resource Properties • Pull – Client can query the RP document, using query engines. • GetResourceProperty • GetMultipleResourceProperties • QueryResourceProperties • Push – Allows services to send changes in their resources’ RPs to interested parties. • WS-Notification USC Viterbi School of Engineering WS-Notification • Subscriber indicates interest in a particular “Topic” by issuing a “subscribe” request • Broker (intermediary) permits decoupling Publisher and Subscriber • “Subscriptions” are WS-Resources • Publisher need NOT be a Web Service • Notification may be “triggered” by: – WS Resource Property value changes – Other “situations” • Broker examines current subscriptions • Brokers may – “Transform” or “interpret” topics – Federate to provide scalability USC Viterbi School of Engineering WS-Notification • Characteristics of WS-Notification: – Web services integration of traditional enterprise publish/subscribe messaging patterns • Composes with other Web services technologies • Facilitates integration between different messaging middleware environments – Standardizes the role of Brokers, Publishers, Subscribers and Consumers – Provides two forms of publish/subscribe: direct publishing and brokered publishing – Standardizes Web service message exchanges for publishing, subscribing and notification delivery – Defines XML model of Topics and TopicSpaces to categorize and organize notification messsages USC Viterbi School of Engineering WS-Resource Lifetime • Creating new resources. • Destroying old resources. – Immediate destruction. – Scheduled destruction using termination time. • Soft-state lifetime management – Lifetime extension. • Example : – jobs in a batch submission system could be represented as resources – submitting a new job causes a new resource to be created – when the job is completed, the resource is destroyed. USC Viterbi School of Engineering WS-ServiceGroups • Service groups maintain information about collection of services or resources. • ServiceGroupRegistration – Add new members to group using WS invocation. • Represent service groups as resources. • MembershipContentRules – Imposes restrictions on services that can become part of service group like implementing an interface. • WS-Notifications for service group changes. • For example – Resource registries etc USC Viterbi School of Engineering WS-BaseFaults • XML based fault transmission. • Associated with an operation in WSDL. • Includes standard datatypes for transmitting webservice faults – Originator, Timestamp etc.. Example : <wsdl:portType name="pt"> <wsdl:operation name="op"> <!-- WSDL operation fault elements for each distinct fault --> <wsdl:input … /> <wsdl:output … /> <wsdl:fault name=“aFault" message="tns:aFaultMessage"/> <wsdl:fault name="BaseFault" message="wsbf:BaseFaultMessage"/> </wsdl:operation> </wsdl:portType> USC Viterbi School of Engineering Globus Toolkit implementation of WSRF USC Viterbi School of Engineering Globus Toolkit: Open Source Grid Infrastructure Globus Toolkit v4 www.globus.org Data Replication Credential Mgmt Replica Location Grid Telecontrol Protocol Delegation Data Access & Integration Community Scheduling Framework WebMDS Python Runtime Community Authorization Reliable File Transfer Workspace Management Trigger C Runtime Authentication Authorization GridFTP Grid Resource Allocation & Management Index Java Runtime Security Data Mgmt Execution Mgmt Info Services Common Runtime USC Viterbi School of Engineering Tools for building WSRF services GT4 WS Core in a Nutshell Service EPR EPR EPR GetRP GetMultRPs Resource SetRP QueryRPs RPs Implementation of WSRF: Resources, EndpointReferences, ResourceProperties Operation Providers: pre-build implementations of WSRF operations Subscribe Notification implementation: Topics, TopicSet, Embedded Notification Consumer service SetTermTime Destroy USC Viterbi School of Engineering GT4 WS Core in a Nutshell Service Container Service Service Service GetRP GetRP GetRP GetMultRPs EPR GetMultRPs EPR GetMultRPs EPR EPR SetRP EPR EPRResource SetRP EPRResource SetRP Resource QueryRPs QueryRPs RPs QueryRPs Subscribe RPs Subscribe RPs Subscribe SetTermTime SetTermTime ResourceHome SetTermTime Destroy ResourceHome Destroy ResourceHome Destroy Service Container: host multiple services in container; one JVM process …more details: based on AXIS service container, processes SOAP messages USC Viterbi School of Engineering GT4 WS Core in a Nutshell Service Container Service Service Service Secure Communication: Transport, Message, Conversation (Transport demonstrates best performance) PIP GetRP GetRP GetRP GetMultRPs EPR GetMultRPs EPR GetMultRPs EPR EPR SetRP EPR EPRResource SetRP EPRResource SetRP Resource QueryRPs QueryRPs RPs QueryRPs Subscribe RPs Subscribe RPs Subscribe SetTermTime SetTermTime ResourceHome SetTermTime Destroy ResourceHome Destroy ResourceHome Destroy PDP Configurable Security Policies: Policy Information Points (PIPs), Policy Decision Points (PDP) -- chained Example authorization PDPs: GridMap, SAML implementations USC Viterbi School of Engineering GT4 WS Core in a Nutshell Service Container PIP Service Service Service GetRP GetRP GetRP GetMultRPs EPR GetMultRPs EPR GetMultRPs EPR EPR SetRP EPR EPRResource SetRP EPRResource SetRP Resource QueryRPs QueryRPs RPs QueryRPs Subscribe RPs Subscribe RPs Subscribe SetTermTime SetTermTime ResourceHome SetTermTime Destroy ResourceHome Destroy ResourceHome Destroy WorkManager DB Conn Pool PDP WorkManager: “thread pool”, site independent “work” manager Apache Database Connection Pool library (JDBC “DataSource” implementation) JNDI Directory: manages internal, shared objects (ResourceHomes, WorkManager, Configuration objects,…) JNDI Directory USC Viterbi School of Engineering GT4 WS Core in a Nutshell Apache Tomcat Service Container PIP Service Service Service GetRP GetRP GetRP GetMultRPs EPR GetMultRPs EPR GetMultRPs EPR EPR SetRP EPR EPRResource SetRP EPRResource SetRP Resource QueryRPs QueryRPs RPs QueryRPs Subscribe RPs Subscribe RPs Subscribe SetTermTime SetTermTime ResourceHome SetTermTime Destroy ResourceHome Destroy ResourceHome Destroy WorkManager DB Conn Pool PDP JNDI Directory USC Viterbi School of Engineering Deploy Service Container “standalone” or within Apache Tomcat Relationship Between OGSA, GT4, WSRF & Web Services USC Viterbi School of Engineering Dealing with Data OGSA-DAI USC Viterbi School of Engineering OGSA-DAI • • • • • An extensible framework accessed via web services that executes data-centric workflows involving heterogeneous data resources for the purposes of data access, integration, transformation and delivery within a Grid • and is intended as a toolkit for building higher-level application-specific data services USC Viterbi School of Engineering Motivation • Grid is about sharing resources • OGSA-DAI is about sharing structured data resources Relational Database XML Database Indexed File USC Viterbi School of Engineering Sharing data via website download • ZIP up data and put it on a website • Pros – Easy distribution for providers – Easy access for consumers • Cons – Consumers have to download all the data – Consumers have to load data into local databases to use it – Static snapshot – Security USC Viterbi School of Engineering Sharing data via direct access • Providers tell consumers – Database URL – mycomputer.epcc.ed.ac.uk:3306 – Username – userID – Password – password • Pros – Consumers have direct access, so it should be faster • Cons – – – – – Firewall issues User and password management is hard No consistent security model Hard to use in grid/web service workflows Continued on next slide… USC Viterbi School of Engineering Sharing data via direct access • Cons (continued) – No server-side layer in which to standardize database heterogeneities – Client needs to know, and have installed, correct driver for the database. – Different drivers for Java, C#, C++, Fortran etc. – Totally different API for different database types, e.g. JDBC for Relational, XMLDB for XML, Lucene for indexed files. USC Viterbi School of Engineering Domain-specific web services • Manipulate data using domain-specific operations, e.g. – Book findByISBN(ISBN) – List<Book> findByAuthor(Author) – List<Book> findByKeyword(Word) • Pros – – – – Fits with grid/web service approach Abstraction hides back-end database details Web services are programming language neutral Operations likely to map well to authorization policies USC Viterbi School of Engineering • Cons Domain-specific web services – Slower than direct access • Web service layer • SOAP transport overhead – especially for large result sets – Domain-specific API prevents use of generic data exploration, mining and manipulation tools Books Cancer Generic Data Linking Application Books written by University employees University Employees University employees in 1932 who have since died of cancer USC Viterbi School of Engineering OGSA-DAI generic web services • Manipulate data using OGSA-DAI’s generic web services • Clients sees the data in its ‘raw’ format, e.g. – Tables, columns, rows for relational data – Collections, elements etc. for XML data • Clients can obtain the schema of the data • Clients send queries in appropriate query language, e.g. SQL, XPath Relational Database request OGSA-DAI XML Database data Indexed File USC Viterbi School of Engineering OGSA-DAI • Pros: – Fits with grid/web service approach. – Web services are programming language neutral. – Access to schema and raw data supports generic tools. • Cons: – Slower than direct connection mainly due to SOAP overhead. – One more layer between client and data – Data not transferred in efficient binary format. USC Viterbi School of Engineering Reducing the SOAP effect – workflows OGSA-DAI Transform Web Service OGSA-DAI Query -> Transform -> DeliverToFTP FTP Server USC Viterbi School of Engineering FTP Server