Download slides - Extreme! Computing

On Using BPEL Extensibility to
Implement OGSI and WSRF
Grid Workflows
Aleksander Slomiski
Department of Computer Science
Indiana University
www.extreme.indiana.edu
Indiana University Extreme! Lab
Workflow
• WfMC defined a workflow as “The automation of a
business process, in whole or parts, where documents,
information or tasks are passed from one participant to
another to be processed, according to a set of
procedural rules”
– This definition is document office automation oriented
• Workflow Management System (WFMS): “a system that
defines, creates and manages the execution of
workflows through the use of software, running on one or
more workflow engines, which is able to interpret the
process definition, interact with workflow participants
and, where required, invoke the use of IT tools and
applications”.
Indiana University Extreme! Lab
Historical Perspective
• ’65-’75 Decompose Applications
– Data And Code Separated
• ’75-’85 Database Management
– DBMS Used To Share Data
• ’85-’95 User Interface Management
– UIMS User Interface Separated
• ’95-’05 Workflow Management
– Isolate Business Process
“Workflow Management” Aalst, van Hee
Indiana University Extreme! Lab
Categories Of Workflows
Production
Ad Hoc
Administrative
Business Value 
Collaborative
Repetition 
“Production Workflows” Leyman, Roller
Indiana University Extreme! Lab
Orchestration and Web Services
• WSFL
– IBM: Web Services Flow Language, May 2001
• XLANG
– Microsoft, May 2001
• GSFL
– Grid Services Flow Language, July 2002
• WSCL / WSCI / W3C WS Choreography WG
– HP WS Conversation Language, March 2002
– Web Service Choreography Interface, August 2002
• BEA, SAP, Sun, Intalio
• BPEL4WS / OASIS WSBPEL
– Replaces WSFL and XLANG, August 2002
Indiana University Extreme! Lab
BPEL4WS
• OASIS WSBPEL group:
• BEA, Choreology Ltd, Collaxa, EDS, HP, IBM,
Intalio, NEC, Novell, Microsoft, Oracle, SAP, Sun,
Sybase, Workflow Management Coalition (WfMC),
and many more ...
• Unique mix of two different paradigms
– XLANG: hierarchical structure with
specialized control constructs
– WSFL: graph structure with control patterns
based on transition and join conditions.
Indiana University Extreme! Lab
BPEL4WS Overview
• Specifies how to connect multiple web service to
provide new web service
• The same language is defined to define executable and
abstract process (contract)
• Executable process describes everything needed to
execute workflow
• Abstract process describes required observable
behavior of workflow based on message exchange (this
allows to verify contracts between business partners)
• Provide support for basic Web Service activities: invoke,
receive, reply
• Implicit lifecycle: workflow process instance is created
when a message is marked as "start" and arrives to
workflow engine
Indiana University Extreme! Lab
Motivation:
XML/Web/Grid Services Fusion
RPC
style
Grid
WS/XML
Services
WSRF
Framework
Indiana University Extreme! Lab
Integration Goals
• Leverage Web Services Workflow
Specification
– BPEL4WS suitable with built-in extensibility
• Compatibility with BPEL 1.1
– Extend Not Replace!
– Can execute BPEL 1.1 examples without any
change
• Seamless integration with WS-RF (OGSI)
– Grid/Semantic Web can use GPEL as
“executable”
Indiana University Extreme! Lab
Additional Goals
• Fully Observable State
– Monitoring tools
• Steering: Dynamically Modifiable
• Modular extensibility
– For example: module to work with XCAT
components
• Easy to integrate inside portals
Indiana University Extreme! Lab
Example Workflow Process
Web Services
(Partners)
Grid Services
(Grid Partners)
Component
Services
(Create, Connect)
Receive
Web Service
Invoke
Invoke
WSRF
Service
Another BPEL
Process
Reply
CoG Services
(security, GridFTP)
Pub/Sub
Messaging
Indiana University Extreme! Lab
BPEL And OGSI Integration
• Leveraging BPEL abstractions
– BPEL Partner is known by interface (PortType)
– Very powerful integration mechanism!
• Understands natively GSH/Locator as
service reference
– Handle automatically GSH to GSR conversion
• Supports Explicit and Implicit Lifecycle
– GPEL workflow engine is also OGSI Factory
Indiana University Extreme! Lab
Issues in OGSI Integration
• GSH/GSR and WS-Addressing
– Endpoint Reference (EPR) is not GSH or GSR …
– Semantic differences: EPR has WS-Police, …
• BPEL and OGSI Factory
– BPEL Engine acts as factory
– BPEL has already implicit creation model
• BPEL Engine that is NOT OGSI Factory
– Will not fit into common OGSI usage pattern
– OGSI Factory is optional …
Indiana University Extreme! Lab
WSRF support
• WS-Addressing (WSA)
– Workflow Instance identification
• WSA is already used by BPEL …
– No need to do conversions
• WS-Resource properties
– Similar to OGSI SDEs
• WS-Notification
– Monitoring workflow state
Indiana University Extreme! Lab
Integration with Today Grid
• GridFTP, GSI Security, Condor, …
• Add set of pseudo partners (services)
– No need to extend BPEL syntax!
– They represent built-in functionality (handy!)
– Can be implemented as real Web Services
• Will make easy to write workflow that is:
–
–
–
–
Working on Grid (CoG services)
Manipulating components (XCAT)
Large data transfers (GridFTP, RFT service)
…
• Similarly could have OGSI partners
– Natively understands GSH/GSR/locator
– Maintains soft state etc.
Indiana University Extreme! Lab
Very Long Running Processes
• Some processes are short lived and some not
– Long running simulations
– Iterative Computation
– Permanent searches
• Workflow Engine must be able to survive host
restart
– All state stored in database
– Lends itself to scalability: per instance state is easy to
share
Indiana University Extreme! Lab
Synchronous vs. Asynchronous
• <receive/>  <reply/>
– Typical binding is SOAP/HTTP
• Use WS-Addressing <ws:ReplyTo/>
– User can specify its return address!!!
– Receive connection can be one and closed!
– Reply can be executed
Indiana University Extreme! Lab
Reliability (Under Development)
• Use WS-ReliableMessaging
– WS-RM enabled Workflow endpoint
– Invoking external services with WS-RM
– What if workflow engine is restarted?
• Allow to annotate partners
– Designate operations as idempotent
– Allow to specify exponential back-off retries
Indiana University Extreme! Lab
Additional Issues
• Workflow Monitoring
– How much state exposed as XML is needed?
– Need a common framework to filter
• WS-Notification Topics?
• Fault Tolerance
– Selection of alternative services
– Dealing with “unexpected” exceptions
• Supporting Large Data and Streaming
– Suitable metaphor (in BPEL)?
Indiana University Extreme! Lab
Summary
• BPEL can be adapted to work in Grids
– Key: deployment is not part of BPEL
– However there are many levels of adaptation …
• There are issues beyond OGSI/WSRF
– Supporting large data and streaming
– Dealing with faults and error recovery
– Monitoring and managing workflows
– Workflow deployment
– Security
–…