Download Remarks on Grids e-Science

Architecture of
Web Service Grids
Beijing
Beihang University
August 27 2004
Geoffrey Fox
Community Grids Lab
Indiana University
[email protected]
e-Business e-Science and the Grid






e-Business captures an emerging view of corporations as
dynamic virtual organizations linking employees, customers
and stakeholders across the world.
• The growing use of outsourcing is one example
e-Science is the similar vision for scientific research with
international participation in large accelerators, satellites or
distributed gene analyses.
The Grid integrates the best of the Web, traditional
enterprise software, high performance computing and Peerto-peer systems to provide the information technology
infrastructure for e-moreorlessanything.
A deluge of data of unprecedented and inevitable size must
be managed and understood.
People, computers, data and instruments must be linked.
On demand assignment of experts, computers, networks and
storage resources must be supported
e-Science



e-Science is about global collaboration in key areas of
science, and the next generation of infrastructure that
will enable it. This is a major UK Program
e-Science reflects growing importance of international
laboratories, satellites and sensors and their integrated
analysis by distributed teams
CyberInfrastructure is the analogous US initiative
Grid Technology
supports e-Science
and
DATA
ACQUISITION
CyberInfrastructure
ADVANCED
VISUALIZATION
,ANALYSIS
QuickTime™ and a
decompressor
are needed to see this picture.
IMAGING INSTRUMENTS
COMPUTATIONAL
RESOURCES
LARGE-SCALE DATABASES
DAME
In flight data
~5000 engines
~ Gigabyte per aircraft per
Engine per transatlantic flight
Airline
Global Network
Such as SITA
Ground
Station
Engine Health (Data) Center
Maintenance Centre
Internet, e-mail, pager
Rolls Royce and UK e-Science Program
Distributed Aircraft Maintenance Environment
NASA Aerospace Engineering Grid
Wing Models
•Lift Capabilities
•Drag Capabilities
•Responsiveness
Airframe Models
Stabilizer Models
•Deflection capabilities
•Responsiveness
Crew
Capabilities
- accuracy
- perception
- stamina
- re-action
times
- SOP’s
Human Models
Engine Models
•Braking performance
•Steering capabilities
•Traction
•Dampening capabilities
Landing Gear Models
•Thrust performance
•Reverse Thrust performance
•Responsiveness
•Fuel Consumption
simulations
are produced
by coupling
ItWhole
takes asystem
distributed
virtual organization
to design,
simulate
andall
build
a complex
system simulations
like an aircraft
of the
sub-system
Virtual Observatory Astronomy Grid
Integrate Experiments
Radio
Far-Infrared
Visible
Dust Map
Visible + X-ray
Galaxy Density Map
e-Chemistry Laboratory
Experiments-on-demand
Grid-enabled Output Streams
Simulation
Video
Diffractometer
Properties
Analysis
Structures
Database
GridGlobus
Resources
X-Ray
e-Lab
Properties
e-Lab
Fig. 23: A Combinatorial Chemistry Grid (Chapter 42)
CERN LHC Data Analysis Grid
Philosophy of Web Service Grids
• Much of Distributed Computing was built by natural
extensions of computing models developed for sequential
machines
• This leads to the distributed object (DO) model represented
by Java and CORBA
– RPC (Remote Procedure Call) or RMI (Remote Method
Invocation) for Java
• Key people think this is not a good idea as it scales badly
and ties distributed entities together too tightly
– Distributed Objects Replaced by Services
• Note CORBA was considered too complicated in both
organization and proposed infrastructure
– and Java was considered as “tightly coupled to Sun”
– So there were other reasons to discard
• Thus replace distributed objects by services connected by
“one-way” messages and not by request-response messages
Programs
Computational resources
service logic
BPEL, Java, .NET
Databases
SOAP and WSDL
• Web Services build
loosely-coupled,
distributed applications,
based on the SOA
principles.
• Web Services interact
by exchanging
messages in SOAP
format
• The contracts for the
message exchanges that
implement those
interactions are
described via WSDL
interfaces.
Humans
<env:Envelope>
<env:Header>
...
</env:header>
<env:Body>
...
</env:Body>
</env:Envelope>
SOAP messages
message processing
Web services
resources
Devices
What is a Grid?
• You won’t find a clear description of what is Grid and how
does differ from a collection of Web Services
– I see no essential reason that Grid Services have different
requirements than Web Services
– Geoffrey Fox, David Walker, e-Science Gap Analysis, June 30
2003. Report UKeS-2003-01,
http://www.nesc.ac.uk/technical_papers/UKeS-2003-01/index.html.
– Notice “service-building model” is like programming language –
very personal!
• Grids were once defined as “Internet Scale Distributed
Computing” but this isn’t good as Grids depend as much if
not more on data as well as simulations
• So Grids can be termed “Internet Scale Distributed
Services” and represent a way of collecting services
together to solve problems where special features and
quality of service needed.
e-Infrastructure



e-Infrastructure builds on the inevitable increasing performance
of networks and computers linking them together to support new
flexible linkages between computers, data systems and people
• Grids and peer-to-peer networks are the technologies that
build e-Infrastructure
• e-Infrastructure called CyberInfrastructure in USA
We imagine a sea of conventional local or global connections
supported by the “ordinary Internet”
• Phones, web page accesses, plane trips, hallway conversations
• Conventional Internet technology manages billions of
broadcast or low (one client to Server) or broadcast links
On this we superimpose high value multi-way organizations
(linkages) supported by Grids with optimized resources and
system support
• Low multiplicity fully interactive real-time sessions
• Resources such as databases supporting (larger) communities
N plus N Community Resources


Grid Community databases have analogy to Television and the
News Web that allow individuals to communicate instantly with
each other via Web Pages and Headline News acting as proxies
N resources deposit information and N can view  Call N plus N
Large and Small Grids






N resources in a community (N is billions for the world and 100010000 for many scientific fields)
Communities are arranged hierarchically with real work being
done in “groups” of M resources – M could be 10-100 in e-Science
Metcalfe’s law: value of network grows like square of number of
nodes M – we call Grids where this true Metcalfe or M2 Grids
Nature of Interaction depends on size of M or N
• N plus N Shared Information Grids for largish N
• M2 Metcalfe Grids for smaller M < N
Technology support depends on M/N – might use a relatively
static DHT (Distributed Hash Table) for large N and a distributed
shared memory for small M
Grids must merge with peer-to-peer networks to support both N
plus N and M2 Systems
M2 Interactions
• Superimpose M way
“Grids” on the sea
(heatbath) of “2 by N”
or N plus N
“ordinary”
interactions
Grids also support
many community
N plus N resources
Implement Grids
as a software
overlay network
Dynamic light-weight Peer-to-peer
Collaboration Training Grid
Enterprise Grid
Students
Information Grid
R1
R2
Compute Grid
Teacher
Campus Grid
4 Overlay Networks
With a 5th superimposed
Architecture of (Web Service) Grids



Grids built from Web Services communicating through
an overlay network built in SOFTWARE on the
“ordinary internet” at the application level
Grids provide the special quality of service (security,
performance, fault-tolerance) and customized services
needed for “distributed complex enterprises”
We need to work with Web Service community as they
debate the 60 or so proposed Web Service specifications
•
•
•
•
Use Web Service Interoperability WS-I as “best practice”
Must add further specifications to support high performance
Database “Grid Services” for N plus N case
Streaming support for M2 case
Web Services
• Java is very powerful partly due to its many “frameworks” that
generalize libraries e.g.
– Java Media Framework
– Java Database Connectivity JDBC
• Web Services have a correspondingly collections of specifications
that represent critical features of the distributed operating systems
for “Grids of Simple Services”
– Some 60 active WS-* specifications for areas such as
– a.
Core Infrastructure Specifications
– b.
Service Discovery
– c.
Security
– d.
Messaging
– e.
Notification
– f.
Workflow and Coordination
– g.
Characteristics
– h.
Metadata and State
– i.
User Interfaces
A List of Web Services I
• a) Core Service Architecture
• XSD XML Schema (W3C Recommendation) V1.0 February 1998,
V1.1 February 2004
• WSDL 1.1 Web Services Description Language Version 1.1, (W3C
note) March 2001
• WSDL 2.0 Web Services Description Language Version 2.0, (W3C
under development) March 2004
• SOAP 1.1 (W3C Note) V1.1 Note May 2000
• SOAP 1.2 (W3C Recommendation) June 24 2003
• b) Service Discovery
• UDDI (Broadly Supported OASIS Standard) V3 August 2003
• WS-Discovery Web services Dynamic Discovery (Microsoft,
BEA, Intel …) February 2004
• WS-IL Web Services Inspection Language, (IBM, Microsoft)
November 2001
A List of Web Services II
• c) Security
• SAML Security Assertion Markup Language (OASIS) V1.1 May
2004
• XACML eXtensible Access Control Markup Language (OASIS)
V1.0 February 2003
• WS-Security 2004 Web Services Security: SOAP Message
Security (OASIS) Standard March 2004
• WS-SecurityPolicy Web Services Security Policy (IBM,
Microsoft, RSA, Verisign) Draft December 2002
• WS-Trust Web Services Trust Language (BEA, IBM, Microsoft,
RSA, Verisign …) May 2004
• WS-SecureConversation Web Services Secure Conversation
Language (BEA, IBM, Microsoft, RSA, Verisign …) May 2004
• WS-Federation Web Services Federation Language (BEA, IBM,
Microsoft, RSA, Verisign) July 2003
A List of Web Services III
• d) Messaging
• WS-Addressing Web Services Addressing (BEA, IBM, Microsoft)
March 2004
• WS-MessageDelivery Web Services Message Delivery (W3C
Submission by Oracle, Sun ..) April 2004
• WS-Routing Web Services Routing Protocol (Microsoft) October 2001
• WS-RM Web Services Reliable Messaging (BEA, IBM, Microsoft,
Tibco) v0.992 March 2004
• WS-Reliability Web Services Reliable Messaging (OASIS Web Services
Reliable Messaging TC) March 2004
• SOAP MOTM SOAP Message Transmission Optimization Mechanism
(W3C) June 2004
• e) Notification
• WS-Eventing Web Services Eventing (BEA, Microsoft, TIBCO) January
2004
• WS-Notification Framework for Web Services Notification with WSTopics, WS-BaseNotification, and WS-BrokeredNotification (OASIS)
OASIS Web Services Notification TC Set up March 2004
• JMS Java Message Service V1.1 March 2002
A List of Web Services IV
• f) Coordination and Workflow, Transactions and Contextualization
• WS-CAF Web Services Composite Application Framework including WS-CTX,
WS-CF and WS-TXM below (OASIS Web Services Composite Application
Framework TC) July 2003
• WS-CTX Web Services Context (OASIS Web Services Composite Application
Framework TC) V1.0 July 2003
• WS-CF Web Services Coordination Framework (OASIS Web Services Composite
Application Framework TC) V1.0 July 2003
• WS-TXM Web Services Transaction Management (OASIS Web Services
Composite Application Framework TC) V1.0 July 2003
• WS-Coordination Web Services Coordination (BEA, IBM, Microsoft) September
2003
• WS-AtomicTransaction Web Services Atomic Transaction (BEA, IBM, Microsoft)
September 2003
• WS-BusinessActivity Web Services Business Activity Framework (BEA, IBM,
Microsoft) January 2004
• BTP Business Transaction Protocol (OASIS) May 2002 with V1.0.9.1 May 2004
• BPEL Business Process Execution Language for Web Services (OASIS) V1.1 May
2003
• WS-Choreography (W3C) V1.0 Working Draft April 2004
• WSCI (W3C) Web Service Choreography Interface V1.0 (W3C Note from BEA,
Intalio, SAP, Sun, Yahoo)
• WSCL Web Services Conversation Language (W3C Note) HP March 2002
A List of Web Services V
• h) Metadata and State
• RDF Resource Description Framework (W3C) Set of recommendations expanded
from original February 1999 standard
• DAML+OIL combining DAML (Darpa Agent Markup Language) and OIL
(Ontology Inference Layer) (W3C) Note December 2001
• OWL Web Ontology Language (W3C) Recommendation February 2004
• WS-DistributedManagement Web Services Distributed Management Framework
with MUWS and MOWS below (OASIS)
• WSDM-MUWS Web Services Distributed Management: Management Using Web
Services (OASIS) V0.5 Committee Draft April 2004
• WSDM-MOWS Web Services Distributed Management: Management of Web
Services (OASIS) V0.5 Committee Draft April 2004
• WS-MetadataExchange Web Services Metadata Exchange (BEA,IBM,
Microsoft, SAP) March 2004
• WS-RF Web Services Resource Framework including WS-ResourceProperties,
WS-ResourceLifetime, WS-RenewableReferences, WS-ServiceGroup, and
WS-BaseFaults (OASIS) Oasis TC set up April 2004 and V1.1 Framework March
2004
• ASAP Asynchronous Service Access Protocol (OASIS) with V1.0 working draft G
June 2004
• WS-GAF Web Service Grid Application Framework (Arjuna, Newcastle
University) August 2003
A List of Web Services VI
• g) General Service Characteristics
• WS-Policy Web Services Policy Framework (BEA, IBM,
Microsoft, SAP) May 2003
• WS-PolicyAssertions Web Services Policy Assertions
Language (BEA, IBM, Microsoft, SAP) May 2003
• WS-Agreement Web Services Agreement Specification
(GGF under development) May 2004
• i) User Interfaces
• WSRP Web Services for Remote Portlets (OASIS)
OASIS Standard August 2003
• JSR168: JSR-000168 Portlet Specification for Java
binding (Java Community Process) October 2003
WS-I Interoperability
• Critical underpinning of Grids and Web Services is the
gradually growing set of specifications in the Web Service
Interoperability Profiles
• Web Services Interoperability (WS-I) Interoperability
Profile 1.0a." http://www.ws-i.org. gives us XSD,
WSDL1.1, SOAP1.1, UDDI in basic profile and parts of
WS-Security in their first security profile.
• We imagine the “60 Specifications” being checked out and
evolved in the cauldron of the real world and occasionally
best practice identifies a new specification to be added to
WS-I which gradually increases in scope
– Note only 4.5 out of 60 specifications have “made it” in this
definition
Web Services Grids and WS-I+
• WS-I Interoperability doesn’t cover all the capabilities need to
support Grids
• WS-I+ is designed to minimal extension of WS-I to support
“most current” Grids: it adds support for
– Enhanced SOAP Addressing (WS-Addressing)
– Fault tolerant (reliable) messaging
– Workflow as in IBM-Microsoft standard BPEL
• Security and Notification best practice and support will probably
get added soon
– There are Web Service frameworks here but various IBM v Microsoft v
Globus differences to be resolved
• Portlet-based User Interfaces could be added
• UK OMII Open Middleware Infrastructure Institute is adopting
this approach to support UK e-Science program
– Currently UK e-Science largely either uses GT2 (as in EDG) or Simple
Web Services for “database Grids”
– http://www.omii.ac.uk/
Importance of SOAP
• SOAP defines a very obvious message structure
with a header and a body
• The header contains information used by the
“Internet operating system”
– Destination, Source, Routing, Context, Sequence
Number …
• The message body is only used by the application
and will never be looked at by “operating system”
except to encrypt, compress it etc.
• Much discussion in field revolves around what is in
header!
– e.g. WSRF adds a lot to header
Application Specific Grids
Generally Useful Services and Grids
Workflow WSFL/BPEL
Service Management (“Context etc.”)
Service Discovery (UDDI) / Information
Service Internet Transport  Protocol
Service Interfaces WSDL
Base Hosting Environment
Protocol HTTP FTP DNS …
Presentation XDR …
Session SSH …
Transport TCP UDP …
Network IP …
Data Link / Physical
Higher
Level
Services
Service
Context
Service
Internet
Bit level
Internet
(OSI
Stack)
Layered Architecture for Web Services and Grids
Working up from the Bottom





We have the classic (CISCO, Juniper ….) Internet routing the
flood of ordinary packets in OSI stack architecture
Web Services build the “Service Internet” or IOI (Internet on
Internet) with
• Routing via WS-Addressing not IP header
• Fault Tolerance (WS-RM not TCP)
• Security (WS-Security/SecureConversation not IPSec/SSL)
• Information Services (UDDI/WS-Context not
DNS/Configuration files)
• At message/web service level and not packet/IP address level
Software-based Service Internet possible as computers “fast”
Familiar from Peer-to-peer networks and built as a software
overlay network defining Grid (analogy is VPN)
SOAP Header contains all information needed for the “Service
Internet” (Grid Operating System) with SOAP Body containing
information for Grid application service
Consequences of Rule of the Millisecond
• Useful to remember critical time scales
– 1) 0.000001 ms
– CPU does a calculation
– 2a) 0.001 to 0.01 ms – Parallel Computing MPI latency
– 2b) 0.001 to 0.01 ms – Overhead of a Method Call
– 3) 1 ms
– wake-up a thread or process
– 4) 10 to 1000 ms – Internet delay
• 2a), 4) implies geographically distributed metacomputing
can’t in general compete with parallel systems
• 3) << 4) implies a software overlay network is possible
without significant overhead
– We need to explain why it adds value of course!
• 2b) versus 3) and 4) describes regions where method and
message based programming paradigms important
Linking Modules
Closely coupled Java/Python …
Module
B
Module
A
Method Calls
.001 to 1 millisecond

Coarse Grain Service Model
Service
B
Messages
Service
A
0.1 to 1000 millisecond latency
From method based to RPC to message based to event-based
publish-subscribe Message Oriented Middleware
“Listener”
Subscribe
to Events
Service B
Publisher
Post Events
Message Queue in the Sky
Service A
What is a Simple Service?
• Take any system – it has multiple functionalities
– We can implement each functionality as an independent distributed
service
– Or we can bundle multiple functionalities in a single service
• Whether functionality is an independent service or one of many
method calls into a “glob of software”, we can always make them as
Web services by converting interface to WSDL
• Simple services are gotten by taking functionalities and making as
small as possible subject to “rule of millisecond”
– Distributed services incur messaging overhead of one (local) to
100’s (far apart) of milliseconds to use message rather than method
call
– Use scripting or compiled integration of functionalities ONLY
when require <1 millisecond interaction latency
• Apache web site has many projects that are multiple functionalities
presented as (Java) globs and NOT (Java) Simple Services
– Makes it hard to integrate sharing common security, user profile,
file access .. services
IOI and CIE
• Let us study the two layers IOI (Service Internet On the Bit Internet)
and CIE (Context and Information Environment)
• IOI is most “straightforward” as it is providing reasonably well
understood capabilities at a new “level”
• CIE is roughly the inter-service “shared memory” used to manage and
control them at “distributed operating system level
– Critical is “shared” (a database service) versus message based CIE
Application Specific Grids
Generally Useful Services and Grids
Workflow WSFL/BPEL
Service Management (“Context etc.”)
Service Discovery (UDDI) / Information
Service Internet Transport  Protocol
Service Interfaces WSDL
Higher
Level
Services
CIE
IOI
NaradaBrokering
Audio/Video
Conferencing Client
Computer
Modem
Minicomputer
Server
Web Service B
Peers
NaradaBrokering Broker
Network
Firewall
Queues
Stream
Server-enhanced
Messaging
Workstation
Laptop computer
Peers
PDA
Audio/Video
Conferencing Client
NB supports messages
and streams
Current NaradaBrokering Features
Multiple transport support
In publish-subscribe
Paradigm with different
Protocols on each link
Transport protocols supported include TCP, Parallel TCP streams,
UDP, Multicast, SSL, HTTP and HTTPS.
Communications through authenticating proxies/firewalls &
NATs. Network QoS based Routing
Subscription Formats
Subscription can be Strings, Integers, XPath queries, Regular
Expressions, SQL and tag=value pairs.
Reliable delivery
Robust and exactly-once delivery of messages in presence of
failures
Ordered delivery
Producer Order and Total Order over a message type
Time Ordered delivery using Grid-wide NTP based absolute time
Recovery and Replay
Recovery from failures and disconnects.
Replay of events/messages at any time.
Security
Message-level WS-Security compatible security
Message Payload options
Compression and Decompression of payloads
Fragmentation a nd Coalescing of payloads
Messaging Related
Compliance
Java Message Service (JMS) 1.0.2b compliant
Support for routing P2P JXTA interactions.
Grid Application Support
NaradaBrokering enhanced Grid-FTP. Bridge to the Globus TK3.
Web Service reliability
Prototype implementation of WS-ReliableMessaging
Virtualizing Communication



Communication specified in terms of user goal and Quality of
Service – not in choice of port number and protocol
Bit Internet Protocols have become overloaded e.g. MUST use
UDP for A/V latency requirements but CAN’t use UDP as
firewall will not support ………
A given “Service Internet” communication can involve multiple
transport protocols and multiple destinations – the latter
possibly determined dynamically
NB Brokers
A
Satellite
UDP
Firewall
HTTP
Software Multicast
NB Broker
Client Filtering
Fast
Link
B1
Hand-Held
Protocol
Dial-up
Filter
B2
B3
Performance Monitoring


Every broker incorporates a Monitoring service that
monitors links originating from the node.
Every link measures and exposes a set of metrics
• Average delays, jitters, loss rates, throughput.



Individual links can disable measurements for
individual or the entire set of metrics.
Measurement
Broker
Broker
Monitoring
intervals can
Node
Node
Service
also be varied
Link
Link
Monitoring Service,
Data
Data
returns measured
Aggregates info
metrics to
Control Message
from nodes in a
Exchange
Performance
certain domain
Aggregator.
Performance Aggregation
Service
Transit Delay (Milliseconds)
Mean transit delay for message samples in
NaradaBrokering: Different communication hops
9
8
7
6
5
4
3
2
1
0
hop-2
hop-3
hop-5
hop-7
100
1000
Message Payload Size (Bytes)
Pentium-3, 1GHz,
256 MB RAM
100 Mbps LAN
JRE 1.3 Linux
Standard Deviation for message samples in NaradaBrokering
Different communication hops - Internal Machines
0.8
hop-2
hop-3
hop-5
hop-7
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
1000
1500
2000
2500
3000
3500
Message Payload Size
(Bytes)
4000
4500
5000
NaradaBrokering Service Integration
Proxy Messaging
Handler Messaging
S?
Service
S1
P1
S1
P2
S2
S2
P?
Proxy
Transport Controlled by Overlay Network
Standard Transport over conventional Internet
Internal to Service: SOAP Handlers/Extensions/Plug-ins Java (JAXRPC) .NET Indigo and special cases: PDA's gSOAP, Axis C++
Custom Message Reliability
2 second PDA reply latency!
Different endpoints may
well need different
reliability schemes.
Another reason to use
application layer.
Need to define easy to
use “standard reliability
profiles
Wireless
Optimized
WS-RM
Filter 2
Narada
Broker
WS-Reliability
Filter 1
WS-RM
Fast Web Service Communication I
• IOI Application level Internet allows one to optimize
message streams at the cost of “startup time”, Web Services
can deliver the fastest possible interconnections with or
without reliable messaging
• Typical results from Grossman (UIC) comparing Slow
SOAP over TCP with binary and UDP transport (latter gains
a factor of 1000)
Record
Count
SOAP/XML
Pure SOAP
WS-DMX/ASCII
SOAP
over UDP
WS-DMX/Binary
Binary
over UDP
MB
µ
σ/µ
MB
µ
σ/µ
MB
µ
σ/µ
10000
50000
150000
375000
1000000
5000000
0.93
4.65
13.9
34.9
93
465
2.04
8.21
26.4
75.4
278
7020
7020
6.45%
1.57%
0.30%
0.25%
0.11%
2.23%
0.5
2.4
7.2
18
48
242
1.47
1.79
2.09
3.08
3.88
8.45
0.61%
0.50%
0.62%
0.29%
1.73%
6.92%
0.28
1.4
4.2
10.5
28
140
1.45
1.63
1.94
2.11
3.32
5.60
5.60
0.38%
0.27%
0.85%
1.11%
0.25%
8.12%
Fast Web Service Communication II
• Mechanism only works for streams – sets of related
messages
• SOAP header in streams is constant except for
sequence number (Message ID), time-stamp ..
• One needs two types of new Web Service
Specification
• “WS-StreamNegotiation” to define how one can use
WS-Policy to send messages at start of a stream to
define the methodology for treating remaining
messages in stream
• “WS-FlexibleRepresentation” to define new
encodings of messages
Fast Web Service Communication III
• Then use “WS-StreamNegotiation” to negotiate stream in
Tortoise SOAP – ASCII XML over HTTP and TCP –
– Deposit basic SOAP header through connection – it is part of
context for stream (linking of 2 services)
– Agree on firewall penetration, reliability mechanism, binary
representation and fast transport protocol
– Naturally transport UDP plus WS-RM
• Use “WS-FlexibleRepresentation” to define encoding of a Fast
transport (On a different port) with messages just having
“FlexibleRepresentationContextToken”, Sequence Number,
Time stamp if needed
– RTP packets have essentially this structure
– Could add stream termination status
• Can monitor and control with original negotiation stream
• Can generate different streams optimized for different end-points
CIE: Common Service Information and Metadata
• Consider a collection of services working together
– Workflow tells you how to specify service interaction but more
basically there is shared information or context
specifying/controlling collection
• WS-RF and WS-GAF have different approaches to contextualization –
supplying a common “context” which at its simplest is a token to
represent state
• More generally core shared information includes dynamic service
metadata and the equivalent of configuration information.
• One can supports such a common context either as pool of messages
or as message-based access to a “database” (Context Service)
• Two services linked by a stream are perhaps simplest example of a
collection of services needing context
• Note that there is a tension between storing metadata in messages and
services.
– This is shared versus distributed memory debate in parallel
computing
Stateful Interactions
• There are (at least) four approaches to specifying
state
– OGSI use factories to generate separate services for each
session in standard distributed object fashion
– Globus GT-4 and WSRF use metadata of a resource to
identify state associated with particular session
– WS-GAF uses WS-Context to provide abstract context
defining state. Has strength and weakness that reveals less
about nature of session
– WS-I+ “Pure Web Service” leaves state specification the
application – e.g. put a context in the SOAP body
• I think we should smile and write a great metadata
service hiding all these different models for state and
metadata
Notification Architecture
• Point-to-Point
Service B
Publish
Subscribe
Service A
• Or Brokered
Subscribe
Service B
Broker
Queues Messages
Supports creation
and subscription of topics
Publish
Service A
• NaradaBrokering will support both WS-Eventing
and WS-Notification as well as Java Message
Service JMS that is Java Notification standard
Collaboration and Web Services

Collaboration has
a) Mechanism to set up members (people, devices) of a
“collaborative sessions”
b) Shared generic tools such as text chat, white boards, audiovideo conferencing
c) Shared applications such as Web Pages, PowerPoint,
Visualization, maps, (medical) instruments ….

b) and c) are “just shared objects” where objects
could be Web Services but rarely are at moment
•

We can port objects to Web Services and build a general
approach for making Web services collaborative
a) is a “Service” which is set up in many different
ways (H323 SIP JXTA are standards supported by
multiple implementations) – we should make it a WS
Shared Event Collaboration





All collaboration is about sharing events defining state changes
• Audio/Video conferencing shares events specifying in
compressed form audio or video
• Shared display shares events corresponding to change in
pixels of a frame buffer
• Instant Messengers share updates to text message streams
• Microsoft events for shared PowerPoint (file replicated
between clients) as in Access Grid
Finite State Change NOT Finite State Machine architecture
Using Web services allows one to expose update events of all
kinds as message streams
Need publish/subscribe approach to share messages (NB) plus
System to control “session” – who is collaborating and rules
• XGSP is XML protocol for controlling collaboration building
on H323 and SIP
XGSP Web Service MCU Architecture
Use Multiple Media servers to scale to many codecs and many
versions of audio/video mixing
Session Server
XGSP-based Control
NaradaBrokering
All Messaging
NB Scales as
distributed
Admire
Web
Services
SIP
H323
Media Servers
Filters
High Performance (RTP)
and XML/SOAP and ..
Access Grid
Gateways convert to uniform XGSP Messaging
NaradaBrokering
Native XGSP
Global-MMCS Community Grid




We are building an open source protocol independent Web
Service “MCU” which will scale to an arbitrary number of users
and provide integrated thousands of simultaneous users
collaboration services.
The function of A/V media server is distributed using
NaradaBrokering architecture.
• Media Servers mix and convert A/V streams
Open XGSP MCU based on the following open source projects
• openh323 is basis of H323 Gateway
• NIST SIP stack is basis of SIP Gateway
• NaradaBrokering is open source messaging
• Java Media Framework basis of Media Servers
• Helix Community http://www.helixcommunity.org for Real
Media
http://www.globalmmcs.org open source “non advertised”
release
Break up into Web Services

Monolithic MCU becomes many different “Simple Services”
•
•
•
•
•
•
•
•



Session Control
Thumbnail “image” grabber
Audio Mixer
Video Mixer
Codec Conversion
Helix Real Streaming
PDA Conversion
H323/SIP Gateways
As independent can replicate particular services as needed
• Codec conversion might require 20 services for 20 streams
spread over 5 machines
1000 simultaneous users could require:
• 1 session controller, 1 audio mixer, 10 video mixers, 20 codec
converters, 2 PDA converters and 20 NaradaBrokers
Support with a stream optimized Grid Farm in the sky
• Future billion way “Video over IP” serving 3G Phones and home media
centers/TV’s could require a lot of computing
GlobalMMCS and NaradaBrokering








All communication – both control and “binary” codecs are
handled by NaradaBrokering
Control uses SOAP and codecs use RTP transport
Each stream is regarded as a “topic” for NB
Each RTP packet from this stream is regarded as an “event” for
this topic
Can use replay and persistency support in NB to support
archiving and late clients
Can build customized stream management to administer replay,
and who gets what stream in what codec
NaradaBrokering supports unicast and multicast
Use firewall penetration and network monitoring services in NB
to improve Q0S
Average delays per packet for 50 video-clients
NaradaBrokering Avg=2.23 ms, JMF Avg=3.08 ms
60
NaradaBrokering-RTP
JMF-RTP
Delay (Milliseconds)
50
40
30
20
10
0
0
200 400 600 800 1000 1200 1400 1600 1800 2000
Packet Number
Average jitter (std. dev) for 50 video clients.
NaradaBrokering Avg=0.95 ms, JMF Avg=1.10 ms
8
NaradaBrokering-RTP
JMF-RTP
Jitter (Milliseconds)
7
6
5
4
3
2
1
0
0
200 400 600 800 1000 1200 1400 1600 1800 2000
Packet Number
Polycom, Access Grid
and RealVideo views of
video-mixed streams
using GlobalMMCS
Integration of PDA, Cell phone and Desktop Grid Access
NB Support for optimized
PDA Communication
Grids and e-globalcommunity

Peer-to-peer networks already are a good example of
value of Information Technology supporting broad
global communities
• File sharing, text chats, bulletin boards



Grids must include these capabilities and extend in
terms of increased functionality and quality of service
This will support business and cultural interactions
between nations
Several interesting applications can be supported by
• Replacing files by multi-media streams so can collaborate in
real-time
• Adding traditional tools like audio-video conferencing and
shared applications to P2P set

This integration of P2P and Grid to give M2 Grids
impacts e-Business as well as e-globalcommunity
Streaming







2
M
Grids
e-Textilemanufacturing involves Clothes designers in USA and
manufacturers in Hong Kong exchanging designs which are
streams of images
e-Sports is a possible collaboration between Indiana University and
Beijing Sport University
• Basket ball coaches (teacher) interact with aspiring NBA players
in China
• Martial Arts masters in China train neophytes in Indiana
• Faculty recreational sports adviser works from university with
faculty exercising at home
• Hope to have working incredibly well by the 2008 Olympics
Interactive TV Grid: allows anybody to discuss professional or
home video (of sports or other events) within a custom Grid
Multi-player distributed games which should be supported with
exactly the same overlay Grid
Video Game Production Grid links artistic direction (design) in one
country with digital animation (manufacturing) in another
e-Science: Physics and Environmental Science Sensors
Surveillance Grid enables security personnel to annotate and
discuss suspicious remote camera images/streams
Some Technology for Streaming M2 Grids





Basic capability is collaborative annotatable multimedia
tool for images, sensors and real-time video streams
• Allow Grid participants to view real-time streams,
rewind on the fly and add text and graphical
comments
• Similar to instant replay on TV but far more flexible
Need rich metadata system to label and correlate streams,
images and annotations
Extend Grid and P2P file access paradigms to stream
storage, browsing and access
Core Technologies shared with distance education
Using http://www.globalmmcs.org for multimedia
services and http://www.naradabrokering.org for overlay
network
GlobalMMCS Futures I
• 1. New Collaboration tools
–
–
–
–
SVG game ( stable )
Whiteboard ( stable )
e-Sport ( prototype)
Jabber IM client ( prototype)
• 2. JMF Audio/Video client ( stable)
–
–
–
–
–
performance enhancement finished
new codec ( MPEG4 finished; try H.264)
support different platform ( Linux, Mac )
support NAT/firewall
screen codec for shared display ( prototype )
• 3. Replay & Archive (prototype)
– Replay Engine based on NaradaBroker Storage Service
– XGSP-RTSP gateway
– Extend RTSP and NaradaBrokering for Instant Replay
• 4. Web Server ( stable)
– Standard calendar service ( iCalendar, vCalendar)
– Flexible conference management
GlobalMMCS Futures II
• 5. Conferencing Media Processing Service ( Stable)
– Fix the bugs and add scheduling
– Support new codec ( MPEG4 )
• 6. H.323 Gateway ( Stable)
– Import it to Linux platform
• 7. RealStreaming Gateway ( Stable )
– Import it to Linux
– Support Mobile clients
• 8. Global-MMCS deployment & test
– Test under the setting of multiple NaradaBroker and
NAT/Firewall
– support deployment for AFRL, NASA, DOE portals
– test with remote sites
• 9. PDA Clients (prototype)
P2P and Server based solutions






Peer-to-peer architectures have advantage that they can be deployed
just using client resources and no system commitment is needed
Typically clients do not have good network QoS and it is hard for
example to support rich multi-point audio video conferencing in this
way
M2 Grids typically require multicast so average load in P2P case on
client legs goes like O(M)
Grid
Farm
in theload
Sky on
(clouds)
• Server-side multicast
puts
O(M)
backbone and O(1) load
on clients and can lead to much better scaling and performance
• N plus N Grids may not see such large improvements with server
Grid Servers
side support
So Grids should support initial P2P deployment with a seamless
upgrade to add better QoS using Servers.
Extend familiar P2P paradigms like BitTorrent to Grids and
Streaming
P2P
Grid and peer-to-peer linkage combines scalable performance with
ease of deployment