Download Part II Middleware: Transport Protocols

Introduction to Middleware
The protocols used for conference media
streams
The protocols to setup and control various
aspects of a conference, such as
membership, session information, media
activity, floor control and so on.
Introduction to Middleware
Here middleware is the glue software try
to combine the multimedia system together,
normally it includes
Transport Protocols
Session Directories, Advertisement and
Invitation Protocols;
Conference Control Specifications.
Transport Protocols
Difference between TCP and UDP at the
viewpoint of multimedia
Multimedia transport protocols vs.
conventional data services
A basic framing service is needed
Multiplexing may be needed
Timely delivery and trade off against
reliability
Synchronization
Transport Protocols
RTP, the preferred carrier/framing protocol
for multimedia traffic, is carried on top of
UDP. It has the potential for further
multiplexing information, since it carries
source identifiers.
RTCP, the Real Time Control Protocol, is
used to convey additional information
such as participant details and statistics of
packet arrivals and loss.
Roadmap
The transport protocol for multimedia
provides some functions that are different
than those for elastic reliable data transfer
RTP provides payload identification,
multiplexing, feedback from receivers to
senders of reception conditions and hooks
for synchronization. This chapter will also
talk about transportation reliability of
multimedia.
TCP Adaption Algorithms
 TCP will smoothly degrade performance of
everyone as load increases(rather than blocking
access). This is done by adaption algorithms.
 Adaption in TCP is both to RTT and send rate.
 Based on the idea of adaption, multimedia
protocols adapt to delay at the receiver to:
Alter adaptive playout buffer to smooth playback
Synchronize streams from different sources at a
receiver
TCP Adaption Algorithms
 Algorithm named Exponential Weighted Moving Average:
Assume we measure the arrival time for each (i'th)
packet as IATi, then the simple average would be:
Av(IATi) = [ sumi=1 to n ( IATi) ]/n. But since the
average is not fixed, the most recent measurements are
more confidential, so use:
Av(IATi) = IATi*alpha +(1- alpha)*Av(IATi-1), where
1> alpha > 0.5
 Once have a mean IAT, required playback buffer can be
calculated( roughly twice of the variation of IAT)
TCP Adaption Algorithms
 Synchronization between two sites by exchange
of packets
send a packet from s to d with source time in t1, its
arrives at d when clock of d reads t2
send the packet back to s with t1, t2 and t3(the time
at d when sending)
s gets the packet at the time when its clock is at t4
if s and d have clocks that differ by offset, and
network delay is delay
t4=t3+offset+delay, t2=t1-offset+delay
so: delay=(t4+t2-t3-t1)/2, offset=(t4+t1-t2-t3)/2
Playout Algorithms
 For a variety of reasons, samples may arrive at
different times than they were sent. The
variation of arrival time is a second order effect
compared with the delivery delay. This can be
coped with playout buffer with the size of the
variation of the mean of the arrival time.
Audio, video and other interactive application
receivers generally use an adaptive playout scheme.
Senders are generally adaptive to reported network
conditions. The report should be constrained to be a
constant percentage of the total session.
MPEG Systems
 MPEG systems is the control part of the MPEG
standard. It addresses the combining of one or
more streams of video and audio as well as
other data into a single or multiple streams
which are suitable for storage or transmission.
The stream is Packetised Elementary
Stream(PES).
Transport and Program
Streams
 Transport Stream(TS), can carry multiple
programs simultaneously, and is optimized for
use in applications where data loss may be
likely.
 Program Stream(PS), is optimized for
multimedia applications, for performing systems
processing in software and for MPEG-1
compatibility.
Transport and Program
Streams
 Synchronization
The basic principle of MPEG system coding is the use
of time stamps provided by RTP, which specify the
decoding and display time of audio/video and the
time of reception of the multiplexed coded data at
the decoder.
This method allows a great deal of flexibility in such
areas as decoder design, the number of streams,
multiplex packet lengths, video/audio frame rates
digital storage medium or network performance.
RTP
 RTP is the Internet Standard for conveying
media streams between interactive participants.
RTP Packet Format
RTP Header Compression
RTP Multiplexing
RTCP Packet Format
Payloads
RTCP Scaling Properties and Timer Considerations
RTP Packet Format
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC
|M|
PT
|
sequence number
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
timestamp
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
synchronization source(SSRC) identifier
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
contributing source(CSRC) identifiers
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The first twelve octets are present in every RTP packet, while
the list of CSRC identifiers is present only when inserted by
a mixer.
RTP Header Compression
 The combination of the IP,UDP and RTP control
information adds up to a significant overhead
for small media samples. IP(20 bytes)+UDP(8
bytes)+RTP(12 bytes)=40 bytes
 A technique for reducing the overhead of
packets, designed for compressing TCP/IP
headers specified in RFC 1144 by Jacobson, was
adapted by Casner and Jacobson to RTP
headers, shows a typical reduction to a header
size of 3-4 bytes for all control overhead.
RTP Header Compression
 The technique consists two parts:
Noting fields in the packet headers that do not
change over the life of a flow. These are the second
order information which can stored in the
compression state
Noting that there are few flows at the edge of the
network, so such information can be conveyed over
the first hop by a single packet, and subsequently
referred to by a short “connection identifier, which
serves to index the full state so that the first op
router can reconstruct the full packet.
RTP Header Compression
 The features of the header compression:
The compression for RTP header is basically the same
as TCP/IP except in exceptional circumstances
The compression state is a soft state. It can be
recovered if lost since the packet conveys enough
implicit information that end-to-end checksums are
still computed, and hop-wise recomputed from the
state vector and from the remaining data in the
compressed header. That is to say, when the router
resets/changes or the network alters states, the
invalid checksum causes a reset of the compressed
state so that a new packet is created again.
RTP Multiplexing
 There are a number of circumstances that
multiple media flows within a single RTP data
payload between two points is needed. There
are at least two ways to multiplex data:
Assuming all samples have the same payload types,
so just use a mapping table to save offsets for each
end-to-end flow.
A more generic method suggested by Mark Handley
is to adapt the ideas from previous section about RTP
header compression and to allow for multiple
compressed headers within a single RTP packet, one
for each of the samples.
RTCP Packet Format
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P| RC
|
PT=SR=200
|
length
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
SSRC of sender
|
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
|
NTP timestamp, most significant Word (sender info ) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
NTP timestap, lest significant word (sender info ) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
RTP timestamp
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
sender’s packet count
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
sender’s octet count
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
RTCP Packet Format
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
|
SSRC_1
(SSRC of first source )
(report block 1 ) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| fraction lost |
cumulative number of packets lost
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
extended highest sequence number received
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
interarrival jitter
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
last SR(LSR)
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
sender’s packet count
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
delay since last SR(DLSR)
|
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
|
SSRC_2 (SSRC of second source) ... (report block 2 )
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Payloads
 There are quite a few payloads defined for RTP,
ranging from owens designed to carry
video/audio formerly available on hardware
CODECs only, such as MPEG, H.261, and the
various CCITT/ITU voice coding schemes,
through to some more generic payloads
designed to carry redundantly coded audio such
as that used by INRIA’s Freephone.
RTCP Scaling Properties &
Timer Considerations
 The intention and original design goal of RTCP
messages was to act as a distributed source of
lightweight session data that would allow a
range of highly fault tolertant, and seasonable
scale mechanisms to be built including:
Membership RTCP reports carry the identifier of
participants.
Loss statistics RTCP reports convey information
about packet losses.
RTCP Scaling Properties &
Timer Considerations
 RTCP reports are designed to be sent
periodically, with a frequency inversely
proportional to the number of members.
Though this was set to constrain the bandwidth
consumed by RTCP to be a small fixed fraction
of the total capacity of the session, there are
three circumstances where this approach
creates a problem:
A flood of initial packets of reports at the start
The flood of “BYE” packet at the end of session
The reports arrives at all participants
Synchronization
 Some kinds of synchronization:
Intra-stream Synchronization inside a stream, for
media like video/audio, whiteboard inside a stream
Inter-stream To see two streams’ reactions to the
occurrence of a third stream
inter-media Just lip-synch
Intra-stream Synch
 Intra-stream synchronization is a base part of
the H.261 and MPEG coding systems. H.221 and
MPEG systems specify an encapsulation of
multiple streams as well as carrying timing
information in the stream.
 In the Internet, the RTP media specific
timestamp provides a general way of carrying
out the same above function.
Inter-stream Synch
 The easiest way of synchronizing between
streams at different sites is based on providing a
globally synchronized clock.
Use a single clock which is propagated around the
network
Use a clock synchronization protocol, such as
NTP(Network Time Protocol), DTS(Digital Time
Service). The network continually exchanges
messages about clock offsets and network delays.
 Or use timestamps to adjust clocks and measure
delays accordingly, and then insert a baseline
delay into the adaptive playout algorithms.
Inter-media Synch
 Use options like Inter-stream Synch approaches
 More generally, encapsulate the media in the
same transmission stream. It is effective but
may cause expensive computation at the receive
side to decode all media at the same time.
 Alternatively, use much the same scheme as to
synchronize different sources from different
places. This approach uses timestamps,
assuming the media are timestamped accurately
at the “real” source points which the receiver
can use to synchronize multiple media.
Reliable Multicast Transport
 “Reliable Multicast” is not people normally think
to be a single protocol at a single ‘layer’ of a
protocol stack. The differences are possibly:
Fate sharing
Performance
Semantics
 In this section we will also talk about congestion
avoidance for reliable multicast applications and
reliable multicast framework.
Fate Sharing
 Fate sharing in unicast applications means that
if two applications has set up connection, then
TCP can keep the connection as long as they
like. However, if either party fails, the
connection certainly fails.
 Fate sharing between multicast endpoints is a
more subtle idea. It is a application-oriented
question. Normally we must permit any recipient
timeouts/Fails.
Performance
 The performance measurement is different for a
talks to b, from a talks to b,c,d,e,f.
The previous situation is limited by 1 path, what can
be done to improve the throughput(or delay bound)
is done by IP.
For the second situation, the throughput or delay
may not be that sustainable even in the slowest
transferring state.
Semantics
 Applications such as software distribution are
cited as classic 1-n service, telemetry is given as
a n-1 reliable protocol, shared whiteboards are
cited as examples of n-m.
 1-n and n-1 services are those where reliability
can be dealt with by rounding up the missing
bits at the end of the transfer.
 N-m processes need timely recovery. For each
recipient, the timely order is very important.
Congestion Avoidance for
Reliable Multicast
 One of the big research areas not quite
complete is how to achieve flow and congestion
control for reliable multicast applications.
 There are a lot options when confronted with a
congested link and sending to a group. The best
option depends on the nature of the application.
The options can be viewed as sending at the
slowest,average or fastest rate.
 The global congestion picture is not at all clear
for either reliable or unreliable multicast.
Reliable Multicast Framework
 Handley proposed a framework about reliable
multicast protocol. He described the categories
of parameters of the system:
Number of source: 1-to-n, n-to-m or other.
Transmitting start time: Is there such a time? Is there
one for each receiver marked at join time?
Real time-Consideration: About the latency
constraints on delivery of data.
Consistency.
 He listed congestion control and ordering as
non-factors.
Reliable Multicast Framework
 He grouped applications into five main areas:
Bulk Transfer(1-n,relaxed real time, synchronized
start time, object level consistency)
Live data feeds(1 source, no start time, quasi real
time, may need synchronized consistency)
Resilient(1 source, no start time, nearly real time,
non consistency)
Share applications(distributed data set, relaxed
consistency, many changing sources)
Hybrid - some applications maybe a hybrid of all of
the above.
 This will be discussed further in chapter eight.