Download Multimedia Networking Quality of Service

Multimedia Networking Quality of
Services
Hongli Luo, IPFW
Topics
 Multimedia networking applications
 Stored, live, interactive multimedia applications
 How should Internet better support QoS
 Internet multimedia streaming
 Web server or streaming server
 TCP or UDP
 Packet loss, delay and jitter
 How to alleviate it
Multimedia and Quality of Service: What is it?
multimedia applications:
network audio and video
(“continuous media”)
QoS
network provides
application with level of
performance needed for
application to function.
MM Networking Applications
Classes of MM applications:
1) stored streaming
2) live streaming
3) interactive, real-time
Fundamental characteristics:
 typically delay sensitive


end-to-end delay
delay jitter
 loss tolerant: infrequent
losses cause minor
glitches
 Elastic applications

Jitter is the variability
of packet delays within
the same packet stream

Web, e-mail, FTP, telnet
loss intolerant but delay
tolerant.
Streaming Stored Multimedia
Stored streaming:
 Applications: CNN, Microsoft Video, YouTube
 media stored at source
 transmitted to client
streaming: client playout begins before all data has arrived
 timing constraint for still-to-be transmitted data: in time for
playout
 Continuous playout
 Streaming multimedia clients: RealPlayer, QuickTime,
Windows Media
Streaming Stored Multimedia:
What is it?
1. video
recorded
2. video
sent
network
delay
3. video received,
played out at client
streaming: at this time, client
playing out early part of video,
while server still sending later
part of video
time
Streaming Stored Multimedia: Interactivity

VCR-like functionality: client can
pause, rewind, FF, push slider bar
 10 sec initial delay OK
 1-2 sec until command effect OK
 timing constraint for still-to-be transmitted
data: in time for playout
Streaming Live Multimedia
Examples:
 Internet radio talk show
 live sporting event
 IPTV
Streaming (as with streaming stored multimedia)
 playback buffer
 playback can lag tens of seconds after transmission
 still have timing constraint
Interactivity
 fast forward impossible
 rewind, pause possible!
Streaming Live Multimedia
 Live, broadcast like applications often have many
clients who are receiving the same audio/video
program
 How to efficiently deliver the contents





Multiple separate server-to-client unicast streams
IP multicast
Application-layer multicast – multicast overlay networks
Content Distribution Networks (CDNs)
Peer-to-peer networks
Real-Time Interactive Multimedia
 applications: IP telephony, video
conference, distributed
interactive worlds
 end-end delay requirements:
 audio: < 150 msec good, < 400 msec OK
• includes application-level (packetization) and network
delays
• higher delays noticeable, impair interactivity
 session initialization

how does callee advertise its IP address, port
number, encoding algorithms?
Requirements for Data Transport
 Delay
 Some small delay at the beginning is acceptable
 E.g., start-up delays of a few seconds are okay
 Jitter
 Variability of packet delay within the same packet stream
 Client cannot tolerate high variation if the buffer starves
 Loss
 Small amount of missing data does not disrupt playback
 Retransmitting a lost packet might take too long anyway
Multimedia Over Today’s Internet
TCP/UDP/IP: “best-effort service”

no guarantees on delay, loss
?
?
?
?
?
?
But you said multimedia apps requires ?
QoS and level of performance to be
?
? effective!
?
?
Today’s Internet multimedia applications
use application-level techniques to mitigate
(as best possible) effects of delay, loss
Challenges to the Current Internet (1)
 TCP/UDP/IP suite provides best-effort, no guarantees
on expectation or variance of packet delay
 Streaming applications delay of 5 to 10 seconds is
typical and has been acceptable, but performance
deteriorate if links are congested (transoceanic)
 Most router implementations use only First-ComeFirst-Serve (FCFS) packet processing and
transmission scheduling
Challenges to the Current Internet (2)
 To mitigate impact of “best-effort” protocols, we can:
 Use UDP to avoid TCP and its slow-start phase…
 Buffer content at client and delay playback to remedy jitter
 Prefetch data during playback when client storage and extra
bandwidth are available
 Adapt compression level to available bandwidth
 Different treatment of packets at the router
• Different classes of packets
How should the Internet evolve to better support
multimedia?
Integrated services philosophy:
 fundamental changes in Internet so that apps can reserve
end-to-end bandwidth
 Applications receive a guarantee on its end-to-end
performance
 Hard guarantee – received QoS with certainty
 Soft guarantee – received QoS with high probability
 requires new, complex software in hosts & routers




Application makes reservation
Modify scheduling policies at the router
Description of traffic
Admission control
How should the Internet evolve to better support
multimedia?
Laissez-faire
 no major changes
 more bandwidth when needed
 content distribution,

Replicate stored content and put the replicated contents at the
edges of the Internet
 application-layer multicast


Multicast Overlay networks can be deployed
Application layer
Differentiated services philosophy:
 fewer changes to Internet infrastructure, yet provide 1st and 2nd
class service
A few words about audio compression
 analog signal sampled
at constant rate


telephone: 8,000
samples/sec
CD music: 44,100
samples/sec
 each sample quantized,
i.e., rounded

28=256
e.g.,
possible
quantized values
 each quantized value
represented by bits

8 bits for 256 values
 example: 8,000
samples/sec, 256
quantized values -->
64,000 bps
 receiver converts bits
back to analog signal:

some quality reduction
Example rates
 CD: 1.411 Mbps
 MP3: 96, 128, 160 kbps
 Internet telephony: 5.3
kbps and up
A few words about video compression
 video: sequence of
images displayed at
constant rate

e.g. 24 images/sec
 digital image: array of
pixels

each pixel represented by
bits
 redundancy
 spatial (within image)
 temporal (from one image
to next)
Examples:
 MPEG 1 (CD-ROM) 1.5
Mbps
 MPEG2 (DVD) 3-6 Mbps
 MPEG4 (often used in
Internet, < 1 Mbps)
Research:
 layered (scalable) video

adapt layers to available
bandwidth
Streaming Stored Multimedia
 Client-server system
 Server stores the audio and video files
 Clients request files, play them as they download, and
perform VCR-like functions (e.g., rewind and pause)
 Playing data at the right time
 Server divides the data into segments
 labels each segment with timestamp or frame id
 so the client knows when to play the data
 Avoiding starvation at the client
 The data must arrive quickly enough
 otherwise the client cannot keep playing
Streaming Stored Multimedia
application-level streaming
techniques for making the best
out of best effort service:
 client-side buffering
 use of UDP versus TCP
 multiple encodings of
multimedia
Media Player
 jitter removal
 decompression
 error concealment
 graphical user interface
w/ controls for interactivity
Streaming stored multimedia
 Through a Web server
• Deliver the video/audio to the client over HTTP

Through a streaming server
• Using HTTP or some other protocol

HTTP popular – pass through firewall while proprietary
protocols are blocked
Internet multimedia: simplest approach –
through a Web Server
 audio or video stored in file
 files transferred as HTTP object
TCP connection between client and server
 received in entirety at client
 then passed to player
 Client invokes the media player




Content-type indicates the encoding
Browser launches the media player
Media player then renders the file
Internet multimedia: through a Web Server
 User clicks on a hyperlink for an audio/video file
 browser GETs metafile describing the object
 browser launches player, passing metafile
 player sets up its own connection to the Web server
 server streams audio/video to player
Streaming from a streaming server
 Web server returns a meta file describing the object via HTTP
 Player requests the data using a different protocol
 allows for non-HTTP protocol between server, media player
 To control playback, protocol is needed to exchange playback control
information – RTSP protocol
Streaming Multimedia: Client Buffering
How to deliver media from the streaming server to the media player
 Media is sent over UDP at a constant rate equal to the drain rate


As soon as the client receives media, it decompresses it and plays
it back
The media player delays playout for 2 to 5 seconds in order to
eliminate network-induced jitter.
 Media is sent over TCP





Fill rate x(t) fluctuates with time – due to TCP congestion control and
window flow control
X(t) may be less that d for long periods because of congestion
Empty client buffer - starvation/underflow
Introduces undesirable pause during the playback
X(t) very much depends on the size of the client buffer
• Large client buffer – TCP make use of the available bandwidth to
prfetch, less likely starvation
• Small client buffer – more client starvation
Streaming Multimedia: Client Buffering
variable
network
delay
client video
reception
constant bit
rate video
playout at client
buffered
video
constant bit
rate video
transmission
client playout
delay
 client-side buffering, playout delay compensate for
network-added delay, delay jitter
time
Streaming Multimedia: Client Buffering
 client-side buffering,


Store the data as it arrives from the server
Play data for the user in a continuous fashion
 playout delay



Client typically waits a few seconds to start playing
Allow some data to build up in the buffer
compensate for network-added delay, delay jitter
Streaming Multimedia: UDP or TCP?
TCP
 send at maximum possible rate under TCP
 fill rate fluctuates due to TCP congestion control


Slow down after a packet loss,
May cause starvation at the client
 HTTP/TCP passes more easily through firewalls
 TCP’s Reliable delivery not needed for multimedia
 Retransmission of lost packets even though retransmission
may not be useful
 Late packet can be thrown away
 Protocol overhead
 TCP header of 20 bytes in every packet
Streaming Multimedia: UDP or TCP?
UDP
 server sends at rate appropriate for client (oblivious to
network congestion !)
 often send rate = encoding rate = constant rate
 then, fill rate = constant rate - packet loss

No automatic adaptation of sending rate
 short playout delay (2-5 seconds) to remove network jitter
 error recover: time permitting
 No automatic retransmission of lost packets
 Retransmit if packet deadline not past
 Smaller packet header
Streaming Multimedia: UDP or TCP?
UDP
 Do not respond to congestion
 Not sharing bandwidth fairly
 Possible to cause congestion collapse
 Put flow control, congestion control into application
 UDP leaves many things to the application
 When to transmit data
 How to encapsulate the data
 Whether to retransmit lost data
 Whether to adapt the sending rate
 Whether to adapt the quality of the audio/video encoding
Streaming Multimedia: UDP or TCP?
UDP
 Even when using UDP, applications should
respond to congestion end-to-end.
 Need to promote “TCP-friendly” behavior.
Rate-based Adaptation
 adjust rate to avoid congestion
 reduce rate before packet loss or at the detection of
packet loss.
TCP-Friendly
 Throughput of a TCP connection
 1.2P /(


p  RTT )
P: the packet size
p: the lost probability of a packet
 Limit flows to TCP-style BW
 Don’t know RTT exactly
YouTube: HTTP, TCP, and Flash
 Flash videos
 All uploaded videos converted to Flash format
 Nearly every browser has a Flash plug-in
 avoids need for users to install players
 HTTP/TCP
 Implemented in every browser
 Easily gets through most firewalls
 Keep It Simple, Stupid
 Simplicity more important than video quality
Real-time interactive applications
 PC-2-PC phone
Skype
PC-2-phone
 Dialpad, Net2phone, Skype
videoconference with webcams
 Skype, Polycom
Video game
QoS requirements:








Going to now look at
a PC-2-PC Internet
phone example in
detail
Strict delay constraints
Much harder than streaming applications
Receiver can not introduce much playback delay
Interactive Multimedia: Internet Phone
Introduce Internet Phone by way of an example
 speaker’s audio: alternating talk spurts, silent
periods.

64 kbps during talk spurt

pkts generated only during talk spurts

20 msec chunks at 8 Kbytes/sec: 160 bytes data
 application-layer header added to each chunk.
 chunk+header encapsulated into UDP segment.
 application sends UDP segment into socket every 20
msec during talkspurt
Internet Phone: Packet Loss and Delay
 network loss: IP datagram lost due to network
congestion (router buffer overflow)
 delay loss: IP datagram arrives too late for playout at
receiver
 loss tolerance: depending on voice encoding, losses
concealed, packet loss rates between 1% and 10%
can be tolerated.
 Most Internet phone applications that run over UDP
do not retransmit lost packets.
Recovering From Packet Loss
 Loss is defined in a broader sense
 Does a packet arrive in time for playback?
 A packet that arrives late is as good as lost
 Retransmission is not useful if deadline passed
 Selective retransmission
 Sometimes retransmission is acceptable
 E.g., if client has not already started playing data
 Data can be retransmitted within time constraint
 Could do Forward Error Correction (FEC)
 Send redundant info so receiver can reconstruct
Internet Phone: Packet Loss and Delay
 Packet delay
 delays: processing, queueing in network; endsystem (sender, receiver) delays
 For highly interactive audio applications, e.g.,
Internet phone
• End-to-end delays smaller than 150 ms are not
perceived by a human listener
• Delays between 150 and 400 ms acceptable
but not ideal
• typical maximum tolerable delay: 400 ms
Removing Jitter at the Receiver for Audio
 Provide synchronous playout of voice chunks in the
presence of random network jitter
 To alleviate the jitter



Sequence number is added to each chunk
Timestamp – sender stamps each chunk with the time at
which the chunk was generated
Delaying playout
• Playout delay must be long enough so that most of the packets
are received before the scheduled playout times
• Fixed playout delay
• Adaptive playout delay
Internet Phone: Fixed Playout Delay
 receiver attempts to playout each chunk exactly q
msecs after chunk was generated.
 chunk has time stamp t: play out chunk at t+q .
 chunk arrives after t+q: data arrives too late for
playout, data “lost”
 tradeoff in choosing q:
 large q: less packet loss
• When large variations in end-to-end delay

small q: better interactive experience
• Delay and variations in delay are small
• .e.g., < 150 ms
Delay Jitter
variable
network
delay
(jitter)
client
reception
constant bit
rate playout
at client
buffered
data
constant bit
rate
transmission
client playout
delay
time
 consider end-to-end delays of two consecutive
packets: difference can be more or less than 20 msec
(transmission time difference)
Fixed Playout Delay
• sender generates packets every 20 msec during talk spurt.
• first packet received at time r
• first playout schedule: begins at p
• second playout schedule: begins at p’
packets
loss
packets
generated
packets
received
playout schedule
p' - r
playout schedule
p-r
time
r
p
p'
Adaptive Playout Delay
 Goal: minimize playout delay, keeping late loss rate
low
 Approach: adaptive playout delay adjustment:
 estimate network delay, adjust playout delay at
beginning of each talk spurt.
 silent periods compressed and elongated.
 chunks still played out every 20 msec during talk
spurt.
Recovery from packet loss (1)
Forward Error Correction
 playout delay: enough
(FEC): simple scheme
time to receive all n+1
 for every group of n chunks
packets
create redundant chunk by  tradeoff:
exclusive OR-ing n original
 increase n, less
chunks
bandwidth waste
 send out n+1 chunks,
 increase n, longer
increasing bandwidth by
playout delay
factor 1/n.
 increase n, higher
 can reconstruct original n
probability that 2 or
chunks if at most one lost
more chunks will be
chunk from n+1 chunks
lost
Recovery from packet loss (2)
2nd FEC scheme
 “piggyback lower
quality stream”
 send lower resolution
audio stream as
redundant information
 e.g., nominal
stream PCM at 64 kbps
and redundant stream
GSM at 13 kbps.
whenever there is non-consecutive loss,
receiver can conceal the loss.
 can also append (n-1)st and (n-2)nd low-bit rate
chunk

Recovery from packet loss (3)
Interleaving
 chunks divided into smaller
units
 for example, four 5 msec units
per chunk
 packet contains small units
from different chunks
 if packet lost, still have most of
every chunk
 no redundancy overhead, but
increases playout delay