Download Playout Buffering

Adaptive Playout
1
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
You are Here
Encoder
Decoder
Middlebox
Receiver
Sender
Network
2
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
How to recv and play?
open socket
while not done
if socket is readable
read packet from socket
remove RTP header
decode
play back
3
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
What’s Wrong?
 packet ordering
 packet loss
 next packet arrive in-time?
Especially bad for audio applications
4
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Overview
Decode
RTP
Classifier
Network
5
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Packet Buffer
 Sorted by sequence number
 When ADU is complete, send to
decoder
RTP
Classifier
7
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Playout Buffer
 Stored decoded data in playout
order
 Post-processing/Mixing may happen
Decode
8
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Why Buffer?
9
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Sending Packets
Packet
Time
10
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Receiving Packets
Packet
t0
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
t1
Time
11
With Jitter
Packet
Time
12
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
With Jitter
Packet
t0
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
t2
Time
13
What causes Jitter?
 Network delay =
Transmission Delay (fixed) +
Propagation Delay (fixed) +
Queuing Delay (variable)
 Delay jitter is caused by variable
queuing delay
14
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Delay Jitter
Transit
Time
small jitter
large jitter
Time
15
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Spike
Transit
Time
Time
16
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Today’s Question
 How big is the playout buffer?
 When to play back?
17
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Types of Applications
 Non-interactive
 Buffer
can be large
 Interactive
 As
small as possible
18
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Types of Applications
 Video
 Frames
are discrete (easier problem)
 Audio
 Samples
are “continuous”
19
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Naive Answer
 How big is a buffer?
 Fixed
at a small value, to reduce
latency
 When to playback?
 Playback
as soon as possible, to
reduce latency
20
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Client Buffer
Management
Bandwidth Smoothing for Noninteractive Applications
21
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Multi-Threshold Flow Control
R. Zimmermann, K. Fu, M.
Jahangiri, C. Shahabi
MTAP 2006
22
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Video-on-Demand (VoD)
 High-quality playback required
 Buffer can be large
 Encoding may be VBR for high visual
quality
 Playback time may be very long (2+
hours)
23
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Two Approaches (1)
 Server-controlled
 Pre-compute
transmission schedule
 Piece-wise linear approximation:
compute a number of constant-rate
segments
 Different optimization criteria

Minimize: # of rate changes, #
utilization of client buffer, peak rate,
etc.
24
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Two Approaches (2)
 Client-controlled
 Client
adaptively informs server
 Advantages:
No rate history necessary (VBR)
 Can adjust to network conditions
 Works easily with interactive
commands such as FF, FR, Pause

25
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Robust Stream Delivery
 Smoothing of VBR media traffic has
the following quality benefits:
 Better
resource utilization (less
bursty)
 More streams with the same network
capacity
26
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Objectives
 Multi-Threshold Flow Control
(MTFC) algorithm objectives:
 Online
operation
 Content independence
 Minimizing feedback
control signaling
 Rate smoothing
27
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Example
28
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
MTFC Buffer Management
 Multiple Thresholds: goal is middle
of buffer
 Send rate
adjust
command
to server
whenever
threshold
is crossed
29
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
How to Set Thresholds?
 Simple: equi-distant
 m:
number of thresholds
 Overflow and underflow thresholds
30
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
How to Calculate Sending Rate?
31
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Threshold Spacing Strategies
 Linear vs. arithmetic vs. geometric
32
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Threshold Spacing Strategies
 Geometric and arithmetic spacing:
33
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Sending Rate Computation
 Relevant factors:




Target buffer level (i.e., 50%): THN
Current buffer level: bwobsv
Predicted duration to reach target level
Data consumed during predicted duration
34
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
MTFC Results
35
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
MTFC Results: # of Thresholds
36
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
MTFC Results: # of Thresholds
Number of Thresholds
3
Number
Of
Rate
Changes
5
9
17
100
90
80
70
60
50
40
30
20
10
0
8MB
16MB
32MB
37
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
A Brief Introduction to
Audio Conferencing
38
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Audio Conferencing
 Live, interactive application
 Latency
is important
 Typically each packet contains
20-30ms of audio
39
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Silence Suppression
 Silence Detection
 If
no sound, no need to send
 Talkspurt
 Consecutive
audio packets (between
silence)
 Hundreds of ms
40
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Recall: RTP Header
 marker bit:
 depends on payload
 e.g. beginning of frame
41
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
RTP and Talkspurt
 First packet of a talkspurt will have
marker bit set to 1
42
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
RTP and Talkspurt
 Deduce talkspurt from sequence
number and timestamp
1
20
2
40
3
60
5
190
SeqNo
TimeStamp
43
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Consequences of Talkspurt
 Opportunity to adjust playout delay
 if
jitter is large, increase delay
 if jitter is small, decrease delay
44
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Fixed Playout Delay
Talkspurt 2
Talkspurt 1
SEND
RECV
PLAY
Time
45
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Adaptive Playout Delay
SEND
RECV
PLAY
46
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Adaptive Playout Delay
SEND
RECV
PLAY
47
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Trade-Off
Latency vs. Packet Loss
48
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Latency vs Loss-Rate
Loss
Rate
Latency
49
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Latency vs Loss-Rate
Loss
Rate
• Note: Late packets
are considered lost
Latency
50
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Adaptive Playout Mechanisms
for Packetized Audio
Applications in WAN
R. Ramjee, J. Kurose, D. Towsley,
H. Schulzrinne
INFOCOM 1994
51
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Variables and Notations
52
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Variables and Notations
Tnet(i)
Tbuf(i)
Tarrive(i)
Tplay(i)
Tdelay(i)
Tsend(i)
53
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
1st Packet in Talkspurt
We can estimate as
Mean and variation of the end-to-end delay.
54
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
How to estimate Vnet(i)
55
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
How to estimate Enet(i)
 Method 1: Jacobson’s Method
 (Delay estimate as in RFC 793, TCP
re-transmit timer.)
56
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Spike
Tnet
Arrival Time
57
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Problems
 Described algorithm does not react
to spike fast enough.
58
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
How to estimate Enet(i)
 Ramjee’s Method:
SPIKE
NORMAL
59
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Ramjee’s Idea
if Tnet(i) suddenly increases
SPIKE
NORMAL
if “slope” is small enough
60
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
In Spike Mode
SPIKE
Tnet
i
61
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
In Normal Mode
NORMAL
Tnet
i
62
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Evaluations
Loss
Rate
Delay
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
63
Evaluations
Loss
Rate
Delay
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
64
Problems with Ramjee’s Method
Transit
Time
Time
65
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Packet Audio Playout Delay
Adjustment: Performance
Bounds and Algorithms
S. Moon, J. Kurose, D. Towsley
Multimedia Systems 1998
66
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Recall Previous Methods
67
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
How to Set Tdelay(i)
 Moon’s Method
 Collect statistics on packets that
have arrived.
 Find t such that q% of last w packets
have Tnet(i) < t.
68
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Example (w =50, q = 90%)
num of
packets
1
2
3
4
5
6
7
8
9
10
11 12
delay
69
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Setting Tdelay(i)
NORMAL
SPIKE
70
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Setting Tdelay(i)
Transit
Time
Time
71
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Performance Bound
 Given a trace of packets, and a loss
rate, find the minimum average
playout delay.
 Use Dynamic Programming
72
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
A Packet Trace
k
M talkspurts
1,k
Tnet
2,k
13 15
3,k
j,k
nk,k
10 …
73
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
More Notations
 M:

Number of Talkspurt
 Npacket(k) or nk
 Number
of packets in talkspurt k
 Ntotal
 Total
number of packets
74
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Definition
minimum average playout delay for
choosing i packets to be played out
from k-th talkspurt
k
M talkspurts
75
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
How to find
76
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Definition
minimum average playout delay for
choosing i packets to be played out
from k-th to M-th talkspurt
k
..
M
M talkspurts
77
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Base Case
D(k, 0) =
D(M, i) =
minimum average playout delay for
choosing i packets to be played out
from k-th to M-th talkspurt
78
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Recursive Case
..
1,k
2,k
..
k
..
3,k
M
nk,k
j
79
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Performance Bound
 Given a trace of M talkspurts and n
packets, and a loss rate e, find the
minimum average playout delay.
 Answer: Minimum possible average
playout delay is D(1, (1-e)n)
minimum average playout delay for
choosing i packets to be played out
from k-th to M-th talkspurt
80
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Evaluations
Loss
Rate
Delay
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
81
Summary
 Playout Adjustment for Audio
Conferencing
 Weighted Average Methods vs.
Statistical Methods
 An Analysis of Minimum Playout
Delay
82
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Practical Complications
 Clock Drifts
 Route Change
83
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Advanced Techniques
 Speed-up Playback
84
NUS.SOC.CS5248-2015
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)