Download Proxy Cache Management for Fine-Grained Scalable Video Streaming

CoolStreaming/DONet: A Data-Driven
Overlay Network for Peer-to-Peer Live
Media Streaming
Jiangchuan Liu
with Xinyan Zhang, Bo Li, and T.S.P.Yum
Infocom 2005
March -05
1
Some Facts
DONet – Data-driven Overlay Network
CoolStreaming – Cooperative Overlay Streaming
First release (CoolStreaming v0.9)
 May 2004
Till March 2005
 Downloads: >100,000
 Average online users: 6,000
 Peak-time online users: 14,000
 Google entries (CoolStreaming): 5130
March -05
2
Outline





Motivation
Background and related work
Design of DONet/CoolStreaming
Implementation and empirical Study
Future work
March -05
3
Motivation

Enable large-scale live broadcasting in the
Internet environment

Capacity limitation
‫־‬
‫־‬


March -05
Streaming: 500Kbps, server outbound band: 100Mbps
200 concurrent users only
Network heterogeneity
No QoS guarantee
4
Client/Server: Poor scalability
Client
Client
Client
Client
Client
March -05
Client
5
IP multicast: Limited deployment
Client
Client
Client
Client
Client
Client
Client
March -05
6
Collaborative Communications
Client
Client
Client
Client
Client
March -05
Client
7
Outline





Motivation
Background and related work
Design of DONet/CoolStreaming
Implementation and empirical Study
Future work
March -05
8
Related Solutions

Content distribution networks



Expensive
Not quite scalable for a large number of
audiences
Self-organized overlay networks


March -05
Application layer multicast
Peer-to-peer communications
9
Related Solutions

Content distribution networks



Expensive
Live streaming (?)
Self-organized overlay networks


March -05
Application layer multicast
Peer-to-peer communications
10
Application Layer Multicast


Issue: Structure construction
Tree


Mesh


NICE, CoopNet, SpreadIt, ZIGZAG
Narada and its extension
F
Multi-tree

A
C
B
SplitStream
G
F
D
E
A
B
C
G
D
E
H
F
B
G
C
D
E
H
I
(a)
A
H
I
(b)
(c)
A3
A3
B1
B1
C0
A0
A0
C0
B0
A1
B2
A2
(a)
March -05
B0
A7
A1
B2
A2
A7
(b)
11
Application Layer Multicast (cont’d)


Issue: Node dynamics
Structure maintenance


Passive/proactive repairing algorithms
Advanced coding


March -05
PALS (layered coding)
CoopNet (multiple description coding)
12
Gossip-based Dissemination

Gossip

Iteration
‫־‬
‫־‬



Sends a new message to a random set of nodes
Each node does similarly in the next round
Pros: Simple, robust
Cons: Redundancy, delay
Related

March -05
Peer-to-peer on-demand streaming
13
Outline





Motivation
Background and related work
Design of DONet/CoolStreaming
Implementation and empirical Study
Future work
March -05
14
Data-driven Overlay (DONet)

Target



Live media broadcasting
No IP multicast support
Core operations


March -05
Every node periodically exchanges data availability
information with a set of partners
Then retrieves unavailable data from one or more
partners, or supplies available data to partners
15
Features of DONet

Easy to implement


Efficient


no need to construct and maintain
a complex global structure
data forwarding is dynamically
determined according to data
availability, not restricted by
specific directions
Robust and resilient

March -05
adaptive and quick switching
among multi-suppliers
16
Key Modules

Membership manager



Partnership manager



mCache – partial overlay view
Update by gossip
Random selection
Partner refinement
Transmission Scheduler
March -05
17
Transmission Scheduling
Problem: From which partner to fetch which
data segment ?

Constraints



March -05
Data availability
Playback deadline
Heterogeneous partner bandwidth
18
Scheduling algorithm

Variation of Parallel machine scheduling


NP-hard
Heuristic

Message exchanged
‫־‬
‫־‬




Window-based buffer map (BM): Data availability
Segment request (piggyback by BM)
Less suppliers first
Multi-supplier: Highest bandwidth within deadline first
Simpler algorithm in current implementation
Network coding ?
March -05
19
Analysis on DONet

Coverage ratio for distance k



E.g. 95% nodes are covered in 6 hops for M=4
Average distance O(logN)
DONet vs Tree-based overlay

March -05
Much lower outage probability
20
Outline





Motivation
Background and related work
Design of DONet/CoolStreaming
Implementation and empirical Study
Future work
March -05
21
PlanetLab Experiments

Distributed experimental system




DONet Module
Console and automation
Command dispatching and report collection
Caveats



Scalability
Reproducibility
Representability
Console
Command
Queue
Automaton
Report Collector
Command Dispatcher
DONet System
March -05
22
Geographical Node Distribution
May 24, 2004
# of Active Node: 200-300
March -05
23
Planet-Lab Result

Data continuity, 200 nodes, 500 kbps streaming
March -05
24
Control overhead
March -05
25
Implementation: CoolStreaming



First release: May 30, 2004
Source code: 2000-line Python
Programming time:



Support formats:



PlanetLab prototype: 2 weeks
Export from prototype: 2 weeks
Real Video/Windows Media
Platform/media independent
Scale and capacity



March -05
Total downloads:
Peak time: 14000 concurrent users
Streaming rate: 450-700kbps
26
User Distribution (June 2004)

Heterogeneous network environment

March -05
LAN, DSL, CABLE...
27
Online Statistics (Jun 21, 2004)
Average Packet Loss around 1% - 5%
March -05
28
Observations

Current Internet has enough available band
to support TV-quality streaming (>450Kbps)


Bottleneck: server, end-to-end bandwidth
Larger data-driven overlay
 better streaming quality

March -05
Capacity amplification
29
Outline





Motivation
Background and related work
Design of DONet/CoolStreaming
Implementation and empirical Study
Future work
March -05
30
Future of DONet/Coolstreaming

Content



Solution: DONet/Coolstreaming as a capacity
amplifier between content provider and clients
Virtually part of network infrastructure
Enhancement

Scheduling algorithm
‫־‬
‫־‬

Transport protocol
‫־‬
March -05
Simplified version
Network coding
TCP (?)
31
Future of DONet/Coolstreaming

Enhancement (cont’d)



User interface
Combined with caching
Combined with CDN
‫־‬

March -05
Provide world-wide reliable media
streaming service
On-demand streaming
32
Thanks
Q&A
March -05
33