Download CMPT 880: P2P Systems

School of Computing Science
Simon Fraser University
CMPT 765/408: P2P Systems
Instructor: Dr. Mohamed Hefeeda
1
P2P Computing: Definitions

Peers cooperate to achieve desired functions
- Peers:
• End-systems (typically, user machines)
• Interconnected through an overlay network
• Peer ≡ Like the others (similar or behave in similar manner)
- Cooperate:
• Share resources, e.g., data, CPU cycles, storage, bandwidth
• Participate in protocols, e.g., routing, replication, …
- Functions:
• File-sharing, distributed computing, communications,
content distribution, …

Note: the P2P concept is much wider than file
sharing
2
When Did P2P Start?

Napster (Late 1990’s)
- Court shut Napster down in 2001

Gnutella (2000)

Then the killer FastTrack (Kazaa, ...)

BitTorrent, and many others

Accompanied by significant research interest

Claim
- P2P is much older than Napster!

Proof
- The original Internet!
- Remember UUCP (unix-to-unix copy)?
3
What IS and IS NOT New in P2P?

What is not new
- Concepts!

What is new
- The term P2P (may be!)
- New characteristics of
• Nodes which constitute the
• System that we build
4
What IS NOT New in P2P?

Distributed architectures

Distributed resource sharing

Node management (join/leave/fail)

Group communications

Distributed state management

….
5
What IS New in P2P?

Nodes (Peers)
- Quite heterogeneous
• Several order of magnitudes difference in resources
• Compare the bandwidth of a dial-up peer versus a
high-speed LAN peer
- Unreliable
• Failure is the norm!
- Offer limited capacity
• Load sharing and balancing are critical
- Autonomous
• Rational, i.e., maximize their own benefits!
• Motivations should be provided to peers to cooperate
in a way that optimizes the system performance
6
What IS New in P2P? (cont’d)

System
- Scale
• Numerous number of peers (millions)
- Structure and topology
• Ad-hoc: No control over peer joining/leaving
• Highly dynamic
- Membership/participation
• Typically open 
- More security concerns
• Trust, privacy, data integrity, …
- Cost of building and running
• Small fraction of same-scale centralized systems
• How much would it cost to build/run a super computer
with processing power of that 3 Million SETI@Home PCs?
7
What IS New in P2P? (cont’d)

So what?

We need to design new lighter-weight
algorithms and protocols to scale to
millions (or billions!) of nodes given the
new characteristics

Question: why now, not two decades ago?
- We did not have such abundant (and
underutilized) computing resources back then!
- And, network connectivity was very limited
8
Why is it Important to Study P2P?

P2P traffic is a major portion of Internet
traffic (50+%), current killer app

P2P traffic has exceeded web traffic
(former killer app)!

Direct implications on the design,
administration, and use of computer
networks and network resources
- Think of ISP designers or campus network
administrators

Many potential distributed applications
9
Sample P2P Applications

File sharing
- Gnutella, Kazaa, BitTorrent, …

Distributed cycle sharing
- SETI@home, Gnome@home, …

File and storage systems
- OceanStore, CFS, Freenet, Farsite, …

Media streaming and content distribution
- PROMISE
- SplitStream, CoopNet, PeerCast, Bullet,
Zigzag, NICE, …
10
P2P vs. its Cousin (Grid Computing)

Common Goal:
- Aggregate resources (e.g., storage, CPU
cycles, and data) into a common pool and
provide efficient access to them

Differences along five axes [Foster & Imanitchi 03]
- Target communities and applications
- Type of shared resources
- Scalability of the system
- Services provided
- Software required
11
P2P vs Grid Computing (cont’d)
Issue
Grid
P2P

Established
Communities communities, e.g.,
scientific institutions
and

Applications

Computationallyintensive problems

Powerful and Reliable
machines, clusters


Resources
Shared
High-speed
connectivity

Specialized
instruments

Grass-root
communities
(anonymous)
Mostly, fileswapping
PCs with limited
capacity and
connectivity

Unreliable

Very diverse
12
P2P vs Grid Computing (cont’d)
Issue
System
Scalability
Grid
Hundreds to thousands
of nodes

Sophisticated services:
authentication, resource
discovery, scheduling,
access control, and
membership control
P2P
Hundreds of
thousands to
Millions of nodes


Services
Provided
Members usually trust
others

Software
required
Sophisticated suit: e.g.,
Globus, Condor

Limited services:
resource discovery

limited trust
among peers

Simple: (screen
saver), e.g., Kazza,
SETI@Home
13
P2P vs Grid Computing: Discussion

The differences mentioned are based on the
traditional view of each paradigm
- It is conceived that both paradigms will converge and
will complement each other [e.g., Butt et al. 03]

Target communities and applications
- Grid: is going open

Type of shared resources
- P2P: is to include various and more powerful resources

Scalability of the system
- Grid: is to increase number of nodes

Services provided
- P2P: is to provide authentication, data integrity, trust
management, …
14
P2P Systems: Simple Model
P2P Application
Middleware
P2P Substrate
Operating System
Hardware
System architecture: Peers form an
overlay according to the P2P
Substrate
Software architecture
model on a peer
15
Overlay Network

An abstract layer built on top of physical network

Neighbors in overlay can be several hops away in
physical network

Why do we need overlays?
- Flexibility in
• Choosing neighbors
• Forming and customizing topology to fit application’s
needs (e.g., short delay, reliability, high BW, …)
• Designing communication protocols among nodes
- Get around limitations in legacy networks
- Enable new (and old!) network services
16
Overlay Network (cont’d)
17
Overlay Network (cont’d)

Some applications that use overlays
- Application level multicast, e.g., ESM, Zigzag, NICE, …
- Reliable inter-domain routing, e.g., RON
- Content Distribution Networks (CDN)
- Peer-to-peer file sharing

Overlay design issues
- Select neighbors
- Handle node arrivals, departures
- Detect and handle failures (nodes, links)
- Monitor and adapt to network dynamics
- Match with the underlying physical network
18
Overlay Network (cont’d)
Recall: IP Multicast
source
19
Overlay Network (cont’d)
Application Level Multicast (ALM)
source
20
Peer Software Model


A software client
installed on each peer
P2P Application
Three components:
- P2P Substrate
- Middleware
- P2P Application
Middleware
P2P Substrate
Operating System
Hardware
Software model on peer
21
Peer Software Model (cont’d)

P2P Substrate (key component)
- Overlay management
• Construction
• Maintenance (peer join/leave/fail and network
dynamics)
- Resource management
• Allocation (storage)
• Discovery (routing and lookup)

Ex: Pastry, CAN, Chord, …

More on this later
22
Peer Software Model (cont’d)

Middleware
- Provides auxiliary services to P2P applications:
• Peer selection
• Trust management
• Data integrity validation
• Authentication and authorization
• Membership management
• Accounting (Economics and rationality)
• …
- Ex: CollectCast, EigenTrust, Micro payment
23
Peer Software Model (cont’d)

P2P Application
- Potentially, there could be multiple applications
running on top of a single P2P substrate
- Applications include
•
•
•
•
File sharing
File and storage systems
Distributed cycle sharing
Content distribution
- This layer provides some functions and
bookkeeping relevant to target application
• File assembly (file sharing)
• Buffering and rate smoothing (streaming)

Ex: Promise, Bullet, CFS
24
P2P Substrate

Key component, which
- Manages the Overlay
- Allocates and discovers objects

P2P Substrates can be
- Structured
- Unstructured
- Based on the flexibility of placing objects at
peers
25
P2P Substrates: Classification

Structured (or tightly controlled, DHT)
− Objects are rigidly assigned to specific peers
− Looks like as a Distributed Hash Table (DHT)
− Efficient search & guarantee of finding
− Lack of partial name and keyword queries
− Maintenance overhead
− Ex: Chord, CAN, Pastry, Tapestry, Kademila (Overnet)

Unstructured (or loosely controlled)
− Objects can be anywhere
− Support partial name and keyword queries
− Inefficient search & no guarantee of finding
− Some heuristics exist to enhance performance
− Ex: Gnutella, Kazaa (super node), GIA [Chawathe et al. 03] 26
Structured P2P Substrates

Objects are rigidly assigned to peers
− Objects and peers have IDs (usually by
hashing some attributes)
− Objects are assigned to peers based on IDs

Peers in overlay form specific geometrical
shape, e.g.,
- tree, ring, hypercube, butterfly network

Shape (to some extent) determines
− How neighbors are chosen, and
− How messages are routed
27
Structured P2P Substrates (cont’d)

Substrate provides a Distributed Hash
Table (DHT)-like interface
− InsertObject (key, value), findObject (key), …
− In the literature, many authors refer to
structured P2P substrates as DHTs

It also provides peer management (join,
leave, fail) operations

Most of these operations are done in O(log
n) steps, n is number of peers
28
Structured P2P Substrates (cont’d)

DHTs: Efficient search & guarantee of
finding

However,
− Lack of partial name and keyword queries
− Maintenance overhead, even O(log n) may be too
much in very dynamic environments

Ex: Chord, CAN, Pastry, Tapestry, Kademila
(Overnet)
29
Example: Content Addressable Network (CAN)
[Ratnasamy 01]
−
Nodes form an overlay in d-dimensional space
− Node IDs are chosen randomly from the d-space
− Object IDs (keys) are chosen from the same d-space
−
Space is dynamically partitioned into zones
−
Each node owns a zone
−
Zones are split and merged as nodes join and
leave
−
Each node stores
− The portion of the hash table that belongs to its zone
− Information about its immediate neighbors in the dspace
30
2-d CAN: Dynamic Space Division
7
n2
n1
n4
n3
n5
0
0
7
31
2-d CAN: Key Assignment
7
K1
K2
n2
n1
K4
n4
n3
K3
n5
0
0
7
32
2-d CAN: Routing (Lookup)
7
K1
K2
n2
n1
K4?
K4?
K4
n4
n3
K3
n5
0
0
7
33
CAN: Routing
−
Nodes keep 2d = O(d) state information
(neighbor coordinates, IPs)
− Constant, does not depend on number of
nodes n
−
Greedy routing
- Route to the node that is closest to the
destination
- On average, is done in O(n1/d) = O(log n) when
d = log n /2
34
CAN: Node Join
−
New node finds a node already in the CAN
− (bootstrap: one (or a few) dedicated nodes outside the
CAN maintain a partial list of active nodes)
−
It finds a node whose zone will be split
− Choose a random point P (will be its ID)
− Forward a JOIN request to P through the existing node
−
The node that owns P splits its zone and sends
half of its routing table to the new node
−
Neighbors of the split zone are notified
35
CAN: Node Leave, Fail
−
Graceful departure
− The leaving node hands over its zone to one of its
neighbors
−
Failure
− Detected by the absence of heart beat messages sent
periodically in regular operation
− Neighbors initiate takeover timers, proportional to the
volume of their zones
− Neighbor with smallest timer takes over zone of dead
node
− notifies other neighbors so they cancel their timers (some
negotiation between neighbors may occur)
− Note: the (key, value) entries stored at the failed node
are lost
− Nodes that insert (key, value) pairs periodically refresh (or
re-insert) them
36
CAN: Discussion
−
Scalable
− O(log n) steps for operations
− State information is O(d) at each node
−
Locality
− Nodes are neighbors in the overlay, not in the physical
network
− Suggestion (for better routing)
− Each node measure RTT between itself and its neighbors
− Forward the request to the neighbor with maximum ratio
of progress to RTT
−
Maintenance cost
− Logarithmic
− But, may still be too much for very dynamic P2P
systems
37
Unstructured P2P Substrates
−
−
Objects can be anywhere  Loosely-controlled
overlays
The loose control
− Makes overlay tolerate transient behavior of nodes
− For example, when a peer leaves, nothing needs to be done
because there is no structure to restore
− Enables system to support flexible search queries
− Queries are sent in plain text and every node runs a minidatabase engine
−
But, we loose on searching
− Usually using flooding, inefficient
− Some heuristics exist to enhance performance
− No guarantee on locating a requested object (e.g., rarely
requested objects)
−
Ex: Gnutella, Kazaa (super node), GIA [Chawathe et al.
03]
38
Example: Gnutella
−
−
−
Peers are called servents
All peers form an unstructured overlay
Peer join
− Find an active peer already in Gnutella (e.g., contact
known Gnutella hosts)
− Send a Ping message through the active peer
− Peers willing to accept new neighbors reply with Pong
−
Peer leave, fail
− Just drop out of the network!
−
To search for a file
− Send a Query message to all neighbors with a TTL (=7)
− Upon receiving a Query message
− Check local database and reply with a QueryHit to
requester
− Decrement TTL and forward to all neighbors if nonzero
39
Flooding in Gnutella
Scalability Problem
40
Heuristics for Searching [Yang and Garcia-Molina 02]
−
Iterative deepening
− Multiple BFS with increasing TTLs
− Reduce traffic but increase response time
−
Directed BFS
− Send to “good” neighbors (subset of your neighbors
that returned many results in the past)  need to keep
history
−
Local Indices
− Keep a small index over files stored on neighbors (within
number of hops)
− May answer queries on behalf of them
− Save cost of sending queries over the network
− Index currency?
41
Heuristics for Searching: Super Node
−
Used in Kazaa (signaling protocols are
encrypted)
−
Studied in [Chawathe 03]
−
Relatively powerful nodes play special role
− maintain indexes over other peers
42
Unstructured Substrates with Super Nodes
Super Node (SN)
Ordinary Node (ON)
43
Example: FastTrack Networks (Kazaa)
−
Most of info/plots in following slides are from
Understanding Kazaa by Liang et al.
−
The most popular (~ 3 million active users in a
typical day) sharing 5,000 Terabytes
−
Kazaa traffic exceeds Web traffic
−
Two-tier architecture (with Super Nodes and
Ordinary Nodes)
−
SN maintains index on files stored at ONs
attached to it
− ON reports to SN the following metadata on each file:
− File name, file size, ContentHash, file descriptors (artist
name, album name, …)
44
FastTrack Networks (cont’d)
−
Mainly two types of traffic
− Signaling
− Handshaking, connection establishment, uploading
metadata, …
− Encrypted! (some reverse engineering efforts)
− Over TCP connections between SN—SN and SN—ON
− Analyzed in [Liang et al. 04]
− Content traffic
− Files exchanged, not encrypted
− All through HTTP between ON—ON
− Detailed Analysis in [Gummadi et al. 03]
45
Kazaa (cont’d)
−
File search
− ON sends a query to its SN
− SN replies with a list of IPs of ONs that have
the file
− SN may forward the query to other SNs
−
Parallel downloads take place between
supplying ONs and receiving ON
46
FastTrack Networks (cont’d)
−
Measurement study of Liang et al.
− Hook three machines to Kazaa and wait till one
of them is promoted to be SN
− Connect the other two (ONs) to that SN
− Study several properties
− Topology structure and dynamics
− Neighbor selection
− Super node lifetime
− ….
47
Kazaa: Topology Structure [Liang et al. 04]
ON to SN: 100 - 160 connections  Since there are ~3M
nodes, we have ~30,000 SNs
SN to SN: 30 – 50 connections  Each SN connects to
~0.1 % of total number of SNs
48
Kazaa: Topology Dynamics [Liang et al. 04]
−
Average ON – SN connection duration
− Is ~ 1 hour, after removing very short-lived
connections (30 sec) used for shopping for
SNs
−
Average SN – SN connection duration
− 23 min, which is short because of
− Connection shuffling between SNs to allow ONs to
reach a larger set of objects
− SNs search for other SNs with smaller loads
− SNs connect to each other from time to time to
exchange SN lists (each SN stores 200 other SNs in
its cache)
49
Kazaa: Neighbor Selection [Liang et al. 04]
−
When ON first joins, it gets a list of 200 SNs
− ON considers locality and SN workload in selecting its future SN
−
Locality
− 40% of ON-SN connections have RTT < 5 msec
− 60% of ON-SN connections have RTT < 50 msec
− RTT: E. US  Europe ~100 msec
50
Kazaa: Lifetime and Signaling
Overhead [Liang et al. 04]
−
−
Super node average lifetime is ~2.5 hours
Overhead:
− 161 Kb/s upstream
− 191 Kb/s downstream
−  Most of SNs are high-speed (campus network, or cable)
51
Kazaa vs. Firewalls, NAT [Liang et al. 04]
−
Default port WAS 1214
− Easy for firewalls to filter out Kazaa traffic
−
Now, Kazaa uses dynamic ports
− Each peer chooses its random port
− ON reports its port to its SN
− Ports of SNs are part of the SN refresh list exchanged among
peers
− Too bad for firewalls!
−
Network Address Translator (NAT)
− A requesting peer can not establish a direct connection with a
serving peer behind NAT
− Solution: connection reversal
− Send to SN of NATed peer, which already has a connection with it
− SN tells NATed peer to establish a connection with requesting
peer!
− Transfer occurs happily through the NAT
− Both peers behind NATs?
52
Kazaa: Lessons [Liang et al. 04]
−
−
−
−
−
Distributed design
Exploit heterogeneity
Load balancing
Locality in neighbor selection
Connection Shuffling
− If a peer searches for a file and does not find it, it may try
later and gets it!
−
Efficient gossiping algorithms
− To learn about other SNs and perform shuffling
− Kazaa uses a “freshness” field in SN refresh list  a
peer ignores stale data
−
Consider peers behind NATs and Firewalls
− They are everywhere!
53
Summary

P2P is an active research area with many
potential applications in industry and academia

In P2P computing paradigm:
- Peers cooperate to achieve desired functions

New characteristics
- heterogeneity, unreliability, rationality, scale, ad hoc
-  new and lighter-weight algorithms are needed

Simple model for P2P systems:
- Peers form an abstract layer called overlay
- A peer software client may have three components
• P2P substrate, middleware, and P2P application
• Borders between components may be blurred
54
Summary (cont’d)

P2P substrate: A key component, which
- Manages the Overlay
- Allocates and discovers objects

P2P Substrates can be
- Structured (DHT)
• Example: CAN
- Unstructured
• Example 1: Gnutella,
• Example 2: Kazza
55