Download Implicit groups

Implicit group messaging
in peer-to-peer networks
Daniel Cutting, 30th November 2005
Outline.
• Explicit and implicit groups
• P2P networks
• Implicit group messaging
• My approach
• Addressing
• Grouping
• Routing
• Evaluation
Implicit group messaging
November 30, 2005 Slide 2
Explicit groups.
• Group “Griffin”
•
•
•
•
Lois
Peter
Stewie
Brian
• Group “A-Team”
• Homer
• Peter
Implicit group messaging
November 30, 2005 Slide 3
Implicit groups.
• Implicit groups
• Soccer
• Soccer AND
Argentina
• Australia OR
Argentina
• (Soccer OR
Football) AND
Argentina
Implicit group messaging
November 30, 2005 Slide 4
Explicit and implicit groups.
• Explicit groups: members are named
• Membership managed as a central list or a distributed
structure (e.g. multicast group trees)
• Members explicitly join (individually or by a coordinator)
• Implicit groups: members are described
• E.g. “Everyone interested in soccer”
• Members don’t join, they just match the description
Implicit group messaging
November 30, 2005 Slide 5
P2P networks.
• Overlay networks of hosts (“peers”) on the Internet
• Structured (CAN, Pastry ): peers reside in a logical space
and are connected in some ordered and consistent way
• Unstructured (Gnutella, KaZaA): more ad hoc
• P2P commonly used to swap files, but also good for:
• Distributed data storage, academic collaboration tools,
large multiplayer games, message forums
• Want support for messaging groups within these
networks (for searches, requests, events, etc.)
• Implicit groups are good for these!
Implicit group messaging
November 30, 2005 Slide 6
P2P networks.
Hand of God
(Soccer OR Football)
AND Argentina
Implicit group messaging
November 30, 2005 Slide 7
Implicit group messaging.
• AIM: Deliver messages from any peer to any
implicit group at any time in a P2P network
• Assumptions:
• Each peer describes itself with attributes (strings indicating
capabilities, interests, services, …),
e.g. “Soccer”, “Argentina”
• Implicit groups are specified as logical expressions of
attributes, e.g. “(Soccer OR Football) AND Argentina”
• System delivers messages from a source to all peers
matching the expression
Implicit group messaging
November 30, 2005 Slide 8
My approach.
• A fully distributed, structured overlay network
• Peers build and maintain a logical Cartesian surface
• Each peer resides at a logical address on surface
• Each peer owns part of the surface and knows its
neighbouring peers
• Key features
• Addressing: a peer’s address encodes its attributes
• Grouping: a group’s description encodes all possible
addresses of matching peers
• Routing: source uses group description to reactively
construct a multicast tree to all possible addresses
Implicit group messaging
November 30, 2005 Slide 9
What does a surface look like?
Implicit group messaging
November 30, 2005 Slide 10
But what does it mean?!
Peter {Australia}
Lois {Soccer, Australia}
Brian {Soccer}
Homer {Football, Argentina, Donuts}
Stewie {Soccer, Argentina}
Implicit group messaging
November 30, 2005 Slide 11
JOINing the network.
• New peer calculates its address
• Routes a JOIN request to that address from a bootstrap
• Peer that currently owns the
address partitions its part of
the surface
• All neighbours are informed
• To leave the network give
your parts of the surface
to your neighbours
Implicit group messaging
November 30, 2005 Slide 12
Getting around on the surface.
• Each peer knows
its neighbours
• When given a
message with a
destination
address, pass it on
to geometrically
nearest neighbour
Implicit group messaging
November 30, 2005 Slide 13
Addressing.
• How does a peer determine its location on the surface?
• Each peer has a set of attributes (its interests, say)
• Encode these into the address of the peer using a Bloom
Filter
• E.g. {Soccer, Argentina}  01101
01100 | 01001
• Map the address to a part of the surface
• Location on surface encodes attributes
Implicit group messaging
November 30, 2005 Slide 14
Addressing.
• Map from an address to
the surface using a
quadtree decomposition
• Quadrants called extents
• E.g. {Soccer, Argentina}
 01101
• Pad with 0s at end
• 01 10 1(0) = 122
Implicit group messaging
November 30, 2005 Slide 15
Grouping.
• How do we find a group of peers given a description?
• E.g. all peers with attributes “Soccer AND Argentina”
• Convert to
a Bloom Filter, but wildcards (*) replace 0s
• {Soccer, Argentina }  *11*1
01100 | 01001
• So any peer with both attributes must have (at least)
the 2nd, 3rd and 5th bits set in their address
• The wildcards may match 1s or 0s depending on what
other attributes the peer has
• *11*1 matches addresses 01101, 11101, 01111, 11111
Implicit group messaging
November 30, 2005 Slide 16
Grouping.
• Need to find extents
where the 2nd, 3rd and
5th bits are set
• {Soccer, Argentina}
 *1 1* 1(*)
• **  00, 01, 10, 11
(extents 0, 1, 2, 3)
• *1  01 and 11 (1, 3)
• 1*  10 and 11 (2, 3)
• 11  just 11 (3)
Implicit group messaging
November 30, 2005 Slide 17
Grouping.
• ORs can be treated as a set of ANDs
• E.g. “(Soccer OR Football) AND Argentina”
• Equivalent to “(Soccer AND Argentina) OR (Football AND
Argentina)”
• {Soccer, Argentina}  *11*1
01100 | 01001
• {Football, Argentina}  11*11
11010 | 01001
• All peers with address having 2nd, 3rd and 5th OR 1st,
2nd, 4th and 5th bits set are part of this group
Implicit group messaging
November 30, 2005 Slide 18
Grouping.
• {Soccer, Argentina}
 *1 1* 1(*)
• {Football, Argentina}
 11 *1 1(*)
• **  00, 01, 10, 11
(extents 0, 1, 2, 3)
• *1  01 and 11 (1, 3)
• 1*  10 and 11 (2, 3)
• 11  just 11 (3)
Implicit group messaging
November 30, 2005 Slide 19
Routing.
• A peer wants to send a message to an implicit group
• Creates a message: “Got any Hand of God photos?”
• Specifies an appropriate implicit group: “Soccer AND
Argentina”
• Chooses the best neighbour(s) to forward the message
• Knows extents yet to be visited (everything initially)
• Intersects these with extents matching group
description
• Clusters what’s left and sends a message towards each
cluster
Implicit group messaging
November 30, 2005 Slide 20
Routing.
• If many targets in same
direction, only route one
copy: i.e. cluster based
on their direction
• Message splits as it gets
closer to clusters since
relative angles increase
• Clustering threshold
angle can be variable
• Guarantees delivery
Implicit group messaging
November 30, 2005 Slide 21
Evaluation.
• Simulation
• OMNeT++ implementation simulating campus- and
world-scale physical networks
• Thousands of peers
• In progress
• Compare to alternative models
• IP multicast flooding (optimised physical routing, but all
peers receive all messages)
• Centralised server (unfair to some peers/links but only
member peers receive messages)
Implicit group messaging
November 30, 2005 Slide 22
Evaluation.
• Metrics
• Normalised overall network traffic for messaging
• Peer fairness (variance of computation and storage)
• Network link fairness (variance of link stress)
• Expected results
• Flood should have good peer and link fairness, poor total
traffic for small implicit groups
• Centralised should have poor peer and link fairness, good
total traffic for all groups
• My approach should have good peer and link fairness, and
good total traffic for small groups, poor for large
Implicit group messaging
November 30, 2005 Slide 23
Evaluation.
• Evaluation of basic features
• Peer storage fairness
• Average cost of unicast
routing between two
random addresses
Peer fairness
Unicast ROUTE cost
Implicit group messaging
November 30, 2005 Slide 24
Questions?
Implicit group messaging
November 30, 2005 Slide 25
Ancillary slides.
Implicit group messaging
November 30, 2005 Slide 26
Related work.
• Explicit group systems
• IP multicast, Usenet (consumers explicit join channels)
• Email (publisher lists recipients by name)
• SCRIBE (on Pastry), CAN Multicast, Bayeux (on Tapestry)
• Implicit group systems
• Khambatti et al: interest-based communities (but don’t
support arbitrary cross-cutting groups)
• Interest management (virtual environment updates)
• Content-based publish/subscribe (but different semantics)
Implicit group messaging
November 30, 2005 Slide 27
Content-based publish/subscribe.
• Conceptually similar: messages are delivered to implicit
groups based on a match at time of publication
• Pub/sub: consumers select the type of message they
receive. Implicit group messaging: publishers select
type of consumer of message
• Converse semantics lead to differing expressiveness
• Pub/sub good for consumers who need to be notified of
specific types of events from any publisher: e.g. GUI
components
• Implicit group messaging good for publishers who need to
reach specific types of consumer: e.g. distributed search
engines
Implicit group messaging
November 30, 2005 Slide 28
Future work.
• Very important to get physical network simulations
running
• Testing with various attribute distributions, higher
dimensional surfaces
• Random shortcuts through the surface to reduce routing
cost (can be inserted when peers JOIN)
• Prefixing addresses with bits that place peers on surface
with some approximation of underlying network may
improve physical network usage
Implicit group messaging
November 30, 2005 Slide 29
Attribute distribution.
Peers with popular attributes
{Soccer, Argentina, Sport}
{Soccer, Argentina, Beer}
{Rugby, Australia, Sport}
Peers with unpopular attributes
{Football, Argentina}
Uniform attribute distribution
Implicit group messaging
November 30, 2005 Slide 30
Zipf attribute distribution
Research plan.
• Technical Report awaiting Smart Internet Technology
CRC approval
• Short Letter expressing basic concepts in next week
• Journal paper with network results by end of year
• Conference paper with future work early next year
• Complete around July
Implicit group messaging
November 30, 2005 Slide 31
P2P networks.
•
Messaging in P2P networks is often many-to-many
•
E.g. any peer can initiate a multicast query to search for files or
services
•
Typically handled by flooding (Gnutella), superpeer registries
(KaZaA), plus many other shortcuts.
•
Some structured networks have multicast capabilities
•
•
Peers can subscribe to multicast channels and receive all
messages sent to that channel
Need messaging between peers for:
•
•
•
Storing/retrieving data or files
Searching for particular data
Searching for particular kinds of peers
Implicit group messaging
November 30, 2005 Slide 32
P2P networks.
• P2P needs multicast (for searches, requests, events)
• Allows a peer to send a message to a group of recipients
• Often will know names of recipients, e.g. when some
peers have explicitly requested notification of an event
• However, there are times when it won’t, e.g. searching
for peers matching some criteria
• Often just flood the network, but may be more targeted
• Difference is the way multicast groups are defined:
explicitly or implicitly
Implicit group messaging
November 30, 2005 Slide 33
Implicit group messaging.
• In an ideal system:
•
•
•
•
•
All implicit group members should receive messages
Non-members shouldn’t receive them
Dynamic membership should be supported
Minimal total network load
Fairness across peers/network links
Implicit group messaging
November 30, 2005 Slide 34