Download Powerpoint

A Distributed Database System
- in a dynamic peer-to-peer environment
A project by:
Claus R. Thrane,
Uffe Sørensen,
Morten Dahl Jørgensen,
Martin Clemmensen
Agenda
• Introduction (Uffe)
• Motivation
• Model
• Networking
• MANETs
• P2P and Overlays (Morten)
• Database
• Querying (Martin)
• Data Replication Versus Caching
• Distributed Transactions (Claus)
• Conclusion
• Model Results
• Experiments and Testing (Morten)
• Demonstration
8 January 2006
AALBORG UNIVERSITET
Page 2 of 30
Motivation
•
•
•
•
Terma
Client/Server
Ad-hoc P2P (MANET)
“The Travelling Wheelbarrow Problem”
8 January 2006
AALBORG UNIVERSITET
Page 3 of 30
Project Goals
• Analyze case network requirements and derive
theoretical and practical solutions.
• Develop an effective routing technique based on
the requirements of the case.
• Develop an effective query scheme based on the
requirements of the case.
• Design, implement, test and analyze results of the
presented solutions.
8 January 2006
AALBORG UNIVERSITET
Page 4 of 30
Model
Proof-of-concept
Groups
Proem
ZRP
Any
8 January 2006
AALBORG UNIVERSITET
Page 5 of 30
MANETs
• M(obile)
• Dynamic topology
• Limited bandwidth, interference – wireless
• A(d-hoc)
• No fixed infrastructure, preferred paths,
designated routers.
• NET(works)
• Maintain intra-MANET connectivity
• Maintain outside connectivity
8 January 2006
AALBORG UNIVERSITET
Page 6 of 30
MANET Application
• Vehicular Networking
• Trucks as in our case
• Mobile Phones
• Festivals – Roskilde 2002
• Cities – Context awareness
8 January 2006
AALBORG UNIVERSITET
Page 7 of 30
Ad-hoc Network Anatomy
• Architectural view:
• No distinction between router and host
• Dynamic network configuration
No pre-established links
The network forms the links
8 January 2006
AALBORG UNIVERSITET
Page 8 of 30
Agenda
• Introduction (Uffe)
• Motivation
• Model
• Networking
• MANETs
• P2P and Overlays (Morten)
• Database
• Querying (Martin)
• Data Replication Versus Caching
• Distributed Transactions (Claus)
• Conclusion
• Model Results
• Experiments and Testing (Morten)
• Demonstration
8 January 2006
AALBORG UNIVERSITET
Page 9 of 30
Peer-to-Peer (P2P)
• Made up by the peers in the network
• Built on top of an existing network
• Motivated by its high scalability/reliability
• No predefined roles (client/server)
• Abstraction over network - overlay
• Free flat “address space”
• Exact location of data not known until “run-time”
• Logical routing and querying
8 January 2006
AALBORG UNIVERSITET
Page 10 of 30
P2P Network
8 January 2006
AALBORG UNIVERSITET
Page 11 of 30
P2P Overlay Types
• 1th and 2nd generation: unstructured
• Application specific
• Little abstraction, more framework
• E.g. Napster and Gnutella
• 3rd generation: structured
•
•
•
•
General purpose
High abstraction: DHT and DOLR
Upper bound on routing (in ideal state)
E.g. Pastry
8 January 2006
AALBORG UNIVERSITET
Page 12 of 30
P2P and MANETs
• MANETs can be highly mobile
• Structures in MANETs should be locality-aware
• P2P overlays assume a “stable” infrastructure
• Upper bound is on logical routing in ideal state
• P2P networks designed to span the globe
• Safari handles large MANETs
• Uses a DHT-like structure
• Distribute/share responsibility
• Small structures
8 January 2006
AALBORG UNIVERSITET
Page 13 of 30
Agenda
• Introduction (Uffe)
• Motivation
• Model
• Networking
• MANETs
• P2P and Overlays (Morten)
• Database
• Querying (Martin)
• Data Replication Versus Caching
• Distributed Transactions (Claus)
• Conclusion
• Model Results
• Experiments and Testing (Morten)
• Demonstration
8 January 2006
AALBORG UNIVERSITET
Page 14 of 30
Querying
• Query language
• Like SQL
• Query Interface
• Strictly defined interface
• Case Database Schema
• Case databases are heterogeneous
• Objects represent physical items
8 January 2006
AALBORG UNIVERSITET
Page 15 of 30
Query Techniques
• Groups
• “Optimized” flooding
• Depth-First Search
• Sequential queries
• Breadth-First Search
• Flooding
• Local Indices
• Answering queries on behalf of others
8 January 2006
AALBORG UNIVERSITET
Page 16 of 30
Group Based Querying
• This solution was chosen in our model
• Optimized flooding
• Performance depends highly on group
construction
• Optimizations to group construction can be
made by performing statistical analysis of
queries
• In a network with few nodes, performance
will be slightly better than flooding
8 January 2006
AALBORG UNIVERSITET
Page 17 of 30
Data Replication Versus Caching
Replication
• Advantages
• Availability
• Fault tolerance
• Disadvantages
• Traffic increase
• Storage usage
increase
8 January 2006
Caching
• Advantages
• Good enough
answers
• No significant traffic
increase
• Disadvantages
• Storage usage
increase
• Not 100% up-to-date
AALBORG UNIVERSITET
Page 18 of 30
Data Replication
• Full replication
• Defeats the purpose of P2P/MANETs
• Uniform replication
• Fixed number of replicas
• Proportional replication
• Replicas of requested objects
• Our Case
• Data objects represents physical objects
• Large amount of traffic on the network
8 January 2006
AALBORG UNIVERSITET
Page 19 of 30
Caching
• Caching Queries & Results
• Used to provide good-enough answers and to
optimize future queries
• Eavesdropping
• Caching results of queries made by others and
heard by the current node
• Our Case
• Does not generate excessive amount of traffic
8 January 2006
AALBORG UNIVERSITET
Page 20 of 30
Agenda
• Introduction (Uffe)
• Motivation
• Model
• Networking
• MANETs
• P2P and Overlays (Morten)
• Database
• Querying (Martin)
• Data Replication Versus Caching
• Distributed Transactions (Claus)
• Conclusion
• Results
• Experiments and Testing (Morten)
• Demonstration
8 January 2006
AALBORG UNIVERSITET
Page 21 of 30
Concepts of Distributed Transactions
• Simulate atomic operations
• Preserve object state consistency and
maximize concurrency
• All or Nothing (in case of crashes)
• Client, Coordinator and Recoverable objects
8 January 2006
AALBORG UNIVERSITET
Page 22 of 30
Distributed Transaction Techniques
• Traditional and distributed
• Locks, Optimistic and Timestamp
• One/two/three-phase commit
• A transaction in our system
• Reserve tire, frame, barrow and mounting
screws
8 January 2006
AALBORG UNIVERSITET
Page 23 of 30
Transaction Example
Pseudo-code:
• T= openTransaction(abc)
• reserveItem(A,tire)
• reserveItem(B,frame)
• reserveItem(C,barrow)
• reserveItem(C,screws)
• closeTransaction(T)
Two-Phase commit
•Issues in ad-hoc
•Who is coordinator?
•Dead-lock detection?
•Server crashes / disconnect?
8 January 2006
AALBORG UNIVERSITET
Page 24 of 30
Model Results
• Goals and Achievements
• Overall project
• Project results
• Implementation – proof of concept
• Ad-hoc and Peer-to-Peer.
8 January 2006
AALBORG UNIVERSITET
Page 25 of 30
Agenda
• Introduction (Uffe)
• Motivation
• Model
• Networking
• MANETs
• P2P and Overlays (Morten)
• Database
• Querying (Martin)
• Data Replication Versus Caching
• Distributed Transactions (Claus)
• Conclusion
• Model Results
• Experiments and Testing (Morten)
• Demonstration
8 January 2006
AALBORG UNIVERSITET
Page 26 of 30
Ad-hoc Testing
• Setup and Environment
• Two algorithms: proactive and reactive
• Three cases: average case and two “optimized”
• Modelled in ns2
8 January 2006
AALBORG UNIVERSITET
Page 27 of 30
Simulation results
8 January 2006
AALBORG UNIVERSITET
Page 28 of 30
Query Testing
• Setup and Environment
•
•
•
•
•
20%/60% items in storage
Broadcast/group model
7/49 nodes
Tightly connected nodes
Implemented using FreePastry with Scribe
8 January 2006
AALBORG UNIVERSITET
Page 29 of 30
Test Results
• Results
• Tightly connected nodes unrealistic
• Future Tests
• More realistic environment
• More realistic scenario; real-world test
• Longer running tests
8 January 2006
AALBORG UNIVERSITET
Page 30 of 30
Demonstration
DEMO
8 January 2006
AALBORG UNIVERSITET
Page 31 of 30