* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Slide Presentation
Serializability wikipedia , lookup
Entity–attribute–value model wikipedia , lookup
Microsoft Access wikipedia , lookup
Oracle Database wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Functional Database Model wikipedia , lookup
Concurrency control wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Relational model wikipedia , lookup
Database model wikipedia , lookup
A Peer-to-Peer Database Server based on BitTorrent John Colquhoun Paul Watson www.neresc.ac.uk Introduction • When a database server receives queries faster than it can process them, performance becomes unacceptable • Similar problems solved for file-sharing by the use of P2P technologies • Can we utilise the combined CPU, memory, network and disk of individual clients to reduce the server load? • We examine how P2P techniques could be applied to databases and introduce the Wigan P2P database, derived from the BitTorrent file-sharing protocol • Potential applications in e-Science and e-Commerce www.neresc.ac.uk 2 System Architecture Advertise: SELECT * FROM t Query: SELECT id, value FROM t WHERE t.Id < 100 Advertise: SELECT id, value FROM t WHERE t.Id < 100 Tracker Query: SELECT id, value FROM t WHERE t.id < 100 Advertise: SELECT id, value FROM t WHERE t.id < 100 Advertise: SELECT id, value Query: SELECT id, value FROMt t FROM WHEREt.Id t.Id<<10 10 WHERE www.neresc.ac.uk 3 Initial Implementation • A simulator of Wigan • The TPC-H benchmark database was used to evaluate the design • Identified cases where Wigan offered a performance advantage over a Client-Server database and those areas where it did not • Showed Wigan can outperform client-server DB when: • There are enough peers to reduce the load on the seed • There is sufficient overlap between the queries • The system is sufficiently busy so a traditional database server is overloaded www.neresc.ac.uk 4 Live Wigan system • Used algorithms developed for the simulator • Written in Java and uses OGSA-DAI • Also uses the TPC-H benchmark database www.neresc.ac.uk 5 Live System Experiments • Experiments – running the 15 TPC-H queries that MySQL can process • Five peers starting at 10 second intervals • Of these, Wigan could answer eight • Remainder require more advanced SQL features • The remaining queries were sent to the seed in Wigan • Three different scenarios • Standard Client-Server (accessing MySQL via JDBC) • Wigan • Wigan in a warm-cache scenario (to see how Wigan would behave if multiple peers have been available for some time) www.neresc.ac.uk 6 Live System Results 600 Average response time (s) 500 400 300 200 100 0 Client-Server Wigan With No Warming Wigan With Warming Method www.neresc.ac.uk 7 Live System Results • Client-Server is quite slow because the queries are complex, involving many joins and aggregations • Wigan in a cold-cache scenario does not offer a performance advantage • Only the seed is able to provide data • Peers have to send all requests to the seed, hence queues form there • Wigan in a warm-cache scenario outperforms ClientServer • Other peer (s) able to provide the results, so reducing the load on the seed • Peer (s) with the query results do not have so much data to scan or complex joins to perform • Validates results from the simulator regarding when Wigan does and does not perform well www.neresc.ac.uk 8 Future Work • Testing Wigan with “real” e-Science data from existing projects at Newcastle University • Connect Wigan to an open-source implementation of Amazon’s SimpleDB to explore the effect of the Cloud and updates • Example of usage in both three-tier architecture and an e-Science application www.neresc.ac.uk 9 Summary • We designed, implemented & evaluated the Wigan P2P Database System • Derived from the popular BitTorrent file-sharing protocol • The first database server that uses P2P to scale over multiple peers • Showed P2P databases can scale to outperform traditional Client-Server databases www.neresc.ac.uk 10