Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
ElasTraS: Database for the Cloud CSE880 Professor Pramanik Shayne Rashedan & Chongguang Bi 1 Agenda ● ● ● ● ● ● ● Background Motivation and Challenges Examples Architecture Schema Level Partitioning Live Database Migration Evaluation 2 Pay per use model 3 What is cloud computing ● Software as Service, Apps are located in Cloud, software experiences are delivered through the internet ● Platform as Service, Internet based platforms for developers who want to create services but don’t want to build their own cloud ● Infrastructure as Service, Cloud users rent storage, computation and maintenance from cloud providers (pay per use utilities) ● Data as Service, Infrastructure for web-scale data mining and knowledge discovery 4 Challenges ● Large numbers of application databases (or tenants) with diverse schemas, unpredictable load and transactions. ● Scaling out using clusters of commodity servers and sharing resources among tenants (multitenancy) ● Pay-Per-Use: paying a small subscription fee on per usage basis every time the software is being used (e.g. hourly) ● Elastic Scaling: When the load increases, add more servers to the database tier to distribute the load, and vice versa. ● Fault-tolerant, self-managing and highly available to support mission critical applications 5 ElasTranS: Elastic, Scalable, Self-Managing 6 Examples: ● Key-value stores - Bigtable, PNUTS and Dynamo, serving hundreds of thousands of concurrent requests ● Schema Level Partitioning - Microsoft SQL Azure ,Relational Cloud and a prototype from Yahoo!, standard RDBMSs (such as SQL Server or MySQL) ● Decoupled shared storage - Project Deuteronomy and Hyder, persistent data layer is separate from the logic that executes operations on that data,on the other hand in Amazon EC2, each node is data node and the transaction processor 7 Examples: ● Live data migration - VM pages, iterative phase-by-phase copying pages, migrates live processes and network connections using OS and networking primitives - Zephyr, for shared nothing database architectures such as Cloud SQL Server or Relational Cloud. The persistent data is stored in disks locally attached to every node. Hence, the persistent data must also be migrated ● Advanced transactions at Scale, - Megastore, Percolator, ecStore and G-Store - Sinfonia, two phase commit (2PC) protocol to guarantee atomicity of operations in a distributed set of nodes 8 Architecture of ElasTraS Transaction Manager Server Partitions of Data Owning Transaction Manager Row Groups Append-only Multiversion Data 9 Architecture of ElasTraS User / Client Routing Routing Routing Routing OTM Routing Routing OTM OTM OTM Data Storage Data Storage Data Storage Data Storage TM Master 10 Static Partitioning 11 Online Multi-player Game 12 Row Groups 13 Row Groups 14 Row Groups = Partitions 15 Why TPC-C Tree Schema? When amounts of data and transactions are large, assign some jobs to more nodes…. Assign all data and transactions related to Michigan State University Football Team to another node? How?? 16 Why TPC-C Tree Schema? MSU Spartan Stadium UM Michigan Stadium MSU Partition Keys UM MSU Player 1 Information 1 MSU Player 2 Information 2 UM Player 3 Information 3 UM Player 4 Information 4 MSU Player 1 Game 1 Score MSU Player 1 Game 2 ... MSU Player 2 Game 3 ... MSU Player 2 Game 4 ... UM Player 3 Game 5 ... UM Player 3 Game 6 ... UM Player 4 Game 7 ... UM Player 4 Game 8 ... 17 Snapshot & Multiversion Data Problem 1 Distribute Transactions? Read A Manager Data Read B Problem 2 Read Lock? Write Lock? Read Manager Data A Write B 18 Snapshot & Multiversion Data Problem 1 Just Read Snapshot! Read A OTM Data Read B Problem 2 Just Read Snapshot! Read OTM Data A Write B While “A”s are reading, B can update database. 19 Albatross Migration Timeline Source Ownership Steady State 1.Begin Migration Destination Ownership 2.Iterative Copying 3.Atomic Handover Steady State Clients TM Master 20 Challenges ● Minimum node unavailability achieved by iteratively copying database cache and state of active transactions ● Elastic Load Balancing, rule-based model, if OTMs load is above the threshold, new OTMs are added, if load is below a min-threshold, partitions are consolidated into a smaller number of nodes. ● Partition assignment algorithm optimization problem with Greedy Heuristic based on load statistics of partitions ● Atomic ownership handover, ensure efficient and ACID execution of transactions, transactions must execute during migration, synchronizing commit logs of two nodes ● Aborted transactions, minimize loss of data, restart aborted transactions in destination ● Failure Handling, how to recover migration state after failure 21 Elastic Load Balancing Data Data OTM User OTM User TM Master User OTM Data OTM Data 22 Evaluation Benchmarks: Yahoo! Cloud Serving Benchmark Key-value stores 3 tables, 10 VARCHAR(100) column per table TPC-C Benchmark 9 tables describing wholesale supplier 5 Transactions: NewOrder, Payment, OrderStatus, Delivery, StockLevel 23 Evaluation Comparing with Albatross: S&M: Stop and Migrate stop serving, flush, transfer ownership F&M: Flush and Migrate flush, stop serving, transfer ownership 24 Elastic Load Balancing Data Data Data User Data Migration OTM User OTM User TM Master User OTM Data OTM Data 25 Evaluation Albatross: Low Unavailability Low Latency 26 Evaluation Albatross: Almost no additional cost for read operations Migration costs much time 27 Evaluation Albatross: Low Latency Long Migration Time 28 Evaluation Not Enough Scale Scale is Large Enough Load is Low Migrate to Low Scale 29