* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download MySQL Cluster - people.vcu.edu
Entity–attribute–value model wikipedia , lookup
Oracle Database wikipedia , lookup
Serializability wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Functional Database Model wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Relational model wikipedia , lookup
Concurrency control wikipedia , lookup
Versant Object Database wikipedia , lookup
ContactPoint wikipedia , lookup
MySQL Storage Engine Overview Dr. Charles A. Bell Senior Software Developer [email protected] Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 1 Agenda • • • • • • MySQL AB Technology Stacks MySQL Server Architecture Pluggable Storage Engines Server Internals Q&A Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 2 MySQL AB • • • • • • • Founded in 1995 Operations in 22 countries 10+ million product installations 50,000 downloads each day Dramatically reduces TCO of database management Bundled by more than 100 SW and HW companies Sold by partners such as HP, Dell, Novell, and others Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 3 Chosen by Successful, Modern Companies • Embedded – – ”Batteries included” database in software applications Network elements • High Volume Web Sites – – – – Dynamic content eCommerce Gaming & entertainment Scale Out • Enterprise – – – – – Data Warehousing High-Volume Transactions Departmental Intranet/Extranet Scale Out Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 4 MySQL Software Priorities Performance MySQL, Sun and BEA WebLogic Set New World Records for Speed & Price/Performance in SPEC Benchmarks Reliability Ease of Use 15 Minute Rule Study found comparatively fewer bugs in MySQL* Up and running in 15 minutes Lower TCO * Robert Lemos CNET News.com Feb 4, 2005 Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 5 Second Generation Open Source • MySQL AB is a profitable company – Develops the software in-house; community helps test it – Owns source code, copyrights and trademarks – Targets the “commoditized” market for databases • “Quid Pro Quo” dual licensing for OEM market – Cost-effective commercial licenses for commercial use – Open source GPL license for open source projects • Annual MySQL Network subscription for Enterprise, Web and OEM development/testing – Per server annual subscription – Includes support, alert and update advisors, Knowledge Base, Certified/Optimized Binaries • MySQL Support – Worldwide 24 x 7 support – Training and certification – Consulting Copyright 2006 MySQL AB “Reasoning's inspection study shows that the code quality of MySQL was six times better than that of comparable proprietary code. ” The World’s Most Popular Open Source Database 6 Popular Technology Stacks LAMP J2EE .NET Java .net / C# Perl MySQL Copyright 2006 MySQL AB MySQL Apache Apache Tomcat JBoss Linux Linux or Solaris MySQL IIS Apache JBoss Windows The World’s Most Popular Open Source Database 7 Supported Technology Platforms Programming Languages Perl … Database Web & Application Server Internet Information Server Apache Operating System HP-UX Hardware Storage Copyright 2006 MySQL AB Tomcat … … … … The World’s Most Popular Open Source Database 8 MySQL Architecture Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 9 Pluggable Storage Engine Architecture • MySQL supports several storage engines that act as handlers for different table types. • Choose, create, or extend a storage engine that best suits your applications unique requirements. • What is most important to you? - Read Intensive - OLTP - Transactions - Performance - Scalability - Level of Concurrency - Indexes Types - Storage Utilization - High Availability Copyright 2006 MySQL AB - Replication - Online Backups - Data Warehousing - Foreign Keys - Small Footprint - Row Level Locking - Embedded - Table Level Locking - Clustering The World’s Most Popular Open Source Database 10 Pluggable Storage Engine Architecture • Storage Engines are available on a per table basis • Changing from one storage engine to another can be done via a simple SQL command: ALTER TABLE mytable ENGINE=MyISAM; Innodb Copyright 2006 MySQL AB MyISAM The World’s Most Popular Open Source Database 11 Pluggable Storage Engine Architecture * * In MySQL 5.0 transactions are supported, however, the partial rollback of a transaction is not supported. Cluster supports the READ_COMMITTED, REPEATABLE_READ, and SERIALIZABLE transaction isolation levels. Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 12 Storage Engines – MyISAM Fast Facts • • • • • • • • • • • Default MySQL engine No practical limits on data storage Very efficient storage Easily handles high-speed data loads Has B-tree, R-tree, and Full-text Indexes Supported by special index memory caches Offers compressed data option Supports geospatial operations Uses table level locks Does not do transactions Backup/point-in-time recovery supported MySQL Server Best Use Cases • High-traffic Web sites • Data warehouses Copyright 2006 MySQL AB MyISAM The World’s Most Popular Open Source Database 13 Storage Engines – InnoDB Fast Facts • • • • • • • • • • • • Provides ACID transaction support 64TB data storage limit per tablespace Higher storage cost Slower data load speed than most other engines Offers MVCC/Snapshot read Has B-tree and clustered indexes Supported by special data & index memory caches Provides foreign key support Does not offer compressed data option Uses row level locks and has custom isolation levels Has crash recovery Backup/point-in-time recovery supported Best Use Cases MySQL Server Innodb • Online transaction processing applications Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 14 Storage Engines – Cluster (NDB) Fast Facts • • • • • • • • • • • Transaction support All data and index reside in main memory Memory limitation removed for tables in 5.1 High data load speed Offers MVCC/Snapshot read Has B-tree indexes Very fast primary key lookup capabilities Offers 99.999% uptime Shared nothing architecture Has high-speed API for access as well as SQL API Online backup/point-in-time recovery supported MySQL Server Best Use Cases • Highly available, always-on/up applications • Fast directory/key lookup applications Copyright 2006 MySQL AB Cluster (NDB) The World’s Most Popular Open Source Database 15 Storage Engines – Archive Fast Facts • • • • • • • • • • • New in 5.0 Provides automatic data compression Offers storage savings up to 80% No practical storage limit Fastest data load speed of any storage engine Offers MVCC/Snapshot read No index support Has special insert buffer for fast insert speed Only supports INSERT and SELECT operations Uses row level locks Backup/point-in-time recovery supported MySQL Server Best Use Cases • Historical data warehouses • Data archiving applications • Data auditing Copyright 2006 MySQL AB Archive Archive The World’s Most Popular Open Source Database 16 Storage Engines – Federated Fast Facts • • • • • • • • New in 5.0 Allows creation of one logical database from many physical Acts as “pointer” from one database to another target object No special middleware needed for remote data access Speed of operations depends on network/misc. factors Actions constrained by target engine object properties Security handled through federated table definition All SQL operations supported (as per target object) MySQL Server Best Use Cases • Distributed database environments • Data Mart environments Federated Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 17 Storage Engines – Other Fast Facts • • • • • • Memory tables reside in RAM; data is lost on shutdown Memory tables support both B-tree and hash indexes BDB tables offer transaction support with COMMIT/ROLLBACK Merge tables are collections of underlying MyISAM tables Merge tables offer one form of data partitioning Custom storage engines may be plugged into MySQL MySQL Server Best Use Cases • • • • Memory: fast lookups for data objects BDB: Online transaction processing Merge: Large databases with partitioned data Custom: special application situations Copyright 2006 MySQL AB Memory BDB Merge Custom The World’s Most Popular Open Source Database 18 Storage Engines – Coming Soon New Transactional Storage Engines: • Maria • Falcon • others still… Copyright 2006 MySQL AB MySQL Server Memory BDB Merge Custom The World’s Most Popular Open Source Database 19 MySQL Architecture – Parser/Optimizer Parser Optimizer Query Translation, Object Privilege Access Paths, Statistics • Validates user’s privileges on accessing database objects and executing SQL calls. • Converts all SQL calls to internal database language. • Decides how best to service user’s SQL request. • Supports all storage engines, which means no special coding per storage engine for particular needs. Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 20 Q&A Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 21 Query Trees & Query Execution An Alternative Query Execution Mechanism Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 22 Query Trees Alternative‘query tree’ for the query to list all the managers that work in the sales department: (job = ‘Manager’) (name=‘Sales’) (EMP emp.deptno = dept.deptno DEPT) (job = ‘Manager’) (name=‘Sales’) emp.deptno = dept.deptno EMP Copyright 2006 MySQL AB DEPT The World’s Most Popular Open Source Database 23 Query Trees • Tree nodes are atomic operations – – – – Project Restrict Join Sort • Each node has 0-2 inputs – From 0-2 relations directly – From 0-2 children Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 24 Query Trees SELECT Col1, Col2 FROM A JOIN (SELECT * FROM C WHERE ColB = 7) ON ColC WHERE A.Col3 > 14 Π Φ Σ Copyright 2006 MySQL AB Σ The World’s Most Popular Open Source Database 25 Optimization Techniques • Cost-Based – Statistics, statistics, statistics • Manual offline computation • Piggy back – Strategies • Even distribution • Heuristic – Knowledge of operations – “Works Best” • Semantic – Knowledge of schema – Decisions based on behavior/relationships Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 26 Heuristic Optimization Example (job = ‘Manager’) (name=‘Sales’) (emp.deptno = dept.deptno) (job = ‘Manager’) (name=‘Sales’) emp.deptno = dept.deptno X EMP EMP DEPT DEPT emp.deptno = dept.deptno (job = ‘Manager’) EMP Copyright 2006 MySQL AB (name=‘Sales’) Optimised Canonical Query DEPT The World’s Most Popular Open Source Database 27 Heuristic Algorithm void QueryTree::HOptimization() { SplitRestrictWithJoin(Root); SplitProjectWithJoin(Root); SplitRestrictWithProject(Root); pNode = FindRestriction(Root); while (pNode != 0) { PushRestrictions(Root, pNode); nNode = FindRestriction(Root); } pNode = FindProjection(Root); while (pNode != 0) { PushProjections(Root, pNode); nNode = FindProjection(Root); } pNode = FindNaturalJoin(Root); while (pNode != 0) { PushNaturalJoins(Root, pNode); nNode = FindNaturalJoin(Root); } BalanceJoins(Root); PruneTree(0, Root); } Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 28 Optimization Using Query Trees SELECT P.Name FROM Professor P, Teaching T WHERE P.Id = T.ProfId AND P. DeptId = ‘CS’ AND T.Semester = ‘F1994’ Name(DeptId=‘CS’ Semester=‘F1994’(Professor Id=ProfId Teaching)) Name DeptId=‘CS’ Semester=‘F1994’ Id=ProfId Professor Copyright 2006 MySQL AB Teaching The World’s Most Popular Open Source Database 29 Query Optimizer • Heuristic optimization eliminates most inefficiencies and generates near optimal queries which can be executed directly from the internal representation (query tree) without modification. • Test – Replace SELECT-PROJECT-JOIN optimizer in MySQL with Heuristic Query Optimizer. – To find out the results you need to… Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 30 Buy my book! Expert MySQL Since the enormous number of new features made available with MySQL release 5.0, MySQL has been gaining steam as a viable alternative to database behemoths like Oracle and IBM DB2. MySQL users now have the ability to extend MySQL with new SQL commands, optimize query execution, and embed MySQL within low-resource environments like embedded devices and kiosks. Expert MySQL, by Dr. Charles A. Bell, is the first book to examine these opportunities in detail, showing you how to wield maximum control over this powerful open source database. You’ll learn how to create your own custom storage handlers, ensuring maximum flexibility and speed within your specialized applications. You’ll also gain valuable insight into MySQL’s architecture and learn how to tweak its behavior through custom changes to the source code. Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 31 Q&A Copyright 2006 MySQL AB The World’s Most Popular Open Source Database 32