* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Power Point - home.apache.org
Survey
Document related concepts
Commitment ordering wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Oracle Database wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Functional Database Model wikipedia , lookup
Navitaire Inc v Easyjet Airline Co. and BulletProof Technologies, Inc. wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Serializability wikipedia , lookup
Clusterpoint wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Relational model wikipedia , lookup
Transcript
Open Source Database Rises to the Challenge Britt Johnston CTO NuSphere Agenda 2 Rise of Open Source Databases Traditional and Open Source Licensing Properties of Successful Community MySQL Gemini Project Future Trends © 2001, NuSphere Terminology 3 OSDB – Open Source Database OSI – Open Source Initiative OSD – Open Source Definition FSF – Free Software Foundation GPL – General Public License © 2001, NuSphere On Relational Databases… “Relational databases can handle no more than one hundred megabytes of data and maybe ten users.” Database Product Manager Digital Equipment Corporation 1985 4 © 2001, NuSphere Change is Constant Relational databases – Early debate on viability – Clumsy positioning with existing products – Tidal wave of acceptance OSDB is in a similar position – Reluctantly admit usefulness, but – “Commercial databases are required for backend functions.” 5 © 2001, NuSphere Rapid Evolution! Top 3 Selling Database Books 1. SQL for Dummies 2. MySQL 3. Oracle 8i Reference B&N Topsellers 10/28/00 References On the Web • • • • • • Oracle MySQL PostgreSQL SQL Server DB2 Interbase Google Searches 6 3.0 Million 2.3 Million 0.7 Million 0.6 Million 0.5 Million 0.1 Million 3.4 Million 2.9 Million 0.8 Million 0.6 Million 0.5 Million 0.1 Million 10/28/00 01/31/01 +13% +26% +14% © 2001, NuSphere 2000 – First Boxed Commercial Distributions – September 2000 First Open Source Database Summit – October 2000 7 Four companies dedicated to Open Source Database products were launched © 2001, NuSphere 2001 – 140,000 MySQL Books sell in 12 months Oracle: MySQL to Oracle Migration Kit MySQL Track at Major Conferences NuSphere Delivers Gemini Beta 8 © 2001, NuSphere Agenda 9 Rise of Open Source Databases Open Source Licensing Properties of Successful Community MySQL Gemini Project Future Trends © 2001, NuSphere The Open Source License Nine Major Requirements: Free redistribution – cannot require royalty Source code – source and binary distribution Derived works – allow distribution of changes Integrity of author’s code – may require patch No discrimination against persons or groups No discrimination against fields of endeavor License distributed – no new license required License not product-specific – extracted code ok No contamination of other software on same medium 1. 2. 3. 4. 5. 6. 7. 8. 9. Open Source Definition (Bruce Perens) www.opensource.org 10 © 2001, NuSphere MySQL License Prior to June 2000 – Not open source license June 2000 and Future – All releases under GPL 11 © 2001, NuSphere On Open Source… “We recommend that products near the end of their life go open source.” Gartner Group Analyst October 2000 12 © 2001, NuSphere Agenda 13 Rise of Open Source Databases Traditional and Open Source Licensing Properties of Successful Community MySQL Gemini Project Future Trends © 2001, NuSphere It’s Not Only About Technology Feature Wars – Oracle vs. Microsoft: competition not based on user need – Feature bloat will have long term impact 14 Fast, easy-to-use, integrated; with a clean programming model are most important © 2001, NuSphere Signs of Healthy Community 15 Mix of Church and State membership World-wide contributor community Active development process Rich collection of interfaces Support from other products Full service offerings Clear license terms © 2001, NuSphere Modular Architecture 16 Key to scalable community Drive rapid evolution Allow large contributions from multiple sources © 2001, NuSphere MySQL Modular Architecture Table Handler – Specialized storage for individual tables MyISAM – High Speed Read Mostly Heap – In Memory Tables Gemini – Large Scale OLTP – Row-level Locking, Transactions, Recovery More under development MySQL MyISAM 17 Heap Gemini © 2001, NuSphere Agenda 18 Rise of Open Source Databases Traditional and Open Source Licensing Properties of Successful Community MySQL Gemini Project Future Trends © 2001, NuSphere Gemini - Row Level Locking Gemini is NuSphere’s contribution to MySQL project – design targets: Multi-threaded engine Supports 10,000 concurrent transactions Sustains 1 billion tpd on single server Small footprint for PC class hardware SMP support for large systems Concurrent operations on parallel threads Familiar SQL standard programming model used by commercial applications today. – Table and Row-level Locking – Standard Isolation Levels 19 © 2001, NuSphere Gemini – Historical Roots Progress RDBMS MySQL Language Processor and Server Language Processor and Server Storage Engine Gemini Engine 20 © 2001, NuSphere Gemini – Historical Roots Progress RDBMS source for Technology #6 Relational DB Worldwide #1 Embedded DB Worldwide IDC Worldwide Database Market – May 2000 Proven Performance and Reliability Technology is Base for Gemini 21 Recovery, Locking, B-Tree, Concurrency, Cache, I/O and Transaction mechanisms © 2001, NuSphere Important Design Factors Gemini is designed to be: – – – – – 22 Database Schema Independent Record Format Independent Index Key Format Independent Server Architecture Independent Gemini API Closely Matches MySQL Table Handler API © 2001, NuSphere Gemini Properties Targeted squarely at OLTP model – Heavy concurrent update – Online maintenance operations Expands open source web platform – Backend database for e-commerce sites – Reliability with 10+ years of “experience” – Proven enterprise-class technology 23 © 2001, NuSphere Gemini Properties Multi-threaded storage manager – – – – Concurrent read and write operations Concurrent commit support Fine grained locking of internal structures Online Recovery from failed threads Scalable database cache – Dynamic data and index cache size – 128GB capacity (RAM limited) – LRU mechanism with index page priority 24 © 2001, NuSphere Gemini Properties - Transactions Support for ACID Transactions – – – – 25 Atomicity – All or nothing Consistency – Data in consistent state Isolation – Allow independence Durability – Effects persist always © 2001, NuSphere Gemini Properties - Transactions Support for 4 Standard Isolation levels – – – – Read uncommitted Read committed Repeatable read Serializable Table and row lock support – 6 mode lock manager (intent support) – 2 phase with automatic lock acquisition – Delegated delete locks 26 © 2001, NuSphere Transaction Isolation Levels Levels described in terms of possible anomalies – Dirty read - read data written by concurrent uncommitted transaction. – Non-repeatable reads - re-read data previously read and see data modified by another committed transaction – Phantom read - re-run same query and see additional rows inserted by another committed transaction. 27 © 2001, NuSphere Transaction Isolation Levels Level Dirty Read Read uncommitted Yes Read committed No Repeatable read No Serializable No 28 Non-Repeatable Read Yes Yes No No Phantom Read Yes Yes Yes No © 2001, NuSphere Durable Transactions Recovery Log DB on Disk • Previous Record • Transaction Notes • Update Notes Memory Copy • Updated Record 29 © 2001, NuSphere Rollback and Recovery Recovery Log DB on Disk • Previous Record • Transaction Notes • Update Notes Memory Copy • Updated Record 30 © 2001, NuSphere Scalability What is scalability, why does it matter? – Increase workload or hardware More work gets done (concurrency) Work gets done faster (response time) – Improve throughput with additional resources Bottlenecks can be removed – Minimal impact on response time when scaling 31 © 2001, NuSphere Scalability Architectural limits raised as high as possible – Goal: storage engine does not impose limits. – Available hardware and OS are limiting factors. – Example – Concurrent Users: 32 Demonstration 5,000 users Published limit is 10,000 Tested Limit is 32,000 Architecture limit is 4 billion © 2001, NuSphere Scalability A system may be limited by: – Number of disks and controllers – Number of open files, OS kernel – Memory Not the underlying database 33 © 2001, NuSphere Multi-Processor Support Several flavors of spin locks Can directly use 32 CPU SMP Hardware Alpha, IA32, IA64, PA-RISC, Power, Sparc Work with hardware vendors to tune chip specific resource locking primitives. – Non blocking, no unneeded system calls – Account for CPU cache characteristics – Instruction and Data Fence requirements 34 © 2001, NuSphere Gemini Crash Recovery Automated recovery and logging – Log is created at system startup – Space is reused automatically – Log is optionally removed at shutdown Asynchronous checkpoint support – Stable performance under load – Self tuning multi-threaded I/O subsystem 35 © 2001, NuSphere 24 X 7 Availability Powerful High Availability solution: – Automatic Crash Recovery Online in 15 to 30 seconds – Fail over Clusters Eliminate single points of failure – Flexible Backup Solutions Online Backup – non blocking Split Mirror Zero Impact Backup – Table and Site Replication 36 Support Server Farm Model © 2001, NuSphere Gemini – Ease of Use Familiar storage model used by MySQL MySQL native record format Specify table type to use Gemini tables CREATE TABLE … Programming model – – – – 37 TYPE = GEMINI; Table and row-level locking Statement atomicity Multi-statement ACID transactions Standard isolation levels © 2001, NuSphere Gemini – Current Status Beginning Formal Beta Check-in via MySQL community process Basic functions complete (insert, update, delete, select) for all types Active reliability testing – Can run same tests against multiple table types Benchmark work started – Target: High concurrency, heavy update – Goal: Fastest engine for transaction processing 38 © 2001, NuSphere Agenda 39 Rise of Open Source Databases Traditional and Open Source Licensing Properties of Successful Community MySQL Gemini Project Future Trends © 2001, NuSphere It’s All In The Community! A traditional software product rarely has a sustaining community outside the employees of the company. Look for existing communities and create or integrate missing technology. Contributors using community process? 40 © 2001, NuSphere Corporate IT Is Catching On Response to OSDB is no longer to challenge its viability. IT managers know they can get support and packaged distributions. OSDB is proven solution for wide range of web infrastructure. Gemini changes the rules for OLTP systems built with open source software. 41 © 2001, NuSphere Corporate IT Is Catching On New projects are looking at OSDB for significant aspects of a solution. Initial acquisition costs no longer a factor; IT can choose service level on perapplication level based on business needs. IT will increasingly go with an OSDB solution. 42 © 2001, NuSphere Questions… [email protected] www.nusphere.com