Download Relational Database Industry and Technology Trends

RDBMS Industry and
technology trends
Guy Harrison
Chief Architect, Database Solutions
Copyright © 2006 Quest Software
Agenda
• Review of market share and competitive
landscape
• Open source and disruptive technology
• Technical directions
– Grids & clusters
– Self-managing databases
– Application development technologies
• Industry trends:
– Security and compliance
– Outsourcing
– Globalization of data and the database
• Visions for the future of the DBMS
History of RDBMS competition
•
1990: Client server revolution
–
–
–
•
1996: Object Oriented Database distractions
–
–
•
Informix vs. IBM vs. Oracle
OODBMS vs. Object relational vs. relational
2000: Internet gold rush
–
–
–
–
•
Sybase vs. Ingres vs. Oracle
Multi threaded servers (SMP support)
Stored procedures/Client server capabilities
Internet and Java compatibility
Best of breed configurations with EMC, Solaris & Oracle
Infinite scale-up anticipated
Oracle price gouging creates some drift to IBM and SQL Server
2005: ROI/ TCO/ Compliance
–
–
–
–
Battle not over capability but over cost
SQL Server disrupts Oracle – but constrained by Windows OS market
Oracle disrupting IBM/MF and the high end via grid/RAC.
IBM pursues enigmatic Information As A Service (IAAS) strategy
Market share and competitive landscape
2005 RDBMS market share
17%
45%
Oracle Corp.
IBM
Microsoft Corp.
Others
17%
26%
26%
21%
Sybase Inc.
NCR Teradata
Source: IDC 2006
Progress Software Corp.
SAS Institute
0%
M ySQL
1%
Ingres Corp.
Fujitsu
12%
22%
13%
Revenues by platform
2005 RDBMS revenues by platform
Revenue ($M)
Other
Microsoft
Unix
Windows
Linux
Mainframe
Other
IBM
Oracle
0.00
500.00
1,000.00
1,500.00
2,000.00
2,500.00
3,000.00
Revenues by company size
80
SQL Server
70
Oracle
DB2
60
50
40
Less than
$100 million
Source: Forrester
$100 million $500 million to $1 billion to
to less than
less than
$10 billion
$500 million
$1 billion
DB2
SQL Server
Oracle
DB2
SQL Server
Oracle
Oracle
DB2
SQL Server
Oracle
DB2
0
SQL Server
10
DB2
20
SQL Server
Oracle
30
More than $10
billion
Market growth by platform – 2004 predictions
$4,500
$4,000
$3,500
Millions
$3,000
Windows
$2,500
Unix
$2,000
Linux
Mainframe
Other
$1,500
$1,000
$500
$0
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
Source: IDC
Market growth by platform – 2006 predictions
$14,000
$12,000
$10,000
Mainframe
$8,000
Unix
$M
Linux/other open source
Windows 32 and 64
$6,000
Linux+Unix
Other
$4,000
$2,000
$0
2005
2006
2007
2008
2009
2010
Market share conclusions and speculations
• Oracle continues to dominate non-mainframe RDBMS landscape.
• However, most shops support >1 RDBMS type (Oracle/SQLServer)
and DBA managers at least need to understand more than one
technology
• Growth of Windows as a server platform ensures a healthy growth
trend for SQL Server
• Similarly, Oracle stands to be the main beneficiary from the growth of
Linux
• No compelling reason to believe that either vendor is going to dominate
• Server platform choices are key in RDBMS vendor decisions and viceversa.
• The above ignores the possibility of sudden disruption….
Disruptive technology
• “Disruptive innovation” occurs when a technology or
a technical approach emerges that offers a radically
cheaper way of meeting a need.
– Lower cost alternatives to low-end utilization
– Lower cost creates new consumers
• Established players are motivated to move towards
the more profitable high-end of the market.
– Existing customers tend to demand features at the high end
– High end has higher profit margins
– High end is less effected by disruptive technology
• But as both the established and disruptive
technologies advance, the established technology
“overshoots” while the disruptive technology gains
the mainstream.
– See “The Innovators Dilemma”, Clayton Christensen, HUP
Disruptive Technology
Functionality
Functionality demanded at high end of market
Sustaining
Technology
Disruptive
Functionality demanded at low end of market
Technology
Time
The Innovators Dilemma, Clayton Christensen,
Harvard University Press
OSDBMS - MySQL
•
Advantages:
–
–
–
–
–
•
Huge install base
Many mission-critical deployments (Sabre, Yahoo, NASA, etc)
Critical part of the LAMP stack
• Well placed to leverage Linux server growth
• But don’t forget WAMP
Disruptive both as low-cost innovation and competing against nonconsumption
Providing 90s style RDBMS for free (internal) or <10% of Oracle cost
(commercial)
Challenges:
–
–
–
Attempts to monetize the install base not yet successful
• MySQL users are completely satisfied by the free offering
• Many OSS companies are disturbingly reminiscent of Y2000
Dot.coms (millions of non-paying customers; VC funded;
unproven business model)
Commercial vendors have all read “The Innovators Dilemma”
• All have a free version for entry level use
• Oracle aggressively counter-disrupting via strategic acquisition
Many competing demands on MySQL R&D
• Unlike Red Hat, MySQL don’t get the software for free
Industry trends - Outsourcing
• 15% of companies report they plan to outsource DBA roles
(Gartner 2004) b/c of:
– Reduced cost
– Difficulty maintaining expertise in house
• But BIG obstacles to widespread adoption:
– High risk (cost of database failure > savings from outsourcing)
– Security implications
– Quality of service
• Adoption is relatively narrow and shallow:
– Minority of companies (but tend to be large) outsourcing only routine DBA
activities
Industry trends - Security and compliance
• A “perfect storm” accelerated interest in Database
security:
– 9/11
– The legislative response to Enron et al
• Sarbanes-Oxley, HIPPA, VISA
– High profile database break-ins, slammer worm, etc
– Outsourcing (challenge of external DBAs).
• Industry responses:
–
–
–
–
–
Database Encryption
Vulnerability Assessment
Fine grained auditing
Intrusion detection and prevention
Separation of duties: Privileges to administer a database do
not automatically imply privilege to view or alter data
– Oracle leading in inbuilt security features
Industry Trend – Autonomic / “self managing” computing
• All vendors – especially Oracle – are motivated to
compete on ease of administration
– Oracle ADDM (Automatic Database Diagnostic Manager)
– SQL Server DTA (Database Tuning Advisor)
– UDB “Leo” (Learning Optimizer)
• Evolutionary changes for the DBA
• As “legacy” becomes automated, leading edge still
requires intensive manual administration
– Oracle 10g RAC, for instance
• Overall effect of automation will be to slightly reduce
DBA market growth and to shift demand to higher
end skills
Technical trends – grids / utility computing
• Computing resources (IO, storage, memory,
CPU) allocated on demand across the
enterprise
– Analogy to the electricity grid
• Economic benefits will be irresistible once the
technical challenges overcome.
• Grids have been viable only for CPU-bound
applications until recently
• To create a database-enabled grid we need:
– A way to shift CPU/memory (eg blades) efficiently
between databases
– A way to shift IO & storage efficiently between
databases
Grids, RAC and VMs
• Oracle RAC is a step towards CPU on demand for databases
– In some future release (possibly Oracle 11) blades will migrate between RAC
clusters on demand
• ASM provides a disk-grid solution
– Although there are non-Oracle technologies that can achieve this in a
heterogenous manner
• RAC and ASM are not quite there yet
– Nevertheless, RAC changes the economics of providing HA VLDB in a way that
competitors cannot currently address
• Virtualization offers an alternative utility computing vision
– Resources can be shifted between VMs on demand
– However, not able to migrate VMs across hosts instantaneously (so scalability
limited to size of single host)
– Databases currently perform poorly inside VMs
• Hypervisor technologies and Virtual-aware chipsets will possibly correct this
Technical trends – grids
Blade Farm
Blade Rack
RAC Instance
Blade Rack
RAC Instance
Blade Rack
RAC Instance
Blade Rack
RAC Instance
Blade Rack
Instance
RAC Instance
Blade Rack
RAC Instance
Instance
RAC
Blade Rack
RAC Instance
Instance
RAC
RAC Instance
Instance
RAC
Disk
Disk Farm
Farm (ASM)
(ASM?)
Disk
Disk
Disk
Disk
Disk
Disk
Disk
Disk
Disk
Disk
Disk
Disk
Disk
Disk
Disk
Disk
Disk
Disk
Disk
Technical trends – Object-Relational Mapping
• Increasing trend towards ORM
–
–
–
–
Dominant paradigm in Java already (Hibernate)
Increasing in OSS (Ruby on Rails)
Emerging in .NET
Increasing prevalence in OSS and .NET
environments (LINQ, Rails Active Record)
– Obfuscation of relational data in some cases (esp.
JDO)
• Harder to perform Business Intelligence and
Analytical processing
– Container generated SQL
• Harder to tune & debug – though often simpler
access paths
• Tendency towards over-simplified and/or
unnormalized data models to suit programming
models (ORMs tend to prefer single table
accesses)
• RDBMS vendors introducing “tune without
change” and “secondary optimizers” in response
Technology trends – Application development
• Multiple factors converging to reduce the
significance of stored procedures in modern
applications:
– No standardization in stored procedure languages
• Use of SPs increase RDBMS vendor lock-in
– ORM ignores SPs
– Packaged applications want to be heterogenous (except
for Oracle Fusion  )
– Middle tier a better choice for business logic
• However:
– We still see strong growth in the PL/SQL development
tools market
– Oracle is one of the big two ERP vendors and they don’t
want to be heterogenous
– Middle tier/OO languages (and programmers!) not
optimized for data access
Data volume growth
• Drivers
– Because we can
– Fine grained, real world data (RFID)
– Longer term retention policies (Sarb-Ox,
etc)
– Complex and unstructured data (Flickr,
YouTube)
Size of largest production database
Source: Forrester (DBMS Survey 2006)
More than
5 TB 21%
• Implications
– Scale out architectures increasingly more
attractive (unpredictable future demands)
– Demand for archiving solutions
– Suppresses disruptive effect of low end
vendors
Less than
100 GB 12%
100 GB to
499 GB 17%
2 TB to
5 TB 19%
1 TB to less
than 2 TB
7%
500 GB to
less than
1 TB 24%
Growing demand for data search and linking
• Islands of information need to be broken down
• Motivations for “globalising” and unifying data:
– RFID/Supply chain
– Web services/mash-ups/co-operative e-commerce
– Expectations raised by internet content search
• But:
– No clear technical solution
– Significant societal issues in respect of security and
privacy
Visions of the future of DBMS
– Larry Ellison:
• A single, global, logical (Oracle) instance/cluster
tied together with grids and data pump technology
• Both data and computing resources will be made
available across the network on demand
• Data sharing through consolidation
– Web services standards bodies:
• All interactions will occur through well-defined
Web Service interfaces utilizing specific
specifications such as WS-transaction, WSsecurity, etc.
• RDBMS is a local “persistence” store only
– Open Source/Web 2.0 community:
• Same as above, but “mash-ups” not Standardsbased WS
Visions of the future of DBMS
• Adam Bosworth (Google)
– Something radically different is going to emerge.
– Orders of magnitude more data is going to be stored
on the net in the near future and the expectation is
going to be that we can find and possibly modify it
from anywhere
– Formal, tightly coupled web services will give way to
simple, sloppy (maybe RSS based) protocols
– Matching will result not from a worldwide
standardization of data or a “semantic web”, but from
stupid but powerful algorithms (similar to Google spell
check)
– Centralized relational Databases of today are going to
seem “so twentieth century” and “one of those
technologies that never got the internet”
Visions of the future of DBMS
• Michael Stonebraker (creator of Posgres/Ingres) et
al
– One Size Fits All (OSFA) RDBMS architecture cannot meet
the needs of current and emerging demands:
• OLTP, stream processing (telco, web), OLAP/DW,
Unstructured, mobile, embedded, multi-dimensional, etc
– Specialized databases can generated 10x performance
improvements
• For instance, column based organization instead of row
based
– The competing demand of data integration will probably
preclude a re-fragmentation of data
– They suggest either:
• Hybrid system with various underlying storage engines
(a la MySQL)
• Data federation
• A new “from scratch” DBMS system with relational
features but also able to perform column based
operations
Questions?