Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Modern Database Administration
Document related concepts
Transcript
Modern Database Administration The Changing World of the DBA Craig S. Mullins Enterprise Data Management 5/6/99 1 Director, Technology Planning Modern DBA = Complexity The effect of Internet-enabled applications Storing code in your database using stored procedures, user-defined functions, and triggers Procedural DBA requirements Database growth due to object/relational features and data warehousing implementation The impact of ERP, CRM, and packaged applications Summary © Copyright 1999, BMC Software, All rights reserved. The Internet On Being an eDBA 3 5/6/99 It’s About Change... Source: Gartner Group © Copyright 1999, BMC Software, All rights reserved. Increasingly Complex Enterprise Infrastructure © Copyright 1999, BMC Software, All rights reserved. Internet Infrastructure Weaknesses Problem Symptom Effect Sporadic crashes for no apparent reason Unplanned outages Operators do not understand how to resolve problems Simple problems result in long outages Fragile IT mgrs. Must debug innocuous changes Long debugging cycles for new releases Vulnerable Viruses and bugs attack all systems at once Unreliable Complex Source: Forrester Research © Copyright 1999, BMC Software, All rights reserved. Systems must be rolled back to clean backups The Cost of an Outage © Copyright 1999, BMC Software, All rights reserved. E-Business Applications Fail to Deliver Service Because... Unplanned outages - Unplanned Planned and Outages Planned outages driven by problems driven by change Staff Shortages - 18 hr. workdays Short Implementation Times Management Application failure Executive Demands for Web Presence Maintenance Element Failure Management Tool Scalability Migrations Performance Unable to Keep Up With Own Scripts Version Capacity limitsWeb Load/Growth Unpredictable Management Transaction Unknown Success of Site Backouts Propagation Customized Environments Need Flexibility of Management © Copyright 1999, BMC Software, All rights reserved. Where is Application Downtime? 30% of Outages Planned vs. Unplanned Planned outages represents 70% of application downtime. Just 30% is due to unplanned outages and 50% of the unplanned downtime is due to problems during planned downtime. 70% of Outages © Copyright 1999, BMC Software, All rights reserved. Application Availability Planned Outages Unplanned Outages New Techniques Avoid downtime with automated tools “On the fly” operational tuning ALTER SYSTEM - Oracle9i SET SYSPARM - DB2 V7 Monitor performance across multiple platforms High speed transaction recovery Automated database change management Redundant systems HACMP © Copyright 1999, BMC Software, All rights reserved. Online and Real-time The need for more and more availability drives online and real-time maintenance The DBMS begins to allow for more changes to be made during normal operations The DBMS begins to gather statistics and performance metrics during normal operations ISVs deliver more online, real-time features and functionality that the DBMS does not yet deliver Less manual-intervention required © Copyright 1999, BMC Software, All rights reserved. Database Design & Web Time When the Web is involved everything becomes “rush-rush” - do it now! Don’t let database design suffer - take your time and do it right. Apps are temporary but data is forever! If you do not believe this, then consider: “How often has your organization reentered or re-keyed data into a new database when the data already exists elsewhere?” © Copyright 1999, BMC Software, All rights reserved. Prepare for Global Scope There is no downtime - it is always primetime somewhere in the world. Unpredictable volume. Who is accessing the database? Internal and External users Local, National, and International 24 x 7 x 365¼ 24 x 7 x 366 © Copyright 1999, BMC Software, All rights reserved. Web Impact on the DBA Where is the performance problem? Most experts agree that 75% to 80% of SQL-Net performance problems in relational applications is caused by poor SQL or gateway application code, but on the web . . . ISP connection 3GL ASP Java applet HTML Oracle Init.ora CGI instance database schema bridge/router/hub Java operating system DNS SQL network cabling hardware HTTP network software application code XML © Copyright 1999, BMC Software, All rights reserved. Java’s Popularity is Skyrocketing Java Software Market ($U.S. Millions) 2000 1750 1500 1250 1000 750 500 250 0 Source: IDC 1997 1998 1999 2000 © Copyright 1999, BMC Software, All rights reserved. 2001 2002 Why Java? Java can be used to make web pages active Portability - you can write an application on any platform and execute it on any platform (to which the JVM is ported). Java is most useful for applications that require a high productivity development environment and high portability for the resultant programs. If instead the application requires maximum performance, platform specific processing, or the use of robust compiler technology, then C++ (or another language) may be preferable to Java. Java is an interpreted language so performance can be a problem. © Copyright 1999, BMC Software, All rights reserved. Java and Databases: Two Methods JDBC Call level interface Enables Dynamic SQL for Java SQLJ Embedded SQL Enables Static SQL for Java © Copyright 1999, BMC Software, All rights reserved. Impact of Java on DBA Application tuning Must understand Java Java programmers tend to avoid the “relational” way of accessing data To provide guidance during design reviews How can you tune the application if you do not understand the language (Java)? Is the problem in the SQL or the application Optimizing SQL is not enough since it may be embedded in poor application code Must understand the SQL techniques used JDBC and SQLJ © Copyright 1999, BMC Software, All rights reserved. Microsoft .NET ... is a set of Microsoft technologies for connecting people, systems, and devices ... allows Internet Servers to expose functions to any client named as .NET web services … enables software to be delivered as a service over the web … is designed to let many different services and systems interact © Copyright 1999, BMC Software, All rights reserved. versus Java ...designed to enable applications to be deployed on any platform as long as they are written in Java .Net © Copyright 1999, BMC Software, All rights reserved. …designed to enable development in multiple languages as long as the application is deployed on Windows The Rush to XML 21 5/6/99 XML & Database Management XML stands for eXtensible Markup Language. Like HTML, XML is based on SGML (Standard Generalized Markup Language) HTML uses tags to describe the appearance of data on a page, whereas XML uses tags to describe the data itself, instead of just its appearance. Allows documents to be self-describing, through the specification of tag sets and the structural relationships between the tags. XML is actually a meta language - a language used to define other languages. © Copyright 1999, BMC Software, All rights reserved. Sample XML <!DOCTYPE CUSTOMER [ <!ELEMENT CUST (first_name, middle_initial, last_name, company_name, street_address, city, state, zip_code, country*)> <!ELEMENT first_name (#PCDATA)> <!ELEMENT middle_initial (#PCDATA)> <!ELEMENT last_name (#PCDATA)> <!ELEMENT company_name (#PCDATA)> <CUSTOMER> <!ELEMENT street_address (#PCDATA)> <first_name>Craig</first_name> <!ELEMENT city (#PCDATA)> <middle_initial>S.</middle_initial> <!ELEMENT state (#PCDATA)> <last_name>Mullins</last_name> <!ELEMENT zip_code (#PCDATA)> <company_name>BMC Software, Inc.</company_name> <!ELEMENT country (#PCDATA)> <street_address>2101 CityWest Blvd.</street_address> ] <city>Houston</city> <state>TX</state> <zip_code>77042</zip_code> <country>U.S.A.</country> </CUSTOMER> XML Document XML DTD © Copyright 1999, BMC Software, All rights reserved. XML to DBMS XML functionality being added to the DBMS Extender capabilities to allow a relational database to store XML XML document stored in a column – or – XML components stored as parts of multiple columns in multiple tables Formulate XML documents from existing tables Unload to XML Search XML documents text and sections XQuery capabilities © Copyright 1999, BMC Software, All rights reserved. Impact of XML on DBA Database definition A new way to create databases DTDs to build a database schema Create DTD from a database schema Interface to data modeling tools Database extenders XML defines the data it contains Does XML even belong in the database? New XML DBMS products? OO all over again? Over-enthusiasm! © Copyright 1999, BMC Software, All rights reserved. Web DBA Web-enabled DBA tools to administer databases on multiple platforms The web is also a good source of information on Oracle and DBA Mailing lists and Newsgroups Web Sites Portals (dbazine.com, searchdatabase.com) Oracle ISVs (BMC.com) Consultants (TUSC.com) © Copyright 1999, BMC Software, All rights reserved. Logic and the DBMS © Copyright 1999, BMC Software, All rights reserved. Triggers vs. Functions vs. Procs U P D A T E I N S E R T D E L E T E Code Code Code Triggers UDFs Stored Procedures Code Code Code © Copyright 1999, BMC Software, All rights reserved. function( ) if this then that else do this stuff return x end SQL Trigger Impact on DBAs Managing an active database environment Performance Implications Statement vs. “Row-at-a-time” Behavior Cascading triggers can change a lot of the database with one SQL modification Utilities do not activate triggers (loading data) Changing Triggers? – cannot ALTER What about data that already exists? Incompatible heterogeneous implementation Oracle: PL/SQL, SQL Server: Transact-SQL, DB2: SQL Procedures Language, ANSI: SQL/PSM © Copyright 1999, BMC Software, All rights reserved. Procedural DBA Duties Stored Procedures D E V E L O P Triggers Performance Monitors D E B U G Admin. Process Design Review Functions © Copyright 1999, BMC Software, All rights reserved. Oracle Role of the Procedural DBA On Call for DBCO Failures © Copyright 1999, BMC Software, All rights reserved. Data Keeps Growing © Copyright 1999, BMC Software, All rights reserved. “Universal” Data Complex Compound Docs Graphics Video Design Data Spatial Data Images Temporal Data Text Unstructured Structured Audio Existing Databases Seismic Data Simple Source: Gartner Group © Copyright 1999, BMC Software, All rights reserved. Impact on DBA How are complex data types: stored - physically on disk? accessed - by SQL in programs? Logged? optimized - by the DBMS or externally? administered (reorg, load, copy, etc.)? Can the database be administered or is it too large given current technology? How can database objects be generated? Current modeling tools up to the task? Component-driven plug-ins How do you maintain knowledgeable staff who understand your “customized” DBMS? © Copyright 1999, BMC Software, All rights reserved. Multimedia = Big Databases Object Typical Size HD TV 200 MB/second Feature-length, high-resolution movie 5-6 GB High-resolution video 3 GB/hour Feature-length movie 2 GB Video 1 GB/hour Radiologic image 40-60 MB Color image 20-40 MB Large image 200 KB-3 MB Text 30-40 KB/page Check image 45 KB © Copyright 1999, BMC Software, All rights reserved. Phenomenal Data Growth “Global 2000 companies double the amount of data they own every year, while the average dot-com’s data doubles every 90 days.” “Inside IBM we talk about 10 times more connected people, 100 time more network speed, a 1000 times more devices and a million times more data.” Mike Ruettgers, CEO of EMC Corp., Oracle Open World 2001 Lou Gerstner, CEO of IBM Corp., eBusiness Expo 2000 A recent research note from Giga Information Group estimates that there are about 201,000TB, or about 197 petabytes, on the planet. Of course, this is just an estimate that Giga deems to be accurate within an order of magnitude (that is, within a factor of 10). McKinsey & Company reports a CAGR of 76% for data storage. © Copyright 1999, BMC Software, All rights reserved. Relational Database Size Source: Gartner Group © Copyright 1999, BMC Software, All rights reserved. Database Size Issues Technology enables larger databases Web, multimedia, data warehousing, and data mining drive up database size Disk drives increase in capacity but speed of access does not keep up with capacity increases Cost of storage decreasing; so why not store more data? But... What data do users need to store? How long must it be maintained? What are they willing to pay? © Copyright 1999, BMC Software, All rights reserved. 0010 001010100 101011101011 101011010010101 01001001101010111000011 11001010100100101000100101 DBMS to Manage All Kinds of Data Federated DBMS • A federated approach allows the DBMS to manage data where it exists Spreadsheet © Copyright 1999, BMC Software, All rights reserved. DBMS to Manage All Kinds of Data Or… • An alternate approach “sucks” the non-relational data into the DBMS to be managed Spreadsheet © Copyright 1999, BMC Software, All rights reserved. Packaged Applications 41 5/6/99 ERP, CRM, and Packaged Applications Buy Versus Build ERP Human Resources Supply Chain Mgmt Finance CRM Front office Sales management Customer relationship management One-to-one marketing © Copyright 1999, BMC Software, All rights reserved. The Computing “Platform” Evolution “Customer Service Apps” This is the current battleground, NOT the DBMS 2000+ “Base” Applications The DBMS Network (LAN/WAN) Communications 60’s SAP, Oracle, Baan, PeopleSoft Oracle, DB2, SQL Server TCP/IP, Novell, WinNT SNA, TCP/IP, NFS The Operating System MVS, UNIX, WIN (NT,95) Hardware Proprietary (IBM+) to Open (Wintel, HP/Intel) © Copyright 1999, BMC Software, All rights reserved. ERP: Not Your Typical Databases Very large databases No concept of a data model Logical or physical Does not follow “practical, common wisdom” Application program-enforced referential integrity Hard to track users and transactions Backup and Recovery “strategy” - do it all © Copyright 1999, BMC Software, All rights reserved. With a Packaged Application DBA is not necessarily easier with a packaged app Don’t think you can “phone it in” with a packaged application like SAP or Peoplesoft Less information to work with Tuning via index manipulation Work with package’s admin tools? In many cases it is required but the functionality of the tools is weak © Copyright 1999, BMC Software, All rights reserved. So, Then What’s It Like to be a DBA, Today?? 46 5/6/99 The Database Environment Adabas, Teradata, MySQL, Supra, Compaq Non-Stop SQL, Ingres, IDMS, IMS, Datacom, PostgreSQL, others A lot of choices! Vendor, platform, and architecture of DBMS Unix AIX Sun Solaris HP-UX Linux… others Windows NT / 2000 / XP MVS, OS/390, z/OS Others (VSE, VMS, MPE, OS/400, etc.) Desktop OS Windows 98 / ME / XP Linux Mac? © Copyright 1999, BMC Software, All rights reserved. Enterprise - Parallel Edition Departmental Personal Mobile (PDA) Enter the DBA Overworked Curmudgeons over 200,000 new DBA positions in next 4 years 15% turnover rate for DBAs DB2 Source: Butler Group © Copyright 1999, BMC Software, All rights reserved. The DBA must be a “Jack of all Trades” OS/390 VB Oracle SQL*Net Linux gateway ISP DB2 C++ SQL V$ Tables App Server application code Unix Informix MQ Java applet bridge/router/hub CGI TCP/IP VTAM SQL Server CICS connection Tuxedo ASP Java DNS 3GL COBOL network software JCL HTML init.ora database schema operating system HTTP XML network cabling © Copyright 1999, BMC Software, All rights reserved. Windows hardware Qualified Professionals Those with a “do it now” mentality sometimes believe “we don’t need a DBA” – web thinking Demand outweighs supply DBAs are in demand As current professionals retire who will replace them? As demand grows, will the number of DBAs grow to fill the need? © Copyright 1999, BMC Software, All rights reserved. © Copyright 1999, BMC Software, All rights reserved. Impact on the DBA Unrealistic Expectations Impossible to master everything Education is the first thing cut! Impossible to specialize in a heterogeneous shop Reactive mode is encouraged Even though proactive mode is optimal Who looks for more problems when they don’t have enough time in the day to solve the problems of those complaining the loudest. DBA uses YBWJ method © Copyright 1999, BMC Software, All rights reserved. So… The job of database administration is getting increasingly more difficult as database technology rapidly advances adding new functionality, more options, and more complex and complicated capabilities... © Copyright 1999, BMC Software, All rights reserved. So we Need More Automation Software that Functions like a “Consultant” Collect Analyze Act Data Information Recommendation Data Data Data System Object © Copyright 1999, BMC Software, All rights reserved. Application Software Scrubbing Bubbles Intelligent automation of DBA tasks because no one has all the skilled resources they need frees up more DBA time They work hard so you don’t have tooooo… © Copyright 1999, BMC Software, All rights reserved. Contact Information BMC Software http://www.bmc.com Craig S. Mullins Director, EDM Technology Planning [email protected] http://www.craigsmullins.com http://www.craigsmullins.com/dba_book.htm © Copyright 1999, BMC Software, All rights reserved.
