* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Essential Database Maintenance Essential Database Maintenance
Survey
Document related concepts
Entity–attribute–value model wikipedia , lookup
Serializability wikipedia , lookup
Microsoft Access wikipedia , lookup
Oracle Database wikipedia , lookup
Ingres (database) wikipedia , lookup
Concurrency control wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Relational model wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
ContactPoint wikipedia , lookup
Database model wikipedia , lookup
Transcript
Essential Database Maintenance Melbourne SQL Server User Group: October 2009 Paul S. Randal Kimberly L. Tripp Kimberly L. Tripp Author/Instructor: Paul S. Randal Consultant/Trainer/Speaker/Author Managing Director, www.SQLskills.com @ Q Email: [email protected] Blog: http://www.SQLskills.com/blogs/Paul Twitter: @PaulRandal Contributing Editor of TechNet Magazine, author of the SQL Q&A column, bi‐monthly articles on DBA topics, multiple 2008 whitepapers 5 years at DEC responsible for the VMS file‐system and check/repair Almost 9 years as developer/manager in the SQL Storage Engine team through August 2007, ultimately responsible for Core Storage Engine Wrote DBCC INDEXDEFRAG DBCC SHOWCONTIG and rewrote DBCC Wrote DBCC INDEXDEFRAG, DBCC SHOWCONTIG and rewrote DBCC CHECKDB and repair for SQL Server 2005 Regular presenter at worldwide TechEds and other conferences on disaster recovery, HA, maintenance, and internals Wrote and presented SQL Server 2008 training for Microsoft Course author/instructor for Microsoft Certified Master qualifications Co‐Chair of the SQL Connections conference SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 1 Author/Instructor: Kimberly L. Tripp Consultant/Trainer/Speaker/Writer Founder, SYSolutions, Inc. (www.SQLskills.com) e‐mail: [email protected] blog: http://www SQLskills com/blogs/Kimberly blog: http://www.SQLskills.com/blogs/Kimberly Microsoft Regional Director and SQL Server MVP Writer/Editor for SQL Magazine www.sqlmag.com Author/Instructor for SQL Server courses: Designing for Performance, Indexing for Performance, Maintenance, Disaster Recovery/HA Author/Manager of SQL Server 2005 Launch Content, Co‐author/Manager for SQL Server 2008 Jumpstart Content, Author/Speaker at TechEd, SQL Connections, SQLPASS, ITForum, Conference Co‐chair for SQL Connections Author of several SQL Server Whitepapers on MSDN/TechNet: Partitioning, Snapshot Isolation, Manageability and SQLCLR for DBAs Partitioning Snapshot Isolation Manageability and SQLCLR for DBAs Author/Presenter for more than 25 online webcasts on MSDN and TechNet (two series and other individual webcasts) Co‐author MSPress Title: SQL Server 2000 High Availability, SQL Server 2008 Internals, and the SQL Server MVP Project Presenter/Technical Manager for SQL Server 2000 High Availability DVD and various 2005/2008 HOL DVDs I still love this stuff… don’t hesitate to ask questions! SQLskills.com Database Maintenance The Top‐Ten List… 10.Data file management SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 2 Database Structure Up to 32,767 databases per instance Database Up to 32,767 files per database (Total is for both data and log files) Minimum of 2 files – one for data, one for log MDF = Primary (MAIN) Data File NDF = Non-Primary Data File Data file *.mdf (1) *.ndf (0-n) Log file *.ldf (1-n) LDF = Log Data File Should be only 1 log file Just because you can, doesn’t mean you should… SQLskills.com Database Maintenance Capacity Planning Data ● ● ● ● Log ● ● ● Rough estimate of current tables – be sure to calculate: Base tables (8096/average row size ‐> rows/page * rows ‐> pages * 8K LOB data (XML, images (varbinary(max)), etc. SQL Server 2008 FILESTREAM (instead of LOB inside the database) Estimate growth for 1‐2 years (are you going to keep all of this data and/or periodically archive) Indexes – estimate at 2‐3 times your data Space for performance (e.g. maintenance) 1.5 times your largest table as a rough estimate R d l Recovery model Disaster recovery strategy (frequency of transaction log backups) Workload type – and large transactions (log cannot be cleared until they complete) SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 3 Summary: Optimizing Data Files Defrag the physical disks Effective RAID array configuration Pre‐allocate to a reasonable initial size (minimal amount of capacity planning) Don’t let auto‐growth get out of control (monitoring) Turn on instant initialization Turn off auto shrink (and use shrink sparingly) Turn off auto‐shrink (and use shrink sparingly) While a lot of these will help for the log as well, there are more important things to be aware of IN the data portion – tables and indexes ● Index fragmentation SQLskills.com Database Maintenance For more info Blog posts on internals: ● ● ● ● ● Inside the Storage Engine: Anatomy of a record (http://www.sqlskills.com/blogs/paul/2007/10/01/InsideTheSt orageEngineAnatomyOfARecord.aspx) f d ) Inside the Storage Engine: Anatomy of a page (http://www.sqlskills.com/blogs/paul/2007/10/03/InsideTheSt orageEngineAnatomyOfAPage.aspx) Inside the Storage Engine: Anatomy of an extent (http://www.sqlskills.com/blogs/paul/2007/10/03/InsideTheSt orageEngineAnatomyOfAnExtent.aspx) Inside the Storage Engine: GAM, SGAM, PFS, and Other Allocation Maps Allocation Maps (http://www.sqlskills.com/blogs/paul/2008/03/14/InsideTheSt orageEngineGAMSGAMPFSAndOtherAllocationMaps.aspx) Inside the Storage Engine: IAM pages, IAM chains, and allocation units (http://www.sqlskills.com/blogs/paul/2007/10/04/InsideTheSt orageEngineIAMPagesIAMChainsAndAllocationUnits.aspx) SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 4 The Top‐Ten List… 10.Data file management 9. Transaction log file management Transaction log file management SQLskills.com Database Maintenance How the Transaction Log Works (1) virtual log file 1 virtual log file 2 virtual log file 3 virtual log file 4 virtual log file 5 inactive virtual log file inactive virtual log file active virtual log file active virtual log file inactive/unused virtual log file min LSN (start of oldest uncommitted transaction) max LSN (last log record written) Once a VLF no longer contains log records that are required, it can be cleared (sometimes called truncated) This is done by a log backup in FULL or BULK_LOGGED recovery d l db h k i i SIMPLE d l models, and by a checkpoint in SIMPLE recovery model All that happens is that a VLF is marked as inactive Nothing is zeroed, removed, overwritten (“clear” is a misnomer) The log file size does NOT change (“truncation” is a misnomer) Only whole VLFs are marked as inactive SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 5 Minimize Impact of Logging Log cannot clear until the transactional information is inactive Restart recovery will take longer if there were a lot of long i i di h i f f il running transactions pending at the time of failure Always try to avoid long running transactions Could also be: Log backup, database mirroring, transactional replication, log shipping, full/differential backup, change‐data‐capture in 2008 Check log_reuse_wait_desc in sys.databases for g the reason the log can’t be cleared SQLskills.com Database Maintenance How the Transaction Log Works (2) virtual log file 1 virtual log file 2 virtual log file 3 virtual log file 4 virtual log file 5 active virtual log file active virtual log file active virtual log file active virtual log file active virtual log file min LSN (start of oldest uncommitted transaction) max LSN (last log record written) What if all VLFs are active and more log records are written? The log will auto‐grow (if configured) More VLFs will be added and log records will be written into them Log auto grow means a pause while the new area is zeroed Log auto‐grow means a pause while the new area is zeroed If the log cannot auto‐grow, write activity stops Until the log is grown, another log file is added, or some action is taken to clear the log Avoid this situation by allowing the log to clear! SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 6 If the Log File Fills Up… Check in sys.databases Take whatever corrective action can be done: Take a log backup Kill long‐running transaction Alter database mirroring or replication to allow the log to clear Grow the existing log file Add another log file Switch to the SIMPLE recovery model S i h h SIMPLE d l Logged operation in 2000, not logged in 2005+ Take preventative action to stop it happening again SQLskills.com Database Maintenance Log File Considerations (1) Only ONE log file necessary Log activity is NOT parallelized over multiple files so NO performance gain from multiple files Only time another log file may be needed is if the log file runs out of space and a backup/auto‐grow cannot occur Isolate the log file from the data files to avoid disk head contention and performance issues Choose an appropriate RAID level RAID 5 is not recommended RAID 1 is good, RAID 10 is better SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 7 Log File Considerations (2) Pre‐allocate the log to an appropriate size What is ‘appropriate’? Depends on your workload, recovery model, log backup schedule, HA technologies Log file cannot be instant initialized Creation/growth takes time because of zero‐initialization Monitor size to avoid ‘runaway log file’ Allow auto‐grow ‘just in case’ but do not rely on it Every auto‐grow adds more VLFs – balance between too few and too many Too few means each will be very large and affect log clearing Grow/auto‐grow in no larger than 8GB chunks (i.e. 16 x 512MB VLFs) SQLskills.com Database Maintenance For more info Blog posts on the transaction log: 8 Steps to better Transaction Log throughput (http://www.sqlskills.com/BLOGS/KIMBERLY/post/8‐Steps‐to‐better‐Transaction‐Log‐ throughput.aspx) Transaction Log VLFs ‐ too many or too few? (http://www.sqlskills.com/BLOGS/KIMBERLY/post/Transaction‐Log‐VLFs‐too‐many‐or‐ too‐few.aspx) Database Maintenance Best Practices Part III ‐ TLog Maintenance (http://www.sqlskills.com/BLOGS/KIMBERLY/post/Database‐Maintenance‐Best‐ Practices‐Part‐III‐Transaction‐Log‐Maintenance.aspx) Understanding backups and log‐related Trace Flags in SQL 200x (http://www.sqlskills.com/BLOGS/KIMBERLY/post/Understanding‐the‐Transaction‐ log.aspx) Misconceptions around the log and log backups (http://www.sqlskills.com/BLOGS/PAUL/post/Misconceptions‐around‐the‐log‐and‐log‐ backups‐how‐to‐convince‐yourself.aspx) TechNet Magazine article Understanding Logging and Recovery in SQL Server (February 2009) (http://technet.microsoft.com/en‐us/magazine/2009.02.logging.aspx) SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 8 The Top‐Ten List… 10.Data file management 9. Transaction log file management Transaction log file management 8. Tempdb SQLskills.com Database Maintenance tempdb is a Special Case tempdb has always been used for: Worktables (sort, hash tables, order by, …) Query execution operations spilling out to disk User created temporary options – table variables (@) and temp tables (# and ##) DBCC CHECKDB processing of user databases tempdb in SS 2005+ is used by many new features Online index operations MARS (multiple active results sets) MARS (multiple active results sets) Snapshot isolation Triggers (INSERTED and DELETED tables) SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 9 Contention in tempdb Tempdb is very susceptible to contention issues because: there’s only one tempdb per instance to support all user d t b database operations ti Some query workloads cause multiple concurrent threads to repeatedly create/drop small temp tables Causes huge amounts of latch contention on allocation bitmaps and system catalogs SGAM page to manipulate mixed extents p g p g ( ) PFS page to allocate/deallocate pages (IAM and data) System catalogs to record Contention occurs in all versions, but worse in SQL Server 2000 (This can sometimes (rarely) happen in user databases with VERY high‐end workloads) SQLskills.com Database Maintenance tempdb Allocation in 2005+ Tempdb allocation algorithms were tweaked in 2005 There is now a cache of temp tables When a temp table is dropped, a single data page, single IAM page, and it’s metadata entry remain The next temp table creation pulls one out of the cache instead of having to create it from scratch Alleviates some contention, but not a very big cache Tempdb contention can (and does) still happen in 2005 and 2008 SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 10 Alleviating tempdb Contention Trace flag 1118 (discussed in KB 328551) turns off the use of mixed extents in tempdb ONLY Removes SGAM contention Removes SGAM contention Still may be needed in 2005+, but not as much as on 2000 Creating multiple data files to spread the contention For 2000, # data files should be equal to # processor cores For 2005+, you can still do 1:1 but you may not need to Start with # data files = ¼ to ½ # processor cores and add more if needed Anecdotal evidence that going beyond 8 data files in 2005+ gives no more gain pp y g Note: this does not apply to log files Separate tempdb data and log files from other databases SQLskills.com Database Maintenance For more info Blog posts on tempdb: Understanding TempDB, table variables v. temp tables and Improving throughput for TempDB throughput for TempDB (http://www.sqlskills.com/BLOGS/KIMBERLY/post/Understanding‐TempDB‐table‐ variables‐v‐temp‐tables‐and‐Improving‐throughput‐for‐TempDB.aspx) Changing Database Collation and dealing with TempDB Objects (http://www.sqlskills.com/blogs/kimberly/post/Changing‐Database‐Collation‐and‐ dealing‐with‐TempDB‐Objects.aspx) Comprehensive tempdb blog post series (http://www.sqlskills.com/BLOGS/PAUL/post/Comprehensive‐tempdb‐blog‐post‐ series.aspx) Misconceptions around TF 1118 (http://www.sqlskills.com/BLOGS/PAUL/post/Misconceptions‐around‐TF‐1118.aspx) Whitepapers on tempdb: Working with tempdb in SQL Server 2005 (http://www.microsoft.com/technet/prodtechnol/sql/2005/workingwithtempdb.mspx ) SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 11 The Top‐Ten List… 10.Data file management 9. Transaction log file management Transaction log file management 8. Tempdb 7. Are your indexing strategies working? SQLskills.com Database Maintenance Indexing Best Practices Don’t go crazy adding indexes Just because you can index every column (and/or INCLUDE every column) – INCLUDE l ) doesn’t mean you should! d ’t h ld! Start by setting up effective base table structures Continue with workload analysis to determine workload characteristics and query priority... Over‐indexing can be worse than under indexing Indexes that are not maintained can be more trouble in the long run (be sure to automate index maintenance) g ( ) General indexing trends/strategies A narrow index has very few uses… A wider index has many more uses – you might be able to make some of your existing indexes more useful by just making them wider (ok, you can’t directly add columns but you can create new indexes and then drop the ones you’re not using) SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 12 Finding the Right Balance Effective Base Table Strategies Determine primary usage patterns of the table ● OLTP – fewer indexes ● OLAP OLAP – more and wider indexes more and wider indexes Create the clustering key (choose a GOOD one!) Create constraints – primary key and alternate/candidate keys Manually add indexes to foreign key columns Capture a workload(s) and analyze with Database g Tuning Advisor Over time (and based on workload analysis), add specific [wider/covering] indexes for low selectivity/high‐priority queries to help improve SARGs, joins, aggregations… SQLskills.com Database Maintenance Finding the Right Balance Good Clustering Key Choices Identity column Composite key of (date, identity) ● Not date alone as it Not date alone as it’ss rarely unique rarely unique GUID − Populated by client‐side call to .NET client guid generator NOT as the CL key but OK as a non‐clustered primary key − Populated by server‐side newid() function No pattern Populated by NEW server‐side newsequentialid() opu ated by se e s de e seque t a d() function Creates a better (but not ideal) pattern Key points: Key points: unique, narrow, static and ever‐ increasing! SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 13 Finding the Right Balance Workload analysis and further tuning… Get rid of any unused indexes before you do anything… Prune redundant indexes Remove unused indexes (sys.dm_db_index_usage_stats) Remove unused indexes (sys dm db index usage stats) 2. Verify the health of the existing (and useful) indexes… 1. Determine the level of fragmentation 1. 1. 2 2. (sys.dm_db_index_physical_stats) Determine splits/latch waits (sys.dm_db_index_operational_stats) Add indexes to improve poorly performing queries? 1. Ask DTA what it thinks? 2. Ask SQL Server what it thinks? Q 2. 3. 1. 2. 3. SQL Server 2005: DMVStats, Performance Dashboard Reports (SP2+), RML Utilities from PSS SQL Server 2008: [Performance] Data Collector Both SQL 2005/2008: DMV queries (missing index DMVs) Bart Duncan’s blog: (search “Bart Duncan missing”) SQLskills.com Database Maintenance Finding the Right Balance Effective Indexing Strategies Are you done? Are you done? Optimizer needs good statistics Data access needs better data contiguity Data access needs better data contiguity You need more maintenance… SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 14 For more info Blog categories (each with multiple posts): ● Indexes ((http://www.sqlskills.com/blogs/kimberly/category/Indexes.aspx) ttp // sq s s co /b ogs/ be y/catego y/ de es asp ) ● Fillfactor (http://www.sqlskills.com/blogs/kimberly/category/FILLFACTOR.aspx) ● Indexes from Every Angle (http://www.sqlskills.com/blogs/paul/category/Indexes‐From‐Every‐ Angle.aspx) Whitepapers on indexes: ● Online Indexing Operations in SQL Server 2005 (http://www.microsoft.com/technet/prodtechnol/sql/2005/onlineindex.msp (http://www microsoft com/technet/prodtechnol/sql/2005/onlineindex msp x) ● Improving Performance with SQL Server 2005 Indexed Views (http://www.microsoft.com/technet/prodtechnol/sql/2005/impprfiv.mspx) SQLskills.com Database Maintenance The Top‐Ten List… 10.Data file management 9. Transaction log file management Transaction log file management 8. Tempdb 7. Are your indexing strategies working? 6. Statistics SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 15 Statistics: Why are they important? For indexes to be used, the optimizer needs to be able to estimate their cost (to choose the lowest expected cost) expected cost) Cost‐based optimization is done by assessing the usefulness of an index by looking at: The query submitted The conditions used (search arguments, join conditions, group by, order by, etc…) The selectivity of the data (based on the arguments) Selectivity is determined by statistics S l i i i d i db i i Statistics on indexes are of primary importance Statistics on the secondary columns of indexes can provide some interesting uses… SQLskills.com Database Maintenance Statistics: When are they created? Statistics are automatically created For all indexes For other columns, when the database option auto For other columns when the database option “auto create statistics” is ON AND when the optimizer thinks that statistics would be a good idea Statistics can be created manually sp_createstats Using CREATE STATISTICS Tips Leave the database option: auto create statistics on Leave the database option: auto create statistics on Add more statistics SQL Server 2000 supports 249 total indexes/stats per table SQL Server 2005 supports 2,000 statistics per table SQL Server 2008 supports 10,000 statistics per table SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 16 sp_createstats sp_createstats @indexonly = 'indexonly', @fullscan = 'fullscan', @norecompute = 'norecompute' @indexonly: only create statistics for the d l l f h secondary columns of indexes. This can help to make non‐clustered indexes more useful. @fullscan: requires more time but will create more accurate statistics. If off‐hours this is a good idea @norecompute: the statistics will NOT get @norecompute: the statistics will NOT get automatically updated as distribution of data changes. Generally, not recommended. Recommendation: sp_createstats 'indexonly', indexonly', 'fullscan 'fullscan' ' SQLskills.com Database Maintenance Statistics: What if the data changes? Automatically updated If auto update statistics is ON (for both the DB and the index wasn’t created with “norecompute”) th i d ’t t d ith “ t ”) If a percentage of the data changes Complete details in TechNet whitepaper Manually update statistics For highly volatile tables where distribution isn’t changing significantly and you see a lot of “statistics” events Turn off “auto update” OR turn off auto update by adding STATISTICS_NORECOMPUTE on the index definition (better control) Executing UPDATE STATISTICS SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 17 Auto Update Statistics SQL Server 7.0 Invalidated when sysindexes.rowmodctr reached U d t d h i lid t d Updated when invalidated SQL Server 2000 Invalidated when sysindexes.rowmodctr reached Updated when needed SQL Server 2005 Invalidated when sysrowsetcolumns.rcmodified reached Updated when needed SQL Server 2008 Invalidated when sysrscols.rcmodified reached Updated when needed SQLskills.com Database Maintenance For more info Blog posts on statistics: ● Auto update statistics and auto create statistics ‐ should you leave them on and/or turn them on?? them on and/or turn them on?? (http://www.sqlskills.com/blogs/kimberly/post/Auto‐update‐statistics‐and‐ auto‐create‐statistics‐should‐you‐leave‐them‐on‐andor‐turn‐them‐on.aspx) ● Index rebuilds depend on stats, which are updated by index rebuilds?!? (http://www.sqlskills.com/blogs/paul/post/Index‐rebuilds‐depend‐on‐stats‐ which‐are‐updated‐by‐index‐rebuilds!.aspx) Whitepaper on statistics: ● Statistics Used by the Query Optimizer in Microsoft SQL Server 2005 Statistics Used by the Query Optimizer in Microsoft SQL Server 2005 (http://www.microsoft.com/technet/prodtechnol/sql/2005/qrystats.mspx) Chapter of interest: ● Chapter 6: Index Internals and Management (Microsoft® SQL Server® 2008 Internals (Pro ‐ Developer) ) SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 18 The Top‐Ten List… 10.Data file management 9. Transaction log file management Transaction log file management 8. Tempdb 7. Are your indexing strategies working? 6. Statistics 5. Index fragmentation SQLskills.com Database Maintenance Fragmentation Defined Two broad types: g Internal fragmentation Wasted space on data/index pages Sometimes called ‘physical’ fragmentation Logical fragmentation Out‐of‐order pages within an index leaf level affecting range‐scan performance Both can have very detrimental effects on Both can have very detrimental effects on performance SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 19 Page Density (Internal Fragmentation) Page fullness is below the required fillfactor More pages required to hold the same rows How does this happen? Poor schema design leading to very large rows and wasted space on each page Random deletes leaving space on pages Page splits Effect is: Increased IOs for range scans I d IO f Greater memory usage if most of the index is memory resident Performance must be decreased because of page splits!! SQLskills.com Database Maintenance What Causes Fragmentation? Schemas/workloads that cause page splits on densely packed pages GUID as high‐order key (or any other random key) Can even affect nonclustered indexes Updates to variable‐length columns Badly configured FILLFACTOR Wide schemas that only fit a few records per page and waste space E.g. a fixed‐size 5000 byte row = 3000 bytes lost per page! Real‐world example: Social networking site that has a homepage comments table with the member ID as the high‐order key Needs fragmentation removed twice a day! SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 20 Detecting Fragmentation Keys to success are: Knowing which indexes to look at Which are used for range scans? Which have very low page density? Knowing which options to use of the various methods Knowing how to interpret the results Use sys.dm_db_index_physical_stats DMV in 2005 Use DBCC SHOWCONTIG in 2000 Still there in 2005 but deprecated Don’t just rebuild all indexes every day! SQLskills.com Database Maintenance How to Correct Fragmentation? 3 realistic choices CREATE INDEX WITH DROP_EXISTING Rebuild the index – ALTER INDEX … REBUILD Defrag the index – ALTER INDEX … REORGANIZE Most people rebuild or reorganize Trade‐offs between the two methods Log space required Di k i d Disk space required Online vs. offline Time to completion Interruptible without loss of work SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 21 Index Rebuild Misconceptions Rebuilding a clustered index rebuilds all nonclustered indexes No, only in 2000 and before for non‐unique clustered indexes There was a bug in 2000 SP1 that *did* There was a bug in 2000 SP1 that did rebuild always rebuild always Index rebuild pre‐allocates the necessary space No, it allocates space as it goes so may run out Rebuilding in BULK_LOGGED reduces the size of the log backups No, only the size of the transaction log. Remember how log backups work after a minimally‐logged operation? Index rebuilds in a multi‐file filegroup only use a single file Index rebuilds in a multi file filegroup only use a single file No, the allocations will follow regular round‐robin proportional fill Online index build doesn’t take any locks at all No, online is a slight misnomer SQLskills.com Database Maintenance For more info Blog categories of interest: Fragmentation (http://www.sqlskills.com/BLOGS/PAUL/category/Fragmentation.aspx) Webcasts on Index Fragmentation: MSDN Webcast Series: Part 5: New Features in Indexing and Index Maintenance Best Practices (Level 200) (http://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?EventID=1032278599) TechEd 2005: Index Defragmentation with SQL Server 2005 (Level 300) (http://www.microsoft.com/emea/spotlight/sessionh.aspx?videoid=30) Whitepaper on Fragmentation (still relevant for concepts even though written ffor SQL Server 2000): Microsoft SQL Server 2000 Index Defragmentation Best Practices (http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/ss2kidbp.mspx) Chapter of interest: Chapter 6: Index Internals and Management (Microsoft® SQL Server® 2008 Internals (Pro ‐ Developer) ) SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 22 The Top‐Ten List… 10.Data file management 9. Transaction log file management Transaction log file management 8. Tempdb 7. Are your indexing strategies working? 6. Statistics 5. Index fragmentation 4 Detecting corruption 4. D t ti ti SQLskills.com Database Maintenance I/O Errors Three types 823 – a hard I/O error 824 824 – a soft I/O error a soft I/O error 825 – a read‐retry error Nice error messages in 2005+ Msg 824, Level 24, State 2, Line 1 SQL Server detected a logical consistency‐based I/O error: incorrect checksum (expected: 0x7232c940; actual: 0x720e4940). It occurred during a read of page (1:143) in database ID 8 at offset 0x0000000011e000 in file 'c:\sqlskills\broken.mdf'. Additional messages in the SQL Server error log or system event log may provide more detail This is a severe error condition that system event log may provide more detail. This is a severe error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB). This error can be caused by many factors; for more information, see SQL Server Books Online. Logged in msdb..suspect_pages Input into single‐page restore operations SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 23 Page Checksums (1) Per‐page checksum Written as last thing SQL Server does on a physical write Checked as first thing SQL Server does on a physical read Provides the “smoking gun” that the error is not due to SQL Server as it detects all changes to a page Superset of torn‐page detection On by default for new databases in SQL Server 2005+ Checksum failures result in an 824 error CPU overhead of approx 1‐2% on heavy load Error‐detecting, not error correcting SQLskills.com Database Maintenance Page Checksums (2) Checked when: Page is read normally Page is read during DBCC CHECKDB Page is read during BACKUP WITH CHECKSUM Page is contained within a checksum’d backup There are two problems though: Not available for tempdb until SQL Server 2008 Switching it on doesn’t do anything until pages are read, h d i changed, written… Blog post: http://www.sqlskills.com/blogs/paul/post/How‐to‐tell‐if‐the‐IO‐ subsystem‐is‐causing‐corruptions.aspx SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 24 DBCC CHECKDB The only way to read all allocated pages in the database Use to force page checksums to be checked Many algorithms to minimize runtime and run ONLINE since Many algorithms to minimize runtime and run ONLINE since SQL Server 2000 Compared with 6.5 or 7.0 Variety of ways to perform consistency checks depending on resource constraints New features in 2005 Progress reporting, data purity, last known good New features in 2008 Long‐running checks moved under WITH EXTENDED_LOGICAL_CHECKS Blog post series: http://www.sqlskills.com/blogs/paul/category/CHECKDB‐From‐Every‐ Angle.aspx SQLskills.com Database Maintenance Consistency Checking How often should CHECKDB be run? How long will CHECKDB take to run? How best to consistency check a VLDB? H b tt it h k VLDB? What does all the CHECKDB output mean? Answer to all of these is “It depends!” Is it better to restore or just run repair? Restore usually guarantees zero data loss, but whether its faster depends on backup strategy B li l i h k b Bottom line: regular consistency checks must be run to proactively find corruption Blog post series: http://www.sqlskills.com/blogs/paul/category/CHECKDB‐From‐ Every‐Angle.aspx SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 25 For more info Category on corruption and CHECKDB (*many* posts!): CHECKDB From Every Angle ( ttp // (http://www.sqlskills.com/blogs/paul/category/CHECKDB‐From‐Every‐ sq s s co /b ogs/pau /catego y/C C o ey Angle.aspx) Specific post of interest: CHECKDB From Every Angle: Complete description of all CHECKDB stages Page Checksums (http://www.sqlskills.com/BLOGS/PAUL/category/Page‐Checksums.aspx) Specific post of interest: How to tell if the IO subsystem is causing corruptions? Webcast (top‐rated at TechEd US/EMEA in 2008): TechEd IT Pro 2008: Surviving Database Corruption (Level 300) (http://www.microsoft.com/emea/teched2008/itpro/tv/default.aspx?vid=78) (htt // i ft / /t h d2008/it /t /d f lt ? id 78) Chapter of interest: Chapter 11: DBCC Internals (Microsoft® SQL Server® 2008 Internals (Pro ‐ Developer) ) SQLskills.com Database Maintenance The Top‐Ten List… 10.Data file management 9. Transaction log file management Transaction log file management 8. Tempdb 7. Are your indexing strategies working? 6. Statistics 5. Index fragmentation 4 Detecting corruption 4. D t ti ti 3. Problem notification SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 26 Did Something Go Wrong? So you setup a regular job to run consistency checks, and turned on page checksums – how can you tell if it goes wrong? Th There has to be some kind of monitoring otherwise you’ll h t b ki d f it i th i ’ll never know! Manual monitoring is time‐consuming and prone to being forgotten Solution: Agent alerts, or other monitoring (e.g. SCOM) Create alerts for: Severity 19 errors and above Severity 19 errors and above Any user‐defined errors (e.g. flagging that the consistency checks job failed) Anything else you’re interested in SQLskills.com Database Maintenance For more info Blog posts on alerts: Easy monitoring of high‐severity errors: create Agent alerts ((http://www.sqlskills.com/BLOGS/PAUL/post/Easy‐monitoring‐of‐high‐ p // q / / /p / y g g severity‐errors‐create‐Agent‐alerts.aspx) A little‐known sign of impending doom: error 825 (http://www.sqlskills.com/BLOGS/PAUL/post/A‐little‐known‐sign‐of‐ impending‐doom‐error‐825.aspx) Conference Questions Pot‐Pourri #7: How to create Agent alerts (http://www.sqlskills.com/BLOGS/PAUL/post/Conference‐Questions‐Pot‐ Pourri‐How‐to‐create‐Agent‐alerts.aspx) SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 27 The Top‐Ten List… 10.Data file management 9. Transaction log file management Transaction log file management 8. Tempdb 7. Are your indexing strategies working? 6. Statistics 5. Index fragmentation 4 Detecting corruption 4. D t ti ti 3. Problem notification 2. Taking backups SQLskills.com Database Maintenance Recovering Using Backups Best way to avoid data loss Not necessarily the best way to avoid downtime Depends what kind of backups are available Although backup compression in SQL Server 2008 helps… Plethora of options available Full database backup is required starting point Series of log backups as well to reduce data loss Remember: Backups have to exist to be useful Backups have to be valid to avoid data loss SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 28 Full Database Backup Only Strategy Only as good as your last full backup Significant amount of work loss exposure N No possibility of point‐in‐time recovery ibilit f i t i ti What happens if a crash occurs at 23:59 on Saturday? 00:00 Sunday 00:00 Sunday F u l l F u l l B a c k u p B a c k u p time secondary filegroup Primary File1 File2 File3 Log D B C R E A T E SQLskills.com Database Maintenance Full + Log Backup Strategy Can get up‐to‐the‐minute and point‐in‐time recovery Restoring a large number of logs is hard to manage and can take a significant amount of time t k i ifi t t f ti May not have much work loss exposure at all What happens if crash occurs at 23:59 on Saturday? 00:00 Sunday 00:00 Sunday F u l l F u l l B a c k u p B a c k u p time secondary filegroup Primary File1 File2 File3 Log D B C R E A T E T L O G T L O G T L O G ………………………………………….. T L O G T L O G T L O G SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 29 Full + Diff + Log Backup Strategy Ultimate in availability, but less common Periodic differential backups reduce the recovery time by removing the need to apply all log backups since last full i th dt l ll l b k i l t f ll Does not *remove* the need for log backups What happens if a crash occurs at 23:59 on Saturday? 00:00 Sunday 00:00 Monday 00:00 Sunday F u l l D i f f D i f f F u l l B a c k u p B a c k u p B a c k u p B a c k u p 00:00 Saturday time secondary filegroup Primary File1 File2 File3 Log D B C R E A T E T L O G T L O G T L O G ………………... T L O G T L O G T L O G SQLskills.com Database Maintenance For more info Blog posts on backup & restore: Database Maintenance Best Practices Part III ‐ Transaction Log Maintenance (http://www.sqlskills.com/BLOGS/KIMBERLY/post/Database‐Maintenance‐Best‐ Practices‐Part‐III‐Transaction‐Log‐Maintenance.aspx) P i P III T i L M i ) Backup and Restore (many posts!) (http://www.sqlskills.com/BLOGS/PAUL/category/BackupRestore.aspx) Webcasts on Backup/Restore: Part 1 of MSDN Webcast Series: Creating a Recoverable Database (Level 200) Part 2 of MSDN Webcast Series: Creating a Reliable and Automated Backup Strategy (Level 200) Both at: Both at: http://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?EventID=103 2278586 TechNet Magazine article Understanding SQL Server Backups (July 2009) (http://technet.microsoft.com/en‐us/magazine/dd822915.aspx) SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 30 The Top‐Ten List… 10.Data file management 9. Transaction log file management Transaction log file management 8. Tempdb 7. Are your indexing strategies working? 6. Statistics 5. Index fragmentation 4 Detecting corruption 4. D t ti ti 3. Problem notification 2. Taking backups 1. Testing SQLskills.com Database Maintenance It’s All About Testing Test your: Disaster recovery plan Backups are valid Indexing strategy Alerts Test, test, test , , And then test again in a few months. And again. Did we mention testing? SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 31 For more info Check out our website (no advertising! It’s only free content and links to more!) ● www SQLskills com www.SQLskills.com Check out our links to whitepapers: ● www.SQLskills.com/whitepapers.asp Check out our blogs: ● ● www.SQLskills.com/blogs/Kimberly www.SQLskills.com/blogs/Paul Check out our online webcasts (6 individual webcasts and 2 multi‐ part series for a total of 27 webcasts part series for a total of 27 webcasts – over 50 hours of FREE over 50 hours of FREE online/downloadable content with more coming…): ● www.SQLskills.com/webcasts.asp SQLskills.com Database Maintenance Thank You! Questions? SQLskills.com Database Maintenance SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery Page 32