Download Essential Database Maintenance Essential Database Maintenance

Essential Database Maintenance
Melbourne SQL Server User Group: October 2009
Paul S. Randal
Kimberly L. Tripp
Kimberly L. Tripp
Author/Instructor:
Paul S. Randal
Consultant/Trainer/Speaker/Author
Managing Director, www.SQLskills.com
@ Q
Email: [email protected]
Blog: http://www.SQLskills.com/blogs/Paul
Twitter: @PaulRandal
Contributing Editor of TechNet Magazine, author of the SQL Q&A column, bi‐monthly articles on DBA topics, multiple 2008 whitepapers
5 years at DEC responsible for the VMS file‐system and check/repair
Almost 9 years as developer/manager in the SQL Storage Engine team through August 2007, ultimately responsible for Core Storage Engine
Wrote DBCC INDEXDEFRAG DBCC SHOWCONTIG and rewrote DBCC
Wrote DBCC INDEXDEFRAG, DBCC SHOWCONTIG and rewrote DBCC CHECKDB and repair for SQL Server 2005
Regular presenter at worldwide TechEds and other conferences on disaster recovery, HA, maintenance, and internals
Wrote and presented SQL Server 2008 training for Microsoft
Course author/instructor for Microsoft Certified Master qualifications
Co‐Chair of the SQL Connections conference
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 1
Author/Instructor:
Kimberly L. Tripp
Consultant/Trainer/Speaker/Writer
Founder, SYSolutions, Inc. (www.SQLskills.com)
e‐mail: [email protected]
blog: http://www SQLskills com/blogs/Kimberly
blog: http://www.SQLskills.com/blogs/Kimberly
Microsoft Regional Director and SQL Server MVP Writer/Editor for SQL Magazine www.sqlmag.com
Author/Instructor for SQL Server courses: Designing for Performance, Indexing for Performance, Maintenance, Disaster Recovery/HA
Author/Manager of SQL Server 2005 Launch Content, Co‐author/Manager for SQL Server 2008 Jumpstart Content, Author/Speaker at TechEd, SQL Connections, SQLPASS, ITForum, Conference Co‐chair for SQL Connections
Author of several SQL Server Whitepapers on MSDN/TechNet: Partitioning, Snapshot Isolation, Manageability and SQLCLR for DBAs
Partitioning Snapshot Isolation Manageability and SQLCLR for DBAs
Author/Presenter for more than 25 online webcasts on MSDN and TechNet (two series and other individual webcasts)
Co‐author MSPress Title: SQL Server 2000 High Availability, SQL Server 2008 Internals, and the SQL Server MVP Project
Presenter/Technical Manager for SQL Server 2000 High Availability DVD and various 2005/2008 HOL DVDs
I still love this stuff… don’t hesitate to ask questions!
SQLskills.com
Database Maintenance
The Top‐Ten List…
10.Data file management
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 2
Database Structure
Up to 32,767 databases per instance
Database
Up to 32,767 files per database
(Total is for both data and log files)
Minimum of 2 files – one for data, one for log
MDF =
Primary (MAIN) Data File
NDF =
Non-Primary Data File
Data file
*.mdf (1)
*.ndf (0-n)
Log file
*.ldf (1-n)
LDF = Log Data File
Should be only 1 log file
Just because you can, doesn’t mean you should…
SQLskills.com
Database Maintenance
Capacity Planning
Data
●
●
●
●
Log
●
●
●
Rough estimate of current tables – be sure to calculate:
Base tables (8096/average row size ‐> rows/page * rows ‐> pages * 8K
LOB data (XML, images (varbinary(max)), etc.
SQL Server 2008 FILESTREAM (instead of LOB inside the database)
Estimate growth for 1‐2 years (are you going to keep all of this data and/or periodically archive)
Indexes – estimate at 2‐3 times your data
Space for performance (e.g. maintenance)
1.5 times your largest table as a rough estimate
R
d l
Recovery model
Disaster recovery strategy (frequency of transaction log backups)
Workload type – and large transactions (log cannot be cleared until they complete)
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 3
Summary: Optimizing Data Files
Defrag the physical disks
Effective RAID array configuration
Pre‐allocate to a reasonable initial size (minimal amount of capacity planning)
Don’t let auto‐growth get out of control (monitoring)
Turn on instant initialization
Turn off auto
shrink (and use shrink sparingly)
Turn off auto‐shrink
(and use shrink sparingly)
While a lot of these will help for the log as well, there are more important things to be aware of IN the data portion – tables and indexes
● Index fragmentation
SQLskills.com
Database Maintenance
For more info
Blog posts on internals:
●
●
●
●
●
Inside the Storage Engine: Anatomy of a record
(http://www.sqlskills.com/blogs/paul/2007/10/01/InsideTheSt
orageEngineAnatomyOfARecord.aspx)
f
d
)
Inside the Storage Engine: Anatomy of a page
(http://www.sqlskills.com/blogs/paul/2007/10/03/InsideTheSt
orageEngineAnatomyOfAPage.aspx)
Inside the Storage Engine: Anatomy of an extent
(http://www.sqlskills.com/blogs/paul/2007/10/03/InsideTheSt
orageEngineAnatomyOfAnExtent.aspx)
Inside the Storage Engine: GAM, SGAM, PFS, and Other Allocation Maps
Allocation Maps
(http://www.sqlskills.com/blogs/paul/2008/03/14/InsideTheSt
orageEngineGAMSGAMPFSAndOtherAllocationMaps.aspx)
Inside the Storage Engine: IAM pages, IAM chains, and allocation units
(http://www.sqlskills.com/blogs/paul/2007/10/04/InsideTheSt
orageEngineIAMPagesIAMChainsAndAllocationUnits.aspx)
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 4
The Top‐Ten List…
10.Data file management
9. Transaction log file management
Transaction log file management
SQLskills.com
Database Maintenance
How the Transaction Log Works (1)
virtual log file 1
virtual log file 2
virtual log file 3
virtual log file 4
virtual log file 5
inactive
virtual log file
inactive virtual log file
active virtual log file
active virtual log file
inactive/unused
virtual log file
min LSN (start of oldest uncommitted
transaction)
max LSN (last log record written)
Once a VLF no longer contains log records that are required, it can be cleared (sometimes called truncated)
This is done by a log backup in FULL or BULK_LOGGED recovery d l
db
h k i i SIMPLE
d l
models, and by a checkpoint in SIMPLE recovery model
All that happens is that a VLF is marked as inactive
Nothing is zeroed, removed, overwritten (“clear” is a misnomer)
The log file size does NOT change (“truncation” is a misnomer)
Only whole VLFs are marked as inactive
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 5
Minimize Impact of Logging
Log cannot clear until the transactional information is inactive
Restart recovery will take longer if there were a lot of long i
i
di
h i
f f il
running transactions pending at the time of failure
Always try to avoid long running transactions
Could also be:
Log backup, database mirroring, transactional replication, log shipping, full/differential backup, change‐data‐capture in 2008
Check log_reuse_wait_desc in sys.databases for g
the reason the log can’t be cleared
SQLskills.com
Database Maintenance
How the Transaction Log Works (2)
virtual log file 1
virtual log file 2
virtual log file 3
virtual log file 4
virtual log file 5
active
virtual log file
active virtual log file
active virtual log file
active virtual log file
active virtual log file
min LSN (start of oldest uncommitted
transaction)
max LSN (last log record written)
What if all VLFs are active and more log records are written?
The log will auto‐grow (if configured)
More VLFs will be added and log records will be written into them
Log auto grow means a pause while the new area is zeroed
Log auto‐grow means a pause while the new area is zeroed
If the log cannot auto‐grow, write activity stops
Until the log is grown, another log file is added, or some action is taken to clear the log
Avoid this situation by allowing the log to clear!
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 6
If the Log File Fills Up…
Check in sys.databases
Take whatever corrective action can be done:
Take a log backup
Kill long‐running transaction
Alter database mirroring or replication to allow the log to clear
Grow the existing log file
Add another log file
Switch to the SIMPLE recovery model
S i h
h SIMPLE
d l
Logged operation in 2000, not logged in 2005+
Take preventative action to stop it happening again
SQLskills.com
Database Maintenance
Log File Considerations (1)
Only ONE log file necessary
Log activity is NOT parallelized over multiple files so NO performance gain from multiple files
Only time another log file may be needed is if the log file runs out of space and a backup/auto‐grow cannot occur
Isolate the log file from the data files to avoid disk head contention and performance issues
Choose an appropriate RAID level
RAID 5 is not recommended
RAID 1 is good, RAID 10 is better
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 7
Log File Considerations (2)
Pre‐allocate the log to an appropriate size
What is ‘appropriate’?
Depends on your workload, recovery model, log backup schedule, HA technologies
Log file cannot be instant initialized
Creation/growth takes time because of zero‐initialization
Monitor size to avoid ‘runaway log file’
Allow auto‐grow ‘just in case’ but do not rely on it
Every auto‐grow adds more VLFs – balance between too few and too many
Too few means each will be very large and affect log clearing
Grow/auto‐grow in no larger than 8GB chunks (i.e. 16 x 512MB VLFs)
SQLskills.com
Database Maintenance
For more info
Blog posts on the transaction log:
8 Steps to better Transaction Log throughput
(http://www.sqlskills.com/BLOGS/KIMBERLY/post/8‐Steps‐to‐better‐Transaction‐Log‐
throughput.aspx)
Transaction Log VLFs ‐ too many or too few?
(http://www.sqlskills.com/BLOGS/KIMBERLY/post/Transaction‐Log‐VLFs‐too‐many‐or‐
too‐few.aspx)
Database Maintenance Best Practices Part III ‐ TLog Maintenance
(http://www.sqlskills.com/BLOGS/KIMBERLY/post/Database‐Maintenance‐Best‐
Practices‐Part‐III‐Transaction‐Log‐Maintenance.aspx)
Understanding backups and log‐related Trace Flags in SQL 200x
(http://www.sqlskills.com/BLOGS/KIMBERLY/post/Understanding‐the‐Transaction‐
log.aspx)
Misconceptions around the log and log backups
(http://www.sqlskills.com/BLOGS/PAUL/post/Misconceptions‐around‐the‐log‐and‐log‐
backups‐how‐to‐convince‐yourself.aspx)
TechNet Magazine article
Understanding Logging and Recovery in SQL Server (February 2009)
(http://technet.microsoft.com/en‐us/magazine/2009.02.logging.aspx)
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 8
The Top‐Ten List…
10.Data file management
9. Transaction log file management
Transaction log file management
8. Tempdb SQLskills.com
Database Maintenance
tempdb is a Special Case
tempdb has always been used for:
Worktables (sort, hash tables, order by, …)
Query execution operations spilling out to disk
User created temporary options – table variables (@) and temp tables (# and ##)
DBCC CHECKDB processing of user databases
tempdb in SS 2005+ is used by many new features
Online index operations
MARS (multiple active results sets)
MARS (multiple active results sets)
Snapshot isolation
Triggers (INSERTED and DELETED tables)
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 9
Contention in tempdb
Tempdb is very susceptible to contention issues because: there’s only one tempdb per instance to support all user d t b
database operations
ti
Some query workloads cause multiple concurrent threads to repeatedly create/drop small temp tables
Causes huge amounts of latch contention on allocation bitmaps and system catalogs
SGAM page to manipulate mixed extents
p g
p
g (
)
PFS page to allocate/deallocate
pages (IAM and data)
System catalogs to record
Contention occurs in all versions, but worse in SQL Server 2000
(This can sometimes (rarely) happen in user databases with VERY high‐end workloads)
SQLskills.com
Database Maintenance
tempdb Allocation in 2005+
Tempdb allocation algorithms were tweaked in 2005
There is now a cache of temp tables
When a temp table is dropped, a single data page, single IAM page, and it’s metadata entry remain
The next temp table creation pulls one out of the cache instead of having to create it from scratch
Alleviates some contention, but not a very big cache
Tempdb contention can (and does) still happen in 2005 and 2008
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 10
Alleviating tempdb Contention
Trace flag 1118 (discussed in KB 328551) turns off the use of mixed extents in tempdb ONLY
Removes SGAM contention
Removes SGAM contention
Still may be needed in 2005+, but not as much as on 2000
Creating multiple data files to spread the contention
For 2000, # data files should be equal to # processor cores
For 2005+, you can still do 1:1 but you may not need to
Start with # data files = ¼ to ½ # processor cores and add more if needed
Anecdotal evidence that going beyond 8 data files in 2005+ gives no more gain
pp y
g
Note: this does not apply to log files
Separate tempdb data and log files from other databases
SQLskills.com
Database Maintenance
For more info
Blog posts on tempdb:
Understanding TempDB, table variables v. temp tables and Improving throughput for TempDB
throughput for TempDB
(http://www.sqlskills.com/BLOGS/KIMBERLY/post/Understanding‐TempDB‐table‐
variables‐v‐temp‐tables‐and‐Improving‐throughput‐for‐TempDB.aspx)
Changing Database Collation and dealing with TempDB Objects
(http://www.sqlskills.com/blogs/kimberly/post/Changing‐Database‐Collation‐and‐
dealing‐with‐TempDB‐Objects.aspx)
Comprehensive tempdb blog post series
(http://www.sqlskills.com/BLOGS/PAUL/post/Comprehensive‐tempdb‐blog‐post‐
series.aspx)
Misconceptions around TF 1118
(http://www.sqlskills.com/BLOGS/PAUL/post/Misconceptions‐around‐TF‐1118.aspx)
Whitepapers on tempdb:
Working with tempdb in SQL Server 2005
(http://www.microsoft.com/technet/prodtechnol/sql/2005/workingwithtempdb.mspx )
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 11
The Top‐Ten List…
10.Data file management
9. Transaction log file management
Transaction log file management
8. Tempdb
7. Are your indexing strategies working?
SQLskills.com
Database Maintenance
Indexing Best Practices
Don’t go crazy adding indexes
Just because you can index every column (and/or INCLUDE every column) –
INCLUDE
l
) doesn’t mean you should!
d
’t
h ld!
Start by setting up effective base table structures
Continue with workload analysis to determine workload characteristics and query priority...
Over‐indexing can be worse than under indexing
Indexes that are not maintained can be more trouble in the long run (be sure to automate index maintenance)
g
(
)
General indexing trends/strategies
A narrow index has very few uses…
A wider index has many more uses – you might be able to make some of your existing indexes more useful by just making them wider (ok, you can’t directly add columns but you can create new indexes and then drop the ones you’re not using)
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 12
Finding the Right Balance
Effective Base Table Strategies
Determine primary usage patterns of the table
● OLTP – fewer indexes
● OLAP OLAP – more and wider indexes more and wider indexes
Create the clustering key (choose a GOOD one!)
Create constraints – primary key and alternate/candidate keys
Manually add indexes to foreign key columns
Capture a workload(s) and analyze with Database g
Tuning Advisor
Over time (and based on workload analysis), add specific [wider/covering] indexes for low selectivity/high‐priority queries to help improve SARGs, joins, aggregations…
SQLskills.com
Database Maintenance
Finding the Right Balance
Good Clustering Key Choices
Identity column
Composite key of (date, identity)
● Not date alone as it
Not date alone as it’ss rarely unique
rarely unique
GUID
− Populated by client‐side call to .NET client guid
generator
NOT as the CL key but OK as a non‐clustered primary key
− Populated by server‐side newid() function
No pattern
 Populated by NEW server‐side newsequentialid() opu ated by
se e s de e seque t a d()
function
Creates a better (but not ideal) pattern
Key points: Key points: unique, narrow, static and ever‐
increasing!
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 13
Finding the Right Balance
Workload analysis and further tuning…
Get rid of any unused indexes before you do anything…
Prune redundant indexes
Remove unused indexes (sys.dm_db_index_usage_stats)
Remove unused indexes (sys dm db index usage stats)
2. Verify the health of the existing (and useful) indexes…
1. Determine the level of fragmentation 1.
1.
2
2.
(sys.dm_db_index_physical_stats)
Determine splits/latch waits (sys.dm_db_index_operational_stats)
Add indexes to improve poorly performing queries?
1. Ask DTA what it thinks?
2. Ask SQL Server what it thinks?
Q
2.
3.
1.
2.
3.
SQL Server 2005: DMVStats, Performance Dashboard Reports (SP2+), RML Utilities from PSS
SQL Server 2008: [Performance] Data Collector
Both SQL 2005/2008: DMV queries (missing index DMVs)
Bart Duncan’s blog: (search “Bart Duncan missing”)
SQLskills.com
Database Maintenance
Finding the Right Balance
Effective Indexing Strategies
Are you done?
Are you done?
Optimizer needs good statistics
Data access needs better data contiguity
Data access needs better data contiguity
You need more maintenance…
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 14
For more info
Blog categories (each with multiple posts):
●
Indexes
((http://www.sqlskills.com/blogs/kimberly/category/Indexes.aspx)
ttp //
sq s s co /b ogs/ be y/catego y/ de es asp )
●
Fillfactor
(http://www.sqlskills.com/blogs/kimberly/category/FILLFACTOR.aspx)
●
Indexes from Every Angle
(http://www.sqlskills.com/blogs/paul/category/Indexes‐From‐Every‐
Angle.aspx)
Whitepapers on indexes:
●
Online Indexing Operations in SQL Server 2005
(http://www.microsoft.com/technet/prodtechnol/sql/2005/onlineindex.msp
(http://www microsoft com/technet/prodtechnol/sql/2005/onlineindex msp
x)
●
Improving Performance with SQL Server 2005 Indexed Views
(http://www.microsoft.com/technet/prodtechnol/sql/2005/impprfiv.mspx)
SQLskills.com
Database Maintenance
The Top‐Ten List…
10.Data file management
9. Transaction log file management
Transaction log file management
8. Tempdb
7. Are your indexing strategies working?
6. Statistics
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 15
Statistics: Why are they important?
For indexes to be used, the optimizer needs to be able to estimate their cost (to choose the lowest expected cost)
expected cost)
Cost‐based optimization is done by assessing the usefulness of an index by looking at:
The query submitted
The conditions used (search arguments, join conditions, group by, order by, etc…)
The selectivity of the data (based on the arguments)
Selectivity is determined by statistics
S l i i i d
i db
i i
Statistics on indexes are of primary importance
Statistics on the secondary columns of indexes can provide some interesting uses…
SQLskills.com
Database Maintenance
Statistics: When are they created?
Statistics are automatically created
For all indexes For other columns, when the database option auto For other columns when the database option “auto
create statistics” is ON AND when the optimizer thinks that statistics would be a good idea
Statistics can be created manually
sp_createstats
Using CREATE STATISTICS
Tips
Leave the database option: auto create statistics on
Leave the database option: auto create statistics on
Add more statistics
SQL Server 2000 supports 249 total indexes/stats per table
SQL Server 2005 supports 2,000 statistics per table
SQL Server 2008 supports 10,000 statistics per table
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 16
sp_createstats
sp_createstats @indexonly = 'indexonly',
@fullscan = 'fullscan',
@norecompute = 'norecompute'
@indexonly: only create statistics for the d
l
l
f h
secondary columns of indexes. This can help to make non‐clustered indexes more useful.
@fullscan: requires more time but will create more accurate statistics. If off‐hours this is a good idea
@norecompute: the statistics will NOT get @norecompute: the statistics will NOT get
automatically updated as distribution of data changes. Generally, not recommended.
Recommendation:
sp_createstats 'indexonly',
indexonly', 'fullscan
'fullscan'
'
SQLskills.com
Database Maintenance
Statistics: What if the data changes?
Automatically updated
If auto update statistics is ON (for both the DB and the index wasn’t created with “norecompute”)
th i d
’t
t d ith “
t ”)
If a percentage of the data changes
Complete details in TechNet whitepaper
Manually update statistics
For highly volatile tables where distribution isn’t changing significantly and you see a lot of “statistics” events
Turn off “auto update” OR turn off auto update by adding STATISTICS_NORECOMPUTE on the index definition (better control)
Executing UPDATE STATISTICS
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 17
Auto Update Statistics
SQL Server 7.0
Invalidated when sysindexes.rowmodctr reached
U d t d h i lid t d
Updated when invalidated
SQL Server 2000
Invalidated when sysindexes.rowmodctr reached
 Updated when needed
SQL Server 2005
 Invalidated when sysrowsetcolumns.rcmodified
reached
Updated when needed
SQL Server 2008
Invalidated when sysrscols.rcmodified reached
Updated when needed
SQLskills.com
Database Maintenance
For more info
Blog posts on statistics:
●
Auto update statistics and auto create statistics ‐ should you leave them on and/or turn them on?? them on and/or turn them on??
(http://www.sqlskills.com/blogs/kimberly/post/Auto‐update‐statistics‐and‐
auto‐create‐statistics‐should‐you‐leave‐them‐on‐andor‐turn‐them‐on.aspx)
●
Index rebuilds depend on stats, which are updated by index rebuilds?!?
(http://www.sqlskills.com/blogs/paul/post/Index‐rebuilds‐depend‐on‐stats‐
which‐are‐updated‐by‐index‐rebuilds!.aspx)
Whitepaper on statistics:
●
Statistics Used by the Query Optimizer in Microsoft SQL Server 2005
Statistics Used by the Query Optimizer in Microsoft SQL Server 2005
(http://www.microsoft.com/technet/prodtechnol/sql/2005/qrystats.mspx)
Chapter of interest:
●
Chapter 6: Index Internals and Management
(Microsoft® SQL Server® 2008 Internals
(Pro ‐ Developer) )
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 18
The Top‐Ten List…
10.Data file management
9. Transaction log file management
Transaction log file management
8. Tempdb
7. Are your indexing strategies working?
6. Statistics
5. Index fragmentation
SQLskills.com
Database Maintenance
Fragmentation Defined
Two broad types:
g
Internal fragmentation
Wasted space on data/index pages
Sometimes called ‘physical’ fragmentation
Logical fragmentation
Out‐of‐order pages within an index leaf level affecting range‐scan performance
Both can have very detrimental effects on
Both can have very detrimental effects on performance
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 19
Page Density (Internal Fragmentation)
Page fullness is below the required fillfactor
More pages required to hold the same rows
How does this happen?
Poor schema design leading to very large rows and wasted space on each page
Random deletes leaving space on pages
Page splits
Effect is:
Increased IOs for range scans
I
d IO f
Greater memory usage if most of the index is memory resident
Performance must be decreased because of page splits!!
SQLskills.com
Database Maintenance
What Causes Fragmentation?
Schemas/workloads that cause page splits on densely packed pages
GUID as high‐order key (or any other random key)
Can even affect nonclustered indexes
Updates to variable‐length columns
Badly configured FILLFACTOR
Wide schemas that only fit a few records per page and waste space
E.g. a fixed‐size 5000 byte row = 3000 bytes lost per page!
Real‐world example:
Social networking site that has a homepage comments table with the member ID as the high‐order key
Needs fragmentation removed twice a day!
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 20
Detecting Fragmentation
Keys to success are:
Knowing which indexes to look at
Which are used for range scans?
Which have very low page density?
Knowing which options to use of the various methods
Knowing how to interpret the results
Use sys.dm_db_index_physical_stats DMV in 2005
Use DBCC SHOWCONTIG in 2000
Still there in 2005 but deprecated
Don’t just rebuild all indexes every day!
SQLskills.com
Database Maintenance
How to Correct Fragmentation?
3 realistic choices
CREATE INDEX WITH DROP_EXISTING
Rebuild the index – ALTER INDEX … REBUILD
Defrag the index – ALTER INDEX … REORGANIZE
Most people rebuild or reorganize
Trade‐offs between the two methods
Log space required
Di k
i d
Disk space required
Online vs. offline
Time to completion
Interruptible without loss of work
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 21
Index Rebuild Misconceptions
Rebuilding a clustered index rebuilds all nonclustered indexes
No, only in 2000 and before for non‐unique clustered indexes
There was a bug in 2000 SP1 that *did*
There was a bug in 2000 SP1 that did rebuild always
rebuild always
Index rebuild pre‐allocates the necessary space
No, it allocates space as it goes so may run out
Rebuilding in BULK_LOGGED reduces the size of the log backups
No, only the size of the transaction log. Remember how log backups work after a minimally‐logged operation?
Index rebuilds in a multi‐file filegroup only use a single file
Index rebuilds in a multi file filegroup only use a single file
No, the allocations will follow regular round‐robin proportional fill
Online index build doesn’t take any locks at all
No, online is a slight misnomer
SQLskills.com
Database Maintenance
For more info
Blog categories of interest:
Fragmentation
(http://www.sqlskills.com/BLOGS/PAUL/category/Fragmentation.aspx)
Webcasts on Index Fragmentation:
MSDN Webcast Series: Part 5: New Features in Indexing and Index Maintenance Best Practices (Level 200) (http://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?EventID=1032278599)
TechEd 2005: Index Defragmentation with SQL Server 2005 (Level 300)
(http://www.microsoft.com/emea/spotlight/sessionh.aspx?videoid=30)
Whitepaper on Fragmentation (still relevant for concepts even though written ffor SQL Server 2000):
Microsoft SQL Server 2000 Index Defragmentation Best Practices
(http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/ss2kidbp.mspx)
Chapter of interest:
Chapter 6: Index Internals and Management
(Microsoft® SQL Server® 2008 Internals
(Pro ‐ Developer) )
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 22
The Top‐Ten List…
10.Data file management
9. Transaction log file management
Transaction log file management
8. Tempdb
7. Are your indexing strategies working?
6. Statistics
5. Index fragmentation
4 Detecting corruption
4.
D t ti
ti
SQLskills.com
Database Maintenance
I/O Errors
Three types
823 – a hard I/O error
824 824 – a soft I/O error
a soft I/O error
825 – a read‐retry error
Nice error messages in 2005+
Msg 824, Level 24, State 2, Line 1
SQL Server detected a logical consistency‐based I/O error: incorrect checksum (expected: 0x7232c940; actual: 0x720e4940). It occurred during a read of page (1:143) in database ID 8 at offset 0x0000000011e000 in file 'c:\sqlskills\broken.mdf'. Additional messages in the SQL Server error log or system event log may provide more detail This is a severe error condition that
system event log may provide more detail. This is a severe error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB). This error can be caused by many factors; for more information, see SQL Server Books Online.
Logged in msdb..suspect_pages
Input into single‐page restore operations
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 23
Page Checksums (1)
Per‐page checksum
Written as last thing SQL Server does on a physical write
Checked as first thing SQL Server does on a physical read
Provides the “smoking gun” that the error is not due to SQL Server as it detects all changes to a page
Superset of torn‐page detection
On by default for new databases in SQL Server 2005+
Checksum failures result in an 824 error
CPU overhead of approx 1‐2% on heavy load
Error‐detecting, not error correcting
SQLskills.com
Database Maintenance
Page Checksums (2)
Checked when:
Page is read normally
Page is read during DBCC CHECKDB
Page is read during BACKUP WITH CHECKSUM
Page is contained within a checksum’d backup
There are two problems though:
Not available for tempdb until SQL Server 2008
Switching it on doesn’t do anything until pages are read, h
d
i
changed, written…
Blog post:
http://www.sqlskills.com/blogs/paul/post/How‐to‐tell‐if‐the‐IO‐
subsystem‐is‐causing‐corruptions.aspx
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 24
DBCC CHECKDB
The only way to read all allocated pages in the database
Use to force page checksums to be checked
Many algorithms to minimize runtime and run ONLINE since
Many algorithms to minimize runtime and run ONLINE since SQL Server 2000
Compared with 6.5 or 7.0
Variety of ways to perform consistency checks depending on resource constraints
New features in 2005
Progress reporting, data purity, last known good
New features in 2008
Long‐running checks moved under WITH
EXTENDED_LOGICAL_CHECKS
Blog post series:
http://www.sqlskills.com/blogs/paul/category/CHECKDB‐From‐Every‐
Angle.aspx
SQLskills.com
Database Maintenance
Consistency Checking
How often should CHECKDB be run?
How long will CHECKDB take to run?
How best to consistency check a VLDB?
H b tt
it
h k VLDB?
What does all the CHECKDB output mean?
Answer to all of these is “It depends!”
Is it better to restore or just run repair?
Restore usually guarantees zero data loss, but whether its faster depends on backup strategy
B
li
l
i
h k
b
Bottom line: regular consistency checks must be run to proactively find corruption
Blog post series: http://www.sqlskills.com/blogs/paul/category/CHECKDB‐From‐
Every‐Angle.aspx
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 25
For more info
Category on corruption and CHECKDB (*many* posts!):
CHECKDB From Every Angle
( ttp //
(http://www.sqlskills.com/blogs/paul/category/CHECKDB‐From‐Every‐
sq s s co /b ogs/pau /catego y/C C
o
ey
Angle.aspx)
Specific post of interest: CHECKDB From Every Angle: Complete description of all CHECKDB stages
Page Checksums
(http://www.sqlskills.com/BLOGS/PAUL/category/Page‐Checksums.aspx)
Specific post of interest: How to tell if the IO subsystem is causing corruptions?
Webcast (top‐rated at TechEd US/EMEA in 2008):
TechEd IT Pro 2008: Surviving Database Corruption (Level 300)
(http://www.microsoft.com/emea/teched2008/itpro/tv/default.aspx?vid=78)
(htt //
i
ft
/
/t h d2008/it /t /d f lt
? id 78)
Chapter of interest:
Chapter 11: DBCC Internals
(Microsoft® SQL Server® 2008 Internals
(Pro ‐ Developer) )
SQLskills.com
Database Maintenance
The Top‐Ten List…
10.Data file management
9. Transaction log file management
Transaction log file management
8. Tempdb
7. Are your indexing strategies working?
6. Statistics
5. Index fragmentation
4 Detecting corruption
4.
D t ti
ti
3. Problem notification
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 26
Did Something Go Wrong?
So you setup a regular job to run consistency checks, and turned on page checksums – how can you tell if it goes wrong?
Th
There has to be some kind of monitoring otherwise you’ll h t b
ki d f
it i
th
i
’ll
never know!
Manual monitoring is time‐consuming and prone to being forgotten
Solution: Agent alerts, or other monitoring (e.g. SCOM)
Create alerts for:
Severity 19 errors and above
Severity 19 errors and above
Any user‐defined errors (e.g. flagging that the consistency checks job failed)
Anything else you’re interested in
SQLskills.com
Database Maintenance
For more info
Blog posts on alerts:
Easy monitoring of high‐severity errors: create Agent alerts
((http://www.sqlskills.com/BLOGS/PAUL/post/Easy‐monitoring‐of‐high‐
p //
q
/
/
/p / y
g
g
severity‐errors‐create‐Agent‐alerts.aspx)
A little‐known sign of impending doom: error 825
(http://www.sqlskills.com/BLOGS/PAUL/post/A‐little‐known‐sign‐of‐
impending‐doom‐error‐825.aspx)
Conference Questions Pot‐Pourri #7: How to create Agent alerts
(http://www.sqlskills.com/BLOGS/PAUL/post/Conference‐Questions‐Pot‐
Pourri‐How‐to‐create‐Agent‐alerts.aspx)
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 27
The Top‐Ten List…
10.Data file management
9. Transaction log file management
Transaction log file management
8. Tempdb
7. Are your indexing strategies working?
6. Statistics
5. Index fragmentation
4 Detecting corruption
4.
D t ti
ti
3. Problem notification
2. Taking backups
SQLskills.com
Database Maintenance
Recovering Using Backups
Best way to avoid data loss
Not necessarily the best way to avoid downtime
Depends what kind of backups are available
Although backup compression in SQL Server 2008 helps…
Plethora of options available
Full database backup is required starting point
Series of log backups as well to reduce data loss
Remember:
Backups have to exist to be useful
Backups have to be valid to avoid data loss
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 28
Full Database Backup Only Strategy
Only as good as your last full backup
Significant amount of work loss exposure
N
No possibility of point‐in‐time recovery
ibilit f i t i ti
What happens if a crash occurs at 23:59 on Saturday?
00:00
Sunday
00:00
Sunday
F
u
l
l
F
u
l
l
B
a
c
k
u
p
B
a
c
k
u
p
time
secondary
filegroup
Primary
File1
File2
File3
Log
D
B
C
R
E
A
T
E
SQLskills.com
Database Maintenance
Full + Log Backup Strategy
Can get up‐to‐the‐minute and point‐in‐time recovery
Restoring a large number of logs is hard to manage and can take a significant amount of time
t k
i ifi t
t f ti
May not have much work loss exposure at all
What happens if crash occurs at 23:59 on Saturday?
00:00
Sunday
00:00
Sunday
F
u
l
l
F
u
l
l
B
a
c
k
u
p
B
a
c
k
u
p
time
secondary
filegroup
Primary
File1
File2
File3
Log
D
B
C
R
E
A
T
E
T
L
O
G
T
L
O
G
T
L
O
G
…………………………………………..
T
L
O
G
T
L
O
G
T
L
O
G
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 29
Full + Diff + Log Backup Strategy
Ultimate in availability, but less common
Periodic differential backups reduce the recovery time by removing the need to apply all log backups since last full
i th
dt
l ll l b k
i
l t f ll
Does not *remove* the need for log backups
What happens if a crash occurs at 23:59 on Saturday?
00:00
Sunday
00:00
Monday
00:00
Sunday
F
u
l
l
D
i
f
f
D
i
f
f
F
u
l
l
B
a
c
k
u
p
B
a
c
k
u
p
B
a
c
k
u
p
B
a
c
k
u
p
00:00
Saturday
time
secondary
filegroup
Primary
File1
File2
File3
Log
D
B
C
R
E
A
T
E
T
L
O
G
T
L
O
G
T
L
O
G
………………...
T
L
O
G
T
L
O
G
T
L
O
G
SQLskills.com
Database Maintenance
For more info
Blog posts on backup & restore:
Database Maintenance Best Practices Part III ‐ Transaction Log Maintenance
(http://www.sqlskills.com/BLOGS/KIMBERLY/post/Database‐Maintenance‐Best‐
Practices‐Part‐III‐Transaction‐Log‐Maintenance.aspx)
P
i
P III T
i L M i
)
Backup and Restore (many posts!)
(http://www.sqlskills.com/BLOGS/PAUL/category/BackupRestore.aspx)
Webcasts on Backup/Restore:
Part 1 of MSDN Webcast Series: Creating a Recoverable Database (Level 200)
Part 2 of MSDN Webcast Series: Creating a Reliable and Automated Backup Strategy (Level 200)
Both at:
Both at:
http://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?EventID=103
2278586
TechNet Magazine article
Understanding SQL Server Backups (July 2009)
(http://technet.microsoft.com/en‐us/magazine/dd822915.aspx)
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 30
The Top‐Ten List…
10.Data file management
9. Transaction log file management
Transaction log file management
8. Tempdb
7. Are your indexing strategies working?
6. Statistics
5. Index fragmentation
4 Detecting corruption
4.
D t ti
ti
3. Problem notification
2. Taking backups
1. Testing
SQLskills.com
Database Maintenance
It’s All About Testing
Test your:
Disaster recovery plan
Backups are valid
Indexing strategy
Alerts
Test, test, test
,
,
And then test again in a few months.
And again.
Did we mention testing? 
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 31
For more info
Check out our website (no advertising! It’s only free content and links to more!)
●
www SQLskills com
www.SQLskills.com
Check out our links to whitepapers:
●
www.SQLskills.com/whitepapers.asp
Check out our blogs:
●
●
www.SQLskills.com/blogs/Kimberly
www.SQLskills.com/blogs/Paul
Check out our online webcasts (6 individual webcasts and 2 multi‐
part series for a total of 27 webcasts part series for a total of 27 webcasts – over 50 hours of FREE over 50 hours of FREE
online/downloadable content with more coming…):
●
www.SQLskills.com/webcasts.asp
SQLskills.com
Database Maintenance
Thank You!
Questions?
SQLskills.com
Database Maintenance
SQL Server 2005/2008: Database Maintenance – From Performance to Disaster Recovery
Page 32