Download Fragmentation - UK SQL Server User Group

Fragmentation
Tony Rogerson SQL Server MVP
Torver Computer Consultants
www.sqlserverfaq.com
[email protected]
Agenda

Logical Disk Fragmentation




NTFS Backgrounder
Fragmentation
Best practices
SQL Server Fragmentation





Storage internals (Files, Extents and Pages)
Extent and Page fragmentation
Shrinking the database
Scripts to deal with SQL Server fragmentation
Best practices
NTFS Backgrounder


Do not use Compression with SQL Server (not supported)!
Writes/Reads using Clusters, default cluster size is variant on size
of disk (see
http://support.microsoft.com/?kbid=140365) can be
up to 4K, specify values up to 64K (NT defrag only works up


to 4K)
Recommend higher cluster sizes as SQL Server deals with 8K
pages and 64K extents – use a cluster size of 4K so you can
defrag (/A option of FORMAT)
On fresh NTFS volume, Master File Table and a number of
system logs are held near middle of the new volume (prevents
starvation).
Logical Disk Fragmentation



Our problem – we have fewer files but big ones.
If you request 50MB, NT looks for free
contiguous clusters to satisfy that request, that
could be anywhere on the disk (fills bigger gaps
first).
Causes increased disk head movement
More fragmentation – More head movement –
Slower disk read/write.
 Increased wear on disk – reduces life expectancy

Fragmentation Example
Based on a
1GB logical
volume:
BEFORE create database (SQL programs after a defrag)
AFTER create database (250MB MDF)
AFTER create database (550MB MDF)
Fragment 2
(end of database)

Fragment 1
(start of database)
Cannot defrag 550MB because of NTFS system
files in middle of volume
Extending the Database


If you request 50MB, NT looks for free
contiguous clusters to satisfy that request, that
could be anywhere on the disk (fills bigger gaps
first).
If lots of other files on the disk then chances of
free contiguous clusters diminish –
fragmentation occurs.
Best Practices





Pre allocate your databases (do not let them
auto-grow) – be in control!
Keep backups on another volume (especially
transaction logs as there are lots of them and
vary in size)
Check fragmentation once a week/month
(depends on your system)
Use allocation size of 4K
To defrag you need to stop SQL Server
SQL Server
Fragmentation
Storage Internals


Refer to Inside SQL Server 2000 pages 192-196
8 Pages in an extent; 1 page = 8K; 1 extent = 64K
MDF File
Extent (64K) Extent (64K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Extent (64K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Page (8K)
Index Allocation Map (IAM)


Holds details of the extents and pages that
belong to the table or index.
Entries are held in the order they appear on the
disk thereby aiding scanning performance (less
disk head movement).
Fragmentation



Fragmentation occurs at the extent and page
level, for indexes it can also occur within
contiguous pages.
SQL Server doesn’t pre-allocate extents.
Causes of Fragmentation
Inserting and Updating data (page splits for existing
data and acquiring free extents).
 Shrinking databases (talked about later).

Table Fragmentation Example
Extent
Page 0 – FULL; A
Page 1 – FULL; B
Page 2 - FREE
Page 3 - FREE
Page 4 – FULL; C
Page 5 – FULL; D
Page 6 – FULL; F
Page 7 – FULL; G




INSERT VALUE ‘E’
Clustered index on our Letter column.
On Index Scan (ordered) reads 0, 1, 4, 5, 2, 6, 7
For un-ordered access doesn’t matter as the IAM (Index
Allocation Map) holds pages in the order they are held on disk.
For index seek, doesn’t matter either - getting back a single value
so very little pages need to be read.
Table De-fragment
Extent
Page 0 – FULL; A
Page 1 – FULL; B
Page 2 – FULL; C
Page 3 – FULL; D
Page 4 – FULL; E
Page 5 – FULL; F
Page 6 – FULL; G
Page 7 – FREE


Free pages are bubbled to
the end of the table; any
extents becoming free are
released to the pool.
Methods of defrag include




DBCC INDEXDEFRAG
DBCC DBREINDEX
BCP/BULK INSERT
DROP/CREATE INDEX
(DROP_EXISTING)
Combating Database Fragmentation






Tables grow and use up free extents
Extents free up when data is deleted
Enter FILLFACTOR to give more free space
for new data/updates that extend the row size.
Page numbers are unique to a file (start at 0).
Consider placing volatile tables (work tables,
staging tables) on their own File Group.
Consider one table per file group (very radical!)
TEST1
TEST1
TEST1
TEST1
TEST1
TEST1
TEST1
TEST1
page_id
75
77
78
79
80
81
82
83
pg_alloc
1
1
1
1
1
1
1
1
ext_size
1
1
1
1
1
1
1
1
TEST2
TEST2
TEST2
84
86
87
1
1
1
1
1
1
TEST1
88
8
8
TEST2
TEST2
TEST2
TEST2
TEST2
TEST2
TEST2
96
97
98
99
100
104
112
1
1
1
1
1
8
8
1
1
1
1
1
8
8
TEST1
TEST1
120
128
8
7
8
8
TEST2
TEST2
TEST2
136
144
152
8
8
1
8
8
8
Example: \fragmentation - dbcc extentinfo.sql
DBCC
EXTENTINFO



TEST2 is mixed in with
TEST1 (fragmentation)
ext_size of 1 are pages
on mixed extents
Missing page number is
the IAM (Index
Allocation Map) which
isn’t reported by this
DBCC command.
DBCC SHOWCONTIG
-




Pages Scanned................................:
Extents Scanned..............................:
Extent Switches..............................:
Avg. Pages per Extent........................:
Scan Density [Best Count:Actual Count].......:
Logical Scan Fragmentation ..................:
Extent Scan Fragmentation ...................:
Avg. Bytes Free per Page.....................:
Avg. Page Density (full).....................:
40
7
6
5.7
71.43% [5:7]
0.00%
28.57%
1018.2
87.42%
Shows degree of fragmentation for an index / heap.
DBCC SHOWCONTIG( ‘test2’, 1 )
Extent Scan Fragmentation is the most important one to watch.
Scan Density doesn’t work when the table is on multiple files.
DBCC DBREINDEX




Locks table and rebuild the index (moves the data as well –
proper defrag!)
DBCC INDEXDEFRAG doesn’t lock everything but only the
pages its working on; can be stopped as well and work done is
kept.
DBCC INDEXDEFRAG only defrags existing leaf pages of the
index, causes no new extent allocations.
DBREINDEX does more than INDEXDEFRAG
-
Pages Scanned................................:
Extents Scanned..............................:
Extent Switches..............................:
Avg. Pages per Extent........................:
Scan Density [Best Count:Actual Count].......:
Logical Scan Fragmentation ..................:
Extent Scan Fragmentation ...................:
Avg. Bytes Free per Page.....................:
Avg. Page Density (full).....................:
40
6 (was 7)
5
6.7
83.33% [5:6]
0.00%
0.00% (was 28.57%)
1018.2
87.42%
What happened?



Because database is full (no free extents), database is
extended and data is put there, which is good –
extents are contiguous.
Object moved off the ‘mixed extent’.
Free extents left within the database.
DBCC DBREINDEX
MDF File
MDF File
Extent: FREE
Extent Table: ‘1’
Extent: FREE
Extent Table: ‘1’
Extent Table: ‘2’
Extent Table ‘2’
Extent: FREE
Extent Table ‘1’
Extent: FREE
Extent Table ‘1’
Extent Table: ‘2’
Extent Table: ‘2’
Extent Table: ‘2’
Extent Table: ‘2’
Extent Table: ‘2’
Extent Table: ‘2’
Extent: FREE
Extent (9): FREE
Extent: FREE
Extent (10): FREE
Extent Table: ‘1’
Extent Table: ‘1’
Extent Table ‘1’
Extent Table ‘1’
Example
DBREINDEX –Data Movement
Example: \fragmentation - dbcc extentinfo.sql
Shrinking the Database






DBCC SHRINKDATABASE
Data is moved into free extents at the start of the
MDF.
Everything is probably fragmented again.
Run DBREINDEX on important tables after
SHRINKDATABASE or don’t SHRINKDATABASE!
Good way of getting a reasonably contiguous database
without BCPing everything out and back in again.
Make sure you have plenty of free extents at the end of
your database!
Example
SHRINKDATABASE – Data
Movement
Example: \fragmentation - dbcc extentinfo.sql
Script
DBREINDEX fragmented tables
Example: \fragmentation - dbcc dbreindex.sql
Summary (Best Practices)


Use File Groups for volatile tables
DBREINDEX
Requires free extents
 Moves data



INDEXDEFRAG defrags index structure
(leaves extents where they are)
Careful if ever using SHRINKDATABASE –
data is moved down file into free extents giving
rise to possible fragmentation.
Further Reading / Q & A

NTFS
http://www.digit-life.com/articles/ntfs/
 http://people.msoe.edu/~taylor/cs384/sitania.pdf


Inside SQL Server 2000 – Kalen Delaney
Tony Rogerson SQL Server MVP
Torver Computer Consultants
www.sqlserverfaq.com
[email protected]