Download StrategiesForWorkingWithTexasSizedDatabases

Strategies for Working with
Texas-sized Databases
Robert L Davis
Database Engineer
www.sqlsoldier.com
@SQLSoldier
PASS Security Virtual Chapter
• http://security.sqlpass.org
• Volunteers needed
Robert L Davis
• Microsoft Certified Master
• Data Platform MVP
Database Engineer
• BlueMountain Capital Management
• 15+ years working with SQL Server
@SQLSoldier
• www.sqlsoldier.com
What is a Texas-sized Database?
What is the smallest database that qualifies as
a VLDB?
5 TB (35%, 7 Votes)
1 TB (25%, 5 Votes)
500 GB (20%, 4 Votes)
100 GB (15%, 3 Votes)
750 GB (5%, 1 Votes)
250 GB (0%, 0 Votes)
50 GB (0%, 0 Votes)
Total Voters: 20
What is a Texas-sized Database?
What is the largest db you have worked with?
1 TB to 5 TB (25%, 3 Votes)
Less than 100 GB (17%, 2 Votes)
100 GB to 500 GB (17%, 2 Votes)
500 GB to 1 TB (17%, 2 Votes)
5 TB to 10 TB (8%, 1 Votes)
20 TB to 50 TB (8%, 1 Votes)
More than 50 TB (8%, 1 Votes)
10 TB to 20 TB (0%, 0 Votes)
Total Voters: 12
What is a Texas-sized Database?
What are your biggest challenges with working
with VLDBs? (choose up to 3)
Index maintenance (79%, 11 Votes)
Backups (57%, 8 Votes)
Integrity checks (50%, 7 Votes)
Archiving/purging data (36%, 5 Votes)
Disk/disk space management (29%, 4 Votes)
Partitioning management (21%, 3 Votes)
Inefficient queries (14%, 2 Votes)
Disk performance (7%, 1 Votes)
Total Voters: 14
Index Maintenance
 Very large tables have very large indexes
 Be selective
 Reorganize vs. Rebuild
 Index maintenance at partition level
 Yay for online partition index maintenance!
 Don’t check fragmentation levels
 Check on different day
 Design with index maintenance in mind
 No LOB, XML, or spatial columns in tables that will get
large
 Combine partitioned views with partitioned tables
 Minion Reindex: http://minionware.net/reindex/
Index Maintenance
 Partitioned views with partitioned tables
Backups: What’s Difficult
 Everything takes longer
 Back up time
 Copy time
 Restore time
 Uses too much disk space
 Uses too many resources
 Network throughput
 SAN throughput
Backups: Speeding up
 Multiple dedicated drives
 1 backup thread per drive/mount point
 1 file per drive/mount point
 Differential/Partial backup
 Read-only filegroups/Read-write filegroups
 Per file or filegroup
 Tune backup options
 MaxTransferSize
 BufferCount
 http://sirsql.net/content/2012/12/13/20121212automated-backup-tuning/
 Fast drives
 Crank up the SAN throughput
 Maximize Queue Depth
 Increase the SAN paths
 BONUS: works for restores too
Integrity Checks
 Takes a very long time
 Restore backup to another server and run there
 Run with PHYSICAL_ONLY option
 Break it up into smaller parts over multiple nights
 Per filegroup
 Per table
 See Paul Randal’s post:
http://www.sqlskills.com/BLOGS/PAUL/post/CHECKDB-FromEvery-Angle-Consistency-Checking-Options-for-a-VLDB.aspx
Archiving / Purging Data
 Takes a long time
 Causes excessive blocking (lock escalation)
 Deleted space difficult to recover
 Partitioning addresses all of this
 Switch partitions out
 Drop table/remove filegroup
Archiving / Purging Data
 No partitioning?
 Delete in small batches
 Ensure that log backups or Checkpoints are
running between batches
 Use targeted deletes
 Identify the PK columns to be deleted
 No good solution for reclaiming space
 Try running DBCC SHRINKFILE with
TRUNCATEONLY
 If you must shrink, plan to rebuild indexes
Replication/High Availability
 Log shipping and database mirroring require
initializing with a full backup
 Replication requires that the data get to the
subscriber
 Initialize from backup
 Log shipping is the gateway
 Set up log shipping
 Convert to replication or mirroring
Reversing Log Shipping











Disable all backup jobs that may hit the database
Disable the log shipping jobs
Run each step of the log shipping jobs in order
Drop log shipping
Manually backup the primary database log using WITH
NORECOVERY
Puts the primary database into a recovering state
Restore log backup on secondary using WITH RECOVERY
Brings the secondary database online
Take log backup of new primary database
Restore log backup on new secondary database using WITH
NORECOVERY
Reconfigure log shipping
Strategies for Working with
Texas-sized Databases
Convert Log Shipping to Replication














Configure replication publication
Disable all backup jobs that may hit the database
Disable the log shipping jobs
Run each step of the log shipping jobs in order
Drop log shipping secondary
Change publication to allow initialization from backup
Back up the log of the publisher
Restore the log backup using WITH RECOVERY to recover the
secondary
Create subscription on publisher
@backupdevicename = path/name of log backup created above
@sync_type = initialize with backup
Create subscription on subscriber
Re-enable any backup jobs
Drop log shipping primary if desired
Before: 2 TB – Log shipped to 2 servers
After: Reverse log shipping and convert 1 server to
replication
RESOURCES
VLDB Series for SQL University:
http://www.sqlsoldier.com/wp/sqlserver/sqluvldbwe
ekintrotovldbs
MinionIndex: http://minionware.net/reindex/
Automated Backup Tuning:
http://sirsql.net/content/2012/12/13/20121212auto
mated-backup-tuning/
Strategies for Working with
Texas-sized Databases
Thanks!
Thank you for coming!
 My blog: www.sqlsoldier.com
 Twitter: twitter.com/SQLSoldier