Download SQL_Clone_TheSQLGuru

SQL Clone
Redgate's New Tool for
SQL Server Database Provisioning
Who am I?
Kevin G. Boles
SQL Server Consultant
“The Wolfe” for the Relational Engine
Indicium Resources, Inc.
@TheSQLGuru
[email protected]
MCITP, MCT, MVP 2007-2012, yada-yada
What do I do?
• World-class Better than average relational engine expert 
• Over 45,000 man-hours invested in SQL Server
• Absolutely LOVE doing targeted performance analysis and
tuning work
• VERY good at making others better at interacting with SQL
Server
• Couldn’t SSAS my way out of a paper bag
… with both ends open! 
Agenda
• Why database provisioning?
• Who does SQL Clone benefit?
• How does SQL Clone work?
• Demos
• Usage Scenarios
SQL Server Provisioning Study - 2017
Database Sizes
NOTE: This is just dev/test.
There are undoubtedly many large
databases not being dropped down
due to time/space constraints
Why Database Provisioning?
• Quality (at speed). Realistic testing with
repeatable processes. Move problems ‘left’.
• Replace current (painful) processes.
• It’s a job that’s ‘delegated up’.
Problems: Quality at Speed
• Matching production volumes
• Mirroring data distribution of real data
• Repeatable processes
• Time taken
Problems: Drive Space
• Multiple teams and multiple
branches
• Realistic data
• Isolated environments
• N times the amount of production
data
Problems: Pressure on Environments
• Multiple development teams
• Multiple developers per team
• Multiple branches of code
•
Main branch
•
Hot fixes
•
New Development
• Automated testing and QA
Problems: Regulatory Compliance
• Business regulations are increasing
•
Health Care
•
Banking
•
Insurance
• International compliance (GDPR)
• Cost of data breaches!
Challenges of Using Realistic Data
• Shouldn’t be on production (it is a pet peeve of
mine to see test databases on production box!)
• Moving/restoring large amounts of data takes time
• Storing large amounts of data costs money
• Data may be sensitive
Primary Target Audience
• Who is responsible for data provisioning?
• DBA
• IT Manager
• Head of Data Services
• Chief Data Officer
Who Benefits from Production-realistic Data?
• Developers
• Testers/QA
• BI/Reporting teams
• Performance Troubleshooting staff
What Is SQL Clone?
“SQL Clone is a SQL Server database cloning tool that
allows you to create editable copies of databases within
seconds while using very little additional disk space.”
Technical details and
architecture
Volume Shadow Copy (VSS)
• 64-bit Windows technology
• Back up/Copy a file/disk at a point in time while in use
• Go to that point in time quickly
• Other uses:
•
SQL Server snapshots
•
VMs
Virtual Disk Service?
• 64-bit Windows technology
• Allocate logical storage without taking disk space
• Mount remote storage as local
• Mount read-only with differencing disks
• Virtual hard drive ‘chaining’
Image Creation In Action
•
Create an initial, small VHD, at the data
image location (shown as 10MB in size in
Figure 8)
•
Create a mount point to this VHD on the local
operating system of the machine hosting the
source database
•
Use the VSC service to “remote copy” the
MDF and LDF files to the data image location
•
Un-mount the VHD
DEMOS
Solutions: Realistic Data
• Clean the data as necessary
• Create a SQL Clone database copy
• Share that copy, including realistic
data, with all teams
• Troubleshoot not on production
Solutions: Drive Space
• Clones are small (save up to 99% of disk space)
• Differential storage only
• Refresh of an environment resets space used
Solutions: Shared Environments
Isolated Environments!
• Database is a mutable object. Multiple people
can cause conflicts, breakages, etc
• Multiple clones per data image
• Multiple environments
• Clones are very small
• Creating clones takes seconds
Solutions: Regulatory Compliance
• Clean data in a clone or a copy of database
• Create new data image
• Distribute the cleaned data image as clones
• Works with database internal processes such as
Dynamic Data Masking (if your security is the same)
Solutions: Data Obfuscation
Clean data prior to clone process
OR
1. Create a clone from a data image
2. Clean the clone
3. Create a data image of the cleaned clone
4. Clone the new data image
(Pursuing clone chaining)
Solutions: A/B Testing
Clone A
Series A Mods
Comparative Analytics or
Testing
\\FileShare\Images\Financial
Clone B
Series B Mods
Solutions: Do More
• Faster than normal restores
• Smaller than normal restores
• Fully automatable
• Self-service
Features & limitations
V1 features
• Central Management Server
• Image from live DB
• Image from backup file
• Image from Clone – full sized
• Automation – PowerShell cmdlets
• Permissions – Basic
V1 limitations
• 2TB image limit
• Image from backup with passwords
• 64 bit Windows 2008R2/7 upwards only
• SQL Server 2008 upwards only*
• Beware default image location in C drive
* Clones do not circumvent Express 10GB database size limit since SQL Server still perceives
the clone as its original size.
Data Masking Workflows
• Image > Clone > Mask > Image
• In V1 and Image of Clone is full size
• Data Generator
• Custom script
• Active Research for Masking and Chaining
Roadmap
• Advanced access and permissions
• Create new image from a Clone using chaining
• Potential:
•
Incrementing images from changes
•
Extending the 2TB limit
•
Data masking
SQL Clone Goals
1. Encourage you to give your developers, analysts,
troubleshooters, etc. the necessary data for more
accurate and timely work
2. Provide a mechanism for automating and managing
provisioning of development/test/reporting databases
3. Enable you to better support a DevOps approach within
your environment
Bonus Round
• SQL Clone Competition (25 Amazon Echo Dots, 5
Clone Licenses!!). Open until April 30!
• https://www.red-gate.com/blog/sql-clone-competition
What About TDE??
• Extracting image from live database works fine
• Creating a clone database requires a Database
Master Key (can be different from source server)
and restores of the original Certificate and
Database Encryption Key
• Extracting from backup also requires DMK and Cert
Packaging and Pricing
• The product is clearly team focused, so it is sold in user
packs
• Pricing scales as number of seats goes up
• UNLIMITED usage regarding Images and Clones
• Primary other entry in this market is Delphix. It is
upscale and does already have data masking, but it also
runs six-figures
SQL Clone Licensing
[email protected]
Questions?
Links
http://www.red-gate.com/products/dba/sql-clone/database-provisioning-report
https://www.red-gate.com/blog/database-lifecycle-management/sql-clone-anddatabases-protected-with-tde
https://www.red-gate.com/blog/database-lifecycle-management/sql-clone-and-backupsprotected-with-tde
Tell them I sent you!!! I may get a small spiff - and I may split it with you! 
[email protected]
@TheSQLGuru