Download document 12658492

Orders of magnitude:
Scale-Out your SQL Server Data
Mark Broadbent
@retracement
A bit more about me!
•
•
•
•
•
•
•
•
•
•
•
More than 20 years in IT and more than 14 years using SQL Server.
Worked at many large global corporations and SMEs such as Microsoft,
Nokia, Hewlett Packard and Encyclopaedia Britannica.
Presented at SQLBits 7 and 8
MCITP Database Development SQL 2008
MCITP Database Administrator SQL 2008/ 2005 and MCDBA SQL 2000
Microsoft Certified Application Developer (C# .net)
Microsoft Certified Systems Engineer + Internet
Participate on #sqlhelp ,MSDN Forums, Stackoverflow & Serverfault.
Used to be active in the MS newsgroups until their demise :(
Run the LinkedIn groups
Linux Mint User Group http://www.linkedin.com/groups?gid=2989801
SQL Server Scripting http://www.linkedin.com/groups?gid=3033621
Orders of magnitude: Scale-Out your
SQL Server Data
2
Agenda
•
•
•
•
•
Why Scale-Out and what should we scale
Before you Scale-Out… Scale-In
Lets Scale-Out
Hybrid Scale-Out
Taming the Beast
Orders of magnitude: Scale-Out your
SQL Server Data
3
Traditional Synchronous Activity
Request B
Wasted
Processor
cycles
during waits
Request C
A
B
Time
Request A
“Database”
C
Request D
Even
distribution
of database
load from
application
D
Request E
E
User
Orders of magnitude: Scale-Out your
SQL Server Data
4
Asynchronous Activity?
“Database”
Request A
Request B
Request C
Request D
Request E
Request x
B
C
Time
User
experience
more
productive
application
thread more
active
A
D
E
Competing
loads on
database
from
application
faster all
round
execution
User
Orders of magnitude: Scale-Out your
SQL Server Data
5
Separation of Function…
Request A
Request B
Request C
Request D
C
B
E
Request E
Request x
User
D
Time
Intensive
reporting
queries
A
“Database”
Separated
loads across
database
based on
function,
even faster
execution
Units of Scale
Orders of magnitude: Scale-Out your
SQL Server Data
6
Separation of Requirements…
Request A
A
“Database”
B
Request B
Request C
Request D
C
D
Time
Queued,
Scheduled or
Governed
either by the
application
server or
instance
E
Request E
B
Request x
User
Impact on
database
lessoned
due to
scheduled
or governed
query
Units of Scale
Orders of magnitude: Scale-Out your
SQL Server Data
7
OLTP vs DW…
Distribution of System Data
Interactive (realtime)
Integrated
(24hrs to 1 month)
Near line (3-4 years)
Probability
of access
Static Data
Aggregates
Denormalized
Archival (5-10 years)
Orders of magnitude: Scale-Out your
SQL Server Data
8
Abstracting your NAME
Why do we need to keep it hidden?
What is the SQL Instance name?
How can we do it?
• Flexible DR
• Good for fast switch over
• Avoids conflict with company standards
Orders of magnitude: Scale-Out your
SQL Server Data
9
Why is Scale-Out important #1
• Availability
– Portions of data can go offline but doesn't effect the whole
• Performance
–
–
–
–
Load balance
Mix and match (low consumers and big consumers)
Separation of workload types (olap and oltp)
Parallelise a System (separate system requests across multiple
hardware)
– Overcome contentious parts of the db server such as tempdb
• Capacity
– backup time (reduce time to backup)
– limited resources
Session Title
10
Why is Scale-Out important #2
• Disaster
– Recovery time (reduce time to restore - reduced when less to
recover)
– Large Disk Partitions can take long time to fix
– Limit the impact of total disaster (i.e. when your DR strategy
does not work)
• Cost
– Reuse commodity hardware for less important data
– The higher we Scale-Up the more expensive it becomes. Scaling
out can be cheaper
Session Title
11
Before you Scale-Out…Scale-In #1
• Keep statistics accurate and up to date
– Avoid big temp vars, they have no indexes or stats
– Stats can get skewed, ensure they are maintained
• Query Tuning
–
–
–
–
–
–
–
Avoid table scans
Use indexes correctly, and remove duplicates
Is parallelism right for your query (OLAP vs OLTP)
Reduce size of the result set
Always use a WHERE clause
DON’T use SELECT * replace with precise column list
Sensible clustered key, avoid large covered index and prefer
include option
Orders of magnitude: Scale-Out your
SQL Server Data
12
Before you Scale-Out…Scale-In #2
• Remove your LOBs from tables
– Use other technologies such as FILESTREAM
– Vertically partition
• Reduce page io
–
–
–
–
Filtered indexes
Sparse columns
Use correct data types
Use table/ row and page compression
• Reduce contention on shared resources
– Denormalize
– Vertically Partition
– Filegroups
Orders of magnitude: Scale-Out your
SQL Server Data
13
Lets Scale-Out
•
•
•
•
•
•
•
Scalable Shared Database
Partitioned Tables
Partitioned Views
Federated Database Servers
Peer to Peer Replication
Synonyms
Service Broker
Orders of magnitude: Scale-Out your
SQL Server Data
14
Scalable Shared Database
Report
Server
Report
Server
Database
SQL Server
Instance A
Report
Server
Database
Application
Server
SQL Server
Instance B
Instance C
Orders of magnitude: Scale-Out your
SQL Server Data
15
Peer to Peer Replication
Read-Write for User
Applications
Read-Write for ETL
SQL Server
Instance B
Database X
SSIS
Application
Server
SQL Server
Instance A
Database X
Read-Only for Reporting
SQL Server
Instance C
Database X
Report
Server
Orders of magnitude: Scale-Out your
SQL Server Data
16
Partitioned tables, views and filegroups
Table
payments
Partitioned
View
payments2011
Partitioned
Tables
payments2011
Filegroup1
Filegroup2
Filegroup3
payments2010
payments2010
Filegroup4
payments2009
payments2009
Filegroup5
Orders of magnitude: Scale-Out your
SQL Server Data
17
With Distributed Partitioned views
Table
payments
Distributed
Partitioned View
Partitioned
Tables
payments2011
FilegroupA1
payments2011
FilegroupA2
SQL Server
Instance A
payments2010
FilegroupA3
payments2010
FilegroupB1
SQL Server
Instance B
payments2009
payments2009
FilegroupB2
Orders of magnitude: Scale-Out your
SQL Server Data
18
Hybrid Scale-Out
• Database Mirroring with rolling snapshots
• SQL Failover Cluster using over-provisioned
failover node “hot-swap Scale-Up”.
• Use Hyper-Visor, migrate to over-provisioned
host.
• Clusters, Peer to Peer, Mirroring on Hyper-Visor
Orders of magnitude: Scale-Out your
SQL Server Data
19
Bringing it all together
Virtual Cluster Nodes
Report
Server
Cluster A
SQL Failover Clusters
Instance A
Database X
Virtual Cluster Nodes
Cluster B
Instance E
Database Y
Mirror
Instance B
Database Y
SQL Failover Clusters
Instance C
Database YP
Instance D
Database X
Partition Views
Hyper-Visor A
Hyper-Visor B
Peer to Peer Replication
Orders of magnitude: Scale-Out your
SQL Server Data
20
Taming the Beast
• Standards
– Policy based management
– Source control
• Monitoring
–
–
–
–
MDW
Performance condition alerts
Resource Governor or WRSM
Extended Events
• Naming
– Aliases
– DNS
Orders of magnitude: Scale-Out your
SQL Server Data
21
In Summary
• We discussed
–
–
–
–
–
Why we should start thinking about Scaling-Out
Scaling in before you Scale-Out
Scale-Out strategies
Hybrid Scale-Out strategies
Keeping your scaled environment under control
Orders of magnitude: Scale-Out your
SQL Server Data
22
Further References
•
Books
– Apress - Pro SQL Server 2008 Service broker – Klaus Aschenbrenner
– Apress - Pro SQL Server 2008 Replication - Sujoy Paul
– Morgan Kaufman - DW 2.0 - The Architecture for the Next Generation of Data
Warewhousing – William Inmon, Derek Strauss and Genia Neushloss
– MS Press - Improving .NET Application Performance and Scalability
•
Blogs/ Websites
–
–
–
–
•
Partitioned Table & Index Strategies Using SQL Server 2008 http://bit.ly/g28zQa
VoltDB.NET: Synchronous vs. Asynchronous Request Processing http://bit.ly/k3rY2N
Data Warehousing 2.0 and SQL Server: Architecture and Vision http://bit.ly/4tRXB4
Performance Considerations of Data Types – Michelle Ufford http://bit.ly/aq9Wyr
Video/ Webcasts
– PASS Summit 2010: AD270S Database Design Fundamentals - Louis Davidson
– MCM #17 SQL Server Partitioning – SQLSkills http://bit.ly/ea3C6e
Session Title
23
Please Complete the Evaluation Form
Pick up your evaluation form:
• In each presentation room
Drop off your completed form
• Near the exit of each presentation room
• At the registration area
Presented by Dell
Orders of magnitude: Scale-Out your
SQL Server Data
24
GET IN TOUCH!
Contact Me:
Blog: tenbulls.co.uk
Email: [email protected]
Twitter: @retracement
LinkedIn: http://linkd.in/gylT4I
Orders of magnitude: Scale-Out your
SQL Server Data
25
THANK YOU!
Presented by Dell
For attending this session and
PASS SQLRally Orlando, Florida
26
Session Code | Session Title