Download Query_Store

SQL Server 2016
The Query Store
Author:
Company:
INTELLIGENT DATA SOLUTIONS
Sean Werick
Pragmatic Works
WWW.PRAGMATICWORKS.COM
About Pragmatic Works
Who is Pragmatic Works?
o 100% Microsoft SQL Server Focused
o Team of Microsoft MVPs, Authors, Presenters
o Concentrated on Optimizing your Data Lifecycle
What is Pragmatic Works?
o Software Solutions
o Training Offerings
o Professional Services
Why Pragmatic Works?
o
o
o
o
o
INTELLIGENT DATA SOLUTIONS
Microsoft Gold Partner – Data Platform & Business Intelligence
Microsoft’s only “National Solution Provider” for Business Intelligence
Microsoft Analytics Platform System Implementation Partner of the Year
INC Magazine’s Fastest 5000 Growing Companies
Leader in offerings centered around Microsoft’s emerging technologies
WWW.PRAGMATICWORKS.COM
About Sean
•
•
•
•
Principal Consultant
Community Speaker
Blogs: www.seanwerick.com
Email: [email protected]
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
SQL Server 2016
New Innovations
Cloud first
Speed
Agility
Proven
Feedback
All of this results in a better on-premises SQL Server
Announcing
SQL Server 2016
In-memory enhancements
Operational analytics & enhanced performance
In-memory
SQL Server
ColumnStore
data warehouse
2-24
hrs
ETL
Fraud detected
Real-time
fraud
detection
0100101010110
In-memory
SQL Server
OLTP
Always Encrypted
Help protect data at rest and in motion, on-premises & cloud
Apps
SQL Server
Trusted
SELECT Name FROM
Patients WHERE SSN=@SSN
Client side
SELECT Name FROM
Patients WHERE SSN=@SSN
@SSN='198-33-0987'
Column
Master
Key
Result Set
Name
Jim Gray
Query
@SSN=0x7ff654ae6d
Enhanced
ADO.NET
Library
Result Set
Column
Encryption
Key
Name
Jim Gray
ciphertext
dbo.Patients
Name
SSN
Country
Jane Doe
243-24-9812 USA
1x7fg655se2e
Jim Gray
198-33-0987 USA
0x7ff654ae6d
John Smith
123-82-1095 USA
0y8fj754ea2c
Mission critical performance
Performance
Security
Availability
Scalability
Operational analytics
Always Encrypted
Enhanced AlwaysOn
Enhanced database
caching
In-memory OLTP for
more applications
Row Level
Security
Query data store
Dynamic Data
Masking
Native JSON
Temporal database
support
• 3 synchronous replicas for
auto failover across domains
• Round robin load balancing
of replicas
• DTC for transactional integrity
across database instances
with AlwaysOn
Enhanced online
operations
Support for Windows
Server 2016
• 12 TB memory support
PolyBase
Query relational and non-relational data with T-SQL
Quote:
************************
T-SQL query
**********************
*********************
**********************
***********************
SQL Server
Name
DOB
State
Jim Gray
11/13/58 WA
Ann Smith 04/29/76 ME
Hadoop
$658.39
Built-in advanced analytics
In-database analytics at massive scale
Example Solutions
Extensibility
• Sales forecasting
R Integration
• Warehouse efficiency
• Predictive maintenance
R
?
New R scripts
• Credit risk protection
010010
100100
010101
Analytic Library
010010
100100
010101
Data Scientist
Interact directly with data
010010
100100
010101
T-SQL Interface
Relational Data
Built-in to SQL Server
Data Developer/DBA
Manage data and
analytics together
010010
100100
010101
010010
100100
010101
Microsoft Azure
Marketplace
MANAGE
CREATE
Microsoft Excel
Power BI
CONSUME
Power BI Web Portal
• Power BI Desktop
• Microsoft Excel
Power BI Desktop
• Mobile reports
Windows App
• Paginated reports
Android App
iOS App
“Datazen” Publisher**
SQL Server
Report Builder
SQL Server BI “Reporting
Services”
• Power BI Desktop
Report Server Web portal
(all BI report types)
• Mobile reports
Report Designer in
SQL Server Data Tools
• Paginated reports
SharePoint Web
Deeper insights across data
Access any data
Scale and manage
Powerful insights
PolyBase
Enterprise-grade
Analysis Services
Built-in advanced analytics
Power Query for analytics
and reporting
Enhanced SSIS
• Designer support for previous
SSIS versions
• Support for Power Query
New single SSDT in
Visual Studio 2015
Enhanced MDS
• Excel add-in 15x faster
• More granular security roles
• Archival options for transaction logs
• Reuse entities across models
Business insights through
rich visualizations on
mobile devices
Enhanced Reporting
Services
Introducing Azure SQL Data Warehouse
Separate storage & compute
Elastic scale
Integration with Power BI, ADF,
and Machine Learning services
Scale-out relational
data warehouse
Stretch SQL Server into Azure
Stretch warm and cold tables to Azure with remote query
processing
Microsoft Azure
Jim Gray
Order history
Name
Jane Doe
Jim Gray
John Smith
Bill Brown
ox7ff654ae6d
3/18/2005
Stretch to cloud
SSN
Date
Customer data
2/28/200
cm61ba906fd
5
Product data
3/18/200
ox7ff654ae6d
5
Order History
4/10/200
i2y36cg776rg
5
4/27/200
nx290pldo90l
5
5/12/200

Query
App
Cortana Analytics Suite
Transform data into intelligent action
What is the Query Store?
A SQL Server 2016 (and SQL DB) Feature that can be Enabled
• Provides insight on query plan choice and performance
• Simplifies performance troubleshooting by quickly finding performance differences
caused by query plan changes
• Automatically captures a history of queries, plans, and runtime statistics, and retains
these for review
• Separates data by time windows to better illustrate database usage patterns and
understand when query plan changes happened on the server
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Why Use the Query Store?
Query Performance Insight Helps Satisfy 2 Main Goals
Performance
INTELLIGENT DATA SOLUTIONS
Cost
WWW.PRAGMATICWORKS.COM
Why Use the Query Store?
Performance
• Review the list of top resource-consuming
queries
• Select an individual query to view its details
• Click Settings to customize how data is
displayed or to show a different time period
• Open Index Advisor* (PaaS) and check if any
recommendations are available
Cost
All about the DTUs (PaaS)
A Database Transaction Unit (DTU) is a
blended measure of the resources allocated at
each performance level.
WHAT IS THE REAL COST?
•
•
•
•
Processor - % Processor Time
Logical Disk - Disk Reads/sec
Logical Disk - Disk Writes/sec
Database - Log Bytes Flushed/sec
* Index Advisor - https://azure.microsoft.com/en-us/documentation/articles/sql-database-index-advisor/
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
How the Query Store “Works”
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Does It Work with In-Memory OLTP?
• SQL Server Query Store allows you to monitor the
performance of natively compiled code for workloads
running in-memory OLTP
• Compile and runtime statistics are collected and
exposed the same way as for disk-based workloads
• When you migrate to in-memory OLTP you can
continue using Query Store views in SQL Server
Management Studio as well as custom scripts you have
developed for disk-based workloads before migration
• This saves your investment in learning Query Store
technology and makes it generally usable for
troubleshooting all type of workloads
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
How to Use the Query Store
In Just 3 Easy Steps, You TOO can be a DBA!!
Okay, okay. This may take a bit more effort and knowledge.
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Setting up the Query Store
There are 2 Ways to Enable Query Store
By Using the Query Store Page in SSMS
In Object Explorer, right-click a database, and then click
Properties
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Setting up the Query Store
By Using the Query Store Page in SSMS
In the Database Properties dialog box, select the Query
Store page
Select related attributes
• General
o Operation Mode (Read or Read/Write)
• Monitoring
o Data Flush Interval (Manual Entry in Minutes)
o Statistics Collection (5 mins, 10 mins, 15 mins, 30 mins, 1 hr,
1 day)
• Max Size
o Query Store Capture Mode (All or Auto)
o Size Based Cleanup Mode (Off or Auto)
o Stale Query Threshold (Manual Entry in Days)
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Setting up the Query Store
By Using T-SQL Statements
•
Use the ALTER DATABASE statement
ALTER DATABASE AdventureWorks2012
SET QUERY_STORE = ON;
Ability to
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Query Store Options with T-SQL
ALTER DATABASE [db_name]
SET QUERY_STORE
(
[Query Store options]
);
-- Example
ALTER DATABASE AdventureWorks
SET QUERY_STORE
(
MAX_STORAGE_SIZE_MB = 250
);
INTELLIGENT DATA SOLUTIONS
GUI Option
T-SQL Option
Operation Mode (Requested) OPERATION_MODE
Data Flush Interval (Minutes)
DATA_FLUSH_INTERVAL_SECONDS
Statistics Collection Interval
INTERVAL_LENGTH_MINUTES
Max Size (MB)
MAX_STORAGE_SIZE_MB
Query Store Capture Mode
QUERY_CAPTURE_MODE
Size Based Cleanup Mode
SIZE_BASED_CLEANUP_MODE
Stale Query Threshold (Days)
CLEANUP_POLICY (STALE_QUERY_THRESHOLD_DAYS)
MAX_PLANS_PER_QUERY
WWW.PRAGMATICWORKS.COM
What Information Does It Provide?
DATA MODEL
sys.query_context_settings
context_settings_id
sys.query_store_runtime_stats
plan_id
sys.query_store_plan
INTELLIGENT DATA SOLUTIONS
sys.query_store_query
query_text_id
runtime_stats_interval_id
sys.query_store_runtime_stats_interval
query_id
sys.database_query_store_options
sys.query_store_query_text
WWW.PRAGMATICWORKS.COM
What Information Does It Provide?
Query Store View
Description
sys.database_query_store_options
Query Store options for this database
sys.query_context_settings
Semantics affecting context settings associated with a query (The same
query text compiled under different settings may produce different
results [depending on the underlying data])
sys.query_store_plan
Each execution plan associated with a query
sys.query_store_query
Information about the query and its associated overall aggregated
runtime execution statistics
sys.query_store_query_text
Transact-SQL text and the SQL handle of the query
sys.query_store_runtime_stats
Information about runtime execution statistics information for the query
sys.query_store_runtime_stats_interval
Information about the start and end time of each interval over which
runtime execution statistics information for a query has been collected
*Additional Detail - https://msdn.microsoft.com/en-us/library/dn818146.aspx
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Query Store Best Practices

Information and performance are generally a give and take, but there ARE some
best practices

Every single setting in the Query Store has associated costs and benefits
• Treat Query Store performance tuning like an agile project with Sprints
o Sprint 1 – Keep Operation Mode in READ/WRITE mode for information collection
 This reduces write time, and subsequent overall database performance, for data
collection as no reporting is occurring
o Sprint 2 – Change Operation Mode to READ mode for analysis and actual performance
tuning
 This it increases report performance because no writes are occurring. This also results
in no “dirty reads” because of the asynchronous writes
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Query Store Best Practices

Information and performance are generally a give and take, and it ultimately
relies on each and every business and its processes
• Flush the data at specific business case intervals. For example, if performance of the DB
seems to be poor around end of the month due financial reporting, consider using the Agile
methodology to complete one iteration, then flush the data before the next iteration.
• Keep the Max Size small enough to keep the reporting relatively snappy, however, keep
enough data to capture all of the business needs
• Keep an accurate Cleanup Policy as the business and reporting solution evolves
• Collect statistics as frequently as possible to see exactly which execution plans are being
executed. If the statistics are not updated for some time, you may be looking at an old
execution plan because the statistics have changed
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Query Store Reporting
Azure SQL DB
• Cloud offering (PaaS)
• Learn and adapt dynamically with your app
• Scale performance on the fly, without app
downtime
• Build multitenant apps with customer isolation
and efficiency
• Work within your preferred development
environments
• Help protect and secure your app data
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Query Store Reporting
The top queries view opens and the
top CPU consuming queries are listed
• The top line shows overall DTU% for the
database, while the bars show CPU%
consumed by the selected queries
during the selected interval
• The bottom grid represents aggregated
information for the visible queries
o Average CPU per query during
observable interval
o Total duration per query
o Total number of executions for a
particular query
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Query Store Reporting
Viewing Individual Query Details
Click any query in the list of top queries.
The details view opens and the queries CPU consumption is broken
down over time.
Click around the chart for details.
The top line is overall DTU%, and the bars are CPU% consumed by
the selected query.
Review the data to see detailed metrics including duration, number
of executions, and resource utilization percentage for each interval
the query was running.
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Query Store On Premise Reporting
On Premise Solution?
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Query Store Dashboards
SSMS
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Custom Views and SQL
HOW?
sys.query_context_settings
context_settings_id
sys.query_store_runtime_stats
plan_id
sys.query_store_plan
INTELLIGENT DATA SOLUTIONS
sys.query_store_query
query_text_id
runtime_stats_interval_id
sys.query_store_runtime_stats_interval
query_id
sys.database_query_store_options
sys.query_store_query_text
WWW.PRAGMATICWORKS.COM
Query Store Admin with T-SQL
Stored Procedure
Description
sp_query_store_flush_db
Flushes the in-memory portion of the Query Store data to disk
sp_query_store_reset_exec_stats
Clears the runtime stats for a specific query plan from the query store
sp_query_store_force_plan
Enables forcing a particular plan for a particular query.
When a plan is forced for a particular query, every time SQL Server encounters
the query, it tries to force the plan in the optimizer. If plan forcing fails, a
XEvent is fired and the optimizer is instructed to optimize in the normal way.
sp_query_store_unforce_plan
Enables unforcing a particular plan for a particular query
sp_query_store_remove_plan
Removes a single plan from the query store
sp_query_store_remove_query
Removes the query, as well as all associated plans and runtime stats from the
query store
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Forcing Execution Plans
OLD WAY
1. Run SET SHOWPLAN_XML ON to return the XML
Execution Plan, or alternatively, grab the XML Plan
from the sys.dm_exec_text_query_plan DMF)
2. Create the Plan Guide using the
sp_create_plan_guide Stored Procedure
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Forcing Execution Plans
NEW WAY
1. Click the “Force Plan” button inside one of the
Query Store reports
2. Use the sp_query_store_force_plan SP.
EXEC sp_query_store_force_plan 136, 146;
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Error State
Remember to verify intermittently that the Query Store is still collecting data
SELECT actual_state_desc, desired_state_desc, current_storage_size_mb,
max_storage_size_mb, readonly_reason, interval_length_minutes,
stale_query_threshold_days, size_based_cleanup_mode_desc,
query_capture_mode_desc
FROM sys.database_query_store_options;
If there is a difference between the actual_state_desc and desired_state_desc, this indicates that a change of
operations mode occurred automatically.
The most common change is for the Query Store to silently switch to read-only mode. In extremely rarely
circumstances, Query Store can end up in the ERROR state because of internal errors.
When the actual state is read-only, use the readonly_reason column to determine the root cause. Typically, the
Query Store transitioned to read-only mode because the size quota was exceeded. In that case the
readonly_reason is set to 65536.
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Error State
How to Fix
Reactive
Proactive
Increase the maximum storage size by using the
MAX_STORAGE_SIZE_MB option of ALTER DATABASE
Apply best practices, specifically related to Query
Store size
Clean up Query Store data
ALTER DATABASE [QueryStoreDB]
SET QUERY_STORE CLEAR;
Make sure that most recent data is retained,
configure time-based policy to remove stale
information regularly
Change Operation Mode back to READ/WRITE
ALTER DATABASE [QueryStoreDB]
SET QUERY_STORE (OPERATION_MODE = READ_WRITE);
Consider setting Query Capture Mode to Auto as it
filters out queries that are usually less relevant for
your workload
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM
Questions and Answers
INTELLIGENT DATA SOLUTIONS
WWW.PRAGMATICWORKS.COM