Download SQL Server XTP Engine Internals

SQL SERVER
IN-MEMORY
INTERNALS
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
About Me
Hamid J. Fard
I am SQL Server Data Platform Expert with more than 9 years’ of professional
experience, I am currently Microsoft Certified Master: SQL Server 2008,
Microsoft Certified Solutions Master: Charter-Data Platform, Microsoft Data
Platform MVP and CIW Database Design Specialist. After a few years of being a
production database administrator I jumped into the role of Data Platform
Expert. Being a consultant allows me to work directly with customers to help
solve questions regarding database issues for SQL Server.
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
Agenda
•
What is In-Memory OLTP?
•
In-Memory OLTP Components.
•
Memory-Optimized Table Requirements.
•
Memory Optimized Tables.
•
Memory Optimized Indexes.
•
In-Memory Query Processing.
•
In-Memory Transactions.
•
In-Memory Memory Estimation.
•
In-Memory Filegroup Configuration.
•
In-Memory Data and Delta File Population.
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
What is In-Memory OLTP?
In-Memory Table
Disk Based Table
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
Benefits:
• Eliminate contention.
• Reduce logging.
• Lower latency data retrieval.
• Minimize code execution time.
• Efficient data retrieval.
• Optional IO reduction or removal, when
using non-durable tables
Implementation Scenarios:
• High data insertion rate from multiple
concurrent connections.
• Read performance and scale with periodic
batch inserts and updates.
• Intensive business logic processing in the
database server.
• Low latency.
• Session state management.
In-Memory OLTP Components
In-Memory
Performance
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
Memory-Optimized Table Requirements.
•
•
•
•
•
X64 Architecture Processor
<60 Processor Cores
Enough Memory (x2 Table Size)
Enough Storage (x2 Table Size)
Processor Needs to Support cmpxchg16b
Hardware
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
•
•
•
•
X64 Architecture Edition
Enterprise or Developer Edition
Hyper-V to Support cmpxchg16b (If needed)
Enable Instant File Initialization
Software
Memory Optimized Tables.
In-Memory Table
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
SQL Server CLR
Memory Optimized Indexes.
Made for point lookups
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
Made for range scans and ordered scans
In-Memory Query Processing
Disk Based Query Plan
Memory Optimized Query Plan
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
SQL Server Query Processing for Disk-Based Tables
INTerpreted Transact-SQL Access to Memory-Optimized Tables
In-Memory Query Processing
(Cont.)
Native compilation of stored procedures
Execution of natively compiled stored procedures
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
In-Memory Transactions
Lifetime of a transaction that accesses memory-optimized tables
• Commit Dependency
Regular Processing
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
• Repeatable Read Validation
• Serialization Validation
Validation Phase
Commit Processing
In-Memory Memory Estimation
CREATE TABLE IMOLTP (
col1 INT NOT NULL PRIMARY KEY NONCLUSTERED,
col2 INT NOT NULL INDEX INDEX_1 HASH WITH (BUCKET_COUNT = 5000000),
col3 INT NOT NULL INDEX INDEX_2 HASH WITH (BUCKET_COUNT = 5000000),
col4 INT NOT NULL INDEX INDEX_3 HASH WITH (BUCKET_COUNT = 5000000),
col5 INT NOT NULL INDEX INDEX_4 NONCLUSTERED,
col6 CHAR (50) NOT NULL, col7 CHAR (50) NOT NULL, col8 CHAR (30) NOT NULL, col9 CHAR (50) NOT NULL)
WITH (MEMORY_OPTIMIZED = ON)
Memory for Table: Timestamps + Index Pointers + Row Data => 24 + (8+8+8+8) + (4+4+4+4+4+50+50+30+50) => 256 x 5000000 => 1.28GB
Memory for Hash Indexes: Round Up Bucket Count x Index Pointer => 223 x 8 => 67,108,864 Bytes => 64MB x 3 => 192MB
Memory for Non-Clustered Index: (Index Pointer + SUM(Key Column Data Type Size)) x Unique Rows => (8 + 4) x 5000000 => 57MB
Memory for Row Versioning: (Longest Trans. Duration in Sec. x Peak No. of Row Updates & Deletes in Sec.) x Row Size
=> ( 1 x 10) x 256 => 2560 Bytes => 2560 x (0.3 x 5000000) => 3.66GB
Visit the following blog post to get SP_InMemTableSizeEst
http://fard-solutions.com/sql-server-memory-table-size-estimation-script/
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
In-Memory Filegroup Configuration
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
In-Memory Data & Delta File Population
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.
Questions and Answers
Copyrights © 2016 Fard Solutions Sdn Bhd, All rights reserved.