Download SQL 2005 - Microsoft

SQL 2005
Kevin Ashby
Microsoft EMEA
Enterprise Technology Strategist
[email protected]
Agenda
 Introduction
 Enterprise Data
 Business
Management
Intelligence
 Clients
 Using SQL2005, includes Microsoft IT
Introduction
Today’s IT
Desired IT
20% average annual increase in R&D spend
Note: The FY’04 figure restated to include stock-based compensation (as per SFAS 123)
*FY’04; Includes stock-based compensation as per SFAS 123; Excluding this, the figure would be $4.8B
**IBM’s software R&D is estimated to be $0.8B-$1.9B, using two contribution figures – one based on revenue and the other based on gross margin; HP’s
software R&D is estimated using the % of software revenue within the Enterprise Systems Group in Q4FY'03-Q1FY'04
Source: Company annual reports
SQL Server Generations
of Innovation
1stHistory
Generation
2nd Generation
SQL Server
6.0/6.5



SQL Server
7.0
Differentiation
from Sybase SQL
Server
Windows
integration
First to include
Replication
Cross-release
objectives
•
•



Re-architecture of
relational server
Extensive auto
resource
management
First to include
OLAP & ETL
3rd Generation
SQL Server
2000




Performance,
scalability focus
XML support
First to include
Notification
First to include
Data Mining &
Reporting
Reliability & Security
•
Integrated Business Intelligence •
SQL Server
2005





True High
availability
Security
Developer
productivity
Native XML
First to include
Enterprise ETL &
Deep Data Mining
Lowest TCO
Automatic Tuning
NET Framework
 Common Language Runtime Integration
 User-defined Aggregates
 User-defined Data Types
 User-defined Functions
 SQL Server .NET Data Provider
 Extended Triggers
Data Types
 Managed SQL Types
 New XML Datatype
 Varchar (MAX) Varbinary (MAX)
SQL Server Engine
 New Message Service Broker
 HTTP Support (Native HTTP)
 Database Tuning Advisor
 Multiple Active Result Sets
 Persisted Computed Columns
 Snapshot Isolation Level
 Scale Up Partitioning
 VIA support
 NUMA support
Database Failure and Redundancy
 Fail-over Clustering (up to 8 node)
 Enhanced Multi-instance Support
 Database Mirroring
 Database Snapshots
XML
 XQUERY Support
 XML Data Manipulation Language
 FOR XML Enhancements
 XML Schema (XSD) Support
 MSXML 6.0 (Native)
 .Net XML Framework
Full-text Search
 Indexing of XML Datatype
Database Maintenance
 Backup and Restore Enhancements
 Checksum Integrity Checks
 Dedicated Administrator Connection
 Dynamic AWE
 Fast Recovery
 Highly-available Upgrade
 Online Index Operations
 Online Restore
 Parallel DBCC
 Parallel Index Operations
Management Tools
 New Management Studio
 MDX Query Editor
 Version Control Support
 XML/A
 SQLCMD Command Line Tool
Performance Tuning
 Profiler Enhancements
 Profiling Analysis Services
 Exportable Showplan
 Exportable Deadlock Traces
MDAC
 SNAC
 Microsoft Installer base setup
 Support for Active Directory Deployment
SQL Server Client .NET Data Provider
 Server Cursor Support
 Asynch
Security
 All Permissions Grantable
 Fine Grain Administration Rights
 Separation of Users and Schema
 Data encryption primitives
Replication
 Auto-tuning Replication Agents
 Oracle Publication
 Improved Blob Change Tracking
 Replication Monitor
OLAP and Data Mining
 Analysis Management Objects
 Integrated Backup/Restore with Windows Server
 Web Service/XML for Analysis
 DTS and DM Integration
 New Data Mining Algorithms
 Auto Packaging and Deployment
Integration Services
 New Architecture (DTR + DTP)
 Complex Control Flows
 Control Flow Debugging
 For Each Enumerations
 Property Mappings
 Full Data Flow Designer
 Full DTS Control Flow Designer
 Graphical Presentation of Pkg Execution
 Immediate Mode and Project Mode
 Package (Advanced) Deployment Tools
 Custom Tasks and Transformations
Reporting Services
 Multiple Output Formats
 Parameters (Static, Dynamic, Hierarchical)
 Bulk Delivery of Personalized Content
 Support Multiple Data Sources
 STS (Web Parts, Doc Libraries)
 Visual Design Tool
 Charting, Sorting, Filtering, Drill-Through
 Scheduling, Caching
 Complete Scripting Engine
 Scale Out architecture
 Open XML Report Definition
Notification Services & SQL Server Mobile Ed.






Integrate and Transform Data
Analyze, Store and Mine Data
Report and Interact with Data



Integration with Visual Studio and .NET
Native XML technology
Interoperability via Web Services
High availability for enterprise applications
Key security and performance features
Focus on self manageability and optimization
2004 Relational DB Market
Non-mainframe
DB ($6 Bil)
Non-mainframe DB
($5.4B)
Growth & Shares By OS Platform
2004
Other
11%
IBM
20%
+2
+2%
SQL
Server
26%
Unix/Linux
49%
Windows
51%
+18
+18%
Oracle
43%
2003
Other
12%
IBM
22%
SQL Server
Leads on Windows
SQL
Server
24%
Windows Database
Market ($3.1 Bil)
Passed
IBM in
2003
+15%
+15
Windows is
the leading
Oracle
42% OS platform
Other
8%
IBM
16%
SQL
Server
51%
Oracle
25%
Source: Gartner
Source:
$40K
$25K
10g
Base Product
DB2
$25K
$330K
$232K
Multi-core
Business
Intelligence
High Availability
DB2 OLAP: $35k
BI Bundle
DB2 Warehouse: $75k
$40k
Cubeviews: $9.5k
RAC: $20k
Recovery
Expert: $10k
Tuning Pack: 3k
Diagnostic Pack: 3k
Partitioning: 10k
Performance
Expert: 10k
$25K
Manageability
10g
Base Product
DB2
Enterprise Data Management
Scalable, Reliable, Secure
SQL Server 2005 is 7% better performance at SQL
better
performance
at
SQLServer
Server2005
2005isis38%
162%
better
performance
SQL Server now in the Million tpmC club
37% lower cost than Oracle (non-clustered)”
20%
lowerlower
cost cost
thanthan
Oracle
(non-clustered)”
at 50%
SQL
Server 2000
SQL Server does not Scale?
SQL Server 2005 Scales
With Hardware and Operating
With Database Features
With Application Design
System
Scaling - Hardware Options
 SQL Server optimized for hardware
 Known as the "SQLOS" abstraction

& OS
This enables better support on
 64 bit architectures
 NUMA systems
 Threads managed as tasks
 Enables SQL Server use of new OS features
 Take advantage of Windows Server 2003
 Take advantage of Windows Vista
Remember : Dual core support at no extra cost!!
Windows Server 2003 Enabled
 Password policy check for SQL passwords
 Hot add memory
 Dynamic AWE
 Native 64 bit support
 SOAP support (HTTP in the kernel)
 Instant file initialization
 8 node SQL Server failover cluster
Advanced OS features used because SQLOS abstraction
Scaling Data with Services
 Functionality built-in to
 Asynchronous
SQL Server
Operations - Service Broker

To achieve scalable, resilient large scale Applications
 Uses Queues
 15k-20k messages per second!
 System/Application continues to work with partial outage, things just queue up
 Cache coherency – Query
 Master data management
 Request-response - Web
 Industry standard protocol
Notifications
Services
 Service Programs can be T-SQL or SQLCLR
 Better performance and flexible deployment
 Choose where to run!
SQL Service Broker
 A Platform
for building reliable, asynchronous, loosely coupled
database applications
 Queues are database objects
 DDL and DML is the same as vanilla T-SQL
 Input in one transaction/context, execute in a different one
 Queue locking reduces conflicts and deadlocks
 Locks are based on dialogs (point-to-point conversation)
 Dialogs give unprecedented message ordering
 Reliable,
durable, sequenced communications session between
services

Ordering even across transactions
Dialogs
 Dialogs provide two-way messaging between two services
 Dialogs offer:
 Guaranteed delivery
Messages will get there, only once and in the right
 Exactly-once delivery
order, even if queue is turned off, so no coding to
 In-order delivery
deal with message arrival order
 Secure communications
 Dialogs:
 May be long-lived (years) or short-lived (seconds)
 Are light-weight
 Are persistent sessions (stored in the DB, so moving DB moves the dialog)
Customer
Service
Travel
Service
Dialog
Database A
Database B
Messaging with Service Broker



Inbound messages arrive on protocol pipe
Then the message is:
 Authenticated
Service Program
(decrement_inventory)
 Dispatched to appropriate queue
Then a Service Programs:
 Pick up work from queue
 May run inside or outside server (on application server)
System continues to run
 May send additional messages
if service program
or queue is unavailable!
X
Message
Message
Query Notifications
 Notify Caches When Master
Data Changes
 built into SQL Server 2005
 based on indexed view notifications
Put data closer to the edge!
 built into ADO.NET
 cache listeners (for when data changes) can be scaled to
multiple machines using SQL Express
 delivery via Service Broker
 built into ASP.NET
 automatic cache invalidation
 Known as Cache Sync
 two lines of code
multiple
granular
replicas
master data
CacheSync
Web Request
Query
Results
Subscription
Results read from master data into cache
CacheSync
Web Request
Subscription
Results read from cache as master not changed
CacheSync
Cache updated with new master data
Subscription
UPDATE dbo.Products SET …
SQL Server is not Secure?
Database Security Bulletins
74
20
14
2
13
11
2002
2
1
2003
2004
SQL
Server
Source: Company web sites, http://www.osvdb.org,
http://www.secunia.com (Includes both SQL Server and MDAC)
Oracle DB
2005
Security
SQL
Server
is part of the Trustworthy Computing
initiative
Whether your company is small, medium, or
large, security is not optional
Data is your company's view of "reality“
Make data secure
You must be secure for accurate picture
Database security consists of
Security by design - integrated with policy
Security by default (only turn on what you need)
Secure deployment and maintenance
Secure communications and storage
SQL Server 2005 Security Initiatives
Focus on security, privacy, and tools
 Enhanced security features
Encryption and decryption of data with
Advanced auditing, authentication, and
 Tracing,
key management
authorization
Events, DDL, custom triggers (Data dict is exposed )
 Reduction in surface area
More optional installation options
Minimized Attack surface: Features
require explicit
configuration
 Deployment improvements
Integration with Microsoft Update services
 Common Criteria
Certification targeted at EAL4+
 Security Tools
Continued MBSA investments
Best Practices Analyzer
Off by Default
Off by Default!
DBA must actively decide to turn ON a feature
 User-schema
separation
- database objects need not be tied to users
Secure Data
& Metadata
Fixes "user leaves company" problem
Allows DBA to allow installation of packages with owners
other than DBO
 So users will not need dbo access to access packages
Allows separation of database object owners even within
a single database
 Secure Metadata
You can only see what you can access
No view of objects you are not allowed to see
Consolidation without seeing others' data
 All Permissions Grantable
Granular permissions
Give only the required permissions to the correct users
Encryption and Privilege
Some industries require encryption
Encryption keys securely stored in database
Instance key protected by DPAPI
(data protection API)
Logins are always encrypted
Procedures can be signed or run as certain accounts
Principle of least privilege
Original login always available for auditing
Proxy accounts for SQL Agent jobs
Symmetric Key Encryption
Cryptography
Faster
101
Encryption
Same key to lock and unlock
0x0088840517080E4FA2…
1234-5678-1234-5678
Decryption
Asymmetric Key Encryption
Slower
Encryption with public key
Lock with public key unlock with private (visa versa)
1234-5678-1234-5678
Public Key
0x0088840517080E4FA2…
Decryption with private key
Private Key

Good
SQL Scenario:
Server Encryption
 Encrypting secrets during login
 Using asymmetric keys to generate

session keys
 Using symmetric keys for data encryption

Faster!
 Using
SQL Server certificates from
trusted sources
 Encrypting data as required by law

Not all data, it restricts its use
Bad Scenario:
 Encrypting all
network traffic inside an
organization
 Using asymmetric keys for data encryption

Slow
 Using
symmetric keys for main key
distribution mechanism

Since it is the same key
 Using SQL Server as a certificate server
 Encrypting all data
 SLOW, and data can't be used for indexes and
joins, eg order by would not work
Addressing Barriers to Availability

Database Server Failure



Database Maintenance & Operations




Online Index Operations
Fast Recovery
Fine-Grained Online Repairs
Data Access Concurrency Limitations


Failover Clustering
Database Mirroring
Snapshot Isolation
User, Application Errors

Database snapshots
Database Mirroring

Database Failover
Very fast failover…less than 3 seconds
Automatic or manual failover
Automatic, transparent client redirect

Achieving high availability while managing costs


Application

Database Snapshots for Reporting

Works with standard hardware and storage
Witness
Principal
Mirror
1
5
2
SQL Server
2
Log
3-6
Data
SQL Server
4
3
Log
3-6
Data
Basic Principle of Mirroring
Acknowledge
Acknowledge
Commit
Constantly
Redoing on
Mirror
Transmit to Mirror
Write to
Local
Log
DB
Committed
in Log
Log
Write to
Remote
Log
Log
DB
•Scenario:
•Itanium 8 principal, Itanium 9 mirror, Itanium 10 witness
•To list the existing Endpoints:
•select * from sys.database_mirroring_endpoints
•To delete existing Endpoints:
•drop endpoint "endpoint"
•Start with a backup and create the first end point (from Itanium 8)
•backup database Database1 to disk ='C:\BACKUP1.bkf' with init
•create endpoint endpoint state=started as tcp (listener_port=5022) for database_mirroring (role=partner)
•Restore the backup on intended mirror server and create the second endpoint(from Itanium 9)
•restore database Database1 from disk='C:\backup1.bkf' with norecovery , replace
•create endpoint endpoint state=started as tcp (listener_port=5023) for database_mirroring (role=partner)
•Create the third endpoint on the Witness (from Itanium 10)
•create endpoint endpoint state=started as tcp (listener_port=5024) for database_mirroring (role=WITNESS)
•Set up Database Mirroring
•From ITANIUM9
•ALTER DATABASE DATABASE1 SET PARTNER = 'TCP://ITANIUM8.SQL.NET:5022'
•GO
•From ITANIUM8
•ALTER DATABASE DATABASE1 SET PARTNER = 'TCP://ITANIUM9.SQL.NET:5023'
•GO
•ALTER DATABASE DATABASE1 SET WITNESS = 'TCP://ITANIUM10.SQL.NET:5024'
•GO
•Create the snapshot on the mirrored DB
•CREATE DATABASE DB1SNAP ON ( NAME = Database1, FILENAME = 'C:\dB1SNAP.ss' ) AS SNAPSHOT OF
Database1;
•GO
HP Proliant
Witness
Mirror
Mirror
Principal
Principal
HP Integrity
HP Integrity
SNAC:SQL Native Client
Online Index Operations

Online index maintenance



Create, Rebuild, Reorganize, Drop, Add or drop constraint
Fully parallel
Online/offline are both supported
Other Availability Features

On-Line Page and File restore


Dedicated Admin Connection


Database remains online - Only data being restored is unavailable
Access to Server when regular connections fail
Fast Recovery

Database is available when undo begins
Redo
Undo
Available
Database Snapshots

Snapshot of a database at a point in time



Does not require a complete copy of the data

Shares unchanged pages of the database

Requires extra storage only for changed pages
Uses a “copy-on-write” mechanism


Created instantly
Read only
Recover from User, Application or DBA error

Rewind database to Database Snapshot
Snapshot (Copy on Write)
Command
Northwind
Northwind_SS
Create Northwind_SS
Update Northwind
Read Northwind_SS
Value
Result:
Value
R
DD
D
X
B
H
J
Space Used 12.5%
0%
L
Y
M
D
Oracle Publishing
Transactional Replication

Designed specifically for Oracle Publishers


v8+ on any operating system
Administered like SQL Server, from SQL Server

No Oracle side software install necessary
 Requires minimal knowledge of Oracle

Leverages existing SQL Server skills

Standard Transactional and Snapshot Publications
SQL Server 2005
Distributor
Subscribers
Management Studio
 Integrated management
 Relational DB,
Analysis Services,
Reporting Services,
Notification Services,
SQL Server CE…
 Multiple versions of SQL from one interface
 Rich management & authoring
 Performance
 Manage multiple servers, MANY objects
 Operations: Profiler, Query Analyzer
Manageability - Tools

New: SQL Server Management Studio




New: SQLCMD


New Rich GUI
Integrated Authoring, Management
Support all SQL Server components
High Performance CMD line tool
New: SQLiMail Replacement

SMTP Support, Cluster Support, No Outlook required
Manageability - Tuning
 Database Tuning Advisor
 Time
Bound Tuning
 What-If Analysis
 Scalability
 XML Show
Plan
 Publicly available Schema
 Programmatic Access
 Portability
Manageability – Diagnostics
“What is happening in my DB”
 Dynamic Management Views
Server Transparency
“sys.memory_clerks” – how
much memory is each
component consuming ?
“sys.schedulers” – Is a scheduler hung ?
 SQL Trace
Query data from live traces
 SQL Profiler
Deadlock and ShowPlan Visualization





Range Partitioning breaks a single object into multiple manageable pieces
Range Partitioning
Partitioning is transparent to the application
Partitioning applies to

Tables

Indexes
The row is the unit of partitioning
All partitions reside in a single database
CREATE
PARTITION
FUNCTION
annual_range (DATETIME)
Partition
Function
DDL
as RANGE RIGHT
for values
(
-- Partition 1 -- 2001 and earlier
'2002-01-01',
-- Partition 2 -- 2002
'2003-01-01',
-- Partition 3 -- 2003
'2004-01-01',
-- Partition 4 -- 2004
'2005-01-01'
-- Partition 5 -- 2005 and later
)
64-bit Database Platform
High-performance computing
Scalability
Manageability

Optimized for Windows Server 2003

Support for Xeon & Itanium

Great performance
Large memory addressability (up to 32 TB)

Nearly unlimited virtual memory (up to 8 TB)

I/O savings due to larger memory buffer pools

Numa Support

T-SQL code-compatibility with SQL Server 2000
8 node clustering support
Same on-disk format as 32-bit for easy migration
64bit support for relational engine and BI

Compelling alternative to expensive Unix solutions



Cost Savings

The highly scalable database platform for memory
intensive, performance-critical business applications
Summary: DBA Productivity with SQL Server 2005
 Scalable, Reliable and
 Ease-of-use
 Achieve
 Scales
secure
and TCO leader for database management
high availability without added complexity, cost
from mobile device to the datacenter
Business Intelligence
BI Part of CIO’s Top 10 Priorities
Technology Priorities Clearly Reflect the Business Priorities
Top-Ten Technology Priorities, 2005
Ranking
2005
2004
2003
Security enhancement tools
1
3
1
Business Intelligence applications
2
10
13
Mobile workforce enablement
3
-
-
Workflow management deployment and integration
4
-
-
Enterprise resource planning (ERP) upgrades
5
-
-
Storage management
6
-
-
Voice and data integration over IP
7
-
-
Customer relationship management (CRM)
8
-
-
Business process integration tools
9
-
-
Server virtualization
10
-
-
#1 in Business Intelligence (OLAP)
Unranked in 1998!
Source: olapreport.com
Integrate





Data transformation
and synthesis
Efficient processing of
large data volumes
Reduced hand coding
with pre-built adapters
and transformations
Complex data handling
Smart data cleansing
(DW specific)
Analyze






Data analysis for a
variety of user needs
Centralized business
logic in a unified model
Transparent data storage
Reduced data latency
Enhanced data mining
Streamlined
infrastructure
Report





Data presentation and
distribution
Business user report
design
Report controls
Infinite drill down
Sharepoint integration
Clients
Plan to go Live on CTP
Help-Desk Service application
Mobility application for delivery trucks
Large Retail Data Warehouse
High-transaction trading infrastructure
Large 16-way Tandem Migration
5000 concurrent user OLAP application
Large SAP/R3 Deployment
Multiple internal Systems – High Tech Vertical
Early Customer Feedback
Will go Live by Launch
•
Revenue forecasting and analysis
•
Arizona Supreme Court. AS/400 Migration
Juvenile Online Tracking System
•
Data Warehouse for cement quality analysis
Web services architecture – real-time BI
•
Will migrate multiple systems
Uses SQL Server throughout enterprise
•
Credit risk analysis data warehouse
Uses SQL Server throughout enterprise
•
Austrian TV station automates Sports News
Deep XML technology
Small research firm runs customer database
•
IT Staff=1, not possible in Oracle