Download Portal Presentation - Melbourne SQL Server User Group

SQL Server 2005
What’s new in the Relational & Storage
Engines
Tony Bain
SQL Server Practice Manager
Red Rock Consulting
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Introduction
What I Cover
• Database Admin
• Indexes
• Meta Data
• Query Processor
• Partitioning
• Exception Handling
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
What I Don’t Cover
• Service Broker
– Event Notification
• .NET Integration
• Queues
• XQuery
• Reporting Services
• Data Transformation
Services
• Analysis Services
Agenda
This is a “What’s New” presentation
• What’s new in TSQL
• What’s new in Meta Data
• What’s new in the Storage Engine
• Misc
• Was also going to do security but don’t have time
– Come and see this at Tech Ed
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
TSQL Commands
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Common Table Expressions
• ANSI SQL 99 Feature
• Code simplification
• Two distinct types
– “Run time” views
– Recursion
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Run Time Views
WITH cte_name AS
(
QUERY
)
SELECT *
FROM cte_name
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Recursion CTE’s
WITH recusvice_cte(parentkey(parentkey, key,
value)
AS
(
SELECT parentkey, key, value FROM mytable
UNION ALL
SELECT m.parentkey, m.key, m.value FROM mytable
m
INNER JOIN recusvice_cte cte on
m.key=cte.parentkey
)
SELECT * FROM recusvice_cte
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
TOP
• Top now supports a parameter value
• About time too!
declare @rows int
select @rows=rand()*10
select top(@rows) * from dbo.Employee
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
APPLY
• Similar to JOIN but feeds the left hand rows into
the right
• Two Types
– CROSS APPLY – No rows are returned for the
evaluation where the right hand side table source
does not produce any rows.
– OUTER APPLY – A row is produced for the
evaluation even when the right hand side table
source does not produce any rows.
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
JOIN
FROM TABLEA INNER JOIN TABLEB ON X=Y
ROW A
ROW B
ROW C
ROW D
ROW E
ROW F
ROW G
ROW H
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
ROW AA
ROW AB
ROW AC
ROW AD
ROW AE
ROW AF
ROW AG
ROW AH
ON
X=Y
ROW B
ROW F
ROW AB
ROW AC
ROW B
ROW F
ROW AG
ROW AH
APPLY
FROM TABLEA CROSS APPLY
dbo.fn_tab1(TABLEA.COLA)
ROW A
ROW B
ROW C
ROW D
ROW E
ROW F
ROW G
ROW H
Table Valued
Function
ROW A
ROW B
ROW C
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
ROW AA
ROW AB
ROW AA
ROW AA
ROW AB
ROW AC
ROW AD
ROW AE
ROW AF
ROW AG
ROW AH
TABLESAMPLE
• A random selection of rows
• Returns approximately the number of rows you
specify
• Specify as % or #
SELECT *
FROM tablex
TABLESAMPLE SYSTEM (100 ROWS)
• REPEATABLE(seed) allows you to get same
selection of rows
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
PIVOT / UNPIVOT
• Rotate rows into columns
• Useful if you have a “soft” schema
• For example, ProductAtrribute table
• Current Limitation
– Only the list of columns in the result is static and
cannot be the result of a query.
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
OUTPUT
• Allows result sets to be returned from INSERT,
UPDATE & DELETE commands
• Have INSERTED & DELETED tables similar to
triggers in SQL Server 2000.
• Returns a standard result set that you can access
from your application
– Of course you will have to use a result set handling
method instead of .ExecuteNonQuery
• Could be used for selective audit
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
OUTPUT Example
update dbo.Customers
Set City=UPPER(City)
OUTPUT 'Changed ' + DELETED.City + ' to ' + INSERTED.City
Changed
Changed
Changed
Changed
Changed
Changed
Changed
Changed
Changed
Changed
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Berlin to BERLIN
México D.F. to MÉXICO D.F.
México D.F. to MÉXICO D.F.
London to LONDON
Luleå to LULEÅ
Mannheim to MANNHEIM
Strasbourg to STRASBOURG
Madrid to MADRID
Marseille to MARSEILLE
Tsawassen to TSAWASSEN
Try / Catch
• Not as first appears - Not TRY / CATCH as per
.NET
• Code simplification that only “handles” a
transaction abort exception
• Creates a wrapper that directs control to error
handling function on transaction abort error
• Set XACT_ABORT on, which means any runtime
error will cause transaction abort
• Cannot actually handle the error. The transaction
is dead, all you can is pass the error up the
application stack.
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
RAISERROR WITH TRAN_ABORT
• RAISERROR has been extended to have a
TRAN_ABORT parameter
• Allows you to manually invoke you tran abort
method
• Kills the transaction though, again no handling
possible
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
EXECUTE AT
• EXECUTE has been extended to include the AT
parameter.
• Allows commands to be passed through to a
linked server
• E.g.
EXEC('SELECT * FROM
Northwind.dbo.Products') AT
OurLinkedServer
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
META DATA
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
fn_virtualindexstats()
• Great new feature 
• Helps reduce the problem of the “Every increasing
number of indexes”
• Shows allocation stats & also usage stats
• KPI’s
– # range scans
– # singleton_lookups
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
fn_transactions()
• Returns information on the transactions that are
currently in progress
– Replaces DBCC OPENTRAN() of SQL 2000
• Returns
–
–
–
–
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
SPID
Transaction Name (if any)
Elapsed time
If transaction is using Snapshot isolation level
DDL Triggers
• Triggers can now be created on events other the
UPDATE, INSERT & DELETE
• Includes DDL statements such as CREATE TABLE
• Can be useful in auditing database changes
• The eventdata() function contains an XML
representation of the event data.
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
eventdata()
<EVENT_INSTANCE>
<PostTime>2004-01-12T16:42:50.237</PostTime>
<SPID>52</SPID>
<EventType>CREATE_TABLE</EventType>
<Database>AdventureWorks</Database>
<Schema>dbo</Schema>
<Object>MyTable</Object>
<ObjectType>TABLE</ObjectType>
<TSQLCommand>
<SetOptions ANSI_NULLS="ON" ANSI_NULL_DEFAULT="ON"
ANSI_PADDING="ON" QUOTED_IDENTIFIER="ON"
ENCRYPTED="FALSE" />
<CommandText>CREATE TABLE
MyTable(IDCOLINT)</CommandText>
</TSQLCommand>
</EVENT_INSTANCE>
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
DDL Triggers (cont)
CREATE_TABLE
ALTER_TABLE
DROP_TABLE
CREATE_VIEW
ALTER_VIEW
DROP_VIEW
CREATE_SYNONYM
DROP_SYNONYM
CREATE_FUNCTION
ALTER_FUNCTION
DROP_FUNCTION
CREATE_PROCEDURE
ALTER_PROCEDURE
DROP_PROCEDURE
CREATE_TRIGGER
ALTER_TRIGGER
DROP_TRIGGER
CREATE_EVENT_NOTIFICATION
DROP_EVENT_NOTIFICATION
CREATE_INDEX
ALTER_INDEX
DROP_INDEX
CREATE_STATISTICS
UPDATE_STATISTICS
DROP_STATISTICS
CREATE_ASSEMBLY
ALTER_ASSEMBLY
DROP_ASSEMBLY
CREATE_TYPE
DROP_TYPE
CREATE_USER
ALTER_USER
DROP_USER
CREATE_ROLE
ALTER_ROLE
DROP_ROLE
CREATE_APPLICATION_ROLE
ALTER_APPLICATION_ROLE
DROP_APPLICATION_ROLE
CREATE_SCHEMA
ALTER_SCHEMA
DROP_SCHEMA
CREATE_MESSAGE_TYPE
ALTER_MESSAGE_TYPE
DROP_MESSAGE_TYPE
CREATE_CONTRACT
ALTER_CONTRACT
DROP_CONTRACT
CREATE_QUEUE
ALTER_QUEUE
DROP_QUEUE
CREATE_SERVICE
ALTER_SERVICE
DROP_SERVICE
CREATE_ROUTE
ALTER_ROUTE
DROP_ROUTE
CREATE_REMOTE_SERVICE_BINDING
ALTER_REMOTE_SERVICE_BINDING
DROP_REMOTE_SERVICE_BINDING
GRANT_DATABASE
DENY_DATABASE
REVOKE_DATABASE
CREATE_EXPRESSION
DROP_EXPRESSION
CREATE_XMLSCHEMA
ALTER_XMLSCHEMA
DROP_XMLSCHEMA
CREATE_PARTITION_FUNCTION
ALTER_PARTITION_FUNCTION
DROP_PARTITION_FUNCTION
CREATE_PARTITION_SCHEME
ALTER_PARTITION_SCHEME
DROP_PARTITION_SCHEME
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
DDL Triggers (cont)
CREATE_LOGIN
ALTER_LOGIN
DROP_LOGIN
CREATE_ENDPOINT
DROP_ENDPOINT
GRANT_SERVER
DENY_SERVER
REVOKE_SERVER
CREATE_CERTIFICATE
ALTER_CERTIFICATE
DROP_CERTIFICATE
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Storage Engine
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Persisted Computed Columns
• Traditionally a Computed Column is calculated at
run time
• In SQL Server 2005 we have the option to persist
the values in our computed columns
• Persisted Computed Columns can be indexed
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Non Key Index Column Inclusion
• In SQL Server 2000, to make an index covering all
columns had to form the index key
• INCLUDE allows additional columns to be
included in the index leaf level, but not included in
the index keys
• Uniqueness and sorting only maintained for the
key columns
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Full Text Indexes
• Now brought inside SQL Server
• FT Catalogs now backed up as part of the
database backup
• You can also just backup the catalog
– BACKUP DATABASE dbname FILE=‘fulltext_cat’
TO device
• Now has a Thesaurus
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Disabling Indexes
•
•
•
•
Indexes can be disabled
Stops them being used
Stops them being updated
Can be used during large data loads instead of
drop / recreate
• Debug / index clean up etc
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Database Views
• Database Views allow the state of a database at a
given point in time to be preserved and made
available in a read only form
• I.e. a logical representation of the database at the
time of view creation
CREATE DATABASE …. AS VIEW OF dbname
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Database Views (cont)
• Efficient as they only store the inverse of changes
made to the database being viewed
• Utilize a type of data file called a spares file (NTFS
5 feature).
• Spares files provide a logical representation of a
file separate to the way the file is physically stored
on disk
• Logically – Fully allocated
• Physically – Only actual data
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Database Views (cont)
• Spares Files
Logical File
Free
Free
Data
Free
Data
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Data
Data
Free
Free
Free
Physical File
Database Views (cont)
CREATE DATABASE DB1VIEW
…. AS SNAPSHOT OF DB1
Database View (MDF)
Free
Free
Free
Free
Free
Free
Free
Free
Data
Data
Data
Data
Data
Data
Data
Data
Database (MDF)
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Database Views (cont)
SELECT * FROM Orders WHERE
ORDERID=100
Database View (MDF)
Free
Free
Free
Free
Free
Free
Free
Free
Data
Data
Data
Data
Data
Data
Data
Data
Database (MDF)
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Database Views (cont)
SELECT * FROM Orders
Database View (MDF)
Free
Free
Data
Free
Free
Data
Free
Free
Free
Data
Free
Data
Data
Data
Data
Data
Data
Data
Data
Database (MDF)
UPDATE Orders SET
Price=Price*1.5
WHERE OrderData BETWEEN
20040101 AND 20040131
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Database Views (cont)
Database
Database View
Database View
Database View
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Database Views (cont)
• There is major current limitation…..
– They cannot be backed up
• Makes sense, yes
– But still will seriously limit any practical application
of such views.
– Hope we will see a “synchronized backup” strategy
in place soon.
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Mirrored Backups
• Allow you to create a duplicate backup set
– Not a stripe
• Useful if backing up directly to tape
– Is anyone doing this by the way?
• Improve redundancy of the backup
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Mirrored Backups (cont)
• Assume we backup database XYZ to:
– Mirror A, which consists of backup devices A1, A2
and A3
– Mirror B, which consists of backup devicesB1, C2
and C3
– Mirror C, which consists of backup devices C1, C2
and C3
• If backup devices A1, B2 and C3 where to become
damaged we would still be able to recover our
database by restoring from the combination of B1,
A2 and C3.
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Large Object Data
• TEXT, NTEXT and IMAGE are gone
• About Time Too!!
• Welcome,
• VARCHAR(MAX), NVARCHAR(MAX),
VARBINARY(MAX)
• While they may be internally stored different,
externally we don’t care. They behave the similar
to non-blob types (ala Access).
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Large Object Data (cont)
CREATE TABLE #TABLE_ABC
(
IDCOL
INT PRIMARY KEY,
TextBlob
TEXT
)
INSERT #TABLE_ABC
VALUES(1,‘Short Text Data')
INSERT #TABLE_ABC
VALUES(2,‘Some more Text Data')
--The following causes an error
SELECT *
FROM #TABLE_ABC
WHERE TextBlob='Short Text Data '
Server: Msg 306, Level 16, State 1, Line 13
The text, ntext, and image data types cannot be compared
or sorted, except when using IS NULL or LIKE operator.
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Data Partitioning
• In SQL Server 2005 strategy has changed
• Moved away from impacting the physical data
model
– Sits between data model and physical page
allocations
• Becomes data storage issue
– AS IT SHOULD BE!
• New concepts
– Partition Functions – Routine that chooses “bucket”
– Partitioning Scheme – Bucket to Filegroup mapping
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Data Partitioning (cont)
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Data Partitioning
• Splitting large volumes of data into more
manageable sub sets
• SQL Server 2000
–
–
–
–
Create multiple tables
Assign PK constraints to each table
Create view over tables
Query / Modify view
• Difficult to change
• Difficult to manage (indexes on each table)
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Instant File Initialization
• SQL Server 2005 creates databases without
zeroing out all the pages allocated.
Creation Duration
Database Creation Comparison
SQL 2000
SQL 2005
10
100
Database Size
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
1000
Row Overflows
• Rows can now be over the 8Kb limit
• Comes into play when variable length columns are
over 8kb (excluding blob)
• SQL Server will move the largest variable length
column into it’s own page and link that child page
to the original row.
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Row Overflows (cont)
INSERT TABLE_ABC VALUES(1,’ABC’,’DEF’)
INSERT TABLE_ABC VALUES(1,REPLICATE(’A’,8000),REPLICATE(’B’,8000))
TABLE_ABC
1
ABC
Page
1
AAAAAAAAAAA…
Page
BBBBBBBBBBBB…
Page
Page
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
KEYCOL INT
COL1 VARCHAR(8000)
COL2 VARCHAR(8000)
DEF
BBBBBBBBBBBB…
Column Link
Deferred Drop
• When an object over 128 extents (1024KB) is
dropped SQL Server 2005 does not immediately
reclaim the space allocated by that object
• A background process outside the scope of the
drop transaction handles the physical deallocation
• This improves the performance of maintenance
operations such as index rebuilds on large tables
• Because deferred drops do not deallocate pages
at the time of dropping, the space consumed by
the object may not be immediately available for
resue.
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Statistics On Views
• By reading BOL you won’t get a feeling for how
cool this feature is.
• If you’re keen read “Statistics on Views”
– Galindo-Legaria, Joshi, Wass, Wu
– Will link from www.tonybain.com
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Statistics on Views (cont)
• Cost based optimization evaluates a series of
candidate plans and picks plan with least
estimated cost
• Accuracy of cost estimation = quality of plan
• Incorrect costing may lead the optimizer to
exclude plans deemed as inefficient when they’re
not
• Estimating the size of the result set, uses table
statistics
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Statistics on Views (cont)
• The shortcomings of this approach
– Errors grow as estimation is done on top of
estimation
– Some constructs cannot be estimated
• In these cases the optimizer guesses using “magic numbers”
• 1/3 data reduction factor for !=
• 1/10 data reduction factor for =
– Queries can become unstable with inaccurate
estimation
e.g.
SELECT * FROM LINEITEM
L_PRICE * (1 - DISCOUNT) > 9000
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Statistics on Views (cont)
• Statistics on views allow the optimizer access to
higher level information
• Provides more accurate estimation that
compounded estimation
• Provides statistical information for where a “good
guess” would have otherwise been used
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Snapshot Isolation Level
•
•
•
•
Optimistic concurrency level
Readers do not block writers
Writers do not block readers
When turned on any update stores a version in
tempdb
• Readers can access the last committed value
while it is being updated
• Snapshot will always read coherent data
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Snapshot Isolation (cont)
• To enable
ALTER DATABASE dbname SET
ALLOW_SNAPSHOT_ISOLATION ON
• A server trace flag is need to change the default
isolation from Read Committed to Snapshot Read
Committed
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Summary
• Aside from the “flashy” features SQL Server 2005
has some great grass roots features
• SQL Server 2005 BETA 2 should be privately
available any week now
• Statistics on Views is a great new feature
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
More Information
• Tech Ed 2004 – Canberra
– Come see me Friday August 6 2:30pm
• SQL Server 2005 Security
• SQL Server 2005 Express Tech Preview
• http://lab.msdn.microsoft.com/express/sql/
• www.tonybain.com
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Red Rock SQL Services
• We are the leaders in SQL
Server & Oracle support
and consulting
• 100+ Consultants
• Service clients in 6
countries
• An autonomous business
unit of UXC
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
Red Rock SQL Services
• Rock Solid Managed
Support – A cost effective
DBA support solution
• Specialised SQL Server
Consulting Services
• Project Consulting
Resources
• Development, BI, Data
Warehousing & Reporting
About Your Speaker
• 7+ years SQL Server
experience
• 50+ clients
Copyright (C) 2004 Red Rock Consulting Pty Ltd
All Rights Reserved
• SQL Server MVP