Download Powerpoint - SQL Saturday

Database Unit Testing
Fundamentals
 Kevin G. Boles
 [email protected]
Who am I??
Kevin G. Boles
SQL Server Consultant
“The Wolfe” for the Relational Engine
Indicium Resources, Inc.
AIM/Twitter: TheSQLGuru
Gtalk/GMail: [email protected]
MCITP, MCT, MVP 2007-2012, yada-yada
What do I do?
• World-class Better-than-average relational
engine expert 
• ~45,000 man-hours invested in SQL Server
• Absolutely LOVE doing targeted
performance analysis and tuning work
• VERY good at making others better at
interacting with SQL Server
• Couldn’t SSAS my way out of a paper bag
… with both ends open! 
Agenda
• What is “Unit Testing”
• What does it mean to “Unit Test” a database?
• How to do it in SSMS
– Manually coding
– tSQLt Framework
– SQL Test
What is “Unit Testing”
• Code that exercises a specific portion of your
codebase in a particular context.
• Returns a simple pass/fail based on an
expected known result.
• Inspect the results to know how things stand
and possibly act on it.
Goals
• Catch mistakes as early in the process as
possible.
• Provide repeatable, regression-checking tests
that
– A) validate code performs correctly
– B) are EXTREMELY important during refactors!
Goals
• Keep your units of code as small and testable
as possible.
– Tighter, more maintainable code
• Ensure that you have a good idea what the
code is supposed to do, perhaps even before
you write it. (née Test Driven Development)
Goals
• They give you confidence that your code
works as you expect it to work.
Goals (for Database folks)
Can ensure CONFORMITY with Standards
What to Test?
• Often not feasible to test everything
– You’d like to hit ~75% coverage
• Every method, possibly multiple tests
– Normal Conditions
– Edge Conditions
– Null / Empty parameters
– Out of bound conditions
Problems with Unit Testing
• More up-front time / work
– Less instant gratification
– Pays off on the back end
• Hard to justify to upper management
– “Squishy”
– Can’t prove it worked
Unit Testing is not….
• Integration Testing in which we make sure our
individual modules work well together.
– Does the 'Delete' button on the U/I actually invoke the 'Delete' method in the
ORM properly and handle the result?
– Does calling the 'Delete' method in the ORM actually remove a record from
the database?
Unit Testing is not….
• System Testing, when you deliver your app to
the testers and they check it all out to make
sure it meets the requirements. Eg.
– load testing
– compatibility testing
– accessibility testing
– and many more.
Unit Testing is…
• Peace of Mind
What does this have to do with
DATABASES?
Well, there IS code in and hitting databases
• Stored procedures
• Functions
• Constraints
• Views
• Triggers
• fired at your database by your favorite ORM
Example #1
• Write a unit test to ensure scalar function
returns what we think it should.
• Also, make sure it doesn’t return what we
don’t think it should.
• Test edge cases and boundary conditions!!
• Exercise all regions of code (separate tests)
Demo
Example #2
• Write multiple unit tests for a function…
Demo
Example #3
• How do we test tables?
– Static and outputs from sprocs?
• How compare data in them?
• How compare schemas?
• How to test constraints (or NOT test them)?
–
–
–
–
Embed in a TRANSACTION
Try to add some bad data
See if it threw an error
End TRANSACTION
• Triggers?
Views
Old way of testing
1.
2.
3.
4.
Alter a view
Compile?
Run view
Got data? Good to go!
Views
How to test if a View’s data went bad?
1. You’d have to have ‘saved’ a copy of your
view data before you make changes.
2. Make your changes
3. Get your new view data
4. Compare
um…how do I do that?
Testing “Tabular Ouput”
OUTPUT A
ID
OUTPUT B
FieldA
FieldB
ID
FieldA
FieldB
1
ABC
123
1
ABC
123
2
DEF
456
2
NULL
456
3
HIJ
789
4
KLM
101112
INT, Identity (1,1)
Numeric(18,0), Identity (1,1)
This is getting out of hand!
• We could write the unit tests in Visual Studio
– nUnit
– Sql Server Data Tools
• Requires a separate tool which we may not
have.
• It'd be nice if there was a framework for
writing unit tests that allowed us to remain
within SQL Server Management Studio.
tSQLt Framework
• tSQLt is a database unit testing framework for
Microsoft SQL Server.
• http://tsqlt.org/
• Features:
– Test are run within transactions
– Tests are grouped within a schema
– Output can go to a file that a CI tool can use
– …and more
tSQLt Framework Installation
1. Download zip (tSQLt.org)
2. Add scripts to a project (optional)
Comes w/ an example database…
Adding tSql to your database
1. Prep your database
1. ALTER AUTHORIZATION ON DATABASE TO sa
(Maybe)
2. SET TRUSTWORTHY ON
3. Enable CLR on Server
2. Run tSqlt.class script on each database you
want to generate tests in.
Redo our tests in tSQLt
• Create a test class: tSQLt.NewTestClass
– Creates a new schema with that classname
• Create a test: CREATE PROCEDURE
– Schema should be the new class name
– Prefix the sp name with ‘test’
• Run the Test: tsqlt.Run <testName>
Assertions
•
•
•
•
•
•
•
AssertObjectExists
AssertEmptyTable
AssertEquals
AssertNotEquals
AssertLike
AssertEqualsTable
AssertResultSetsHaveSameMetaData
Handling Dependencies
• ApplyConstraint – Add constraint back to
Fake Table by name. CHECK, single-column
FOREIGN KEY only.
• FakeFunction – Remove complex code. Can
replace with your own code.
• FakeTable – Basic copy of table.
• SpyProcedure – Logs execution of sproc
including parameter values.
Faking It
• FakeTable
– No constraints
– ALL NULLable columns
– Can keep Identity definition
– No triggers (can add)
– Temp tables cannot be faked
tSQLt.FakeTable [@TableName = ] 'table name‘
, [[@Identity = ] 'preserve identity']
, [[@ComputedColumns = ] 'preserve computed columns']
, [[@Defaults = ] 'preserve default constraints']
Handling Known Errors
• ExpectException – Good for cases like catching
a PK violation on UPSERT action
tSQLt.ExpectException [
[@ExpectedMessage= ] 'expected error message']
[, [@ExpectedSeverity= ] 'expected error severity']
[, [@ExpectedState= ] 'expected error state']
[, [@Message= ] 'supplemental fail message']
[, [@ExpectedMessagePattern= ] 'expected error
message pattern']
[, [@ExpectedErrorNumber= ] 'expected error number']
Running Tests
• One test per stored procedure
• tsqlt.Run <testName>
• tsqlt.RunTestClass <className>
• tsqlt.RunAll
Bonus Features!
• Sproc: <className>.Setup
– Is executed at the start of every test execution for that
class.
– Great for D.R.Y. stuff
• Create routinely accessed objects/fakes/spys
• Settings/object validation?
• Populating lookups?
• Last Execution Tracking
– tSQLt stores last test executed for each session (by
SPID/LoginTime)
– EXEC tSQLt.Run (no parameter) will exec last test
Tips
• Filter the Stored Procedures to just see Tests
• Assign a hotkey to run tests (Tools, Options,
Environment, Keyboard,
EXEC tSQLt.RunAll
• Create a template for creating a test
• Keep an eye on tlog size!! LOTS of stuff happens
behind the scenes. Be especially aware because
my clients NEVER seem to properly
setup/maintain their DEV/TEST SQL Server
environments! Heck, I’m lucky if they set up
PRODUCTION servers at all!! 
Demos
Not Enough?
• No Pretty Color Coding?
• Visual cues to what/how many tests failed?
• I can't just doubleclick on my failed test to edit
it?!?
SQL Test
• RedGate
• Not Free
– $369 Standalone
– $1895 in SQL Developer Bundle
• SQL Source Control
• Continuous Integration
• LOTS of useful tools!!
SQL Test
Provides …
• A nice GUI
• Color Coding
• SQL Cop Tests
Demo … (actually run Master Sproc Stub)
SQL Test
Auto-Setup
Demo…
Continuous Integration
• Automatically run Unit Tests
– On a schedule
– When code is checked in
•
•
•
•
Reject if fails
Proceed with other tests if succeeds
Compile the app
Distribute
Conclusion
• Unit Testing is for Database People
• Discipline you to write testable code
• Reward you with easy to maintain code
– Fewer errors – ESPECIALLY on refactors!!
– Save money
– Save time
• Conform to standards
Q&A / Comments
• Contact Information:
– Kevin G. Boles
– [email protected]
– @TheSQLGuru
– LinkedIn: TheSQLGuru
Resources
•
•
•
•
•
•
•
•
tSQLt.org Website
Unit Testing Database with tSQLt
Ten Things I Wish I'd Known
48 SQL Cop Tests (Zip File)
Using SQL Test in Continuous Integration
How to write good unit tests
Are Unit Tests overused?
Test Driven Development
•
Special thanks to Eric Selje for the deck and samples