xUnit Style Database Unit Testing
ACCU London – 20th
January 2011
Chris Oldwood
gort@cix.co.uk
Presentation Outline
• Database Development Process
• The xUnit Testing Model
• Test First Development
• Continuous Integration/Toolchain
• Pub
Legacy Database Development
• Shared development environment
• Only integration/system/stress tests
• No automated testing
• Only real data not test data
• Referential Integrity – all or nothing
• No automated build & deployment
Ideal Development Process
• Isolation
• Scaffolding
• Automation
Example Testable Behaviours
• Default constraint
• Trigger to cascade a delete
• Refactoring to a surrogate key
NUnit Test Model
[TestFixture]
public class ThingTests
{
[Test]
public void Thing_DoesStuff_WhenAskedTo()
{
var input = ...;
var expected = ...;
var result = ...;
Assert.That(result, Is.EqualTo(expected));
}
}
NUnit Test Runner
• Tests packaged into assemblies
• Uses reflection to locate tests
• In-memory to minimise residual effects
• Output to UI/console
SQL Test Model
create procedure test.Thing_DoesStuff_WhenAskedTo
as
declare @input varchar(100)
set @input = ...
declare @expected varchar(100)
set @expected = ...
declare @result varchar(100)
select @result = ...
exec test.AssertEqualString @expected, @result
go
SQL Test Runner
• Tests packaged into scripts (batches)
• Uses system tables to locate tests
• Uses transactions to minimise residual
effects
• Output to UI/console
SQL Asserts
• Value comparisons (string, datetime, …)
• Table/result set row count
• Table/result set contents
• Error handling (constraint violations)
Setup & Teardown
• Per-Fixture (static data)
• Per-Test (specific data)
• Use helper procedures
Default Constraint Test
create procedure test.AddingTask_SetsSubmitTime
as
declare @taskid int
declare @submitTime datetime
set @taskid = 1
insert into Task values(@taskid, ...)
select @submitTime = t.SubmitTime
from Task t
where t.TaskId = @taskid
exec test.AssertDateTimeNotNull @submitTime
go
Trigger Test
create procedure DeletingUser_DeletesUserSettings
as
...
set @userid = 1
insert into AppUser values(@userid, ...)
insert into AppUserSettings values(@userid, ...)
delete from AppUser where UserId = @userid
select @rows = count(*)
from AppUserSettings
where UserId = @userid
exec test.AssertRowCountEqual @rows, 0
go
Unique Key Test
create procedure AddingDuplicateCustomer_RaisesError
as
...
insert into Customer values(‘duplicate’, ...)
begin try
insert into Customer values(‘duplicate’, ...)
end try
begin catch
set @threw = 1
end catch
exec test.ErrorRaised @threw
go
Automation
• Enables easy regression testing
• Enables Continuous Integration
• Performance can be variable
Test First Development
• Start with a requirement
• Write a failing test
• Write production code
• Test via the public interface
The Public Interface
• Stored procedures
• Views
• Tables?
Implementation Details
• Primary keys
• Foreign keys
• Indexes
• Triggers
• Check constraints
• Default constraints
Deployment Testing
Build version N+1
then run unit tests
Build version N
then patch to N+1
then run unit tests
==
Buy or Build?
• Batch file, SQL scripts & SQLCMD
• TSQLUnit & PL/Unit
• Visual Studio
• SQL Server/Oracle Express
“The Oldwood Thing”
http://chrisoldwood.blogspot.com
Chris Oldwood
gort@cix.co.uk

xUnit Style Database Testing

  • 1.
    xUnit Style DatabaseUnit Testing ACCU London – 20th January 2011 Chris Oldwood gort@cix.co.uk
  • 2.
    Presentation Outline • DatabaseDevelopment Process • The xUnit Testing Model • Test First Development • Continuous Integration/Toolchain • Pub
  • 3.
    Legacy Database Development •Shared development environment • Only integration/system/stress tests • No automated testing • Only real data not test data • Referential Integrity – all or nothing • No automated build & deployment
  • 4.
    Ideal Development Process •Isolation • Scaffolding • Automation
  • 5.
    Example Testable Behaviours •Default constraint • Trigger to cascade a delete • Refactoring to a surrogate key
  • 6.
    NUnit Test Model [TestFixture] publicclass ThingTests { [Test] public void Thing_DoesStuff_WhenAskedTo() { var input = ...; var expected = ...; var result = ...; Assert.That(result, Is.EqualTo(expected)); } }
  • 7.
    NUnit Test Runner •Tests packaged into assemblies • Uses reflection to locate tests • In-memory to minimise residual effects • Output to UI/console
  • 8.
    SQL Test Model createprocedure test.Thing_DoesStuff_WhenAskedTo as declare @input varchar(100) set @input = ... declare @expected varchar(100) set @expected = ... declare @result varchar(100) select @result = ... exec test.AssertEqualString @expected, @result go
  • 9.
    SQL Test Runner •Tests packaged into scripts (batches) • Uses system tables to locate tests • Uses transactions to minimise residual effects • Output to UI/console
  • 10.
    SQL Asserts • Valuecomparisons (string, datetime, …) • Table/result set row count • Table/result set contents • Error handling (constraint violations)
  • 11.
    Setup & Teardown •Per-Fixture (static data) • Per-Test (specific data) • Use helper procedures
  • 12.
    Default Constraint Test createprocedure test.AddingTask_SetsSubmitTime as declare @taskid int declare @submitTime datetime set @taskid = 1 insert into Task values(@taskid, ...) select @submitTime = t.SubmitTime from Task t where t.TaskId = @taskid exec test.AssertDateTimeNotNull @submitTime go
  • 13.
    Trigger Test create procedureDeletingUser_DeletesUserSettings as ... set @userid = 1 insert into AppUser values(@userid, ...) insert into AppUserSettings values(@userid, ...) delete from AppUser where UserId = @userid select @rows = count(*) from AppUserSettings where UserId = @userid exec test.AssertRowCountEqual @rows, 0 go
  • 14.
    Unique Key Test createprocedure AddingDuplicateCustomer_RaisesError as ... insert into Customer values(‘duplicate’, ...) begin try insert into Customer values(‘duplicate’, ...) end try begin catch set @threw = 1 end catch exec test.ErrorRaised @threw go
  • 15.
    Automation • Enables easyregression testing • Enables Continuous Integration • Performance can be variable
  • 16.
    Test First Development •Start with a requirement • Write a failing test • Write production code • Test via the public interface
  • 17.
    The Public Interface •Stored procedures • Views • Tables?
  • 18.
    Implementation Details • Primarykeys • Foreign keys • Indexes • Triggers • Check constraints • Default constraints
  • 19.
    Deployment Testing Build versionN+1 then run unit tests Build version N then patch to N+1 then run unit tests ==
  • 20.
    Buy or Build? •Batch file, SQL scripts & SQLCMD • TSQLUnit & PL/Unit • Visual Studio • SQL Server/Oracle Express
  • 21.

Editor's Notes

  • #2 Who am I (1st talk, ACCU member, not DBA – app dev, predominately SQL server) Audience background (any DBAs, any SQL based, any familiarity with xUnit)
  • #3 Middle section is about xUnit, sandwiched between details of the infrastructure and side-effects What makes unit testing successful is as much the other stuff Questions at the end
  • #4 Michael Feathers’ definition of ‘legacy’ Shared environment (data volume growth – slower feedback cycle, conflicting changes causes downtime, organisation policy - no personal db for unit testing) Only integration/system/stress testing Testing done with real data (not requirements driven data – e.g. handling nulls) No RI means testing more important (static vs dynamic analogy) No automated deployment
  • #5 Isolation during implementation (sandbox, atomic commit of changes) Control over test data to verify behaviour Automated regression testing at unit level
  • #6 Default constraint to set the date (simple) Trigger to cascade a delete (simple) Refactoring natural key to surrogate (harder – unique constraint on old key) No real data required – test data is enough Note: these are implementation options – will come back to encapsulation later
  • #7 Fixture setup/teardown Test setup/teardown Test (illustrate the 3 A’s) Tests packaged together in assemblies
  • #9 Fixture setup/teardown Test setup/teardown Test (with Assert) Use Reflection to discover test cases Rollback to avoid per-test residual effects (not-perfect without nested transactions) Separate schema for tests
  • #16 Regression testing Multiple branches (db naming – physical dependencies) Personal database (isolated development) Performance – time to build from scratch and run test suite?
  • #17 Start with a requirement (‘date submitted’ set automatically) Writing failing test proc Consider implementation options (proc, constraint, trigger) Mocking how? Test only publicly detectable behaviour…
  • #18 Don’t test implementation (e.g. default => constraint, proc, trigger, etc.) Do you test accessors (e.g. selects, views) Use schemas & grant permissions
  • #20 From Scratch == Old + Patch (use Unit Tests) Data migration tests are separate tests
  • #22 Scott Ambler’s book Questions