SQL Server Unit Testing      Joel Champagne  President, CodeX  President CodeX Enterprises                 July 2011      ...
Who am I?                 Who am I?• President CodeX Enterprises LLC  President CodeX Enterprises LLC• 20 years consulting...
Why?• Unit Testing…  what’s a “unit”?• U it t ti h  Unit testing has proven valuable in many                              ...
Scenarios for Failure         Scenarios for Failure• Suppose there’s an existing procedure which   has been working for ag...
Integration Points            Integration Points• On Demand  On Demand• Scheduled• S  Strategic         i  – Build / movin...
Visual Studio 2010           Visual Studio 2010• Ability to run tests within VS  Ability to run tests within VS• Setup tes...
Observations…• Quite powerful as we can craft data driven   tests using a bit of C#  tests using a bit of C#  – Not necess...
Data‐driven example
SQL‐Hero• Problems trying to address:   – Developers often don’t like to invest time in writing tests      • At least get ...
Configuration     g
TestingPolicies.xmlExample:  a proc parameter that ends in AreaID, populate using a select from Area table that is filtere...
Notes• The test generation process will look at  The test generation process will look at   captured workloads to see if a...
Creating Tests       g
Running Tests                Running Tests•   Scheduling•   Via Template•   Via Template, with parallel execution!     i  ...
Some Observations…• Presence of touch testing has proven value from   experience   experience ‐ example• SQL‐Hero testing ...
Some Observations…• Presence of touch testing has proven value from   experience   experience ‐ example• SQL‐Hero testing ...
A more complex use case…                  p• Problems trying to address:   –   Data quality in development regions   –   S...
A more complex use case ‐ illustrated… Production                             QA/Development        Prod                  ...
A more complex use case ‐ illustrated…Production                       QA/Development                                    D...
A more complex use case ‐ illustrated…Production                       QA/Development      Prod                           ...
Putting it together…• VS Database Project  VS Database Project   –   “Build” ‐> Deploy (local or shared)   –   Can exercis...
Offers…• Looking for Customer Advisory Board   members  – Free enterprise edition in exchange for on‐going     feedback (i...
To learn more or inquire about speaking opportunities, please contact: o ea     o e o qu e about spea g oppo tu t es, p ea...
Upcoming SlideShare
Loading in …5
×

Microsoft SQL Server Testing Frameworks

1,425 views
1,357 views

Published on

Creator: Joel Champagne, President of CodeX Enterprises

This presentation covers various issues associated with SQL unit testing. We’ll look at end solutions in demo form using Visual Studio 2010 and other third party tools.

You'll learn:

* The value of pursuing SQL testing, early and continually in the development cycle
* Capabilities in Visual Studio 2010 to support SQL unit testing
* Capabilities in other tools to support SQL unit testing


Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,425
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
26
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Microsoft SQL Server Testing Frameworks

  1. 1. SQL Server Unit Testing Joel Champagne President, CodeX President CodeX Enterprises July 2011 Mark Ginnebaugh, User Group Leader,  mark@designmind.com mark@designmind com
  2. 2. Who am I? Who am I?• President CodeX Enterprises LLC President CodeX Enterprises LLC• 20 years consulting experience –LLarge mission critical systems i i iti l t – .NET, SQL 2000‐2008, SSAS, SSRS, SSIS – Architecture, Data Architecture• Focus on developer productivity tools
  3. 3. Why?• Unit Testing…  what’s a “unit”?• U it t ti h Unit testing has proven valuable in many  l bl i languages• Things can break for countless reasons – find Things can break for countless reasons  find  those breakages early• Some may say SQL has gotten the short end of  the stick here for various reasons h i kh f i – Harder:  time/effort to write tests for possibly  thousands of objects (many of which could be CRUD) j ( y ) – Separation of development process – Statefulness:  dependence on specific data to make it  fly (well) fly (well)
  4. 4. Scenarios for Failure Scenarios for Failure• Suppose there’s an existing procedure which  has been working for ages• Someone renames a table column, missed Someone renames a table column, missed  updating this procedure for whatever reason  (e.g. different database) (e.g. different database)• Fails at run‐time…  you would have known,  however, if it had at least been  exercised however if it had at least been “exercised”• Also, what does SSMS tell us when we commit  the broken procedure…? th b k d ?
  5. 5. Integration Points Integration Points• On Demand On Demand• Scheduled• S Strategic i – Build / moving objects• Automation – Includes “on change” – Continuous Integration strategies
  6. 6. Visual Studio 2010 Visual Studio 2010• Ability to run tests within VS Ability to run tests within VS• Setup tests, per object• Can have “expected results” C h “ d l ”• Schema View offers easy way to script, per  object• Demo
  7. 7. Observations…• Quite powerful as we can craft data driven  tests using a bit of C# tests using a bit of C# – Not necessarily testing SQL directly (can be .NET  components) p ) – Next slide example• Hosts tests in “runnable” code (SQL scripts  ( Q p themselves stored in resource file)• Integrates as part of a larger testing  g p g g infrastructure available with Visual Studio – Test rigs, agents, etc. – Can make as complex as desired
  8. 8. Data‐driven example
  9. 9. SQL‐Hero• Problems trying to address: – Developers often don’t like to invest time in writing tests • At least get some “touch tests” as a starting point At least get some  touch tests as a starting point – Want to exercise tests early and often! • On ALTER and scheduled (and on demand) –L Leverage tests we do have at strategic times:  e.g. scripting  d h i i i i changes to a new place • Can include these automatically from the schema compare tool! – Setting of targets, not just thresholds (so we can do  reporting / analysis vs. targets) – Tests can be somewhat “portable” (i.e. do not need to  Tests can be somewhat  portable (i.e. do not need to strictly tie to a single database) – Reality of databases with hundreds of tables, reference  data, etc. – may not be practical to re‐gen the entire thing  data etc may not be practical to re gen the entire thing from scripts, etc.
  10. 10. Configuration g
  11. 11. TestingPolicies.xmlExample:  a proc parameter that ends in AreaID, populate using a select from Area table that is filtered<PARAMETER> <PATTERN>AreaID$</PATTERN> <SQL_LOOKUP>SELECT AreaID FROM Area a WHERE a.AreaAbbr LIKE A%</SQL_LOOKUP> /</PARAMETER>Example:  a proc parameter of @DocTypeID that relates to a table called CommDocumentType<PARAMETER> PARAMETER <NAME>@DocTypeID</NAME> <CHANGE_NAME_TO>CommDocumentType</CHANGE_NAME_TO></PARAMETER>Example:  a sample plug‐in, put in the program filessqlheroplugins directory<PLUGIN> <ASSEMBLY_FILE>SampleTestingPlugIn.dll</ASSEMBLY_FILE> <TYPENAME>SampleTestingPlugIn.MyTestPolicy</TYPENAME></PLUGIN>Example:  all objects in the Debug schema would be excluded from testing<OBJECT> <PATTERN>^[?Debug]?.</PATTERN> <IS_EXCLUDED>True</IS_EXCLUDED></OBJECT>
  12. 12. Notes• The test generation process will look at The test generation process will look at  captured workloads to see if an already‐ executed invocation  will work executed invocation “will work”• Is using a randomized process, but influenced  by testing policy file by testing policy file• No “side‐effects” – Rollback per object – Advantages / Disadvantages
  13. 13. Creating Tests g
  14. 14. Running Tests Running Tests• Scheduling• Via Template• Via Template, with parallel execution! i l ih ll l i !• From SHCommand.exe• From .sqlheroproj Visual Studio project type• Demo
  15. 15. Some Observations…• Presence of touch testing has proven value from  experience  experience ‐ example• SQL‐Hero testing can augment VS testing: – VS for formal unit tests which are part of build  process, complex cases where can use C# – SH for touch tests that are just “there” behind the  scenes to find certain issues for you transparently y p y – Output of SH testing template can be used in VS, if  you like!• Not intending SQL Hero to “do it all” but  Not intending SQL‐Hero to  do it all but considering some new features along the lines of  turning captured workloads into complex test  cases (C#) ( )
  16. 16. Some Observations…• Presence of touch testing has proven value from  experience  experience ‐ example• SQL‐Hero testing can augment VS testing: – VS for formal unit tests which are part of build  process, complex cases where can use C# – SH for touch tests that are just “there” behind the  scenes to find certain issues for you transparently y p y – Output of SH testing template can be used in VS, if  you like!• Not intending SQL Hero to “do it all” but  Not intending SQL‐Hero to  do it all but considering some new features along the lines of  turning captured workloads into complex test  cases (C#) ( )
  17. 17. A more complex use case… p• Problems trying to address: – Data quality in development regions – Security and isolation of production data – In progress work (using external source control?) – Making this easy Making this easy• How does unit testing relate? – Integrated as part of the process to ensure things “still  work” at the end (could be a different process too) work” at the end (could be a different process too) – Overall process can serve as one way to achieve the  coveted “known starting point” – easily• YMMV – may be simpler or more complex (likely  b i l l (lik l supported!) p p g g• Watch for a whitepaper describing this in great detail• Demo
  18. 18. A more complex use case ‐ illustrated… Production QA/Development Prod QANew QA (remote) Backup Backup /  Restore Backup QANew Remove… (remote)Scramble sensitive data
  19. 19. A more complex use case ‐ illustrated…Production QA/Development Detach, Move  Detach, Move Prod Files, Attach… QANew QA (remote) Backup Sync  Pending  Changes QANew Backup Restore QANew FTP Landing  Zone Perform tests
  20. 20. A more complex use case ‐ illustrated…Production QA/Development Prod QAOld (remote) E‐mail  relevant parties about  parties about completion… QA
  21. 21. Putting it together…• VS Database Project VS Database Project – “Build” ‐> Deploy (local or shared) – Can exercise static code analysis – Can include unit testing Can include unit testing – Can use MSBuild, which in turn integrates with TFS / CI options• SQL‐Hero – Precludes none of the above Precludes none of the above – Adds touch testing capability during development cycle, transparently – Source control integration interops with Database Projects! – Large data sets are encouraged and easy to build – Compliance checking incorporated there acts against “real” databases, can  be scheduled, email delivery (advantages!) – Supports automated operations • Example: DB reconcile from branched dev to branched QA (both stateful worked Example:  DB reconcile from branched dev to branched QA (both stateful, worked  pretty flawlessly with a simple .cmd script) – More use cases to support:  testing from a collected workload• Your environment – Do developers do actual data modeling? – Do developers want a shared environment?
  22. 22. Offers…• Looking for Customer Advisory Board  members – Free enterprise edition in exchange for on‐going  feedback (iterative development on v next  feedback (iterative development on v‐next features)• Next release coming soon (along with new  g ( g whitepapers) – Still free! – www.codexframework.com (Library) – @sqlheroguy – joelc@codexframework.com
  23. 23. To learn more or inquire about speaking opportunities, please contact: o ea o e o qu e about spea g oppo tu t es, p ease co tact: Mark Ginnebaugh, User Group Leader mark@designmind.com

×