Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Database continuous integration, unit test and functional test

1,503 views

Published on

Discuss continuous integration for database projects, including building project, deploying project to database, and executing unit tests and functional tests.

This presentation will also discuss database test standards, tips and tricks.

Published in: Software
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1url.pw/VtO5Q ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Database continuous integration, unit test and functional test

  1. 1. Database Continuous Integration, Unit Test and Functional Test
  2. 2. About myself: • Data architect/DBA with 16 years of SQL Server experience • Independent consultant, currently Architecture Lead and Technical Delivery Manager at Government of Alberta • Microsoft Certified System Engineer: MCSE • Oracle Certified Professional: Oracle DBA • IBM Certified Solution Expert: DB2 UDB http://netdbsolutions.com https://twitter.com/HarryZheng http://ca.linkedin.com/in/harryzheng https://www.facebook.com/Harry.H.Zheng 2
  3. 3. Session agenda 1. What is Continuous Integration? 2. Database Continuous Integration Setup 3. Database unit test and functional test 4. Database test standard and best practice 5. Tips & Tricks on writing database tests 6. Q&A 3
  4. 4. What is Continuous Integration • Continuous integration (CI) – the process of continuously integrating developers code in order to find problems quickly • Wikipedia: CI is the practice of merging all developer working copies with a shared mainline several times a day. • CI is often related to Test Driven Development (TDD) and Agile practice 4
  5. 5. Traditional work flow 1. Dev A works on his computer and checks in code when ready 2. Dev B works on his computer and checks in code when his task is done 3. Dev C… 4. When time to release, we start up build process and see if the combined code works 5. The version of code doesn’t build 6. Who is responsible? 5
  6. 6. Continuous Integration Work Flow 6
  7. 7. What does CI check? • Does the solution compile? • Does the code pass our unit tests? • Does the code meet functional requirements? • Does the version meet code quality requirements? 7
  8. 8. What does CI produce? • Automatically produce build artifacts • Ensure same version can be staged to various environments • Setup for automated deployment 8
  9. 9. Benefits of CI • Ensure code in source control works • Less time on integration • Reduce friction within project team • Save time on build and deploy • Increased visibility of progress • Increased overall confidence on code 9
  10. 10. Do you need CI: scenario 1 • Harry works with 3 other Devs on a side project • Project is a web based app with SQL backend • Harry is the only dba on the team • SQL database is relatively simple with 30 tables and a few SPs • Do we need setup CI? 10
  11. 11. Do you need CI: scenario 2 • Harry works in a team with 20 developers, 4 DBAs • Main SQL Server has 200+ tables and many SPs, functions • Some developers have good knowledge of SQL skills, others not so good but can code T-SQL • We used to have a rule that only DBAs can check in SQL code, but DBAs became a bottom neck quickly • Now we have opened database projects to developers to check in code changes, bug fixes, etc. • Do we need setup CI? 11
  12. 12. Software for Running CI • Team City • Cruise Control • Final Builder 12
  13. 13. TeamCity Web Interface 13
  14. 14. Final Builder Server Web Interface 14
  15. 15. Session agenda 1. What is Continuous Integration? 2. Database Continuous Integration Setup 3. Database unit test and functional test 4. Database test standard and best practice 5. Tips & Tricks on writing database tests 6. Q&A 15
  16. 16. Prerequisite for Continuous Integration • Source Control System • Build Server • Manage Database project with SSDT • Database Tests 16
  17. 17. Database CI Work flow 1. Developer check in code to source control 2. Check in triggers a build of the project 3. On successful build, the version of the code is deployed to testing databases 4. Unit tests are executed against test database 5. Functional tests are executed against functional test database 6. If all steps succeed, the version is ready for deployment 7. If any step fails, alert is sent to the team and the cycle starts from step1 again 17
  18. 18. Database CI 18
  19. 19. Database deploy script 19
  20. 20. Substitution/Build configuration file <Environment value="BuildDB_CI" machine="VM-DBServer-1" substitute="false"> <Database environmentsuffix="_CI" BackupFileFolder="VM-DBServer-1Deploy"> <SQLInstances> <SQLInstance name="SQLInst1" host="VM-DBServer-1" sqlinstance="Dev" databaseFolder="d:SQLDataDev" databaseLogFolder="d:SQLLogDev" dbdeployfolder="D:Deploy"> <Projects> <Project name="DBProj1" seedserver="VM-DBServer-1Seed" seeddatabase="DB1_Seed" fullbuild="No"> <Databases> <Database name="DBProj1" targetdatabase="DB1_CI" /> </Databases> </Project> <Project name="DBProj2" seedserver="VM-DBServer-1Seed" seeddatabase="DB2_Seed" fullbuild="No"> <Databases> <Database name="DBProj2" targetdatabase="DB2_CI" /> </Databases> </Project> </Projects> </SQLInstance> </SQLInstances> </Database> <SSIS ServerName="VM-DBServer-1" /> <SSRS ServerName="VM-DBServer-1" ReportServerPath="ReportServer_Dev"/> </Environment> 20
  21. 21. Deploy DB schema with SQLPackage "C:Program Files (x86)Microsoft SQL Server110DACbinSqlPackage.exe" /Action:Publish /SourceFile:"%1" /TargetConnectionString:"Server=%2;Initial Catalog=%3;Integrated Security=True" /p:BlockOnPossibleDataLoss=False /p:CreateNewDatabase=False /p:DisableAndReenableDdlTriggers=True /p:DropConstraintsNotInSource=True /p:DropDmlTriggersNotInSource=True 21
  22. 22. Deploy database and preserve data 1. Run Pre-deployment scripts 2. Execute sqlpackager.exe to deploy database schema changes 3. Run Post-deployment scripts 22
  23. 23. Deploy database and preserve data When to use Pre and Post scripts? •If you change table definition that may cause data loss when database project is deployed, you need to write a pre-deployment script to preserve the data •All other data change scripts will go to Post-deployment script •All Pre and Post scripts should be created so that the scripts can be executed repeatedly 23
  24. 24. Session agenda 1. What is Continuous Integration? 2. Database Continuous Integration Setup 3. Database unit test and functional test 4. Database test standard and best practice 5. Tips & Tricks on writing database tests 6. Q&A 24
  25. 25. What is unit test? • Test on a discrete unit of code • Unit test should not be affected by other units • Isolates the unit under test from the rest of the code • Repeatable 25
  26. 26. My definition of database unit test • Test that runs quickly, and does not rely on existing data with the exception of static data (e.g.., reference data) 26
  27. 27. Unit Testing - AAA •Arrange • Setup data (with TestHelper) •Act • Call Precedure/Function •Assert • Compare results with expectation 27
  28. 28. Create Unit Test From SSOE from VS 28
  29. 29. Unit Test Example for SP CREATE PROCEDURE [dbo].[uspShowMeetingAttendee] @MeetingID int = 0 AS SELECT MT.Name ,M.FirstName ,M.[LastName] From Meeting MT inner join MeetingAttendee MA on MT.MeetingID = MA.MeetingID inner join Member M on M.MemberID = MA.MemberID RETURN 0 29
  30. 30. Unit Test Example -- database unit test for dbo.uspShowMeetingAttendee BEGIN TRANSACTION --Arrange Declare @MeetingID int INSERT INTO [dbo].[Meeting] ([Name]) VALUES ('XXTestMeeting') select @MeetingID = MeetingID from Meeting where Name = 'XXTestMeeting' INSERT INTO [dbo].[Member]([FirstName],[LastName],[BirthDate]) VALUES ('XXFirstName','XXLastName','1970-01-01') INSERT INTO [dbo].[MeetingAttendee] ([MeetingID],[MemberID]) Select MeetingID, MemberID From Meeting M1 inner Join Member M2 on 1=1 where M1.Name = 'XXTestMeeting' and M2.LastName = 'XXLastName' --code to create a dynamic temporary table ##xx{Schema}{SPName}xx -- based of the output of a stored procedure exec [TestHelper].[uspCreateTableForStoredProcedureOutput] @storedProcedure = '[dbo].[uspShowMeetingAttendee]' --ACT --insert into the temporary table created by the SP insert into ##xxdbouspShowMeetingAttendeexx EXECUTE [dbo].[uspShowMeetingAttendee] @MeetingID = @MeetingID; --ASSERT --now you can select specific column without knowledge of the oridinal value select LastName from ##xxdbouspShowMeetingAttendeexx where FirstName = 'XXFirstName' ROLLBACK TRANSACTION 30
  31. 31. What is functional test? • Functional testing is a type of black box testing that bases its test cases on the specifications of the software component under test • Functions are tested by feeding them input and examining the output, and internal program structure is rarely considered 31
  32. 32. My definition of database functional test • Tests with long execution times and they rely on existing non-static data • Functional tests will run on a server where the complete dataset is available. 32
  33. 33. Database Test Tools and Frameworks • Microsoft SSDT • http://msdn.microsoft.com/en-ca/data/tools.aspx • TSQLT • http://tsqlt.org/ • Red-Gate SQL Test • http://www.red-gate.com/products/sql-development/sql-test/ 33
  34. 34. Session agenda 1. What is Continuous Integration? 2. Database Continuous Integration Setup 3. Database unit test and functional test 4. Database test standard and best practice 5. Tips & Tricks on writing database tests 6. Q&A 34
  35. 35. Database test standard and best practice • The folder structure within the database test project will have a structure that closely resembles the structure of the database project. 35
  36. 36. Naming Conventions Database tests must be discoverable. All database tests will follow one of these naming conventions: <Database Object Name>_Main_<Description>.cs <Database Object Name>_State_<Description>.cs <Database Object Name>_Defect_<Defect #>.cs 36
  37. 37. Rules • All database objects must have at least one main test. • Static tables (e.g., reference data) must have one or more state tests. • A defect that results in a change to a database object must have at least one defect test. 37
  38. 38. Rules • All test scripts must not change the state of the database. • This is done by adding a Begin Tran/ Rollback Tran code block to the test script. 38
  39. 39. Test Helpers • In order to assist in creating database tests, the database has a number of stored procedures to create test data. • All of these stored procedures exist in the [TestHelper] schema. • Developers are free to modify existing [TestHelper] stored procedures provided the changes do not alter the original purpose of the stored procedure and the changes do not impact any existing database tests. 39
  40. 40. Documenting Tests All test scripts should be well documented. 1.The comment will indicate which test condition(s) are associated with a particular result set. 2.The comments will indicate what part of the SQL in the database object is being exercised (e.g., which case statements within a where clause are exercised). 40
  41. 41. Session agenda 1. What is Continuous Integration? 2. Database Continuous Integration Setup 3. Database unit test and functional test 4. Database test standard and best practice 5. Tips & Tricks on writing database tests 6. Q&A 41
  42. 42. Writing tests for SP or Function Problem: •A test that calls to the stored procedure returns a result set with many rows and the test involves retrieving one row only •A test is only concerned about testing a single value, but the stored procedure returns a result set with many columns 42
  43. 43. Writing tests for SP or Function There are a couple of ways of making the tests less susceptible to changes: 1.Using a dynamic temporary table 2.Using a declared temporary table The first is fair more desirable but cannot be used for stored procedure or functions that have dynamic SQL. 43
  44. 44. Insert into a dynamic temporary table This approach uses sys.dm_exec_decribe_first_result_set_for_object to dynamically create a temporary table that has the same columns as the first result set of the stored procedure. [TestHelper]. [uspCreateTableForStoredProcedureColumns] creates a global temporary table that can be used to store the results of executing the stored procedure under test. 44
  45. 45. Insert into a dynamic temporary table Begin Tran --code to create a dynamic temporary table ##xx{Schema}{SPName}xx -- based of the output of a stored procedure exec [TestHelper].[uspCreateTableForStoredProcedureColumns] @storedProcedure = '[dbo].[uspRetrieveUserInfo]' --insert into the temporary table created by the SP insert into ##xxdbouspRetrieveUserInfoxx exec [dbo].[uspRetrieveUserInfo] @UserID = 51 --now you can select specific column without knowledge of the oridinal value select FirstName from ##xxdbouspRetrieveUserInfoxx where [BirthDate] > '1995-01-01' Rollback Tran 45
  46. 46. Insert into a dynamic temporary table Create procedure [TestHelper].[uspCreateTableForStoredProcedureOutput] ( @storedProcedure varchar(128) ) as begin declare @sql nvarchar(max) = '', @tablename varchar(100) = '##xx' + replace(replace(replace(@storedProcedure,'[',''),']',''),'.','') + 'xx' set @sql = @sql + 'IF OBJECT_ID(''tempdb..' + @tablename + '''' + ') IS NOT NULL DROP TABLE ' + @tablename + ';' set @sql = @sql + 'create table ' + @tablename + '(' select @sql = @sql + '[' + name + '] ' + coalesce(system_type_name, (D.Data_Type + case when Character_Maximum_Length is not null then '(' + cast(Character_Maximum_Length as varchar) + ')' else '' End)) + ',' from sys.dm_exec_describe_first_result_set_for_object (object_id(@storedProcedure), null) FR Left Outer Join INFORMATION_SCHEMA.DOMAINS D on FR.User_Type_Name = D.Domain_Name Order by FR.Column_Ordinal set @sql = substring(@sql, 0, len(@sql)) + ');' EXEC sp_ExecuteSQL @sql; end 46
  47. 47. Insert into a dynamic temporary table Pros: • Test is not dependent on the order of the columns in the stored procedure, i.e., the order can change • Result set can be further filtered Cons: • Will not work for stored procedures that use dynamic SQL • Will not work for stored procedures that return more that one result set 47
  48. 48. Insert into a declared temporary table Declare a temporary table that has the same columns as the result set of the stored procedure. This is the less desirable of the two options but is the only choice if the stored procedure or function returns the result set using dynamic SQL. 48
  49. 49. Insert into a declared temporary table Begin Tran -- TestHelper SP creates a simple user with a first name of xxFoobarxx declare @UserID Int exec [TestHelper].[uspCreateUser] @id = @UserID output declare @User table ( ID Int, FirstName varchar(50), LastName varchar(50), BirthDate date ) --Insert the SP output to the temp table insert into @User exec dbo.uspSearchUserByName @FirstName = 'xxFoobarxx‘ --this should return a single user --Verify_One_User_Exists select * from @User where FirstName = 'xxFoobarxx‘ Rollback Tran 49
  50. 50. Insert into a declared temporary table Pros: • Further filtering of the result set is possible. Cons: • The order of the columns in the declared temporary table must be the same as the order of the result set returned by the stored procedure or function. 50
  51. 51. Session agenda 1. What is Continuous Integration? 2. Database Continuous Integration Setup 3. Database unit test and functional test 4. Database test standard and best practice 5. Tips & Tricks on writing database tests 6. Q&A 51
  52. 52. 52
  53. 53. Reference and further reading: 1. http://www.pluralsight.com/courses/continuous-integration 2. http://en.wikipedia.org/wiki/Continuous_integration 3. http://www.rackspace.com/blog/the-business-advantages-of-continuous- integration/ 4. http://www.youtube.com/watch?v=PbVKgoAVZjY (Continuous Integration with the Database by Ike Ellis) 5. http://indragunawan.com/2013/09/teamcity-continuous-integration- for-everybody.html 6. http://www.youtube.com/watch?v=4sANX9AhM8c (Introduction to Continuous Integration) 53

×