David Atkinson gives a presentation on automated database build and testing, or continuous integration (CI), for databases. He explains that CI ensures database code is always working by automatically building and testing databases on each change. The demo shows two developers using CI to incrementally keep a test database up to date, run automated tests, and generate deployment scripts. When a fix introduces a breaking change, they address it by adding a custom migration script to the deployment package to safely update data during deployment.
Boost Fertility New Invention Ups Success Rates.pdf
Database automated build and test - SQL In The City Cambridge
1. #sqlinthecity
Automated Build and Test
(or continuous integration)
(for your database)
David Atkinson
Product Manager, Red Gate
David.Atkinson@red-gate.com
2. #sqlinthecity
What will we cover today?
Database build automation:
What, Why, How
Practical Demo
Questions encouraged!
3. #sqlinthecity
Show of hands:
How agile is your database development?
1. Database objects/scripts aren’t in version control
2. Database objects/scripts are in version control
3. An automated build system exists to build and test
the database
4. An automated release system is used to promote
your database to your environments
4. #sqlinthecity
How agile is your database development?
1. Database objects/scripts aren’t in version control
2. Database objects/scripts are in version control
3. An automated build system exists to build and test
the database
4. An automated release system is used to promote
your database to your environments
5. #sqlinthecity
What is continuous integration?
“Continuous Integration is a practice
designed to ensure that your software is always
working, and that you get comprehensive
feedback in a few minutes as to whether any
given change to your system has broken it.”
Jez Humble, ThoughtWorks, author of
“Continuous Delivery”
6. database
#sqlinthecity
What is continuous integration?
^
“Database Continuous Integration is a
practice designed to ensure that your database
software is always working, and that you get
comprehensive feedback in a few minutes as to
whether any given change to your system has
broken it.”
8. #sqlinthecity
What is build?
• For application code = compile
• For database code = database creation script
– But only for a new installation!
• Upgrade scripts required for existing
installations
– Need to preserve the state of the data
9. #sqlinthecity
What is test?
• For .NET code, NUnit.
– Runs on a developer’s machine and build
server
• What about the database?
– tSQLt is an open source framework for testing
SQL Server databases
– SQL Test provides SSMS integration
10. #sqlinthecity
Demo background
Two fictional developers, David and Grant,
are working on the website, www.simpletalk.com, a community website for .NET
and SQL Server developers
The application comprises
• An ASP.Net web application
• A SQL Server database
11. #sqlinthecity
Demo background
Continuous integration has been configured
to:
1. Keep a test database up to date with the
latest changes
2. Run automated tSQLt tests
(on realistic amounts of data)
3. Generate up-to-date documentation
4. Generate deployment scripts
12. #sqlinthecity
Demo background
A few improvements are being made to
Simple Talk.
These changes will be deployed to
production.
But… only when the dev team has
demonstrated that the changes work!
13. #sqlinthecity
Tools used
• TeamCity (CI tool from JetBrains)
• SQL Automation Pack (includes TeamCity plugin)
• Glimpse (open source)
From the SQL Developer Bundle:
•
•
•
•
SQL Source Control
SQL Test (tSQLt unit testing framework)
SQL Data Generator
SQL Doc
20. #sqlinthecity
But not all changes can be inferred
1
2
3
5
column
rename
merge table
8
merge column
split column
9
6
add NOT NULL
column without
DEFAULT
table
rename
split table
7
4
10
11
data transformation
data motion
12
21. #sqlinthecity
SQL Compare and SQL Source Control
support custom migration scripts
1
2
3
4
5
6
11
12
Custom migration script
7
8
9
10
25. #sqlinthecity
Recap: database CI
1. Incrementally keeps a test database up to date
2. Runs tSQLt tests on a database with realistic
amounts of data
3. Generates always-up-to-date documentation
4. Generates and validates deployment scripts
5. Publishes a Deployment Manager database
package
26. #sqlinthecity
Recap of demo: writing a test
1.
2.
3.
4.
5.
A new test is added
Continuous Integration identifies a failure
We reproduce the bug locally
We fix the bug and test it
We commit the fix to source control
27. #sqlinthecity
Recap: migration scripts
1. The fix sets the [PublishDate] column from NULL to
NOT NULL
2. Grant’s “get latest” fails because existing data that
has NULL values
3. The continuous integration build alerts us of the
same issue
4. A migration script is added to UPDATE existing
data prior to changing the column to NOT NULL
5. Get Latest becomes possible and the continuous
integration build once again indicates working
software
CI server is set up by one person and the whole team of developers benefits.
The testing of deployment scripts is customarily left to the end. => Issues with upgrades only found at the end of a project.
Although CI catches issues that arise when developers have failed run tests locally, also solving a different problem:- Two incompatible changes committed by different developers- “Works on my machine” in development.Issues that only arise in more realistic integration environments.
Show of hands for SQL Developer Bundle.TeamCity is used at Red Gate but *any* build server can be used, eg:Jenkins, Hudson, TFS Build, CruiseControl, Bamboo, etc.There is a sqlci.exe command line, and NAnt and MSBuild template wrappers available.