Slides for my presentation about Agile Database Development at SDC2012 (Gothemburg - April 2012)

1. Agile Database development
(with Visual Studio)
Jose Luis Soria
Göteborg, April 2012

2. Jose Luis Soria
• ALM Team Lead at
Plain Concepts
• Professional Scrum
Trainer at scrum.org
jlsoria@plainconcepts.com
http://geeks.ms/blogs/jlsoria
@jlsoriat

3. Agenda
• Database development
• Agile development
• Agile database development
• Practices and tools

4. Database development

5. How databases work
• Databases are different from
application code
– Application code is not changed while
running (even in dynamic languages, changes are not
persisted)
– Databases are changed while running
• Changes in data or database objects, made
while the database is online, have to be taken
into account while developing

7. Databases are changed while running
Take running database changes into
account while developing.
Have a proper policy for changes in place

8. Databases across environments
• We have to deal with several
environments while developing
• Each environment has its own
database
• These databases are used for different
things
• These databases are in different states
and versions, and contain different
data

9. Databases are different
across environments
Be prepared to change different things
in different databases

10. Applying changes to databases
• Most times, any particular change can
only be made one time. Once the
change has been made, it is not
possible to make it again
• Changes are difficult to be undone

11. Changes to databases are lasting
and hard to be undone
Have a proper policy for changes in place.
Have a proper policy for undoing
changes in place.

12. Agile development

13. Agile?

14. Twelve principles of Agile
Our highest priority is to satisfy the customer Working software is the primary measure of
through early and continuous delivery of progress.
valuable software.
Welcome changing requirements, even late Agile processes promote sustainable
in development. Agile processes harness development. The sponsors, developers, and
change for the customer's competitive users should be able to maintain a constant
advantage. pace indefinitely.
Deliver working software frequently, from a Continuous attention to technical excellence
couple of weeks to a couple of months, with and good design enhances agility.
a preference to the shorter timescale.
Business people and developers must work Simplicity--the art of maximizing the amount
together daily throughout the project. of work not done--is essential.
Build projects around motivated individuals. The best architectures, requirements, and
Give them the environment and support they designs emerge from self-organizing teams.
need, and trust them to get the job done.
The most efficient and effective method of At regular intervals, the team reflects on how
conveying information to and within a to become more effective, then tunes and
development team is face-to-face adjusts its behavior accordingly.
conversation.

15. Twelve principles of Agile
Our highest priority is to satisfy the customer Working software is the primary measure of
through early and continuous delivery of progress.
valuable software.
Welcome changing requirements, even late Agile processes promote sustainable
in development. Agile processes harness development. The sponsors, developers, and
change for the customer's competitive users should be able to maintain a constant
advantage. pace indefinitely.
Deliver working software frequently, from a Continuous attention to technical excellence
couple of weeks to a couple of months, with and good design enhances agility.
a preference to the shorter timescale.
Business people and developers must work Simplicity--the art of maximizing the amount
together daily throughout the project. of work not done--is essential.
Build projects around motivated individuals. The best architectures, requirements, and
Give them the environment and support they designs emerge from self-organizing teams.
need, and trust them to get the job done.
The most efficient and effective method of At regular intervals, the team reflects on how
conveying information to and within a to become more effective, then tunes and
development team is face-to-face adjusts its behavior accordingly.
conversation.

16. Agile database development
Early and continuous delivery of Working software is the primary
valuable software. measure of progress.
Welcome changing requirements. Sustainable development, constant
pace.
Deliver working software Continuous attention to technical
frequently. excellence and good design.
Business people and developers Simplicity.
must work together.
Build projects around motivated The best architectures,
individuals. Give them the requirements, and designs emerge.
environment and support they
need.

17. What does it mean?

18. Early and continuous delivery of
valuable software
Ability to deliver in a simple and quick
way

19. Welcome changing requirements
Ability to introduce changes
at any moment

20. Deliver working software frequently
Ability to deliver in a simple and quick
way

21. Business people and developers
must work together
Development depends on business

22. Build projects around motivated individuals.
Give them the environment they need
Proper environment and tools

23. Working software
is the primary measure of progress
Development depends on business.
Keep always
a functional version of the software

24. Sustainable development.
Constant pace
Ability to introduce changes
at any moment

25. Continuous attention to technical
excellence and good design
Best practices: CI, refactoring, TDD…

26. Simplicity
Reuse.
Work in small chunks

27. The best architectures, requirements
and designs emerge
Ability to introduce changes
at any moment

28. Agile database development

29. Agile database development
• Be able to deliver database changes in a simple and quick
way
• Be able to introduce database changes at any moment
• Make database development dependent on business
• Set up a proper environment and tools
• Keep always a functional version of the database
• Use best practices: CI, refactoring, TDD…
• Reuse
• Work in small chunks
• Take running database changes into account while
developing
• Have a proper policy for changes in place
• Have a proper policy for undoing changes in place
• Be prepared to change different things in different databases

30. How to do it?: practices & tools

31. Practices and tools
• Are going to help us to do database
development properly
• Tools and demos for this session deal with
relational databases (SQL Server). But
principles are the same for other
scenarios:
– NoSQL (MongoDB, Azure Storage…)
– ORMs (Entity Framework, NHibernate…)
• Most things will also work with VS 11 and
SQL Server Data Tools

32. Version control
• Treat database code like any other source
code
– Enable several people to work on the same
code base
– Make modifications available to the rest of the
team in a controlled way
– Make it possible to set up Continuous Integration
– Use versioning features: compare, restore old
version, combine…
• Use the same code, schema or approach to
migrate any database in your environment

33. Demo
Version control for databases

34. Coding aids
• Visual Studio provides several aids for
working with database code
– Intellisense
– Tracking and resolving dependencies
– Static analysis
– Refactoring

35. Demo
Coding aids

36. Isolated development
environment
• Have each developer work in his own
environment, which includes the
database
– Self-contained
– Resembling production environment
– Automated (deployment, testing, etc.)

37. Demo
Isolated development
environment

38. Unit tests
• Sometimes, database code can contain
logic
• Database code can benefit from
automated unit testing the same way as
application code
– Find problems early
– Facilitate change
– Simplify integration
– Self-document
– Drive the design

39. Demo
Database unit testing

40. Continuous Integration
• Early warning of conflicting changes
• Immediate unit testing of all changes
• Constant availability of a "current"
build for testing, demo, or release
purposes
• Immediate feedback
• Modular, less complex code
• Quality code

41. Continuous Integration
principles
• Maintain a code repository
• Automate the build
• Make the build self-testing
• Everyone commits to the baseline every day
• Every commit (to baseline) should be built
• Keep the build fast
• Test in a clone of the production environment
• Make it easy to get the latest deliverables
• Everyone can see the results of the latest build
• Automate deployment

42. Continuous Integration
principles
• Maintain a code repository
• Automate the build
• Make the build self-testing
• Everyone commits to the baseline every day
• Every commit (to baseline) should be built
• Keep the build fast
• Test in a clone of the production environment
• Make it easy to get the latest deliverables
• Everyone can see the results of the latest build
• Automate deployment

43. Demo
Database continuous
integration

44. Requirements drive changes
• Make changes in small chunks
• Changes are originated by
requirements
• Make different changes for different
requirements

45. Continuous deployment to any
environment
• Automating database migrations to
any environment helps us:
– Easily roll forward or back
– Minimize disruption for changes
– Maximize reliability of deployment process
– Work incrementally (including DBAs)

46. Continuous deployment process
• Obtain the correct version of the
database
• Prepare the environment
• Initialize database and instance
• Initialize schema
• Initialize user credentials
• Populate database with reference data
• Have a rollback mechanism in place

47. Demo
Continuous deployment

48. Database sync vs. delta scripts
• Syncing databases can lead to
uncontrolled changes across
environments
• It’s better to make changes the same
way for all the environments
• We can use numerated delta scripts
and tools like DBDeploy

49. Database sync vs. delta scripts
• Delta scripts
– Great for greenfield development and Continuous
Integration
• Database sync
– For legacy databases
– For non-agile, non-CI projects
– For changing data
– When you don’t have database under version control
• Other options
– ORMs usually have their own mechanism to deal with
database changes. For example, migrations in Entity
Framework

50. Dealing with data
• Reference data can be scripted
• Changes (delta scripts) should deal
with existing data
• There are several ways to obtain data
for testing
– Using the application API
– Using backups
– Using data comparison tools
– Using data generation tools

51. Have we covered everything?
• Be able to deliver database changes in a simple and quick
way
• Be able to introduce database changes at any moment
• Make database development dependent on business
• Set up a proper environment and tools
• Keep always a functional version of the database
• Use best practices: CI, refactoring, TDD…
• Reuse
• Work in small chunks
• Take running database changes into account while
developing
• Have a proper policy for changes in place
• Have a proper policy for undoing changes in place
• Be prepared to change different things in different databases

52. Any questions?
• Visual Studio Database tools http://bit.ly/dyN3wv
• VSDBCMD http://bit.ly/vB6G1
• Database changes done right http://bit.ly/wTOY01
• Continuous delivery http://continuousdelivery.com/
jlsoria@plainconcepts.com
@jlsoriat