1.
Delivering changes for
applications and databases
“The beginning of a beautiful friendship”
Eduardo Piairo
Miguel Alho

2.
Delivering changes for applications and databases
@EdPiairo
https://pt.linkedin.com/in/jesuspiairo
eduardopiairo@outlook.com
http://www.eduardopiairo.com/
Eduardo Piairo, DBA & DevOps @ Celfinet

3.
Delivering changes for applications and databases
@MytyMyky
https://pt.linkedin.com/in/miguelalho-982331
alho@miguelalho.com
http://www.miguelalho.pt/
Miguel Alho, Software Engineer @ Faro Europe

4.
@DevOpsPorto
Is Coming

5.
Delivering changes for applications and databases
Source
Control
Continuous
Integration
Continuous
Delivery
Database
+
Application

6.
What’s so special about the database?

7.
What’s so special about the database?

8.
DLM – Database Lifecycle Management
Data management and data migration
Data monitoring
Data recovery

9.
Typical problems we want to fix
• Databases are out of pace with application development
• Lack of traceability of database changes (changes history)
• Manual databases processes prevent the CI and CD utilization
• Lack of testability
• Database setup time for new environments
• Bugs in production

10.
Typical problems we want to fix
Databases becoming a
bottleneck in an agile
delivery process

11.
Typical problems we want to fix
Databases becoming an
easy target to blame

12.
Why database automation?
• Enable control over database development
• Increase speed of response to change
• Keep a versioned “history” of database states and changes
• Reliability of the release process
• Increase release frequency though repeatability of processes
• Reduce time spent fixing bugs - automated tests
• Remove/reduce human intervention in the release process

13.
Why database automation?
Fearless database changes

14.
1st step: Source Control
• First step in your database deployment pipeline
• Traceability through change history
• SQL as documentation
• Shared code-base and shared process
• Enforceable standards to reduce conflicts

15.
1st Step: Source Control
Fundamental resource:
SQL Script

16.
Migrations-based approach vs State-based approach
• State based solutions
• Script represents the current database state
• Your source of truth is how the database should be
• Migrations based solutions
• Script represents a migration
• Migration represents how to transition to the next database version
• Your source of truth is how the database should change

17.
Flyway
• Flyway – “Database Migrations made Easy”
http://flywaydb.org/
• Open source database migration tool
• Flyway command-line enabled
Simplicity: easy to setup, no need to install
Zero dependencies (java + jdbc)
Scripts are written in SQL

18.
Flyway – How does it work?
• Flyway commands:
• migrate, clean, info, validate, baseline, repair
• Metadata table:
• Used to track the state of the database
• If the database is empty, flyway won’t find it and will create it
• Flyway scans the migrations directory and check migrations against the metadata table
• Migrations are sorted based on their version number and applied in order

19.
Version Control System
Two components:
Database scripts repository
Flyway – database migrations
Rule 1: Script version (timestamp)
Rule 2: Operation type
Rule 3: Object type
Rule 4: Object name
Example: V20160728.1100__Create_TB_MyTable.sql

20.
Demo

21.
How about applications?
It takes two to tango

22.
Database and Application change alignment
● Database schemas should evolve along with application evolution
○ deploy app => deploy changes / migrations
● (Integration) Test app and db changes together
○ should pass tests while developing,
○ should pass tests while promoting to new environments
● Ideally, one application per database
● Pipeline deployment only!

23.
Integration test at the database layer
● Use TDD techniques
● Limit database integration tests
to data layer
○ faster tests!
○ validate sql and mappings
● Always test with the latest
schema
DatabaseApplication
DatabaseApplication

24.
Demo
https://github.com/MiguelAlho/Purchases-DbMigration-sample

25.
What we just saw

26.
Database and Application change alignment
Fearless database changes

27.
Challenges
• Challenge #1: managing multiple branches
• Manage merge conflicts
• The default branch is the main/master
branch and all the merges for this branch
must be done with caution

28.
Challenges
• Challenge #2: Script expiration date
• Should the migration script have an
expiration date?
• Sometimes, in the next day, the migration
script no longer makes sense

29.
Challenges
• Challenge #3: migration scripts stack
• How can I manage the dependencies
between different databases?

30.
Challenges
• Challenge #4: developer sandbox
• How to supply a disposable development
environment where the database is
included?

31.
What’s Next?

32.
Links
● https://github.com/MiguelAlho/Purchases-DbMigration-sample
https://flywaydb.org/
● http://dbup.github.io/
● http://www.red-gate.com/products/sql-development/sql-compare/
● https://octopus.com/
○ http://docs.octopusdeploy.com/display/OD/Deploying+ASP.NET+Core+Web+Applications
● https://www.visualstudio.com/
●

33.
Reading material

34.
Delivering changes for applications and databases
@EdPiairo
https://pt.linkedin.com/in/jesuspiairo
eduardopiairo@outlook.com
http://www.eduardopiairo.com/
@MytyMyky
https://pt.linkedin.com/in/miguelalho-982331
alho@miguelalho.com
http://www.miguelalho.pt/