SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.
SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.
Successfully reported this slideshow.
Activate your 14 day free trial to unlock unlimited reading.
Presentation used at PortoData (28/07/2016) about manageing database changes in application projects. Demoed Flyway migrations for script packages, and a strategy for using DBUp and localDb for integration testing at the data layer
Presentation used at PortoData (28/07/2016) about manageing database changes in application projects. Demoed Flyway migrations for script packages, and a strategy for using DBUp and localDb for integration testing at the data layer
1.
Delivering changes for
applications and databases
“The beginning of a beautiful friendship”
Eduardo Piairo
Miguel Alho
2.
Delivering changes for applications and databases
@EdPiairo
https://pt.linkedin.com/in/jesuspiairo
eduardopiairo@outlook.com
http://www.eduardopiairo.com/
Eduardo Piairo, DBA & DevOps @ Celfinet
3.
Delivering changes for applications and databases
@MytyMyky
https://pt.linkedin.com/in/miguelalho-982331
alho@miguelalho.com
http://www.miguelalho.pt/
Miguel Alho, Software Engineer @ Faro Europe
8.
DLM – Database Lifecycle Management
Data management and data migration
Data monitoring
Data recovery
9.
Typical problems we want to fix
• Databases are out of pace with application development
• Lack of traceability of database changes (changes history)
• Manual databases processes prevent the CI and CD utilization
• Lack of testability
• Database setup time for new environments
• Bugs in production
10.
Typical problems we want to fix
Databases becoming a
bottleneck in an agile
delivery process
11.
Typical problems we want to fix
Databases becoming an
easy target to blame
12.
Why database automation?
• Enable control over database development
• Increase speed of response to change
• Keep a versioned “history” of database states and changes
• Reliability of the release process
• Increase release frequency though repeatability of processes
• Reduce time spent fixing bugs - automated tests
• Remove/reduce human intervention in the release process
14.
1st step: Source Control
• First step in your database deployment pipeline
• Traceability through change history
• SQL as documentation
• Shared code-base and shared process
• Enforceable standards to reduce conflicts
15.
1st Step: Source Control
Fundamental resource:
SQL Script
16.
Migrations-based approach vs State-based approach
• State based solutions
• Script represents the current database state
• Your source of truth is how the database should be
• Migrations based solutions
• Script represents a migration
• Migration represents how to transition to the next database version
• Your source of truth is how the database should change
17.
Flyway
• Flyway – “Database Migrations made Easy”
http://flywaydb.org/
• Open source database migration tool
• Flyway command-line enabled
Simplicity: easy to setup, no need to install
Zero dependencies (java + jdbc)
Scripts are written in SQL
18.
Flyway – How does it work?
• Flyway commands:
• migrate, clean, info, validate, baseline, repair
• Metadata table:
• Used to track the state of the database
• If the database is empty, flyway won’t find it and will create it
• Flyway scans the migrations directory and check migrations against the metadata table
• Migrations are sorted based on their version number and applied in order
19.
Version Control System
Two components:
Database scripts repository
Flyway – database migrations
Rule 1: Script version (timestamp)
Rule 2: Operation type
Rule 3: Object type
Rule 4: Object name
Example: V20160728.1100__Create_TB_MyTable.sql
22.
Database and Application change alignment
● Database schemas should evolve along with application evolution
○ deploy app => deploy changes / migrations
● (Integration) Test app and db changes together
○ should pass tests while developing,
○ should pass tests while promoting to new environments
● Ideally, one application per database
● Pipeline deployment only!
23.
Integration test at the database layer
● Use TDD techniques
● Limit database integration tests
to data layer
○ faster tests!
○ validate sql and mappings
● Always test with the latest
schema
DatabaseApplication
DatabaseApplication
26.
Database and Application change alignment
Fearless database changes
27.
Challenges
• Challenge #1: managing multiple branches
• Manage merge conflicts
• The default branch is the main/master
branch and all the merges for this branch
must be done with caution
28.
Challenges
• Challenge #2: Script expiration date
• Should the migration script have an
expiration date?
• Sometimes, in the next day, the migration
script no longer makes sense
29.
Challenges
• Challenge #3: migration scripts stack
• How can I manage the dependencies
between different databases?
30.
Challenges
• Challenge #4: developer sandbox
• How to supply a disposable development
environment where the database is
included?