Successfully reported this slideshow.

Delivering Changes for Applications and Databases

0

Share

Loading in …3
×
1 of 35
1 of 35

Delivering Changes for Applications and Databases

0

Share

Download to read offline

Presentation used at PortoData (28/07/2016) about manageing database changes in application projects. Demoed Flyway migrations for script packages, and a strategy for using DBUp and localDb for integration testing at the data layer

Presentation used at PortoData (28/07/2016) about manageing database changes in application projects. Demoed Flyway migrations for script packages, and a strategy for using DBUp and localDb for integration testing at the data layer

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Delivering Changes for Applications and Databases

  1. 1. Delivering changes for applications and databases “The beginning of a beautiful friendship” Eduardo Piairo Miguel Alho
  2. 2. Delivering changes for applications and databases @EdPiairo https://pt.linkedin.com/in/jesuspiairo eduardopiairo@outlook.com http://www.eduardopiairo.com/ Eduardo Piairo, DBA & DevOps @ Celfinet
  3. 3. Delivering changes for applications and databases @MytyMyky https://pt.linkedin.com/in/miguelalho-982331 alho@miguelalho.com http://www.miguelalho.pt/ Miguel Alho, Software Engineer @ Faro Europe
  4. 4. @DevOpsPorto Is Coming
  5. 5. Delivering changes for applications and databases Source Control Continuous Integration Continuous Delivery Database + Application
  6. 6. What’s so special about the database?
  7. 7. What’s so special about the database?
  8. 8. DLM – Database Lifecycle Management Data management and data migration Data monitoring Data recovery
  9. 9. Typical problems we want to fix • Databases are out of pace with application development • Lack of traceability of database changes (changes history) • Manual databases processes prevent the CI and CD utilization • Lack of testability • Database setup time for new environments • Bugs in production
  10. 10. Typical problems we want to fix Databases becoming a bottleneck in an agile delivery process
  11. 11. Typical problems we want to fix Databases becoming an easy target to blame
  12. 12. Why database automation? • Enable control over database development • Increase speed of response to change • Keep a versioned “history” of database states and changes • Reliability of the release process • Increase release frequency though repeatability of processes • Reduce time spent fixing bugs - automated tests • Remove/reduce human intervention in the release process
  13. 13. Why database automation? Fearless database changes
  14. 14. 1st step: Source Control • First step in your database deployment pipeline • Traceability through change history • SQL as documentation • Shared code-base and shared process • Enforceable standards to reduce conflicts
  15. 15. 1st Step: Source Control Fundamental resource: SQL Script
  16. 16. Migrations-based approach vs State-based approach • State based solutions • Script represents the current database state • Your source of truth is how the database should be • Migrations based solutions • Script represents a migration • Migration represents how to transition to the next database version • Your source of truth is how the database should change
  17. 17. Flyway • Flyway – “Database Migrations made Easy” http://flywaydb.org/ • Open source database migration tool • Flyway command-line enabled Simplicity: easy to setup, no need to install Zero dependencies (java + jdbc) Scripts are written in SQL
  18. 18. Flyway – How does it work? • Flyway commands: • migrate, clean, info, validate, baseline, repair • Metadata table: • Used to track the state of the database • If the database is empty, flyway won’t find it and will create it • Flyway scans the migrations directory and check migrations against the metadata table • Migrations are sorted based on their version number and applied in order
  19. 19. Version Control System Two components: Database scripts repository Flyway – database migrations Rule 1: Script version (timestamp) Rule 2: Operation type Rule 3: Object type Rule 4: Object name Example: V20160728.1100__Create_TB_MyTable.sql
  20. 20. Demo
  21. 21. How about applications? It takes two to tango
  22. 22. Database and Application change alignment ● Database schemas should evolve along with application evolution ○ deploy app => deploy changes / migrations ● (Integration) Test app and db changes together ○ should pass tests while developing, ○ should pass tests while promoting to new environments ● Ideally, one application per database ● Pipeline deployment only!
  23. 23. Integration test at the database layer ● Use TDD techniques ● Limit database integration tests to data layer ○ faster tests! ○ validate sql and mappings ● Always test with the latest schema DatabaseApplication DatabaseApplication
  24. 24. Demo https://github.com/MiguelAlho/Purchases-DbMigration-sample
  25. 25. What we just saw
  26. 26. Database and Application change alignment Fearless database changes
  27. 27. Challenges • Challenge #1: managing multiple branches • Manage merge conflicts • The default branch is the main/master branch and all the merges for this branch must be done with caution
  28. 28. Challenges • Challenge #2: Script expiration date • Should the migration script have an expiration date? • Sometimes, in the next day, the migration script no longer makes sense
  29. 29. Challenges • Challenge #3: migration scripts stack • How can I manage the dependencies between different databases?
  30. 30. Challenges • Challenge #4: developer sandbox • How to supply a disposable development environment where the database is included?
  31. 31. What’s Next?
  32. 32. Links ● https://github.com/MiguelAlho/Purchases-DbMigration-sample https://flywaydb.org/ ● http://dbup.github.io/ ● http://www.red-gate.com/products/sql-development/sql-compare/ ● https://octopus.com/ ○ http://docs.octopusdeploy.com/display/OD/Deploying+ASP.NET+Core+Web+Applications ● https://www.visualstudio.com/ ●
  33. 33. Reading material
  34. 34. Delivering changes for applications and databases @EdPiairo https://pt.linkedin.com/in/jesuspiairo eduardopiairo@outlook.com http://www.eduardopiairo.com/ @MytyMyky https://pt.linkedin.com/in/miguelalho-982331 alho@miguelalho.com http://www.miguelalho.pt/

×