The document discusses the importance of integrating database changes into the CI/CD deployment pipeline to enhance collaboration and traceability. It outlines various stages of the deployment process, challenges faced with database management, and the value of automation to improve speed and reliability. Additionally, it presents scenarios for managing changes effectively within the software development lifecycle.

1. TRACK: CI/CD CONTINUOUS EVERYTHING
NOVEMBER 12, 2020
Eduardo Piairo
Why and how to
include database
changes in the
deployment
pipeline

2. TRACK: CI/CD CONTINUOUS EVERYTHING
● Engineering Operations Director @ Deeper Insights
● Co-Founder and Leader @ DevOps Porto
About Me
@EdPiairo
https://pt.linkedin.com/in/eduardopiairo
eduardopiairo@gmail.com
https://www.eduardopiairo.com

3. TRACK: CI/CD CONTINUOUS EVERYTHING
● Deployment pipeline
● Database challenges
● Deployment pipeline stages
● Deployment pipeline scenarios
● Final thoughts
Agenda

4. TRACK: CI/CD CONTINUOUS EVERYTHING
A deployment pipeline should enable collaboration between the various
groups involved in delivering software and provide everyone visibility
about the flow of changes in the system, together with a thorough audit
trail.
by Martin Fowler
@ https://martinfowler.com/bliki/DeploymentPipeline.html
Definition

5. TRACK: CI/CD CONTINUOUS EVERYTHING
● Stage 1 – Change description (Source Control)
● Stage 2 – Change validation (Continuous Integration)
● Stage 3 – Change implementation (Continuous Delivery)
Stages
Source
Control
Continuous
Integration
Continuous
Delivery

6. TRACK: CI/CD CONTINUOUS EVERYTHING
Database & Deployment Pipeline
Source
Control
Continuous
Integration
Continuous
Delivery
Database
+
Application

7. TRACK: CI/CD CONTINUOUS EVERYTHING
What’s so special about
databases?

8. TRACK: CI/CD CONTINUOUS EVERYTHING
Database Challenges
• Database and application changes (most of the time) are
handled differently
• Different deployment pipelines - synchronization needed
• Database is not included (manual work)

9. TRACK: CI/CD CONTINUOUS EVERYTHING
Database Challenges
• Manual work …
• Lack of traceability of database changes (changes history)
• It’s the most expensive and risky work type
• Prevent CI and CD utilization in their full extent
• Promote the fear of changes

10. TRACK: CI/CD CONTINUOUS EVERYTHING
Databases become a bottleneck
in the delivery process

11. TRACK: CI/CD CONTINUOUS EVERYTHING

12. TRACK: CI/CD CONTINUOUS EVERYTHING
The Value Of Automation
• Enable control over database development
• Increase speed of response to change
• Greater reliability, repeatability and consistency
• Enables failing in a controlled and known way
• Remove/reduce human intervention in the release process

13. TRACK: CI/CD CONTINUOUS EVERYTHING
Fearless database changes

14. TRACK: CI/CD CONTINUOUS EVERYTHING
Source Control
• Every database change can be described with an SQL script
• Traceability through change history
• SQL as documentation
• Shared code-base and shared process
• Enforce standards to reduce conflicts

15. TRACK: CI/CD CONTINUOUS EVERYTHING
Migrations vs State
• State based solutions
• How the database should be
• Migrations based solutions
• How the database should change
Migration Migration
State
Delta

16. TRACK: CI/CD CONTINUOUS EVERYTHING
Scripting Guideline
• One script, one operation type, one object (small batches)
• Merge conflicts management
• Patterns identification
• File system scripts history search

17. TRACK: CI/CD CONTINUOUS EVERYTHING
Continuous Integration
• Integrate and validate changes
• Unit and integration tests
• Small batches, less risk
• Considerations
• What should be tested? And when?
• How can the application contribute to this stage?

18. TRACK: CI/CD CONTINUOUS EVERYTHING
Continuous Delivery
• Delivering change(s) in the target environment
• Considerations
• Downtime
• Time to recover
• Affected applications

19. TRACK: CI/CD CONTINUOUS EVERYTHING
Failed Deployment
• Scripts (roll-back, roll-front)
• Very fast
• (Please) Do not make mistakes
• Working backwards

20. TRACK: CI/CD CONTINUOUS EVERYTHING
Failed Deployment
• Backups (before deploy)
• Safer but slower
• Keep databases small
• Avoid shared databases

21. TRACK: CI/CD CONTINUOUS EVERYTHING
Scenario #1
Source Control Continuous Integration Continuous Delivery
APP
DB
• Independent deployment pipelines
• High need of synchronization between DBA and development team

22. TRACK: CI/CD CONTINUOUS EVERYTHING
Scenario #2
Source Control Continuous Integration Continuous Delivery
APP
DB
• Different code repositories
• Independent CI process
• The contact point will be the deployment moment

23. TRACK: CI/CD CONTINUOUS EVERYTHING
Scenario #3
Source Control Continuous Integration Continuous Delivery
APP
DB
• Different code repositories
• Connected/dependent CI process
• Short learning step towards scenario #4

24. TRACK: CI/CD CONTINUOUS EVERYTHING
Scenario #4
Source Control Continuous Integration Continuous Delivery
APP
DB
• Share the some code repository, CI process and CD process
• No need of synchronization
• Promotes learning within teams (database code reviews)

25. TRACK: CI/CD CONTINUOUS EVERYTHING
My Definition
Pipeline: Cultural and technical tool for managing changes in the
software development process (should contemplate databases, applications
and infrastructure).

26. TRACK: CI/CD CONTINUOUS EVERYTHING
DevOps Way
• Increase flow visibility
• Increase feedback
• Increase knowledge base and exploration

27. TRACK: CI/CD CONTINUOUS EVERYTHING
Q&A
@EdPiairo
https://pt.linkedin.com/in/eduardopiairo
eduardopiairo@gmail.com
http://www.eduardopiairo.com/
#2021-ci-cd