This document discusses deployment pipelines for databases. It begins by defining a deployment pipeline and describing its typical stages of source control, continuous integration, and continuous delivery. It then addresses challenges of including databases in deployment pipelines, such as different processes for database and application changes and lack of traceability for database changes. The document advocates for automating database deployments to enable faster, more reliable releases. It presents scenarios for integrating databases into deployment pipelines at different levels and emphasizes the value of a fully integrated pipeline that includes databases in source control, testing, and delivery.

1. DEPLOYMENT PIPELINE FOR DATABASES
Eduardo Piairo
@EdPiairo
#TECHinPORTO

2. ABOUT ME
Deployment pipeline for databases
@EdPiairo, #TECHinPORTO
@EdPiairo
https://pt.linkedin.com/in/jesuspiairo
eduardopiairo@gmail.com
http://www.eduardopiairo.com/
Eduardo Piairo
DevOps Coach @ Natixis
DevOps Porto Founder

3. AGENDA
• Deployment pipeline
• Database challenges
• Deployment pipeline stages
• Deployment pipeline scenarios
• Final thoughts
Deployment pipeline for databases
@EdPiairo, #TECHinPORTO

4. DEPLOYMENT PIPELINE DEFINITION
A deployment pipeline should enable collaboration between the various groups involved in
delivering software and provide everyone visibility about the flow of changes in the
system, together with a thorough audit trail.
by Martin Fowler
@ https://martinfowler.com/bliki/DeploymentPipeline.html
Deployment pipeline for databases
@EdPiairo, #TECHinPORTO

5. DEPLOYMENT PIPELINE STAGES
Deployment pipeline for databases
Stage 1 – Change description (Source Control)
Stage 2 – Change validation (Continuous Integration)
Stage 3 – Change implementation (Continuous Delivery)
Source
Control
Continuous
Integration
Continuous
Delivery
@EdPiairo, #TECHinPORTO

6. DATABASE & DEPLOYMENT PIPELINE
Deployment pipeline for databases
Source
Control
Continuous
Integration
Continuous
Delivery
Database
+
Application
@EdPiairo, #TECHinPORTO

7. WHAT’S SO SPECIAL ABOUT DATABASES?
Deployment pipeline for databases
@EdPiairo, #TECHinPORTO

8. DATABASE CHALLENGES
Deployment pipeline for databases
• Database and application changes (most of the time) are handled
differently
• Different deployment pipeline
• Synchronization needed
• Database is not included (manual work)
@EdPiairo, #TECHinPORTO

9. DATABASE CHALLENGES
Deployment pipeline for databases
• Manual work …
• Lack of traceability of database changes (changes history)
• It’s the most expensive and risky work type
• Prevent CI and CD utilization in their full extent
• Promote the fear of changes
@EdPiairo, #TECHinPORTO

10. DATABASE CHALLENGES
Deployment pipeline for databases
Databases become a bottleneck in an
agile delivery process
@EdPiairo, #TECHinPORTO

11. AUTOMATION
Deployment pipeline for databases
@EdPiairo, #TECHinPORTO

12. THE VALUE OF AUTOMATION
Deployment pipeline for databases
• Enable control over database development
• Increase speed of response to change
• Greater reliability of the release process
• Remove/reduce human intervention in the release process
From
(Re)Learn and forget it
to
Improve and forget it
@EdPiairo, #TECHinPORTO

13. THE VALUE OF AUTOMATION
Deployment pipeline for databases
Fearless database changes
@EdPiairo, #TECHinPORTO

14. SOURCE CONTROL
Deployment pipeline for databases
• First step in your database deployment pipeline
• Traceability through change history
• SQL as documentation
• Shared code-base and shared process
• Enforceable standards to reduce conflicts
SQL Script
@EdPiairo, #TECHinPORTO

15. • State based solutions
• How the database should be
• Migrations based solutions
• How the database should change
MIGRATIONS VS STATE
Deployment pipeline for databases
Migration Migration
State
Delta
@EdPiairo, #TECHinPORTO

16. SCRIPTING GUIDELINES
Deployment pipeline for databases
• Small batches (One script, one operation type, one object)
• Merge conflicts management
• Patterns identification
• File system scripts history search
@EdPiairo, #TECHinPORTO

17. CONTINOUS INTEGRATION
Deployment pipeline for databases
• Integrate and validate changes
• Tests: Unit, Integration
• Small batches, less risk
• Considerations:
• What should be tested? And When?
• Can the application help on this?
@EdPiairo, #TECHinPORTO

18. CONTINUOUS DELIVERY
Deployment pipeline for databases
• Delivering change(s) in the target environment
• Considerations
• Downtime
• Time to recover
• Affected applications
@EdPiairo, #TECHinPORTO

19. FAILED DEPLOYMENT
Deployment pipeline for databases
• Rollback/roll forward scripts
• Very fast
• Please, do not make mistakes
• It’s like working backwards
• Backups (before deploy)
• Safer but slower
• Keep databases small
• Avoid share databases between applications
@EdPiairo, #TECHinPORTO

20. SCENARIO #1 FULLY INDEPENDENT PIPELINES
Deployment pipeline for databases
Source Control Continuous Integration Continuous Delivery
APP
DB
• Independent deployment pipelines
• High need of synchronization between DBA and development team
@EdPiairo, #TECHinPORTO

21. SCENARIO #2 DELIVERING APPLICATION AND DATABASE TOGETHER
Deployment pipeline for databases
Source Control Continuous Integration Continuous Delivery
APP
DB
• Different code repositories
• Independent CI process
• The contact point will be the deployment moment
@EdPiairo, #TECHinPORTO

22. Deployment pipeline for databases
Source Control Continuous Integration Continuous Delivery
APP
DB
• Different code repositories
• Connected/dependent CI process
@EdPiairo, #TECHinPORTO
SCENARIO #3 BUILDING AND DELIVERING APPLICATION AND DATABASE TOGETHER

23. SCENARIO #4 FULLY INTEGRATED PIPELINE
Deployment pipeline for databases
Source Control Continuous Integration Continuous Delivery
APP
DB
• Share the some code repository, CI process and CD process
• No need of synchronization
• Promotes learning within teams (database code reviews)
@EdPiairo, #TECHinPORTO

24. MY DEPLOYMENT PIPELINE DEFINITION
Pipeline: Cultural and technical tool for managing changes in the software development
process (should contemplate databases, applications and infrastructure).
by me ;)
Show me your pipeline and I will tell you how you work!
Deployment pipeline for databases
@EdPiairo, #TECHinPORTO

25. DEVOPS AND DATABASES
Deployment pipeline for databases
• DevOps way
• Increase flow visibility
• Increase feedback
• Increase knowledge base
@EdPiairo, #TECHinPORTO

26. READING MATERIAL
Deployment pipeline for databases
@EdPiairo, #TECHinPORTO

27. Q&A
Deployment pipeline for databases
@EdPiairo
https://pt.linkedin.com/in/jesuspiairo
eduardopiairo@gmail.com
http://www.eduardopiairo.com/
@EdPiairo, #TECHinPORTO