This document provides an overview of a webinar on best practices for continuous delivery and including databases. It discusses how continuous delivery is becoming more important and widespread. However, databases are often left out of continuous delivery processes due to challenges like manual scripting and lack of version control. To fully realize the benefits of continuous delivery, databases need to be brought into the process through practices like source control for all files including database scripts, automation, and ensuring the database is in sync with other code changes. Vendor tools discussed help achieve this through features like database version control, deployment automation, and change impact analysis.
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Challenges and best practices of database continuous delivery
1. Best Practices for Database
Continuous Delivery
DBmaestro and Perforce
Joint-Webinar
October 21st, 2015
2. ▪ You will be on mute for the duration of the event
▪ We are now talking so please type a message
in the Questions box in the Control Panel if you can’t
hear us (please check your speakers and
GoToWebinar audio settings first)
▪ There will be a Q+A session at the end, you can start submitting you
questions on the Q&A bar on your gotowebinar dashboard.
▪ A recording of the full webinar will be put up online
Before We Begin
3. Presenters
Mark Warren
Product Marketing Director, Perforce
Mark has over 20 years experience as a user and developer of tools to
improve software delivery processes.
4. Presenters
Yaniv Yehuda
Co-Founder & CTO at DBmaestro
Spent the last years raising awareness about the challenges around
database development and deployment, and how to support database
Continuous Delivery.
5. Outline
▪ The Adoption of Continuous Delivery
▪ The Reasons why the Database is Often Left Behind
▪ Single Source of Truth for your Database
▪ Best Practices for Including the Database in CD
▪ The Bigger Picture – Automation and Stopping the Line
▪ Q&A
6. Industry Perspective
Waterfall
• Annual releases
• Mostly manual
Agile
• Release more
than once a
year
• Some
automation
Continuous
• Weekly/daily
updates
• Massive
automation
“The days when a successful
organization could release
software once every 12 to 18
months are over.”
“Continuous Delivery is Reshaping the Future of ALM”, Kurt Bittner, Forrester, July 2013
7. Continuous Delivery Is Moving Ahead!
Recently Conducted DBmaestro Survey
• Over 350 participants
9. What is CD? Product Delivery Pipeline
Requirements Develop Build Test Integrate Deploy
10. BuildRequirements Develop IntegrateTest Deploy
Rqmts Doc Licenses/
IP History
Social
Coding
Build
Farms
Database Scripts “Hardware”
(Virtual)
Code Open
Source
Binaries Release
Binaries
Code Open
Source Binaries
Release BinariesDatabase
Product Delivery Pipeline
11. BuildRequirements Develop IntegrateTest Deploy
Rqmts Doc Licenses/
IP History
Social
Coding
Build
Farms
Scripts “Hardware”
(Virtual)
Code Open
Source
Team Collaboration (design, dev, release, devops…)
Binaries
Release BinariesDatabase
Accelerate the Pipeline (code, DB, scripts, binaries, etc.)
Product Delivery Pipeline
12. Less Efficient Product Delivery
Poor visibility between
teams introduce friction and
design errors
Poor component reuse
results in higher production
cost
More delays, less efficient
product delivery
Increased risk of
quality issues
DevOps
14. Continuous Delivery Best Practice
Keep everything in version control… this includes requirement
documents, test scripts, automated test cases, network
configuration scripts, deployment scripts, database creation…
- Continuous Delivery by Jez Humble and David Farley
“
“
16. Hybrid Workflows
• Distributed & Centralized Version control,
code reviews, simple file sharing
• Happy developers & contributors
Every File
• Efficiently handles large, often binary, data
DevOps Stay Happy & Productive
• A mainline source for all builds even with
distributed development
All IP Safe & Secure
• Granular permissions, theft risk monitoring
Perforce Helix
CONTRIBUTORS
CONSUMERS
17. Perforce Manages IP for Market Leaders
13,000
20,000 users
9,500 users
500+ terabytes
5,000+ users
coders & designers
10,000,000
Perforce xact/day
Everything! 11,000+ users
Chips
Games
& Animation
Cloud/SW Electronics Systems Automotive
And > 15,000 other companies across industries
18. Facts– Fortune 1000 (IDC)
▪ Average total cost of unplanned application downtime per year is
$1.25b - $2.5b
Hourly cost of an infrastructure failure is $100K/hour to $500K – $1 M
Average number of deployments/month is expected to double in two
years
DevOps-led projects will accelerate the delivery of capabilities to the
customer by 15%–20%
Continuous Delivery is required!
22. ▪ Based on the survey questions, only 13% from the ones reported
doing CD for DB, are actually performing basic CD practices!!!
The rest are plugging the
automated process with
various manual steps…
▪ Why?
▪ What is so special about the database?
Not 49%, actually DOWN to 13% !!!
24. Manual steps in an automated process don’t work!!
1. At first, you think “This is easy. We can handle it!”
2. Then “I think we’re fighting a losing battle…”
3. And then you have to double your speed !!!
25. Also, manual steps lead to Chaos in Emergency Times
90%
Rate this as a risk factor, but
53%
Break the process and test urgent hot-fixes in pre-
production
26. Old adage but true
– The database is often neglected and therefore can become the
weakest link
– Manual processes
Database/Code Silos exist…
– Don’t always communicate effectively
– Need to follow same procedures & best practices
Essential from a compliance and business point of view
Should be the strongest link
Is database the Weakest link in a chain ???
28. What is the problem?
▪ Root Causes for issues:
− Challenging manual source control process
− Static deployments code overrides
− Dynamic deployments tools unaware of version control
− No release automation red-flags – don’t know when to stop the line…
29. Version Control Process
(file based)
Database
Check-Out
Script
Modify Script
Get updated
Script from DB
Check-In
Script
Compile
Script
in DB
Debug Script
in DB
?
?
?
?
A
A’
Traditional isolated processes
Risk of falling out of sync
30. 90%
Rate this as a risk factor, yet
72%
Admit database may not be in sync with the source repository
31. Scripts & version control
▪ Challenges:
− Code-overrides
− Working on the wrong revisions
− Scripts do not always find their way to the version control solution
− Out of process updates go unnoticed
− Hard to locate outdated update scripts
▪ Playing safe? What we really need:
− The upgrade script
− The actual code of the object
− A roll-back script
33. X
1.11.1.11.11.21.31.41.51.61.7
Scripts… build once deploy many
Int QA Stage Prod
Database Deploy Script
Environment
Re-Base (due to defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.11.11.41.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
Out of Process
Change
X
X
X
X
X
? 1.1.1
X
a
a
34. 60%Of those manually building scripts have to fix or tweak them
regularly as part of a deployment process
35. 80%
Rate this as a risk factor, yet
71%
Follow manual processes to create their database deploy scripts
36. Build & deploy on demand
1.11.21.31.41.51.61.7
*
Int QA Stage Prod
Database Deploy Script
Environment
* Execute the same script being
executed at the Stage environment
Re-Base (due to defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.1 1.4
1.4 1.7
1.1.1 1.7
1.1 1.1 1.11.41.7
File Based
Version Control
Out of Process
Change
1.1.11.7 1.1.11.7
Validate
38. Challenges…
A compare & sync tool:
▪ Is unaware of any changes that occurred before the time it ran
▪ Has no knowledge of changes that took place at the target environment
▪ Does not leverage version control for more information
▪ Unable to deal with conflicts & merges between different teams
▪ Requires manual inspection
▪ Requires detailed knowledge regarding
each change as part of the process
Mistrust AGAIN…
So…no automation…
We fear for automating problems into
production and a major risk!!!
39. 70%of those using compare & sync tools have to review and fix the
results as they can't always trust them to automatically deploy
correctly
40. ■ Understand the nature of the changes
■ Raise red flags on conflicts
■ Support out-of-process changes
■ Utilize baseline aware analysis
Safety Net Deployment Automation
41. Deploying changes if needed
Development Baseline
Previous Label /
Production Golden Copy
Production
If we had the index in the baseline =>
we should take it down from production…
(Deploy Change)
42. Or protecting target environment…
Development Baseline
Previous Label /
Production Golden Copy
Production
BUT… If no index in baseline =>
we should protect the NEW index on production!!!
(Protect Target)
46. Impact Analysis! not Damage Control…
Raise red flags to stop the line… before actual deployment…
if requires human intervention
47. Automate “everything”
– Package the deployment of database changes along with all your other
application components to give a unified picture
Move the process upstream
– Easily promote the same package (including database changes!) from one
environment to the next, handling environment-specific differences
automatically
Create the deployment pipeline
For successful CD:
48. What does DBmaestro offer?
▪ Database Enforced Change Management solution
– Database source control
– Enforce best practices
– Plugs into Perforce
– Database merge & change impact analysis
▪ DevOps & CD Solution for databases
– Baseline aware deployment automation, rollback & recovery
– Reduce database deployment issues
– Plugs into release management & Continuous Delivery
▪ Security
– Control roles & responsibilities when introducing DB changes
▪ Audit & Compliance
– Know who can do what, where, when & why
Allows you to package, verify,
deploy and promote database
changes just as you would do
with application code…
putting you in a position
to build a full delivery
pipeline…
49. ▪ Focusing on changes rather than managing changes and
dealing with re-work, boosted overall productivity of 250
developers. We estimate we were able to do 15% more with the
same resources.
▪We went from several fix-centric deployments a day, to one
feature-centric deployment a week.
▪The amount of incidents in production has declined as well. We
had 20% less incidents.
▪CIO @ Credit Card company
“
Testimonials - Efficiency
“
50. About DBmaestro
▪ The leading provider of DevOps for Database
▪ Database Development and Continuous Delivery
51. Thank you!
Q & A
Mark Warren Yaniv Yehuda
mwarren@perforce.com yanivy@dbmaestro.com
www.perforce.com www.dbmaestro.com
Editor's Notes
Hi everybody and welcome to our webinar, thank you for taking the time to be here with us. Today we will be hearing from Bob Aiello and Yaniv Yehuda, who will be discussing DevOps for Database. But first I would like to go over a few details:
Please note you will be on mute during the event. If you can't hear me now, please check your speakers and GoToWebinar audio settings.
We will have a Q&A session at the end of the presentation, but you can start submitting you questions on twitter, using the hashtag #DevOps4db or the Q&A bar on your gotowebinar dashboard.
Opportunity to poll – how many of you use products from samsung? How many from apple? How many use Amazon? How many use services from google? You touch products built on Perforce, daily…
Click….
We have 10000+ customers across the globe. This is a proven platform for delivering a diverse set of products.
IT organizations that have tried to custom adjust current tools to meet DevOps practices have a failure rate of 80%, thus making tool replacement and/or addition a critical requirement
The average cost percentage (per year) of a single application's development, testing, deployment, and operations life cycle considered wasteful and unnecessary is 25%
Development teams are the leading sponsors of DevOps teams, with operations and architecture teams close behind
There are significant acceleration advantages for IT leaders that decide to create a DevOps team or center of excellence versus a less-organized DevOps organizational approach