Is Your Database Ready for
DevOps?
Modern Application
Development and the Database
Craig S. Mullins
Mullins Consulting, Inc.
http://www.craigsmullins.com
This presentation was prepared by:
Craig S. Mullins
President & Principal Consultant
IBM Champion for Analytics
IBM Gold Consultant
Mullins Consulting, Inc.
15 Coventry Ct
Sugar Land, TX 77479
Tel: 281-494-6153
Twitter: @craigmullins
E-mail: craig@craigsmullins.com
http://www.mullinsconsulting.com
This document is protected under the copyright laws of the United States and other countries as an unpublished work. This document contains information that is proprietary and
confidential to Mullins Consulting, Inc., which shall not be disclosed outside or duplicated, used, or disclosed in whole or in part for any purpose other than as approved by Mullins
Consulting, Inc. Any use or disclosure in whole or in part of this information without the express written permission of Mullins Consulting, Inc. is prohibited.
© 2020 Craig S. Mullins and Mullins Consulting, Inc. (Unpublished). All rights reserved.
Agenda
– Industry Trends
– DevOps and Agile Overview
– The impact on database systems
• Database change management
• SQL quality assurance
– Guidance
Industry Trends
• Reduced Budgets and IT Headcount
• Database Growth and Usage
• Shorter Development Cycles,
Faster Innovation
• Automation is increasingly
important
Data Management Staff
“Growth”
Percentage of DBA versus IT Staff
Things Won’t Be Getting Any Better
v
By the year 2020
Industry Trends
• Reduced Costs and IT Headcount
• Database Growth and Usage
• Shorter Development Cycles,
Faster Innovation
• Automation is increasingly
important
9
DBMS Market Continues to Grow
Most Orgs
Run Multiple
DBMSes
Source: Database Performance: The Need for Speed & Scale,
Database Trends & Applications, 2017
• Polyglot Persistence
• Multiple relational
• Pre-relational
• NoSQL
• Document
• Graph
• Key/Value
• Wide Column Store
• Hybrid
• In-Memory DBMS
• Not just DBMS…
• …but also Hadoop and Spark
• Commercial and open source
Industry Trends
• Reduced Costs and IT Headcount
• Database Growth and Usage
• Shorter Development Cycles,
Faster Innovation
• Automation is increasingly
important
Shorter Development Cycles
• Agile development is de rigueur
– Shorter development cycles
– Sprints
– Early customer feedback
• Higher ROI
• Quicker time to value
Faster Innovation
Source: Getting Agile with Database
Development, 2018
More Frequent Deployments
Source: State of DevOps Report,
DORA (DevOps Research & Assessment)
Industry Trends
• Reduced Costs and IT Headcount
• Database Growth and Usage
• Shorter Development Cycles,
Faster Innovation
• Automation is increasingly
important
Automating the App Dev Process
• GitHub – distributed source code
management, repository
• Jenkins – automated CI/CD server for
automating software delivery
• Docker – container platform (isolates
applications; secure & portable)
• Kubernetes – container orchestration
(automates deployment and scaling for
containerized apps)
16
• UrbanCode – automates continuous
delivery cross-platform; on-premises,
cloud and mainframe
• Puppet – cross-platform configuration
management – manage your
infrastructure as code (see also Ansible,
Chef)
• Nagios – DevOps infrastructure
monitoring
DevOps
DevOps is not a technology, it is a methodology.
But What is DevOps?
• The phrase is an amalgamation of Development
and Operations but it implies more
– Application Release Automation
• Continuous Development, Continuous Integration, Continuous Delivery
• DevOps implies Agile development
– DevOps applies agile/lean methods from source code all the way to
production
• But also:
– Any change anywhere throughout the organization can be
delivered into production quickly and accurately
• In a way that is safe, predictable, scalable and controlled
Shifting Left
• Traditionally, software development progresses from requirements to design to
development to testing and deployment through support, usually depicted in a
graph showing these tasks from left to right (see below).
– Requirements gathering and design are shown to the left, delivery and testing to the right.
• With a shift-left development mentality, processes on the right are moved to the
left, so they are performed earlier in the development lifecycle.
19
But There is a Mindset Mismatch
• Applications teams release code much more
rapidly than operations teams (infrastructure)
Source: The Need for Speed: Drive
Velocity and Quality with DevOps
Forrester Research, 2017
DevOps and the Database
• DBA is part of the Ops in DevOps
when databases are involved
– All too often, Dev is over-emphasized vs. Ops
• Common misconceptions:
– Developers become responsible for all database
development requirements
– Database “code” can be treated like program code
But Developers…
• Gather requirements
• Design application systems
• Write code
• Test programs and applications
Developers are not going to completely replace DBAs
DevOps does not negate the need for DBAs
The PacMan Effect:
Dev Eating Ops
Dev Ops OpsOps
Why is this Happening?
• Misguided belief that DBAs are roadblocks
• Failure to consider the value of DBA
– Misguided belief that developers
can do the DBA job in addition
to being good programmers
• Frequently w/o DBA training
– Lack of integrating DBA
tools into DevOps
Let’s Talk About the Database
• DBA is still required
even with DevOps
– The actions we take on
databases and database
structures are different
than those we take on
application code
DBA Practices & Procedures
• Database provisioning and configuration
• Automated backups and recovery testing
• Database design
• SQL expertise
• SQL testing and performance
• Test data management
• Database change management
– Continuous delivery of database schema migrations
• A good goal: DBAs no longer do repetitive tasks
– They need to be strategic and focus on architecture and
oversight
Database Changes
• Can’t be treated exactly like apps
– The database is stateful
– Application delivery copies code, executables
from environment to environment
– But the database is an entire configuration in
each environment and changes get migrated
• Some changes are very intrusive -> DROP/re-CREATE
• Each environment might be different (however so slightly)
• Differences can “drift” in
– Optimization changes, emergency changes, etc.
Application and Database Design
• DBAs and Application Developers must
cooperate
– DBAs are the database SMEs
– App Devs are the coding SMEs
• A developer can no more design
a database appropriately than a
DBA can code an application
correctly
SQL Expertise
• Write SQL correctly and efficiently
• Many application folks know SQL only at a
cursory level
• Continuous stream of new SQL capabilities
– DBAs have most up-to-date knowledge
• DBA and Apps must cooperate as a team
Production Data Volumes
Test environments rarely have
production-sized databases
• The relational optimizer can, and usually will, produce
different access paths for smaller amounts of data than for
larger
• What works nicely in test may crash and burn in production
– Requires skilled technicians that understand the nuances of SQL and
optimization
SQL Access
Paths and
Performance
SELECT deptno
FROM department
ORDER BY deptno;
A Word About Some Other DBA
Practices and Procedures
• Database provisioning and configuration
• Backup and Recovery
• Test data management
Some Guidance
Helpful next steps
More Database Focus is Needed
• More Ops participation in DevOps
– DBA and performance analysts
– Not oversight and bottlenecks, but real
participation
• More automation of database Ops
– Improve app dev release velocity
– Integrate with existing DevOps tooling and
practices
– Built-in intelligence and rules
A Mindset Change
• Within the organization
• With developers
• With DBAs
Thank You
Contact Details:
Craig S. Mullins
President & Principal Consultant
IBM Champion for Data & AI
IBM Gold Consultant
Mullins Consulting, Inc.
15 Coventry Ct
Sugar Land, TX 77479
Tel: 281-494-6153
Twitter: @craigmullins
E-mail: craig@craigsmullins.com
http://www.mullinsconsulting.com
This document is protected under the copyright laws of the United States and other countries as an unpublished work. This document contains information that is proprietary and
confidential to Mullins Consulting, Inc., which shall not be disclosed outside or duplicated, used, or disclosed in whole or in part for any purpose other than as approved by Mullins
Consulting, Inc. Any use or disclosure in whole or in part of this information without the express written permission of Mullins Consulting, Inc. is prohibited.
© 2020 Craig S. Mullins and Mullins Consulting, Inc. (Unpublished). All rights reserved.
Additional Supporting Materials
Source: The Current State and
Adoption of DevOps
Unisphere Research, 2018
Source: The Current State and
Adoption of DevOps
Unisphere Research, 2018
Source: The Current State and
Adoption of DevOps
Unisphere Research, 2018
3 Critical Requirements for
Successful DevOps
• Automation
• Integration
• Measurement

Geek Sync | Is Your Database Environment Ready for DevOps?

  • 1.
    Is Your DatabaseReady for DevOps? Modern Application Development and the Database Craig S. Mullins Mullins Consulting, Inc. http://www.craigsmullins.com
  • 2.
    This presentation wasprepared by: Craig S. Mullins President & Principal Consultant IBM Champion for Analytics IBM Gold Consultant Mullins Consulting, Inc. 15 Coventry Ct Sugar Land, TX 77479 Tel: 281-494-6153 Twitter: @craigmullins E-mail: craig@craigsmullins.com http://www.mullinsconsulting.com This document is protected under the copyright laws of the United States and other countries as an unpublished work. This document contains information that is proprietary and confidential to Mullins Consulting, Inc., which shall not be disclosed outside or duplicated, used, or disclosed in whole or in part for any purpose other than as approved by Mullins Consulting, Inc. Any use or disclosure in whole or in part of this information without the express written permission of Mullins Consulting, Inc. is prohibited. © 2020 Craig S. Mullins and Mullins Consulting, Inc. (Unpublished). All rights reserved.
  • 3.
    Agenda – Industry Trends –DevOps and Agile Overview – The impact on database systems • Database change management • SQL quality assurance – Guidance
  • 4.
    Industry Trends • ReducedBudgets and IT Headcount • Database Growth and Usage • Shorter Development Cycles, Faster Innovation • Automation is increasingly important
  • 5.
  • 6.
    Percentage of DBAversus IT Staff
  • 7.
    Things Won’t BeGetting Any Better v By the year 2020
  • 8.
    Industry Trends • ReducedCosts and IT Headcount • Database Growth and Usage • Shorter Development Cycles, Faster Innovation • Automation is increasingly important
  • 9.
  • 10.
    Most Orgs Run Multiple DBMSes Source:Database Performance: The Need for Speed & Scale, Database Trends & Applications, 2017 • Polyglot Persistence • Multiple relational • Pre-relational • NoSQL • Document • Graph • Key/Value • Wide Column Store • Hybrid • In-Memory DBMS • Not just DBMS… • …but also Hadoop and Spark • Commercial and open source
  • 11.
    Industry Trends • ReducedCosts and IT Headcount • Database Growth and Usage • Shorter Development Cycles, Faster Innovation • Automation is increasingly important
  • 12.
    Shorter Development Cycles •Agile development is de rigueur – Shorter development cycles – Sprints – Early customer feedback • Higher ROI • Quicker time to value
  • 13.
    Faster Innovation Source: GettingAgile with Database Development, 2018
  • 14.
    More Frequent Deployments Source:State of DevOps Report, DORA (DevOps Research & Assessment)
  • 15.
    Industry Trends • ReducedCosts and IT Headcount • Database Growth and Usage • Shorter Development Cycles, Faster Innovation • Automation is increasingly important
  • 16.
    Automating the AppDev Process • GitHub – distributed source code management, repository • Jenkins – automated CI/CD server for automating software delivery • Docker – container platform (isolates applications; secure & portable) • Kubernetes – container orchestration (automates deployment and scaling for containerized apps) 16 • UrbanCode – automates continuous delivery cross-platform; on-premises, cloud and mainframe • Puppet – cross-platform configuration management – manage your infrastructure as code (see also Ansible, Chef) • Nagios – DevOps infrastructure monitoring
  • 17.
    DevOps DevOps is nota technology, it is a methodology.
  • 18.
    But What isDevOps? • The phrase is an amalgamation of Development and Operations but it implies more – Application Release Automation • Continuous Development, Continuous Integration, Continuous Delivery • DevOps implies Agile development – DevOps applies agile/lean methods from source code all the way to production • But also: – Any change anywhere throughout the organization can be delivered into production quickly and accurately • In a way that is safe, predictable, scalable and controlled
  • 19.
    Shifting Left • Traditionally,software development progresses from requirements to design to development to testing and deployment through support, usually depicted in a graph showing these tasks from left to right (see below). – Requirements gathering and design are shown to the left, delivery and testing to the right. • With a shift-left development mentality, processes on the right are moved to the left, so they are performed earlier in the development lifecycle. 19
  • 20.
    But There isa Mindset Mismatch • Applications teams release code much more rapidly than operations teams (infrastructure) Source: The Need for Speed: Drive Velocity and Quality with DevOps Forrester Research, 2017
  • 21.
    DevOps and theDatabase • DBA is part of the Ops in DevOps when databases are involved – All too often, Dev is over-emphasized vs. Ops • Common misconceptions: – Developers become responsible for all database development requirements – Database “code” can be treated like program code
  • 22.
    But Developers… • Gatherrequirements • Design application systems • Write code • Test programs and applications Developers are not going to completely replace DBAs DevOps does not negate the need for DBAs
  • 23.
    The PacMan Effect: DevEating Ops Dev Ops OpsOps
  • 24.
    Why is thisHappening? • Misguided belief that DBAs are roadblocks • Failure to consider the value of DBA – Misguided belief that developers can do the DBA job in addition to being good programmers • Frequently w/o DBA training – Lack of integrating DBA tools into DevOps
  • 25.
    Let’s Talk Aboutthe Database • DBA is still required even with DevOps – The actions we take on databases and database structures are different than those we take on application code
  • 26.
    DBA Practices &Procedures • Database provisioning and configuration • Automated backups and recovery testing • Database design • SQL expertise • SQL testing and performance • Test data management • Database change management – Continuous delivery of database schema migrations • A good goal: DBAs no longer do repetitive tasks – They need to be strategic and focus on architecture and oversight
  • 27.
    Database Changes • Can’tbe treated exactly like apps – The database is stateful – Application delivery copies code, executables from environment to environment – But the database is an entire configuration in each environment and changes get migrated • Some changes are very intrusive -> DROP/re-CREATE • Each environment might be different (however so slightly) • Differences can “drift” in – Optimization changes, emergency changes, etc.
  • 28.
    Application and DatabaseDesign • DBAs and Application Developers must cooperate – DBAs are the database SMEs – App Devs are the coding SMEs • A developer can no more design a database appropriately than a DBA can code an application correctly
  • 29.
    SQL Expertise • WriteSQL correctly and efficiently • Many application folks know SQL only at a cursory level • Continuous stream of new SQL capabilities – DBAs have most up-to-date knowledge • DBA and Apps must cooperate as a team
  • 30.
    Production Data Volumes Testenvironments rarely have production-sized databases • The relational optimizer can, and usually will, produce different access paths for smaller amounts of data than for larger • What works nicely in test may crash and burn in production – Requires skilled technicians that understand the nuances of SQL and optimization
  • 31.
    SQL Access Paths and Performance SELECTdeptno FROM department ORDER BY deptno;
  • 32.
    A Word AboutSome Other DBA Practices and Procedures • Database provisioning and configuration • Backup and Recovery • Test data management
  • 33.
  • 34.
    More Database Focusis Needed • More Ops participation in DevOps – DBA and performance analysts – Not oversight and bottlenecks, but real participation • More automation of database Ops – Improve app dev release velocity – Integrate with existing DevOps tooling and practices – Built-in intelligence and rules
  • 35.
    A Mindset Change •Within the organization • With developers • With DBAs
  • 36.
    Thank You Contact Details: CraigS. Mullins President & Principal Consultant IBM Champion for Data & AI IBM Gold Consultant Mullins Consulting, Inc. 15 Coventry Ct Sugar Land, TX 77479 Tel: 281-494-6153 Twitter: @craigmullins E-mail: craig@craigsmullins.com http://www.mullinsconsulting.com This document is protected under the copyright laws of the United States and other countries as an unpublished work. This document contains information that is proprietary and confidential to Mullins Consulting, Inc., which shall not be disclosed outside or duplicated, used, or disclosed in whole or in part for any purpose other than as approved by Mullins Consulting, Inc. Any use or disclosure in whole or in part of this information without the express written permission of Mullins Consulting, Inc. is prohibited. © 2020 Craig S. Mullins and Mullins Consulting, Inc. (Unpublished). All rights reserved.
  • 37.
  • 38.
    Source: The CurrentState and Adoption of DevOps Unisphere Research, 2018
  • 39.
    Source: The CurrentState and Adoption of DevOps Unisphere Research, 2018
  • 40.
    Source: The CurrentState and Adoption of DevOps Unisphere Research, 2018
  • 41.
    3 Critical Requirementsfor Successful DevOps • Automation • Integration • Measurement