The Tricky Bits of Deployment Automation:
Mainframe Code, WAS Configuration,
Databases
Eric Minick
IBM DevOps Evangelist
IBM Software, Rational
eminick@us.ibm.com
@ericminick
Robert Reeves
Chief Technology Officer
Datical, Inc.
r2@datical.com
@robertreeves
Agenda
 What makes something hard to deploy?
 Why do it anyway?
 Three examples:
–Database schema updates
–Mainframe code changes
–WebSphere Application Server configuration changes
 Q&A
What makes something hard to deploy?
Where “Something” = Component of an app
Easy things to deploy fit the classic pipeline
Code turned into a
package by a build
A version of the thing
can be stored on the
file system
The stuff in the
environment is
what is in the repo
Rolling back is just
redeploying the old
version
Hard things break the build promotion model
Not source controlled
Changes are incremental
Not owned by development
Deploy process is inconsistent
Agenda
 What makes something hard to deploy?
 Why do it anyway?
 Three examples:
–Database schema updates
–WebSphere Application Server configuration changes
–Mainframe code changes
 Q&A
Why Automate the Hard Stuff?
Because that stuff is part
of our application.
♫ Without Automation:
• We make mistakes
• We forget things
• We schedule based on
engineer availability
Agenda
 What makes something hard to deploy?
 Why do it anyway?
 Three examples:
–Database schema updates
–WebSphere Application Server configuration changes
–Mainframe code changes
 Q&A
Database Change Management Challenges
Test
Development
Build
Code
Database Change Management Challenges
Test
Release
Test
Development
Build
Code
Database Change Management Challenges
ProductionTest
Release
Test
Development
Build
Code
Database Change Management Challenges
Production
SQL Script 1
SQL Script 3
SQL Script 2
Test
Release
Test
Development
Build
Code
Database Change Management Challenges
Production
SQL Script 1
SQL Script 3
SQL Script 2
Test
Release
Test
Development
Build
Code
Database Change Management Challenges
Production
SQL Script 1
SQL Script 3
SQL Script 2
Test
Release
Test
Development
Build
Code
Database Change Management Challenges
Production
SQL Script 1
SQL Script 3
SQL Script 2
Test
Release
Test
Development
Build
Code
Database Change Management Challenges
Production
SQL Script 1
SQL Script 3
SQL Script 2
Test
Release
Test
Development
Build
Code
Database Change Management Challenges
Production
Manual Change Manual Change
SQL Script 1
SQL Script 3
SQL Script 2
Test
Release
Test
Development
Build
Code
Database Change Management w/Datical DB
Test
Development
Build
Test Production
Release
CodeCode DaticalDB
Database Change Management w/Datical DB
Test
Development
Build
Test Production
Release
CodeCode DaticalDB
Model
Easily create and
model database
changes across your
software release
stages.
Database Change Management w/Datical DB
Test
Development
Build
Test Production
Release
CodeCode DaticalDB
Model
Easily create and
model database
changes across your
software release
stages.
Forecast
Proactively scrutinize
the impact of
database changes in
production – or any
other environment –
before you deploy.
Database Change Management w/Datical DB
Test
Development
Build
Test Production
Release
CodeCode DaticalDB
Model
Easily create and
model database
changes across your
software release
stages.
Forecast
Proactively scrutinize
the impact of
database changes in
production – or any
other environment –
before you deploy.
Deploy
Deploys database
schema changes to
multiple databases
and mixed
environments
simultaneously.
Database Change Management w/Datical DB
Test
Development
Build
Test Production
Release
CodeCode DaticalDB
Model
Easily create and
model database
changes across your
software release
stages.
Forecast
Proactively scrutinize
the impact of
database changes in
production – or any
other environment –
before you deploy.
Deploy
Deploys database
schema changes to
multiple databases
and mixed
environments
simultaneously.
Manage
Confidently know the
current state of the
database and how it
got there across the
application release
lifecycle.
Datical Product Overview
Deploy Plan
DEV
QA
PROD
Datical Product Overview
Deploy Plan
DEV
QA
PROD
Datical DB Engine
Datical Product Overview
Deploy Plan
DEV
QA
PROD
ChangeSet 1
ChangeSet 2
ChangeSet 3
ChangeLog
Datical DB Engine
Datical Product Overview
Baseline
Captures the
current state
of the
database
Compare
Provides
schema
differences
between
environments
Forecast
Impacts
analysis of
proposed
changes
Deploy
Executes
changes to the
database
Rollback
Undo select
database
changes
Audit
Provides
visibility into
database
changes
Deploy Plan
DEV
QA
PROD
ChangeSet 1
ChangeSet 2
ChangeSet 3
ChangeLog
Datical DB Engine
Datical Product Overview
Baseline
Captures the
current state
of the
database
Compare
Provides
schema
differences
between
environments
Forecast
Impacts
analysis of
proposed
changes
Deploy
Executes
changes to the
database
Rollback
Undo select
database
changes
Audit
Provides
visibility into
database
changes
C:datialdb.exe
user@host:~$./daticaldb
Datical DB UI Datical DB CLI Integrations
Deploy Plan
DEV
QA
PROD
ChangeSet 1
ChangeSet 2
ChangeSet 3
ChangeLog
Datical DB Engine
Datical Product Overview
Baseline
Captures the
current state
of the
database
Compare
Provides
schema
differences
between
environments
Forecast
Impacts
analysis of
proposed
changes
Deploy
Executes
changes to the
database
Rollback
Undo select
database
changes
Audit
Provides
visibility into
database
changes
C:datialdb.exe
user@host:~$./daticaldb
Datical DB UI Datical DB CLI Integrations
Deploy Plan
DEV
QA
PROD
ChangeSet 1
ChangeSet 2
ChangeSet 3
ChangeLog
Datical DB Engine
Agenda
 What makes something hard to deploy?
 Why do it anyway?
 Three examples:
–Database schema updates
–WebSphere Application Server configuration changes
–Mainframe code changes
 Q&A
WebSphere Config – What are we deploying?
Let’s see… The app needs:
• More memory
• Longer timeout setting on the JDBC connector
• Subscription to a new message queue
WebSphere Config – Why it’s hard
Not source controlled
Changes are incremental
Not owned by development
Deploy process is inconsistent




Strategy: Infrastructure as Code
 Represent Configuration as Files
–Have a “build” than can be versioned
–Can compare versions to see what changed
 Requirements for Automation
–Tokenize / Replace per environment settings
–Extract configuration from a known good target
–Apply to other servers
Reliable Middleware Configuration
Management
Artifact Library
Application
EAR
WAR
DB
Cluster
template
Exemplar
WAS Cell
Plugin
Import
configuration
WAS Configuration Template Creation
+ Template
Assembled
PROD
QA
Dev
Deploy and promote application and
configuration across environments
Agenda
 What makes something hard to deploy?
 Why do it anyway?
 Three examples:
–Database schema updates
–WebSphere Application Server configuration changes
–Mainframe code changes
 Q&A
Mainframe changes – Why it’s hard
Not source controlled
Changes are incremental
Not owned by development
Deploy process is inconsistent
relies on tribal knowledge



Mainframe changes – Why it’s hard
Worse yet…
• We want to keep the files on the box
• Test environments are precious
Strategy: Embrace Incremental
 Capture builds
 Categorize the content
 Apply changes incrementally, tracking order.
Build
System
Post build
script
z/OS DeployToolkit
Create new version
z/OS
CodeStation
in USS
Server
Agent
Download
artifacts
Review PDS in
version and
request deploy
process
Pre-processing
steps TSO,
REXX, SHELL
Deploy
data sets
PDS
PDS/E
Update Inventory
status
High Level Overview of UrbanCode’s z Deployment Capabilities
z/OS LPAR, Build system z/OS LPAR
Note: LPARs can be the same or different LPARs
Store meta data
Store version artifacts
Fetch artifacts
via copy or FTP
Post-processing
steps TSO,
REXX, SHELL
deploy
1
2
3
4
5
6
Summary
 Incremental, non-code changes are hard
 Two strategies for managing
–Smart incremental updates
–Model full versions, incrementally bring into compliance
 Tools exist that help
For more information
IBM UrbanCode Deploy Datical , Inc
IBM developerWorks
https://developer.ibm.com/urbancode
www.ibm.com
Eric Minick
eminick@us.ibm.com
@ericminick
Questions
ucinfo@us.ibm.com
DaticalDB4UDeploy
datical.com/ibm
Bring Agile Development to
the Database
datical.com/agile
info@datical.com
(949) DATICAL (328-4225)
@datical
fb.com/datical
42
© Copyright IBM Corporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of
any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to,
nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing
the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release
dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment
to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the
International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
www.ibm.com/software
TheTricky Bits of Deployment Automation

TheTricky Bits of Deployment Automation

  • 1.
    The Tricky Bitsof Deployment Automation: Mainframe Code, WAS Configuration, Databases Eric Minick IBM DevOps Evangelist IBM Software, Rational eminick@us.ibm.com @ericminick Robert Reeves Chief Technology Officer Datical, Inc. r2@datical.com @robertreeves
  • 2.
    Agenda  What makessomething hard to deploy?  Why do it anyway?  Three examples: –Database schema updates –Mainframe code changes –WebSphere Application Server configuration changes  Q&A
  • 3.
    What makes somethinghard to deploy?
  • 4.
    Where “Something” =Component of an app
  • 5.
    Easy things todeploy fit the classic pipeline Code turned into a package by a build A version of the thing can be stored on the file system The stuff in the environment is what is in the repo Rolling back is just redeploying the old version
  • 6.
    Hard things breakthe build promotion model Not source controlled Changes are incremental Not owned by development Deploy process is inconsistent
  • 7.
    Agenda  What makessomething hard to deploy?  Why do it anyway?  Three examples: –Database schema updates –WebSphere Application Server configuration changes –Mainframe code changes  Q&A
  • 8.
    Why Automate theHard Stuff? Because that stuff is part of our application. ♫ Without Automation: • We make mistakes • We forget things • We schedule based on engineer availability
  • 9.
    Agenda  What makessomething hard to deploy?  Why do it anyway?  Three examples: –Database schema updates –WebSphere Application Server configuration changes –Mainframe code changes  Q&A
  • 10.
    Database Change ManagementChallenges Test Development Build Code
  • 11.
    Database Change ManagementChallenges Test Release Test Development Build Code
  • 12.
    Database Change ManagementChallenges ProductionTest Release Test Development Build Code
  • 13.
    Database Change ManagementChallenges Production SQL Script 1 SQL Script 3 SQL Script 2 Test Release Test Development Build Code
  • 14.
    Database Change ManagementChallenges Production SQL Script 1 SQL Script 3 SQL Script 2 Test Release Test Development Build Code
  • 15.
    Database Change ManagementChallenges Production SQL Script 1 SQL Script 3 SQL Script 2 Test Release Test Development Build Code
  • 16.
    Database Change ManagementChallenges Production SQL Script 1 SQL Script 3 SQL Script 2 Test Release Test Development Build Code
  • 17.
    Database Change ManagementChallenges Production SQL Script 1 SQL Script 3 SQL Script 2 Test Release Test Development Build Code
  • 18.
    Database Change ManagementChallenges Production Manual Change Manual Change SQL Script 1 SQL Script 3 SQL Script 2 Test Release Test Development Build Code
  • 19.
    Database Change Managementw/Datical DB Test Development Build Test Production Release CodeCode DaticalDB
  • 20.
    Database Change Managementw/Datical DB Test Development Build Test Production Release CodeCode DaticalDB Model Easily create and model database changes across your software release stages.
  • 21.
    Database Change Managementw/Datical DB Test Development Build Test Production Release CodeCode DaticalDB Model Easily create and model database changes across your software release stages. Forecast Proactively scrutinize the impact of database changes in production – or any other environment – before you deploy.
  • 22.
    Database Change Managementw/Datical DB Test Development Build Test Production Release CodeCode DaticalDB Model Easily create and model database changes across your software release stages. Forecast Proactively scrutinize the impact of database changes in production – or any other environment – before you deploy. Deploy Deploys database schema changes to multiple databases and mixed environments simultaneously.
  • 23.
    Database Change Managementw/Datical DB Test Development Build Test Production Release CodeCode DaticalDB Model Easily create and model database changes across your software release stages. Forecast Proactively scrutinize the impact of database changes in production – or any other environment – before you deploy. Deploy Deploys database schema changes to multiple databases and mixed environments simultaneously. Manage Confidently know the current state of the database and how it got there across the application release lifecycle.
  • 24.
  • 25.
    Datical Product Overview DeployPlan DEV QA PROD Datical DB Engine
  • 26.
    Datical Product Overview DeployPlan DEV QA PROD ChangeSet 1 ChangeSet 2 ChangeSet 3 ChangeLog Datical DB Engine
  • 27.
    Datical Product Overview Baseline Capturesthe current state of the database Compare Provides schema differences between environments Forecast Impacts analysis of proposed changes Deploy Executes changes to the database Rollback Undo select database changes Audit Provides visibility into database changes Deploy Plan DEV QA PROD ChangeSet 1 ChangeSet 2 ChangeSet 3 ChangeLog Datical DB Engine
  • 28.
    Datical Product Overview Baseline Capturesthe current state of the database Compare Provides schema differences between environments Forecast Impacts analysis of proposed changes Deploy Executes changes to the database Rollback Undo select database changes Audit Provides visibility into database changes C:datialdb.exe user@host:~$./daticaldb Datical DB UI Datical DB CLI Integrations Deploy Plan DEV QA PROD ChangeSet 1 ChangeSet 2 ChangeSet 3 ChangeLog Datical DB Engine
  • 29.
    Datical Product Overview Baseline Capturesthe current state of the database Compare Provides schema differences between environments Forecast Impacts analysis of proposed changes Deploy Executes changes to the database Rollback Undo select database changes Audit Provides visibility into database changes C:datialdb.exe user@host:~$./daticaldb Datical DB UI Datical DB CLI Integrations Deploy Plan DEV QA PROD ChangeSet 1 ChangeSet 2 ChangeSet 3 ChangeLog Datical DB Engine
  • 30.
    Agenda  What makessomething hard to deploy?  Why do it anyway?  Three examples: –Database schema updates –WebSphere Application Server configuration changes –Mainframe code changes  Q&A
  • 31.
    WebSphere Config –What are we deploying? Let’s see… The app needs: • More memory • Longer timeout setting on the JDBC connector • Subscription to a new message queue
  • 32.
    WebSphere Config –Why it’s hard Not source controlled Changes are incremental Not owned by development Deploy process is inconsistent    
  • 33.
    Strategy: Infrastructure asCode  Represent Configuration as Files –Have a “build” than can be versioned –Can compare versions to see what changed  Requirements for Automation –Tokenize / Replace per environment settings –Extract configuration from a known good target –Apply to other servers
  • 34.
    Reliable Middleware Configuration Management ArtifactLibrary Application EAR WAR DB Cluster template Exemplar WAS Cell Plugin Import configuration WAS Configuration Template Creation + Template Assembled PROD QA Dev Deploy and promote application and configuration across environments
  • 35.
    Agenda  What makessomething hard to deploy?  Why do it anyway?  Three examples: –Database schema updates –WebSphere Application Server configuration changes –Mainframe code changes  Q&A
  • 36.
    Mainframe changes –Why it’s hard Not source controlled Changes are incremental Not owned by development Deploy process is inconsistent relies on tribal knowledge   
  • 37.
    Mainframe changes –Why it’s hard Worse yet… • We want to keep the files on the box • Test environments are precious
  • 38.
    Strategy: Embrace Incremental Capture builds  Categorize the content  Apply changes incrementally, tracking order.
  • 39.
    Build System Post build script z/OS DeployToolkit Createnew version z/OS CodeStation in USS Server Agent Download artifacts Review PDS in version and request deploy process Pre-processing steps TSO, REXX, SHELL Deploy data sets PDS PDS/E Update Inventory status High Level Overview of UrbanCode’s z Deployment Capabilities z/OS LPAR, Build system z/OS LPAR Note: LPARs can be the same or different LPARs Store meta data Store version artifacts Fetch artifacts via copy or FTP Post-processing steps TSO, REXX, SHELL deploy 1 2 3 4 5 6
  • 40.
    Summary  Incremental, non-codechanges are hard  Two strategies for managing –Smart incremental updates –Model full versions, incrementally bring into compliance  Tools exist that help
  • 41.
    For more information IBMUrbanCode Deploy Datical , Inc IBM developerWorks https://developer.ibm.com/urbancode www.ibm.com Eric Minick eminick@us.ibm.com @ericminick Questions ucinfo@us.ibm.com DaticalDB4UDeploy datical.com/ibm Bring Agile Development to the Database datical.com/agile info@datical.com (949) DATICAL (328-4225) @datical fb.com/datical
  • 42.
    42 © Copyright IBMCorporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others. www.ibm.com/software