DevOps for Mainframe for IBM Pulse Conference


Published on

Presentation used at IBM Pulse 2013 to describe DevOps for the Mainframe. Presentation done by David Myers, and Rosalind Radcliffe

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

DevOps for Mainframe for IBM Pulse Conference

  1. 1. DevOps for the MainframeLeveraging Continuous Integration, the cloud, and beyond todeliver z/OS applicationsDavid Myers, Product Manager, Continuous Delivery forMainframe and Mobile SoftwareRosalind Radcliffe, Distinguished Engineer, EnterpriseModernization Solution Architect © 2012 IBM Corporation
  2. 2. Please noteIBM’s  statements  regarding  its  plans,  direc3ons,  and  intent  are  subject  to  change  or  withdrawal  without  no3ce  at  IBM’s  sole  discre3on.  Informa3on  regarding  poten3al  future  products  is  intended  to  outline  our  general  product  direc3on  and  it  should  not  be  relied  on  in  making  a  purchasing  decision.    The  informa3on  men3oned  regarding  poten3al  future  products  is  not  a  commitment,  promise,  or  legal  obliga3on  to  deliver  any  material,  code  or  func3onality.  Informa3on  about  poten3al  future  products  may  not  be  incorporated  into  any  contract.  The  development,  release,  and  3ming  of  any  future  features  or  func3onality  described  for  our  products  remains  at  our  sole  discre3on.  Performance  is  based  on  measurements  and  projec3ons  using  standard  IBM  benchmarks  in  a  controlled  environment.    The  actual  throughput  or  performance  that  any  user  will  experience  will  vary  depending  upon  many  factors,  including  considera3ons  such  as  the  amount  of  mul3programming  in  the  user’s  job  stream,  the  I/O  configura3on,  the  storage  configura3on,  and  the  workload  processed.    Therefore,  no  assurance  can  be  given  that  an  individual  user  will  achieve  results  similar  to  those  stated  here.   2
  3. 3. New era systems integrate existing operational systems with rapid delivery of new client-facing apps Social, Local, Mobile Smart Infrastructure Analytics CRM   HR   Legacy   ERP   DB   Manage workloads Rapid innovation User experience and and maintain security in the cloud mobile management 33
  4. 4. Evolving customer and market expectationsCapabilities and User Experience Today Emerging Systems of Record Systems of Engagement (+ Record) Primary Workload Types Transactional Big Data, Analytics, Mobile/Social Channels Time to Value Planned Opportunistic Delivery Model Planned Incremental (DevOps)Development and Operations Team Sizes 100s and Costly 10s with built-in DevOps automation Release Frequency Months to Years Hours to Days, based on business opportunity Integration Frequency Weeks Continuous Service Sourcing Develop Consume and Assemble (Public and Private) Operational Model Systems Management Built in to application, Recovery Oriented Computing, Continuous Availability Infrastructure Deployment Days Minutes 4 Risk Profile Big-Bang (High Risk) Incremental
  5. 5. DevOps is…A set of principles and values that facilitatecollaboration across disciplines to…1. Enable rapid evolution of deployed business services2. Reduce risk, decrease cost, and improve quality acrossthe portfolio People     Process  DevOps Principles§  Collaborate across disciplines Tools  §  Develop and test against a production-like system§  Deploy frequently using repeatable and reliable processes§  Continuously monitor and validate operational quality characteristics§  Amplify feedback loops 5
  6. 6. Solution: A Continuous Delivery Pipeline Development Testing Staging Production Business Owners Customers Ensure applications are production-ready throughout the lifecycle and can be released at any time while minimizing rollback due to quality issues•  Validate  on  more  produc3on-­‐like   •  Standardiza3on  on  processes  and  assets   condi3ons  earlier   between  Dev  and  Ops  •  Automate  hand-­‐offs/promo3ons  to   •  Automated  monitoring  and  dashboarding   increase  velocity  through  the  different   of  quality  and  performance  against  service   stages   level  agreements  at  mul3ple  stages   6
  7. 7. Mainframe Delivery Pains…§  Multiple teams working across §  Too much bad code going into restricted dev and test capacity test and production causes crit lead to conflict, delays, or bad sits and emergency fixes test results in shared environments §  Bottlenecks due to inefficient§  Complex and manual communications between management and configuration disparate platforms and teams tasks result in errors and delays (Dev/Test - System Programmers; mobile – distributed-mainframe) 7
  8. 8. …solutions from IBM Play  Defense  § Provide  cheap,  isolated,  development   § Enforce  base  quality  standards   and  test  environments  for  project   automa3cally  prior  to  promo3on   teams   –  Ra3onal  Development  and  Test  Environment   –  Ra3onal  Test  Workbench   –  Ra3onal  Test  Virtualiza3on  Server   –  Ra3onal  Quality  Manager   –  SmartCloud  Provisioning     –  SmartCloud  Applica3on  Monitoring   –  Cloud  Ready  for  Linux  on  System  z   –  Omegamon      § Automate  consistent  build,  configure,   § Improve  communica3on  and   and  deploy  processes  across  all  stages     collabora3on  with  cross-­‐plaVorm   –  Ra3onal  Team  Concert   release  planning   –  SmartCloud  Con3nuous  Delivery   –  IBM  Collabora3ve  Lifecycle  Management   –  SmartCloud  Orchestrator   –  Smart  Cloud  Control  Desk   Go  on  Offense   8
  9. 9. DevOps for the common mainframe delivery cycle1   2   2   Dev   Test   Dev   Test   Dev   Test   Staging   Prod   4   3   Development" Test" System Programmer" 9
  10. 10. Typical z/OS Testing Architecture Organized by project team, vertically scaled, sharing resources, limited automation z/OS Problems Encountered Test LPAR Project Team 1. Shared resources combined [April Maintain] with overlapping schedules can App elicit conflicts, impede innovation and slow code Project delivery Team [Prototype SOA] 2. Coordination of environmental Test   changes and releases cause App bottlenecks, delays and Project Data   Team additional overhead [June New Func] 3. Shared test data is difficult to … manage and can lead to over testing or incorrect test results Project Team App [Dec Sys Upgrade] 1010 10
  11. 11. Rational Development and Test Environment for System zThe ultimate in modern application development for System z COBOL,  PL/I,  C++,  Java,  EGL,  Batch,     Assembler,  Debug  Tool   IMS     DB2 CICS WAS MQ RDz user RDz user z/OS            x86  PC  running  Linux   RDz user ISPF user RDz user RDz & ISPF user §  Liberate  developers  to  rapidly  prototype  new  applica3ons   §  Develop  and  test  System  z  applica3ons  anywhere,  any3me!   §  Free  up  mainframe  development  MIPS  for  produc3on  capacity   §  Eliminate  costly  delays  by  reducing  dependencies  on  opera3ons  staff     Note:  This  Program  is  licensed  only  for  development  and  test  of  applica3ons  that  run  on  IBM  z/OS.  The  Program  may  not  be  used  to  run  produc3on  workloads  of  any  kind,  nor  more  robust   development  workloads  including  without  limita3on  produc3on  module  builds,  pre-­‐produc3on  tes3ng,  stress  tes3ng,  or  performance  tes3ng.       11
  12. 12. Decouple subsystem teams using Green HatImprove  tes+ng  with  Green  Hat  service  simula+on   Simulate subsystem dependencies –  Test impact of latency –  Test application response to unresponsive services Application changes Create a stable test environment being tested –  Simulate “public” services –  Reduce capacity requirements on infrastructure –  Improve security by reducing access from external services Databases Mainframe Third-party applications Services virtualized services Ra3onal  Test  Virtualiza3on  Server   12
  13. 13. Testing with dependency virtualization Controlled large system testing by isolating components under test •  Easier problem determination •  Lower test environment capacity requirements •  Improved component quality Phase 1 Phase 2 Phase 3 Test  Case   Test  Case   Test  Case  IMSData 3rd Party IMS Data 3rd PartyAccess Call IMS Data 3rd Party App   Access App   Call Access Call App   CICS CICS Commarea CICS Commarea Call Commarea Call Call Virtual   Virtual   Virtual   Services   Services   Services   13
  14. 14. Provision new dev / test systems in minutes•  Fast  VM  provisioning  for  near-­‐instant   deployment  of  100s  of  virtual  machines  and   scale  based  on  business  needs  •  Rich  set  of  web  interfaces  into  the  cloud  that   can  be  driven  by  a  user  or  scripts  for  complete   automa3on  •  Reliable,  non-­‐stop  cloud  capable  of   automa3cally  tolera3ng  and  recovering  from   soeware  and  hardware  failures  •  Near-­‐zero  down=me  due  to  faults,  hypervisor/ management  soeware  upgrades  or  addi3on/ removal  of  hardware   Func3on  Test   Performance   Test   Requested  VMs  will  be  up     and  running  under  a     minute  using  standard  HW   14
  15. 15. Fail fast and drive better quality downstreamReduced delivery time, end-to-end visibility of test activities, safer and faster upgrades (V2V)   Automated, Reusable continuous testing integration with isolated Continuous Integration for z tests simplify development and testing across test environment layers Check  In  +  Build  +  Deploy  +  Run   Code   Automated  Tests     Change     Promote   Request   To  QA   Frequent, rapid feedback earlier Developer Higher quality to improve     going into QA quality Rapid  Feedback   §  Fast, dependable, automatic feedback speeds time to market §  Lower cost of application testing using off-mainframe z/OS test environment §  Enables confidence by automatically tracking and promoting code health 15
  16. 16. Continuous Integration for System z Scenario System z Test ests> pp+zUnit T Environment <Deploys A (RD&T or LPAR) <Drives> SCM/CI <Starts Tests> Test Server Execution (Rational Team Manager Concert) (Ra3onal  Quality   <Results> Manager) Test Automation (Rational Test1.  Check-in code 5.  Kick-off Automated Workbench)2.  Build code and zUnit Test Plan 6.  Run automated tests interface tests against3.  Deploy build results 8.  Report test results Test Environment and test data to Test in dashboard/build 7.  Mark execution Environment results/defect records Pass/Fail in4.  Execute zUnit Tests records in CI server. Test Execution 16 Manager 16
  17. 17. Separate tools makes collaboration inefficient or difficult...§ Each tool came with its own Tool A Dev work Items Tool B •  UI - Web and desktop presentations of views and DB Release Planning UI tasks •  Logic – Workflow, process, LOGIC DB UI search, query, scale, security Tool E and collaboration LOGIC •  Storage – Availability, SCM traceability DB UI •  Privacy, backup/archive LOGIC§ Resulting in... DB UI •  Brittle integrations LOGIC •  Silos everywhere DB UI Tool C •  High cost to maintain and Tool D LOGIC Reporting administer •  Proprietary APIs Incident Management 17
  18. 18. Tie development release to operational releaseRa3onal  Team  Concert  Release  Plan  and  work  items   SmartCloud  Control  Desk     18
  19. 19. Delivery Organized for Flexibility and Speed Organized by application team, horizontally sliced, dedicated resources, highly automated z/OS LPAR Problems Encountered Project 1.  Shared resources Team [April Maintain] LPAR combined with overlapping schedules Data   can elicit conflicts, QA/Integra3on  Test  LPAR   impede innovation and Project Team slow code delivery [Prototype SOA] RD&T 2.  Coordination of Data   Integr.   environmental changes Test   and releases cause Project Data   bottlenecks, delays and Team [June New Func] RD&T additional overhead 3.  Shared test data is … Data   difficult to manage and can lead to over testing Project or incorrect test results Team [Dec Sys Upgrade] LPAR 4.  Provisioning, managing, and synchronizing project Data   test environments including data 1919 19
  20. 20. Deploy application and config changes to test environments in minutesOpera+ons  team   Development  team  establishes  deployment   makes  applica+on  pa;erns  for  use  by   IBM code  and  development  and  test   SmartCloud configura+on   Continuousteams   Delivery changes   Deliver Environments 3x Test against Faster!! production-like environments with Applications & middleware AUTOMATICALLY installed and properly IBM configured IBM SmartCloud Workload 20 Provisioning Deployer
  21. 21. What are we working on inthe lab… 21
  22. 22. …solutions from IBM Play  Defense  § Provide  cheap,  isolated,  development   § Enforce  base  quality  standards   and  test  environments  for  project   automa3cally  prior  to  promo3on   teams   –  Ra3onal  Test  Workbench   –  Ra3onal  Development  and  Test  Environment   –  Ra3onal  Quality  Manager   –  Ra3onal  Test  Virtualiza3on  Server   –  SmartCloud  Applica3on  Monitoring   –  SmartCloud  Provisioning     –  Omegamon   –  Cloud  Ready  for  Linux  on  System  z      § Automate  consistent  build,  provision,   § Improve  communica3on  and   configure,  and  deploy  pof  zLinux   across   collabora3on  with  cross-­‐plaVorm   Cloud-­‐style  deploy   rocesses   all  stages     dev  and  test  paMerns   and  z/OS   release  planning   on  RD&T  or  zHardware   –  Ra3onal  Team  Concert   –  IBM  Collabora3ve  Lifecycle  Management   –  SmartCloud  Con3nuous  Delivery   –  Smart  Cloud  Control  Desk   –  SmartCloud  Orchestrator*   Go  on  Offense   22
  23. 23. Provisioning using Cloud#!/usr/bin/env rubyclass DevopsDeployer def initialize(build_url, build_id) @log = @log.level = LOG_LEVEL @iaas_gateway =,LOG_FILE, LOG_LEVEL) @server_instance = nil .cbl   .cpy   rtc_build_system_provider =, RTC_USER_ID, RTC_PASSWORD_FILE) @build = rtc_build_system_provider.resolve_build(build_url, ENV[buildResultUUID], build_id) Infrastructure   as  Code   @build_system_gateway = .asm  rtc_build_system_provider, LOG_FILE, LOG_LEVEL) end def add_build_stamp template_file = WEB_APP_ROOT +"/app/templates/pages/page.html" "Adding build ID stamp #{} to .rexx  #{template_file}" CLIST   # Read in the files contents as a string, replace # the build_id, then overwrite the original contents # of the file     text = new_text = text.gsub(/{{ build_id }}/, Source  Ar3facts  "<a href="#{@build.uri}">#{}</a>"), "w") { |file| } end Source  Control     file.puts new_text# ... Management   23
  24. 24. Delivery Pipeline Fundamentals Build,  Package,   &  Unit  Test   Applica+on   .cbl   .cpy   Binaries  &   PlaGorm   Configura+on   Deploy   .asm   .rexx   CLIST   Environment       Deployable  Ar3facts   Running  System   Source  Ar3facts  Source  Control     Library  Management   24
  25. 25. Consolidate, standardize, simplify deploymentEnterprise  Applica3ons  have  thousands  of  disparate  parts  •  Currently  maintained  in  separate  systems,  if  maintained  at  all  •  Limited  linkage  between  systems  for  applica3on  and  configura3on  dependencies  •  Missing  assets  and  informa3on  is  rampant    DevOps  force  linkages,  automa3on,  and  standardized  packaging…   Copybooks   HTML   COBOL   JCL   Build,  Package,   WSDL   &  Unit  Test   MQ   Applica+on   Deploy   Java   Queue   Binaries  &   Def   PlaGorm   Configura+on   DB2   IMS  DBD/PSB   DDL   DB2  Plans   Environment   Deployable  Ar3facts   Running  System   Procs   Library   Source  Management   25
  26. 26. Create, secure, and deploy test data Automated Build & Deploy Test Data -Compare z/OS   -Subset -Refresh UT   RD&T   2TB 1 GB Test Data z/OS   Production or UT   LPAR  Production Clone 1 GB Masked Test Data Automated Testing Create “right-size” De-identify sensitive information Automated test dataproduction-like environments De-identify sensitive information with realistic but fictional data for deployment for each for application testing with realistic purposes testing & development but fictional data for testing & development purposes build and test Optim RTC 26
  27. 27. Standardize z/OS region configurations to ease deployment Configura3on   Applica3on   •  Standard topologies exist today (production LPARs) •  Standardized regions which can be Data   repeatedly deployed are rare •  Standardized/Automated deployment of COMPLETE system is spottyCICS  5.1   IMS  11.1   Standard  Middleware  Palern   To adopt the cloud for testing… Standardize and automate provisioning ofZ/OS  1.12   DB2  10.1   everything … 27
  28. 28. Standardize z/OS region configurations to ease deployment RTC/CIz Available Application Capabilities Configura3on   Applica3on   load modules Data   JCL and REXX Test DataCICS  5.1   IMS  11.1   Pattern/Platform Standard  Middleware  Palern  Z/OS  1.12   DB2  10.1   28
  29. 29. Evolving to deploying regions … cloud-style “Cloud” CICS+DB2   RD&T CICS+  IMS   Or DB   LPAR Region/Middleware   RD&T Palerns   Or Batch   LPAR RD&T Or LPAR RD&T Or LPARSmartCloud  Orchestrator   +   <SCD  for  Mainframe   Project>   29
  30. 30. Acknowledgements and Disclaimers:    Availability.    References  in  this  presenta3on  to  IBM  products,  programs,  or  services  do  not  imply  that  they  will  be  available  in  all  countries  in   which  IBM  operates.                        The  workshops,  sessions  and  materials  have  been  prepared  by  IBM  or  the  session  speakers  and  reflect  their  own  views.    They  are  provided  for   informa3onal  purposes  only,  and  are  neither  intended  to,  nor  shall  have  the  effect  of  being,  legal  or  other  guidance  or  advice  to  any  par3cipant.     While  efforts  were  made  to  verify  the  completeness  and  accuracy  of  the  informa3on  contained  in  this  presenta3on,  it  is  provided  AS-­‐IS  without   warranty  of  any  kind,  express  or  implied.  IBM  shall  not  be  responsible  for  any  damages  arising  out  of  the  use  of,  or  otherwise  related  to,  this   presenta3on  or  any  other  materials.  Nothing  contained  in  this  presenta3on  is  intended  to,  nor  shall  have  the  effect  of,  crea3ng  any  warran3es  or   representa3ons  from  IBM  or  its  suppliers  or  licensors,  or  altering  the  terms  and  condi3ons  of  the  applicable  license  agreement  governing  the  use   of  IBM  soeware.              All  customer  examples  described  are  presented  as  illustra3ons  of  how  those  customers  have  used  IBM  products  and  the  results  they  may  have   achieved.    Actual  environmental  costs  and  performance  characteris3cs  may  vary  by  customer.    Nothing  contained  in  these  materials  is  intended   to,  nor  shall  have  the  effect  of,  sta3ng  or  implying  that  any  ac3vi3es  undertaken  by  you  will  result  in  any  specific  sales,  revenue  growth  or  other   results.        ©  Copyright  IBM  Corpora6on  2013.  All  rights  reserved.   §  U.S.  Government  Users  Restricted  Rights  -­‐  Use,  duplica6on  or  disclosure  restricted  by  GSA  ADP  Schedule  Contract  with  IBM  Corp.   §  Please  update  paragraph  below  for  the  par+cular  product  or  family  brand  trademarks  you  men+on  such  as  WebSphere,  DB2,  Maximo,   Clearcase,  Lotus,  etc    IBM,  the  IBM  logo,,  [IBM  Brand,  if  trademarked],  and  [IBM  Product,  if  trademarked]  are  trademarks  or  registered  trademarks  of   Interna3onal  Business  Machines  Corpora3on  in  the  United  States,  other  countries,  or  both.  If  these  and  other  IBM  trademarked  terms  are   marked  on  their  first  occurrence  in  this  informa3on  with  a  trademark  symbol  (®  or  ™),  these  symbols  indicate  U.S.  registered  or  common  law   trademarks  owned  by  IBM  at  the  3me  this  informa3on  was  published.  Such  trademarks  may  also  be  registered  or  common  law  trademarks  in   other  countries.  A  current  list  of  IBM  trademarks  is  available  on  the  Web  at  “Copyright  and  trademark  informa3on”  at    If  you  have  men3oned  trademarks  that  are  not  from  IBM,    please  update  and  add  the  following  lines:      [Insert  any  special  3rd  party  trademark  names/alribu3ons  here]      Other  company,  product,  or  service  names  may  be  trademarks  or  service  marks  of  others.   30
  31. 31.© Copyright IBM Corporation 2012. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind,express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall havethe effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBMsoftware. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilitiesreferenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or featureavailability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business MachinesCorporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others. 31