ITIL and DevOps at War in the Enterprise - DevOpsDays Amsterdam 2014
•Download as PPTX, PDF•
5 likes•3,291 views
The journey from ITIL/CMMi to DevOps in the corporate setting of ING Netherlands. Presentation by Mark Heistek and Jan-Joost Bouwman at DevOpsDays Amsterdam 2014.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (10)
Similar to ITIL and DevOps at War in the Enterprise - DevOpsDays Amsterdam 2014
Similar to ITIL and DevOps at War in the Enterprise - DevOpsDays Amsterdam 2014 (20)
Recently uploaded
Recently uploaded (20)
ITIL and DevOps at War in the Enterprise - DevOpsDays Amsterdam 2014
- 1. ITIL and DevOps at war in the enterprise
- 2. Mark Heistek Background in: • Operations • Project management / Process management • DevOps and CD evangelist since 2 years Current position within ING: • Continuous Delivery team CIO NL • DevOps community manager on internal social platform Interests: • Sports Jan-Joost Bouwman Background in: • Operations • Process management (change) • DevOps and CD evangelist since 2 years Current position within ING: • Process owner Service Operations & Service Transition (and only person that knows what that is) • DevOps community co-manager Interests: • Birdwatching, travel @markheistek Mark.heistek@ing.nl @janjoostbouwman Jan-joost.bouwman@ing.nl
- 4. The enterprise as organism
- 5. OPS DEV Where do we come from?
- 7. Where did we start to come together?
- 8. ITIL service design and tollgates
- 9. Generic Acceptance Criteria PCAB / Tollgate 1 Solution Delivery Clarity code: Service Management Change nr: Nr. Description Expected Output Remarks/Checkpoints Nr. Description Expected Output Remarks/Checkpoints SD1.1 Has the responsible OIB parties been identified, including co-developing parties and are all work packages/team plans defined and incorporated in planning, PID (list of deliverables) and business case (exploitation costs), including infrastructure? PID, work packages, team plans SM1.1 Has Service Management delivered the baseline information from the CMDB to the project? CI Relation report SD1.2 Is the Solution Architecture for the involved infrastructure up to date and approved by Technology Design Authority? Confirmation and approval by TDA that deliverables are in alignment with Solution Architecture SA, TDA confirmation SM1.2 Has Service Management provided the non-functional requirements to the project (including waivers and problems to be solved, and any Specific Acceptance Criteria, ie specific for that department or that environment) SD1.3 Is the Solution Architecture for the involved application up to date and approved by Enterprise Architecture? Confirmation and approval by EA that deliverables are in alignment with Solution Architecture. SA, EA confirmation SM1.3 Has Service Management registered the project and the work packages (including infrastructure) in HPSC and assigned to the domains responsible for support after implementation? HPSC records and linkages SD1.4 Are proposed changes/designs to the IT Services been aligned with the policy to disentangle IT Services for Insurance and Banking? SA, EA confirmation SM1.4 Has Service Management provided the costs for Transition to Support and the expected delta in the exploitation costs as input for the Business Case of the project. updated business case SD1.5 Capacity modelling: has the delta in required capacity for production been specified for Application Support and Infrastructure? Are test plans developed for testing the capacity models? Capacity model; test plans SM1.5 Has Service Management provided baseline Capacity reports as input for Capacity Modelling Performance reports SD1.6 Has infrastructure confirmed the requested delta in required capacity? Confirmation from ISS/TS/BST SD1.7 Have contracts for both Solution Delivery and Service Management with (sub-)contractors and vendors been checked and updated (if necessary) and are the results added to the business case? Finance, Vendor Management SD1.8 Has TDA checked the consequences for the system and service management tooling? TDA confirmation SD1.9 Has the list of items that will be transfered to Service Management in Transition been specified? PID SD1.10 Has the list of items that will be solved by the project (problems, waivers, software/hardware decommissioning etc.) been delivered to Service Management? list SD2.1 Is the current BIA still valid or does it need updating? Necessary updates are approved by the relevant parties Approved BIA SM2.1 Has Service Management determined the Risk & Impact of the change during implementation and operation Initial Risk & Impact To be confirmed in TCAB SD2.2 Are the CIA and PIA still valid or do they need updating? Necessary updates are approved by the relevant parties Approved CIA and PIA SD2.3 Are the ACA, BCDR design, Security Monitoring design, RBAC and IST/SOLL matrix still valid or do they need updating? Necessary updates are approved by the relevant parties Approved ACA, BCDR design, Security Monitoring design, RBAC and IST/SOLL matrix 3 Testing SM3.1 Has Service Management delivered the baseline for performance testing test plan 4 Planning SD4.1 Has the TCAB -as tollgate 2- been planned before start of UAT? planning SD4.2 Has the final DCAB been planned (after UAT, before implementation date)? planning SD4.3 Does the planning cater for rework after testing (both software and infrastructure)? planning SD4.4 Has BCDR delivery and testing been planned? planning SD4.5 Has delivery of an (updated) I and A OSG been planned planning SD4.6 Have all the documents to be updated been identified and has updating the documents been planned. planning SD4.7 Has the need to train or educate staff for supporting and endusers for using new or changed functionality been determined? planning SD4.8 In case you deliver a modification (including new developments) in a web based application: plan the delivery of sign-offs of the security code review scan report before promoting the software into Production environment. planning SD4.9 If the project requires new or adjusted external connections, have penetration tests been planned, staffed as part of UAT/PAT and has the sign-off by RCEC been planned before Tollgate 2? planning 1 Non- functional requirements and deliverables 2 Risk, Continuity and Security Generic Acceptance Criteria 1st DCAB / Tollgate 2 Solution Delivery Clarity code: 0 Service Management Change nr: 0 Nr. Description Expected Output Remarks/Checkpoints Nr. Description Expected Output Remarks/Checkpoints SD1.1 Have all items on the PCAB and TCAB checklist been completed and signed-off before the first DCAB meeting? updated checklist SM1.1 Is the RfC record in HPSC up to date? In case of a Project Exception Report: has the record been updated to reflect changes to deliverables, planning and or business case for Service Management? NB Change owner may reside on SD side confirmation by change owner SD1.2 SM1.2 Have all Specific Acceptance Criteria agreed upon between SoDC and SeDC confirmation by change owner SD1.3 Has Technology Design Authority confirmed that the final deliverables are in line with the approved Infrastructure and Security Architecture and Detail Design sign off on deliverables by TDA SM1.3 Have Service Delivery Contracts been approved? confirmation by RL4/Service Manager SD1.4 Has Entreprise Architecture confirmed that the final deliverables is in line with the approved Solution Architecture and Detail Design? sign off on deliverables by EA SM1.4 Is the SLA for the Service or components that will be changed ready, has it been signed-off by Supplier and Business? SLA with sign off by Supplier and Business SD1.5 Are the final deliverables still in alignment with the policy to disentangle IT Services for Insurance and Banking? confirmation by EA SM1.5 Has all updated documentation (or knowledge management system) been distributed to the relevant parties confirmation by Service manager SD1.6 SM1.6 Has an aftercare period during which the project is in the lead with regards to incident solving been agreed upon? confirmation by Service manager SD2.1 Have external connections been approved by RCEC? Are they agreed upon and signed off by their respective business owner? RCEC minutes, certificates SM2.1 Is the Operational risk for implementation researched, analysed and mitigated? Minutes TCAB SD2.2 Has the (A-)OSG been completed and approved by SM Security Manager? approved (A-)OSG SD2.3 Has OSS/UAC or its custodian confirmed that the soll matrix is in place and up to date, that User Access Model complies with RBAC, and that User Access Management complies with Authorization Management Process? sign off by SM Risk manager SD2.4 Has Security monitoring been implemented in accordance with the Risk Minimum Standards? sign off by SD Risk manager SD3.1 Is the run book for implementation complete, including roll- back and back-out scenarios? Final runbook attached to change record SD3.2 Have FAT results been accepted by Service Management confirmation by Service Manager 4 Planning SM4.1 Do planned start and end date and the outage times of the Request for Change not conflict with other items on the Change Deployment Calendar? OIB Change Calendar 1 Non functional requirements and deliverables 2 Risk, Continuity and Security 3 Testing Generic Acceptance Criteria 2nd DCAB / Tollgate 3 Solution Delivery Clarity code: 0 Service Management Change nr: 0 Nr. Description Expected Output Remarks/Checkpoints Nr. Description Expected Output Remarks/Checkpoints SD1.1 Have all remaining checkpoints on Tollgate 2 been signed off before the second DCAB meeting SM1.1 Have all CMDB changes to be implemented been approved by the CI owner and the Configuration manager? confimation by CI owner and CFG manager SM1.2 Have all Known Errors with their Work arounds delivered by the project been entered in HPSC? confirmation by Problem manager SD2.1 Has the code review report to check whether Secure Coding Guidelines were followed been delivered? Code review report, for Web facing with ORM sign off SD2.2 If the project requires new or adjusted external connections, has the penetration test been completed successfully? sign off on Penetration test results by ORM SD2.3 Has the Business owner (or his delegate) approved the implementation? Business owner approval (if delegated, incl. proof of delegation) SD3.1 Have all in TCAB identified affected Service Management Teams delivered their UAT results? approvals in HPSC SM3.1 Have the testcases of this release been stored for future reference? testcases in Dimensions SD3.2 Has the runbook been successfully tested on the Acceptance environment, including performing a roll back confirmation by Test manager SM3.2 Has the Regression test been updated in accordance to the changes in functionality of this release? updated Regression test SD3.3 Have all changes to ITSM Tooling (i.e. monitoring, logging etc) been tested and accepted, including the documentation and knowledge management system updates? confirmation by RL4 Operation Management team SD3.4 Have UAT and PAT results been accepted by System Management Confirmation by Service Manager SD4.1 Has ORM confirmed that the open iRisk items will be closed based on the release? confirmation by ORM SM4.1 Has the Problem manager confirmed that the problems to be solved in this release will be closed? (i.e. has Problem Management been involved/consulted during testing?) confirmation by Problem Manager SD4.2 Have updates of Test, Acceptance and Disaster Recovery Environments, synchronising them with the production environment, including all documentation, been planned? confirmation by RL4 Operation Management team SD4.3 Has a DR test with the changed Production and DR Environment been planned? Confirmation by BCM coordinator and central DR coordinator SD4.4 When replacing something old: has decommissioning of the old environments been planned Confirmation by Service Manager 4 Planning 1 Non functional requirements and deliverables 3 Testing 2 Risk, Continuity and Security Tollgate TERROR!
- 10. OPS attitude towards DEV
- 11. DCAB PIRTCABPCAB InitiationStart up Execution Process Close Project closure Requirements / Scope High level design Timelines Design Build Test Implement Scope Risk & Impact Plan B&T Release to Production Closure Development Project Configuration Management Maintenance Configuration Management CM MI ITIL SO/SA PID ITIL Change proces: CMMi-ITIL interface Acceptance criteria Config baseline PER
- 12. DCAB* Tollgate2 PCAB* InitiationStart up Execution Project closure CM MI ITIL Generic Acceptance Criteria & Tollgate 1 Generic & Specific Acceptance Criteria & Tollgate 3 Project Board Project Board Project Board Value Chain Value Chain Value Chain * PCAB = Planning Change Advisory Board; TCAB is Technical Advisory Board; DCAB = Deployment Advisoy Board; CB = Control Board Tollgate1 Tollgate3 TCAB* Project & Service Delivery (Tollgates & CABs) CB* CB*CB* Generic Acceptance Criteria & Tollgate 2 Generic Acceptance Criteria
- 13. Meanwhile
- 14. We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more.
- 15. Process managers in panic
- 16. The new bible
- 17. Dev Ops
- 18. Some results DevOps line org Official start DevOps Start Scrum, end process mngrs
- 19. Some results DevOps line org Official start DevOps Start Scrum, end process mngrs
- 20. Some results Start Scrum, end process mngrs Official start DevOps DevOps line org
- 21. Some results DevOps line org Official start DevOps Start Scrum, end process mngrs
- 22. Combine the feedback loop
- 23. Pro tips • Get your whole value chain on board • You still need processes • Tweak ITIL to meet DevOps demands • All engineers need to add value
- 24. Join your forces, use best of both worlds in order to provide the best service and applications for your customers
Editor's Notes
- MH Hi everyone. We are pleased to tell our story about ‘ITIL and DevOps at war in the enterprise’ at DevOpsdays Amsterdam. Before we do so, let us introduce ourselves.
- MH en JJ
- MH We will tell you about our journey from waterfall development and process mindset towards Agile development, continuous delivery and Devops mindset. How did we experience this journey and how do we manage the clash between ITIL and DEVOPS as part of the wall of confusion. This story is only from the IT perspective at ING The refernce is: DEV is softwarer development and Ops is application maintenance
- MH Before we go on you need to know something about our company… Metaphor for ING: yes, we operate as one organism, but it’s not always clear for all starlings who is in charge. That means that we regularly try to fix a problem from different angles, before we discover we are dealing with the same problem! That also happened with the change process, which was being fixed from Dev side, Ops side, but also in different departments. So we were inventing the wheel several times. We did not know of each other what we were doing or how we could reuse great ideas or how not to reuse poor ideas.
- JJ Where we came from? Our development was doing Waterfall development. Some CMMi implementation, but not everywhere. Some deliver large releases, some deliver small releases Ops was doing ITIL based support. Gravity dictates that water falls down (hence the name Waterfall…), so Operations is always on the receiving end, having a difficult time to dictate non functional requirements or getting problems solved by Development. Third partner in the equation: “the business”. Often felt left ‘in the woods’, not involved in discussions between Dev and Ops.
- JJ As Ops guy you can tell Dev that disasters will happen if you do not deliver this and this, but did they listen? And so you end up with things like this… Management also noticed and gave Ops change management the task to stop this by introducing project aligners and tollgates…
- MH Where did we start? The DEV world was represented by developers and project managers. The “classic” approach of adding value to our products and control people and money. The OPS world was represented by process managers, operators and so called project aligners, the linking pin of OPS to DEV in the change process
- MH Based on ITIL we thought that during the ‘design phase’ we introduce tolgates in order to be sure that our service will be running smooth without any serviceloss for the customers So we created a nice ticklist in three phases to be ticked off in the Design phase, before we deploy to our Acceptance and Production environment with the strict instructions. E.g. Every phase Design known as planning CAB coupled with PID phase on PRINCE II side Deploy acceptance known as Technical CAB Coupled to an execution phase on PRINCE II side Deploy producttion known as Deploymemt CAB Coupled to an execution phase on PRINCE II side
- MH
- MH OPS attitude was not to let pass any offending projects into the next phase. Like insurance companies always reject your first claim Since we want to work together this was not the way to work. So what to do? (next slide with all kinds of nice intergrating models in one overview)
- JJ Just to give you an idea what we used to confuse our colleagues with… We tried map ITIL processes with CMMi to make clear to DEV that we wanted to work with them and not against them.
- JJ And now with new and improved tollgates!
- JJ At the same time some starlings on the other side of the flock thought there should be a better way of doing things. At first we laughed at them, because the first Scrum releases delivered about half the functionality for double the cost. But somehow it grew and the added value was seen by other teams. The concept of Agile/SCRUM was adopted by the organisation. But still not throughout the whole organization.
- MH Adopting the new way of work caused posters like these showing up at coffee machines. Telling the way we are heading in order to provide better software.
- MH When these posters started showing up, the Ops people and especially the process managers, started to worry. With reason, because by the end of 2011 Management decided we didn’t need process managers anymore, so starting first of January 2012 doing without.
- MH Management starting pushing people to buy (and read) this book… and by Q3 2012 the first pilot teams working as this new thing called ‘DevOps’ started to appear. By the end of 2012 it was clear we were going to work as DevOps teams, either as Dev Engineer or Ops Engineer by 1/5/2013. Using Agile/Scrum methods.
- MH Selection for new roles on technical skills and mindset. Reaction: Developer: pretty happy, mostly continued in the same team. And Operations: PANIC! Do we have the right technical skills? What is expected from us? How do we control change? What about ITIL?
- Since the start of DevOps all teams working in sprints which lead to potentially shippable software. Resulting is smaller changes which are done more often. Before the start of DevOps the number of changes decreased because people had to apply for a job in the new organisation, so less work got done in the months preceeding the organisational change.
- The average risk value of the changes also decreases because the changes are less big and scary. (decrease preceeding the organisational change is because big changes got postponed)
- Reducing the number of process managers has no influence on the number of incidents. The organisational changes don’t seem to have an effect either.
- When we put the number of incidents in relation to the number of changes you see a steady decrease over the past 8-12 months. This means that our quality is improving!
- MH Feedbackloop DevOps Feedbackloop ITIL
- JJ + MH Business on board: Funding discussion: change (dev) budget vs baseline (ops) doesn’t help Product owners need to understand that being responsible for a product also means being responsible for the CONTINUITY of that service! You still need processes for incident management, change management (regulators!) Do not make your people think that those don’t matter anymore! Tweaking ITIL: Timelines are not interesting, communication and proper testing are Example change management: move towards just implementation control, that is a check if stuff is production ready. Stop trying to influence a design that is not being made anymore anyway! Example incident management: prio 1 and 2 need to be solved immediately, but it is fine to discuss whether prio 3 and 4 incidents need to be added to the backlog and weighed against new functionality with regards to what adds the most value Do not document thing in more than one place!!! If you manage new functionality in your backlog, that is where you do that, so don’t use a problem module. You still need the skills of problem management to find your root cause and to prepare a solution, but the problem record on its own does not add value Stop doing Service level management if you have a business representative as your Product owner in your team. If there is a need for service reporting, let the PO do it. Adding value is a major mindset change for Ops engineer. It is ok not to solve some incidents! It is not ok to go into production with crappy software!
- You need both sides of the equation to be successful. Make use of both dev’s and ops’s knowledge and experience. Let them complement eachother instead of work against eachother. If you succeed in that you will be succesfull for your clients!