Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Cognizant 20-20 Insights | August 2017
Using DevOps’
Intelligent
Insights to
Deliver Greater
Business Value
By applying re...
THE ROLE OF DEVOPS IN
DIGITAL TRANSFORMATION
CIOs are leading their IT organizations through
ongoing digital transformatio...
Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 3
DEVOPS MEASUREMENT
CHALLENGES
There is an old man...
Cognizant 20-20 Insights
Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 4
Metrics to Measure Veloc...
Cognizant 20-20 Insights
Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 5
DATA COLLECTION AND
MEAS...
Cognizant 20-20 Insights
Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 6
LOOKING FORWARD
Most Dev...
Cognizant 20-20 Insights
Sample Dashboards: DevOps by the Numbers
CI Build Status
Builds Per Month
Average Build Duration
...
Cognizant 20-20 Insights
Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 8
Repeating these steps fo...
Cognizant 20-20 Insights
Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 9
Rajkumar
Chandrasekaran
...
World Headquarters
500 Frank W. Burr Blvd.
Teaneck, NJ 07666 USA
Phone: +1 201 801 0233
Fax: +1 201 801 0243
Toll Free: +1...
Upcoming SlideShare
Loading in …5
×

Using DevOps' Intelligent Insights to Deliver Greater Business Value

261 views

Published on

By applying DevOps - with its real-time analytics - to the software development lifecycle, IT can deliver greater business value in velocity, quality and many other measures.

  • Be the first to comment

  • Be the first to like this

Using DevOps' Intelligent Insights to Deliver Greater Business Value

  1. 1. Cognizant 20-20 Insights | August 2017 Using DevOps’ Intelligent Insights to Deliver Greater Business Value By applying real-time analytics and end-to-end traceability enabled by DevOps, IT organizations not only make software delivery more efficient, but they also enhance business value delivered throughout the digital journey. EXECUTIVE SUMMARY For large enterprises, accelerating their velocity to being digital is becoming increasingly difficult. The process is hindered by historical work silos and the inability to transform older, legacy appli- cations and technology infrastructure. While DevOps1 enables business to address market demand for rapid digitization, the approach primarily focuses on end-to-end automation, from application build and deployment, through release and measurement. This is a great first step, but issues are often uncovered while appli- cations are in production. Some of these issues need immediate resolution, increasing the risk of budget and schedule overruns. Thus, diligent and continuous measurement is necessary to ensure enterprise DevOps success. In a DevOps world, where delivery is agile, rapid and conducted in short Sprints, emphasis should be on proactive issue detection rather than reac- tive responses to bugs and challenges. This white paper discusses how organizations can use rapid feedback mechanisms to attain continuous intel- ligent insights that help continuously improve an application’s value. COGNIZANT 20-20 INSIGHTS
  2. 2. THE ROLE OF DEVOPS IN DIGITAL TRANSFORMATION CIOs are leading their IT organizations through ongoing digital transformation that extends to every aspect of the company. Yet, decision-mak- ers are often at a loss as to where to start and sometimes wonder if they have made enough progress based upon their initial targets.2 Effective digital transformation requires organi- zations to possess the agility to innovate, deploy software and accelerate time to market.3 Organi- zations that are able to quickly turn their ideas into consumer products, absorb rapid feedback and then quickly refine their software to launch the suggested improvements are excelling in a world that is moving at the speed of digital.4 DevOps gives these organizations this kind of ability to deliver new digital ideas quickly. Orga- nizations adopt the DevOps philosophy to bring sustained continuous development, integration and innovation. MEASURING DEVOPS FOR DIGITAL SUCCESS However, the greatest challenge today is that continuous software updates are not so simple to deploy. Multiple apps must be rendered on mobile and web interfaces, especially in indus- tries where customers need consistent anytime, anywhere access to information and application services. Any downtime, however minor, can be extremely costly. Despite pre-launch testing, technical issues still arise in production, some of which could be severe enough to warrant immediate remedies. These can totally disrupt development schedules and, more important, mar the business’s reputa- tion and cause financial hardship, if not remedied quickly and effectively. And often, the repercus- sions of such glitches depend on how quickly an issue is discovered, diagnosed and fixed. Issue detection needs to be immediate and automatic, and the information captured must facilitate remedial action. In addition to remediating production issues, there is also a need to continuously improve appli- cation services for the customers at a greater velocity, to fast-track their digital transforma- tion. But the velocity should not come at a cost of quality, and this is one prime area where DevOps can help. CIOs who comprehend DevOps’s value should also understand that DevOps is a journey and not an end state, and thus they should use this approach to continuously evolve and provide better application services to their customers. Next-gen analytics tools and cognitive technol- ogies such as autonomics have opened up the possibility to add adaptive automation within DevOps’ continuous delivery (CD) implementa- tions. This facilitates not only identifying issues but also enables the systems to take mindful decisions in solving some of the most common problems in the software delivery lifecycle. As a result, the velocity of applications changes released to production increases significantly and the customer’s digital journey accelerates. CIOs who comprehend DevOps’s value should also understand that DevOps is a journey and not an end state, and thus they should use this approach to continuously evolve and provide better application services to their customers. Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 2 Cognizant 20-20 Insights
  3. 3. Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 3 DEVOPS MEASUREMENT CHALLENGES There is an old management saying: If you can’t measure it, you can’t improve it. DevOps is not a one-time activity, but an ongoing process where IT organizations improve the various activities across the systems development lifecycle (SDLC) of build, deploy, verify, release and measure. In this way, they work to significantly accelerate time to market and business value. Organizations that are unable to baseline DevOps outcomes are unable to continuously improve; in the long run, they could potentially become laggards.5 This simple thought drives DevOps analytics. Unfortunately, there is no single standard by which we can define or baseline DevOps indices that need to be measured. This is primarily due to the fact that not all parameters that are mea- sured may be applicable to all technology stacks — even if the concept of DevOps may be still appli- cable to them. Therefore, if an enterprise wishes to measure its DevOps journey, it would have to first create baseline measurements on how much time is being spent inside SDLC processes and how DevOps can help reduce that. If these indices show progress once an enterprise adopts DevOps, it could justify the investment. But these measurements and computations of progress are made over a period of time and are not simple to measure due to the following: • The indices change based on the customer and what the customer wants to measure. • The indices change based on the type of the vector that needs to be measured: build, deploy, verify, release, measure and environ- ment (in some cases). • There is no one single tool or mechanism that helps to measure DevOps progress. DEFINING THE METRICS THAT MATTER An obvious first step is to identify the basic per- formance indicators to measure — i.e., define the KPIs. Based on our experience, the metrics in Figures 1 and 2 (see next page) are good starting points. One caveat: The figures offer an indica- tive view of the various performance indicators; they are not intended to be an exhaustive cata- log. These could be used for creating baselines, and the trends, either upward or downward (based upon the metrics), can help reveal if the needle is tipping in the right direction. Organizations that are unable to baseline DevOps outcomes are unable to continuously improve; in the long run, they could potentially become laggards. Cognizant 20-20 Insights
  4. 4. Cognizant 20-20 Insights Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 4 Metrics to Measure Velocity Performance Indicator What It Means Mean Lead Time Time taken for a bit of code to get built, tested and deployed. Daily Change Rate Number of changes committed to the main branch of source control and deployed to the necessary environments. Mean Time to Environment Setup The time taken for developers/testers to create a testing environment for verifying each delivered change. Mean Time to Detect Time elapsed since the original coding until the bug it introduces is detected. Mean Time to Resolve Time taken to resolve an issue after detection. Mean Time to Approve Time taken to approve and verify a release. (This is measured from the moment all release content has been delivered until the release has passed all the defined test and verification cycles.) Figure 1 Metrics to Measure Quality Performance Indicator What It Means Build Failure Rate Ratio of failed builds to overall builds. Deployment Failure Rate Ratio of failed deployments to overall deployments. Infrastructure-Related Failure Rate Ratio of build/deployment failures related to infrastructure issues. Rework Rate Ratio of tickets being reopened to total tickets. Automated Detection Rate Ratio of defects detected by automated testing cycles to overall detects. Figure 2 Velocity Metrics These metrics (in Figure 1) essentially measure the rate at which change is being delivered. And “change” could be how quickly a problem is detected, how quickly it can be fixed, or how quickly a basic need can be assigned to an engi- neering team. Quality Metrics Apart from agility, quality is of paramount impor- tance in the DevOps world. The quality metrics listed in Figure 2 help to define the rate at which quality factors can be measured.
  5. 5. Cognizant 20-20 Insights Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 5 DATA COLLECTION AND MEASUREMENT Once key performance metrics are identified, the next step is to gather the data to make these measurements. After collecting data for the indi- vidual metric, it is necessary to plot each data point to derive meaningful insights. Some of the data can be used directly to derive meaning, while other data might require complex calcu- lations to quantify the value or the outcome. While the numerical value of an isolated data point may be relevant, generally the real impact of such data is realized when the decision-maker conducts trend analysis over a period of time. This can reveal progress — or the lack thereof. Given the complexity of data collection, analysis and interpretation, it is obviously suboptimal to perform perform such processes manually. The principles of DevOps need to be applied to mea- suring outcomes, as well. In Figure 3, we present a sample setup for data collection to create a DevOps measurement baseline. This figure illustrates at a high level how data can be collected, and the top characteristics of such a system. The system basically contains touch points (integration points) with the necessary tools that are deployed to collect data and then portray the full picture of how the application code moves through the various phases. Creating Measurement Baselines Build Provide capability to correlate between data Should be able to work with all the tools Figure out bottlenecks Simple and effective ways to collect data Simple setup Deploy Release Verify Environment DevOps Measurement Platform SCM (GIT) Artifact Repo (Nexus) Artifact Repo (Nexus) Functional and Regression Automation (Selenium) Continuous IntegrationSCM Artifact Mgmt Release Mgmt Continuous Verification Env. Mgmt Continuous Deployment Code checkout Push artifact Pull artifact Pass Fail DEV ST UAT P&V Production Static Code Analysis (SONAR) Build (Maven) Test (Junit) Measure Coverage (Cobertura) Deploy Application (XL Deploy) Automated Smoke Test (Selenium) Rollback Deployment (XL Deploy) Jenkins Inadequate coverage Test failures Major Code Violations Build failures Figure 3
  6. 6. Cognizant 20-20 Insights Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 6 LOOKING FORWARD Most DevOps consultants and architects are challenged to address whether DevOps is more concerned with IT than business outcomes. We believe that this is one of the most challenging questions IT organizations face. Answering this with data-driven measurements and metrics (as described above) is therefore critical. This is the primary goal of large transformation engage- ments wherein DevOps is the key change lever. Quick Take Improving Time to Market, Productivity and Incidence of Bugs A large enterprise client sought to apply DevOps to improve application time to market. Initially, the company’s IT organization was unable to figure out where the bottlenecks were in its SDLC process, which affected devel- oper productivity. By applying key principles of DevOps to identify bottlenecks in its develop- ment process, the company greatly reduced cycle time from development to production release. This also significantly reduced defects, because the IT team was able to more proactively find and treat root cause issues. Overall, the company’s IT organization increased developer productivity by 15%, an outcome measured by its own analytics and correlation engine.
  7. 7. Cognizant 20-20 Insights Sample Dashboards: DevOps by the Numbers CI Build Status Builds Per Month Average Build Duration Jenkins Success Count Jenkins Failure Count Build Duration COUNTTIME(MINUTES) APR 2016 DEC 2016 JUL 2016 JUN 2016 MAR 2016 MAY 2016 NOV 2016 APR 2016 DEC 2016 JUL 2016 JUN 2016 MAR 2016 MAY 2016 NOV 2016 10.0 7.5 5.0 2.5 0.0 10.0 7.5 5.0 2.5 0.0 User Story SCM Commits Files Commited 33 74 502 Figure 4 When business teams are able to realize these outcomes, they will also be part of the trans- formation journey — and hence DevOps will no longer be an “IT thing.” Sample dashboards that can be created based on the identified metrics are illustrated in Figure 4. These provide a sense of what exactly needs to be improved and what can be accomplished via DevOps. By examining the figure’s three reports, DevOps teams can see how a particular Sprint impacted the number of files that changed but didn’t improve the quality of the CI builds and the deployments, given the large number of failures. This basically shows that the team has good velocity but must improve on their quality mea- sures. Inferences like these will help the teams improve in their DevOps journey. DevOps measurement enables organizations to answer the age-old return on investment (ROI) question. The key first step to ensure that DevOps leads to successful digital transformation is to define and measure DevOps metrics. The fol- lowing are the priority items that organizations should bear in mind when defining the metrics: • Identify what needs to be measured. • Which actors contribute to the KPI – these could be tools, process and technology, not just people – or collect data for metrics? • Measurement of the metrics with a defined baseline. • Progress tracking metrics to ensure contin- ued forward motion in adoptign of DevOps principles. Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 7
  8. 8. Cognizant 20-20 Insights Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 8 Repeating these steps for each KPI, and then taking the actual measurements, will allow an organization to objectively measure DevOps’ outcomes and weed out the bottlenecks — and thereby accelerate the organization’s digital jour- ney. Figure 5 presents some additional outcomes that can be achieved from both a business and technical standpoint when the above measure- ments are put in place. Business Benefits Outcome Remarks Improve Team Efficiency By identifying — and then acting on — factors that limit the team’s velocity, the team can deliver changes at a faster pace. Portfolio Performance This helps in measuring the relative maturity of a given portfolio. For the business, it serves as a measuring point for determining which areas need focus given a particular line of business. Time to Market & Trend Time taken for a business feature to be rolled out from build to deployment, and the trend over a particular time frame. This helps the business to make corrective steps and to deliver products more rapidly. Cost of Release The ability to measure the cost vs. value delivered by a release. In an ideal scenario, the cost of the release should taper once DevOps principles are adopted — thus justifying the investment in this approach. Impact of Release The success/failure of features delivered to customers. This measurement gives the business the capability to gauge positive/negative ROI and help make feature delivery more customer-centric. High Friction Zones This helps to identify areas that choke application development velocity. They could be in development, QA or infrastructure; knowing where helps teams take corrective measures. Technical Debt Costs incurred for not following the best code practices, and the related expenses to fix them. This denotes that when headroom projects are not executed in a timely way it impacts the velocity of the delivered code changes. Velocity and Trend This is a direct measure of the team’s capability to deliver changes at a constant speed. The team should be able to increase its velocity of delivering features when the bottlenecks are removed by the application of DevOps principles. Defect Fix & Rate Average time to fix defects and determine the trend. When defects of a particular type proliferate, it indicates a choke point in that area. For example, if environment defects are high, then the environment area needs some attention. This should decrease when DevOps principles are applied in the environment. Figure 5
  9. 9. Cognizant 20-20 Insights Using DevOps’ Intelligent Insights to Deliver Greater Business Value | 9 Rajkumar Chandrasekaran Principal Architect, DevOps Practice Rajkumar Chandrasekaran is a Principal Architect within Cogni- zant’s DevOps Practice. He has 15-plus years of experience in the field of large-scale application development and has played varied roles, from application architecting through reengineering of appli- cations. Rajkumar is currently architecting Cognizant’s OneDevOps platform assets for the company’s DevOps offerings. He can be reached at Rajkumar.Chandrasekaran@cognizant.com. ABOUT THE AUTHORS FOOTNOTES 1 DevOps is a term used to refer to a set of practices that emphasize the collaboration and communication of both software developers and information technology professionals while automating the process of software delivery and infrastructure changes – from Wikipedia. en.wikipedia.org/wiki/DevOps. 2 “Drive Results with Digital Transformation Initiatives: Three Keys for CIOs to Ensure Superior Customer Experience,” by Jennifer Thomson, Carla Arend, Archana Venkatraman; IDC white paper, October 2015. 3 “Digital Transformation and DevOps – A Winning Combination,” DevOpsGuys, www.devopsguys.com/2016/11/03/digital-transformation-devops-winning-combination/. 4 “DevOps: Unlocking the Value from Digital Transformation — A CxO’s Guide,” DevOpsGuys white paper, October 2016. 5 “CA Technologies: DevOps leaders outshine DevOps laggards in the App Economy,” Jessica Twentyman, Oct. 14, 2014. www.itproportal.com/2014/10/14/ca-technologies-devops-leaders-outshine-devops-laggards-app-economy/. Rangarajan Rajamani Senior Manager, DevOps Practice Rangarajan Rajamani is a Senior Manager within Cognizant’s DevOps Practice. He has over 14 years of expertise in multiple indus- tries such as global life sciences, manufacturing and construction, performing business consulting, pre-sales and product evangeli- zation. As a functional expert in digital technologies such as the IoT, cloud, service virtualization, functional QA automation and DevOps, Rangarajan has created point solutions for organizations in life sciences, insurance and retail. He currently anchors platform evangelization for Cognizant’s DevOps Practice. Rangarajan can be reached at Rangarajan.Rajamani@cognizant.com.
  10. 10. World Headquarters 500 Frank W. Burr Blvd. Teaneck, NJ 07666 USA Phone: +1 201 801 0233 Fax: +1 201 801 0243 Toll Free: +1 888 937 3277 European Headquarters 1 Kingdom Street Paddington Central London W2 6BD England Phone: +44 (0) 20 7297 7600 Fax: +44 (0) 20 7121 0102 India Operations Headquarters #5/535 Old Mahabalipuram Road Okkiyam Pettai, Thoraipakkam Chennai, 600 096 India Phone: +91 (0) 44 4209 6000 Fax: +91 (0) 44 4209 6060 © Copyright 2017, Cognizant. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means,electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Cognizant. The information contained herein is subject to change without notice. All other trademarks mentioned herein are the property of their respective owners. TL Codex 2796 ABOUT COGNIZANT Cognizant (NASDAQ-100: CTSH) is one of the world’s leading professionalservices companies, transforming clients’ business, operating and technology models for the digital era. Our unique industry-based, consultative approach helps clients envision, build and run more innova- tive and efficient businesses. Headquartered in the U.S., Cognizant is ranked 205 on the Fortune 500 and is consistently listed among the most admired companies in the world. Learn how Cognizant helps clients lead with digital at www.cognizant.com or follow us @Cognizant.

×