Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Data on DevOps: Making the Case for Awesome

2,110 views

Published on

What’s the value proposition of DevOps? Does culture change show up in the bottom line? What practices predict high IT performance? We hear many stories to inspire and inform us, but the plural of anecdote is not data. Let’s dive into the research and find out which DevOps practices drive optimal IT and business outcomes.

The data shows that the best IT performers have the highest throughput and reliability while contributing to organizational profitability, productivity, and market share goals. Industry trends around security, containers, continuous delivery, and lean management relate to IT performance and quality: let’s talk about how.

Management and practitioners alike will leave with a better understanding of how to achieve the best outcomes, while armed with the data they need to make the case for change.

Published in: Technology

The Data on DevOps: Making the Case for Awesome

  1. 1. Nicole Forsgren, PhD Director of Organizational Performance & Analytics, Chef Research Affiliate, Clemson University The Data on DevOps: Making the Case for Awesome
  2. 2. “IT doesn’t matter.” -- Nicholas Carr, 2003 @nicolefv
  3. 3. IT Does Matter • Times – and IT – have changed • DevOps is good for Organizations • DevOps is good for IT • And then some detail: What drives this change? • Technical practices (hint: Continuous Delivery) • Management practices (hint: from Lean Management) • Culture and identity @nicolefv
  4. 4. @nicolefv
  5. 5. @nicolefv
  6. 6. DevOps is Technical practices seen in Continuous Delivery, Management practices seen in Lean Management principles, and Organizational Culture and Identity @nicolefv Research shows that these drive IT Performance and Organizational Performance
  7. 7. DevOps is good for organizations @nicolefv
  8. 8. High Performing IT organizations 2x More likely to exceed Profitability, Market share, and Productivity goals 50% Higher market cap growth over 3 years* The 2014 and 2015 State of DevOps Report @nicolefv
  9. 9. Devops is good for IT Measuring DevOps and IT Performance - Deploy frequency (Note: NOT delivery) - Lead Time for Changes - Mean Time to Recover (MTTR) - Change Fail Rate @nicolefv
  10. 10. High Performing DevOps teams More agile The 2016 State of DevOps Report @nicolefv
  11. 11. High Performing DevOps teams More reliable The 2016 State of DevOps Report @nicolefv
  12. 12. IT Performance over the years The 2016 State of DevOps Report @nicolefv Deploy Frequency Change LeadTime MeanTime to Recover
  13. 13. DevOps promises – and delivers More throughput More stability In tandem. Without the tradeoffs that ITIL calls for. Let’s talk about what this means for us @nicolefv
  14. 14. High Performing DevOps teams More agile What does this mean for: 200x More frequent deployments The 2015 DevOps Survey of Practice and its resulting database are the property of Puppet Labs, Inc. and Gene Kim and Associates, LLC. Allrights reserved. New content delivery Value/savings around A/B testing Value around speed to market Compliance / regulatory Security2,555x Faster lead times @nicolefv
  15. 15. Evaluating well-designed and executed experiments that were designed to improve a key metric, only about 1/3 were successful at improving the key metric! Online Experimentation at Microsoft, Kohavi et al http://stanford.io/130uW6X @nicolefv
  16. 16. High Performing DevOps teams More reliable What does this mean for: 3x Fewer deploy failures The 2016 State of DevOps Report Value/savings around reliability Value/savings around uptime Compliance Security Reputation around uptime, compliance & security 24x Faster MTTR @nicolefv
  17. 17. @nicolefv
  18. 18. "Fixing it as soon as possible or having compensating controls in place days before could have saved this entire breach from occurring in the first place."
  19. 19. We know: • IT Performance is comprised of throughput and stability, and both are possible without tradeoffs • IT Performance contributes to org performance ($$$) So: What drives IT and Organizational Performance? @nicolefv
  20. 20. IT Performance Org Performance??? @nicolefv
  21. 21. IT Performance Org Performance??? 1. @nicolefv
  22. 22. IT Performance Org Performance??? 2. 1. @nicolefv
  23. 23. IT Performance Org Performance??? 2. $$$1. @nicolefv
  24. 24. Continuous Delivery makes our work better… The 2016 State of DevOps Report @nicolefv
  25. 25. “We never had testability before. We have it now. We have this experience and know this stuff is working, and working with controls.” – Product Owner for Yahoo Chef implementation Automated configuration and deployment of 250,000 nodes Can deploy up to 140k node configurations in 8 hours. Can patch entire infrastructure within 6 hours of a patch being made available @nicolefv
  26. 26. Continuous Delivery makes our work better… and makes us feel better! @nicolefv https://devops-research.com Forsgren, N., J. Humble (2016). "The Role of Continuous Delivery in IT and Organizational Performance." In the Proceedings of the Western Decision Sciences Institute (WDSI) 2016, Las Vegas, NV. Available at SSRN: http://ssrn.com/abstract=2681909
  27. 27. Microsoft Engineering: DevOps Lessons Thiago Almeida -- DevOps Days London, 2016 @nicolefv Work/Life Scores Before CD: 38% After CD: 75% Source: https://vimeo.com/165184757
  28. 28. Continuous Delivery makes our work better… The 2016 State of DevOps Report @nicolefv
  29. 29. But what else drives IT Performance? The 2015 DevOps Survey of Practice and its resulting database are the property of Puppet Labs, Inc. and Gene Kim and Associates, LLC. Allrights reserved. @nicolefv
  30. 30. Lean Management practices The 2015 State of DevOps Report @nicolefv
  31. 31. “If it moves, graph it.” - Michael Rembetsy, Vice President Operations, Etsy @nicolefv
  32. 32. “I was trying to figure out why my team was working themselves to death but not getting anything done… By implementing WIP limits, we were able to focus on our work. Finishing work feels better than sprinting and feeling like a hero in the moment, because that’s only a moment.” - Julia Wester, Development Manager for Turner Sports, Turner Broadcasting @nicolefv
  33. 33. Lean Product Management practices The 2016 State of DevOps Report @nicolefv
  34. 34. Organizational Culture @nicolefv Pathological Power-oriented Bureaucratic Rule-oriented Generative Performance-oriented Low cooperation Modest cooperation High cooperation Messengers shot Messengers neglected Messengers trained Responsibilities shirked Narrow responsibilities Risks are shared Bridging discouraged Bridging tolerated Bridging encouraged Failure leads to scapegoating Failure leads to justice Failure leads to inquiry Novelty crushed Novelty leads to problems Novelty implemented
  35. 35. • I am glad I chose to work for this organization rather than another company. • I talk of this organization to my friends as a great company to work for. • I am willing to put in a great deal of effort beyond what is normally expected to help my organization to be successful. • I find that my values and my organization's values are very similar. • In general, the people employed by my organization are working toward the same goal. • I feel that my organization cares about me. Adapted from adapted from Atreyi Kankanhalli, Bernard C.Y. Tan, and Kwok-Kee Wei (2005), “Contributing Knowledge to Electronic Knowledge Repositories: An Empirical Investigation,“ MIS Quarterly, 29, 113-143. Identity @nicolefv
  36. 36. Google Team Performance Surprise! No magical formula for what makes the perfect team
  37. 37. Intuit “By installing a rampant innovationculture, we performed 165 experiments in the peak three months of tax season. Our business result? Conversion rate of the website is up 50%. Employee result? Everyone loves it, because their new ideas can make it to market. ” - Scott Cook, Intuit founder @nicolefv
  38. 38. Amazon “I think buildingthis culture is the key to innovation. Creativity must flow from everywhere. Whether you are a summer intern or the CTO, any good idea must be able to seek an objective test, preferably a test that exposes the idea to real customers. Everyone must be able to experiment, learn, and iterate.” - Greg Linden @nicolefv
  39. 39. IT Does Matter • Times – and IT – have changed • DevOps is good for Organizations • DevOps is good for IT • And then some detail: What drives this change? • Technical practices (hint: Continuous Delivery) • Management practices (hint: from Lean Management) • Culture and identity @nicolefv
  40. 40. For more information: @nicolefv Sign up for our ROI whitepaper & get peer- reviewed research devops-research.com
  41. 41. nicole@chef.io @nicolefv nicolefv.com Thank you
  42. 42. Appendix
  43. 43. 10 deploys per day Dev & ops cooperation at Flickr John Allspaw & Paul Hammond Velocity 2009 That was then… @nicolefv
  44. 44. Amazon Deployment Stats (production & host environments only) This is now… 1,079Max deploys In a single hour Every 11.6 seconds! 10,000Mean # hosts receiving Deploys simultaneously 30,000Max # hosts receiving Deploys simultaneously @nicolefv
  45. 45. Etsy Code Deployment What once required 6-14 hours and an “Army” …Now takes 15 minutes and 1 person This is now… 2013 Mike Brittain,Continuous Deployment:The Dirty Details 3/2014 Daniel Schauenberg ,Qcon London 4/2014 tweet@philkates 30+ Deploys per day 2013 50 Deploys per day March 2014 QCon London 80-90 Deploys per day April 2014 Chef Conf @nicolefv

×