This document discusses lessons learned from enterprise DevOps implementations. It begins with an overview comparing DevOps to "enterprise DevOps" which involves more complex environments requiring a different balance of automation, visibility and control. It then outlines 5 key lessons: 1) DevOps initiatives require a balance of top-down and bottom-up approaches, 2) cross-cutting concerns like security and compliance must be addressed, 3) standardization is important but too much can stifle innovation, 4) DevOps involves expanding beyond development and operations to include other groups like QA, and 5) organizations tend to focus internally on automation versus externally on customer impact. Examples are provided for how these lessons can be applied to reduce handover time, speed
2. 2
About me
▪ VP DevOps Strategy for XebiaLabs
▪ Been on both sides of the “Dev…Ops” fence
▪ Lots of enterprise software development on
high-performance systems
▪ Active open source contributor and
committer
▪ Regular meetup, conference etc. presenter
Andrew Phillips
17. 17
5 Lessons From Enterprise DevOps
1. Top-down vs. bottom-up
2. Cross-cutting concerns
18. 18
5 Lessons From Enterprise DevOps
1. Top-down vs. bottom-up
2. Cross-cutting concerns
3. Standardization
19. 19
5 Lessons From Enterprise DevOps
1. Top-down vs. bottom-up
2. Cross-cutting concerns
3. Standardization
4. Dev(.+)Ops
20. 20
5 Lessons From Enterprise DevOps
1. Top-down vs. bottom-up
2. Cross-cutting concerns
3. Standardization
4. Dev(.+)Ops
5. Inward vs. outward
21. 21
5 Lessons From Enterprise DevOps
1. Top-down vs. bottom-up
2. Cross-cutting concerns
3. Standardization
4. Dev(.+)Ops
5. Inward vs. outward
6. Means, not goals
22. 22
VISIBILITY
AUTOMATION
CONTROL
Continuous Delivery and
DevOps pioneer, authority
and technology leader
Global team in US,
Europe & APAC
Consistently recognized by
leading industry analysts
Connecting the dots for
Continuous Delivery
at enterprise scale
About XebiaLabs
G l o b a l L e a d e r s D e l i v e r S o f t w a r e w i t h X e b i a L a b s
23. 23
V I S I B I L I T Y
C O N T R O L
AUTOMATION
Award-winning tools,
recognized by leading industry analysts
XebiaLabs Solutions: Connecting the dots for
Continuous Delivery at enterprise scale
45. 45
Inward vs. outward
Two distinct types of DevOps/Agile/CD/etc. organizations
▪ Type 1:
− Automation, automation, automation
46. 46
Inward vs. outward
Two distinct types of DevOps/Agile/CD/etc. organizations
▪ Type 1:
− Automation, automation, automation
▪ Type 2:
− Agile, continuous delivery & small changes
− product teams & experimental organizations
− cultural responsibility shift
49. 49
Inward vs. outward
▪ Most enterprise success stories so far
are type 1
▪ Most of the “unicorn” stories are type 2
▪ Moving to type 2 is significantly harder
than implementing type 1
▪ Requires far more wide-reaching
changes: org structure, company culture
etc.
50. 50
Inward vs. outward
• Waterfall agile: 3 years
• 220 Apps - 1 deployment per month
• “EVERY manual tester does automation”
• “We don’t log bugs. We fix them.”
• Measures are built in & visible to everyone
• Promote your wins! Educate your peers.
• EVERYONE can do continuous delivery.
51. 51
Inward vs. outward
700 deployments / year
10 + deployments / day
50 – 60 deployments / day
Every 11.6 seconds
52. 52
Means, not goals
"I need me some DevOps"
▪ CIO of major organization: “What is DevOps and what do I need
to do about it? I’m concerned about missing the boat, but have
no idea what DevOps means for my organization”
▪ Lack of clear definition and overlapping marketing messages
create confusion
53. 53
Means, not goals
▪ DevOps is a means, not a goal
▪ There is no “standard DevOps template” that to apply
▪ There are common elements to many of the DevOps success
stories that we can learn from
54. 54
Means, not goals
▪ DevOps is a means, not a goal
▪ There is no “standard DevOps template” that to apply
▪ There are common elements to many of the DevOps success
stories that we can learn from
56. 56
Metrics, metrics, metrics
▪ “Concept-to-cash” time
▪ “Commit-to-cash” time
▪ # production deployments/time
▪ # production deployments rolled back/time
▪ MTTR
▪ Business value/time
▪ Developer feedback time
▪ Handover time during release
▪ Time spent providing audit data
57. 57
Metrics, metrics, metrics
▪ “Concept-to-cash” time
▪ “Commit-to-cash” time
▪ # production deployments/time
▪ # production deployments rolled back/time
▪ MTTR
▪ Business value/time
▪ Developer feedback time
▪ Handover time during release
▪ Time spent providing audit data
58. 58
Metrics, metrics, metrics
▪ “Concept-to-cash” time
▪ “Commit-to-cash” time
▪ # production deployments/time
▪ # production deployments rolled back/time
▪ MTTR
▪ Business value/time
▪ Developer feedback time
▪ Handover time during release
▪ Time spent providing audit data
59. 59
Metrics, metrics, metrics
▪ “Concept-to-cash” time
▪ “Commit-to-cash” time
▪ # production deployments/time
▪ # production deployments rolled back/time
▪ MTTR
▪ Business value/time
▪ Developer feedback time
▪ Handover time during release
▪ Time spent providing audit data
60. 60
What our users do…
Standardization
Mobile app
Web frontend
Mainframe change
68. 68
Example: Concept to cash time
▪ Code analysis & inspection
− Building a dependency graph
▪ Increase in test automation
− Mitigate risk of regression
▪ Shadow mode operation
▪ Rank by value
− Determine priority for investment
70. 70
Example: Time to provide audit data
▪ Changed communication process and tooling
− No more requests for deployment by email
▪ Added regular data exports into a centralized audit database
− Had to change tooling to make that possible
▪ Added custom logging to automation tooling
− To allow for correlation of data
▪ Training & internal info material
− Explaining the requirements for audit data to the teams
72. 72
A quick takeaway…
▪ “I am not a crusader for open source, I am a crusader for you
building the best software possible, you shipping the best
software to your customer possible, you having an awesome
software development team.”
▪ “It’s not about open or closed source, it’s about what’s best for
the end user, […] it’s about what’s the best for your business,
your developers, your customers right now.”
73. 73
A quick takeaway…
▪ “I am not a crusader for open source, I am a crusader for you
building the best software possible, you shipping the best
software to your customer possible, you having an awesome
software development team.”
▪ “It’s not about open or closed source, it’s about what’s best for
the end user, […] it’s about what’s the best for your business,
your developers, your customers right now”
Chris Wanstrath, CEO at GitHub
https://a16z.com/2016/01/06/a16z-podcast-what-software-
developers-and-therefore-every-company-need-2/
75. 75
Resources
▪ Get Started with XebiaLabs
www.xebialabs.com
www.xebialabs.com/products
blog.xebialabs.com
@xebialabs
youtube.com/xebialabs
▪ The Periodic Table of DevOps
https://xebialabs.com/periodic-table-of-
devops-tools/
▪ eBook: The IT Manager’s Guide to CD
https://xebialabs.com/resources/
whitepapers/the-it-managers-guide-to-
continuous-delivery/
“Making DevOps compatible with enterprise security, compliance etc. requirements without killing innovation and imposing top-down control”
Skeleton:
Tooling such as we’ve heard about today gives teams a lot of flexibility to accelerate and to innovate
This encourages bottom-up adoption, which empowers teams and is a great driver
In an enterprise environment, however, there are some constants that most bottom-up teams tend not to focus on: reporting, security, audit etc.
Also, if we want to bring the benefits of DevOps to our entire organization, the story needs to go beyond the new build Tiger Teams and embrace a wide spectrum of tools and teams with different levels of skill and ability to make significant changes to their application or tech stack
Further, conversation recently has justifiably focused on the need to bring more than just “Dev” and “Ops” to the table – witness “DevSecOps” or “BizDevOps” conversations
This leads to the following observations and requirements
We need tooling and processes that are flexible enough to accommodate different levels of technical skill, different technologies and different maturity levels
We need to provide an on-ramp that doesn’t require us to boil the ocean
We need to meet the needs of project/programme management, change control, security etc. – not forever, but at least until we can demonstrate that many of the requirements we put in place can be re-thought in a well-functioning DevOps setup
We need to do that without imposing a rigid, “boring”, top-down enterprise tech stack and process…
…but also without requiring every team to re-discover the DevOps wheel in a slightly different way
We need top-down support and “air cover” to be able to handle the obstacles that we can’t “Tiger Team our way around”
Here’s some examples of how our large enterprise users are trying to put this into practice
Highlights from 2015
* DevOps for the rest of us
* compelling stats from real-world organizations
* Growing consulting practice
* "I need me some DevOps"
What have we learned:
* about what practical DevOps is
"Step 1"
* internal IT initiative
* automation, automation, automation
"Step 2"
* Agile, continuous delivery & small changes
* product teams & experimental organizations
* cultural responsibility shift
* about how to implement it
* top-down engagement, bottom-up growth
* automation and coding fundamentals are key for New Ops
* measure things and make them visible
* incremental approach - manager as problem solver
* expect failures and pushback
* spreading knowledge is key
* DOs and DON'Ts
* DO invest in automation training for your Ops team and identify if your staff there have the mindset for change
* DO introduce visibility, communication, training/education and potentially even reward/recognition structures that emphasize the end-to-end picture for the product/project, rather than a silo-based view
* DO define your *own* goals and interpretation of what DevOps means for your organization, and communicate that clearly
* DON'T rely on a bottom-up approach for enterprise-wide adoption
* DON'T lose sight of the fact that DevOps and CD practices are *means to an end* rather than *ends in themselves*
* DON'T forget to include accelerated/automated quality verification in your delivery pipeline
* Open Questions
* role of the business?
* outsourcing?
* what about the silos?
* when platform, when shared?
For 2016
* ensure you have a central group with executive support in place - not to *control* the DevOps initiative, but to *support* it, provide expertise, gather data and facilitate adoption
* figure out whether and where your organization is suited to "Phase 1" or "Phase 2" Devops: are we suited to the delivery of small changes? can we embrace an experimental approach to innovation?
* decide how you want your new DevOps delivery model to interact with your existing "framework" processes of programme/portfolio mgmt, project/backlog management, change/release management etc.
* keep an eye out for containers but resist hasty hype-driven decisions: what is the use case vs. our current cloud plan? Are we going to microservices? Can we handle the immaturity of the space?
* interested in a structured approach? contact us! Don't "go from Dinosaur to DINOsaur"