This document discusses various aspects of DevOps practices and culture. It provides definitions of DevOps, discusses how organizations have implemented DevOps to significantly improve performance metrics like deployment frequency and lead times. It also addresses challenges like change approval boards, security and compliance, adopting an agile mindset, and shifting to see work as developing products rather than projects to better manage risk. Throughout, it emphasizes automating processes, reducing waste, collaborating across teams, and focusing on business outcomes over documentation and processes.

1. excella.com | @excellaco
Doing DevOps vs.
Getting Results
April 29, 2020

2. excella.com | @excellaco
June 23, 2009.

3. excella.com | @excellaco
What is DevOps?

4. excella.com | @excellaco
Everyone had their own definition of DevOps.

5. excella.com | @excellaco
Culture.
Automation.
Lean.
Measurement.
Sharing.

6. excella.com | @excellaco
How should we do DevOps?

7. excella.com | @excellaco
Toolchains. Team structure. Processes. Policies.

8. excella.com | @excellaco
How can I improve performance?

9. excella.com | @excellaco
208x more frequent deploys.
106x faster lead times.
2,604x faster MTTR.
7x lower change failure rate.

10. excella.com | @excellaco
U.S. Citizenship
and Immigration
Services

11. excella.com | @excellaco
Federal agency.
9 two-pizza teams.
Lead time for changes under 60 minutes.
Up to 40 deploys per day.
MTTR less than 3 minutes.
The Art of the
Possible.

12. excella.com | @excellaco
Change how we think about work.

13. excella.com | @excellaco
① Change Approval Boards.

14. excella.com | @excellaco
Traditional Change Approval Boards/Change Control Boards
hurt software delivery performance.

15. excella.com | @excellaco
“We found that external approvals were
negatively correlated with lead time,
deployment frequency, and restore time,
and had no correlation with change fail
rate. In short, approval by an external
body (such as a manager or CAB)
simply doesn’t work to increase the
stability of production systems,
measured by the time to restore service
and change fail rate. However, it
certainly slows things down. It is, in fact,
worse than having no change approval
process at all.”

16. excella.com | @excellaco
1. Reduce dependencies (and complexity) in your architecture.
2. Use heavy doses of automation.
3. Improve monitoring and observability.
4. Institute effective peer reviews.
Instead of the CAB.

17. excella.com | @excellaco
Preserve emphasis on finding
issues before going to
production.
Increase emphasis on reducing
impact of finding issues after
going to production.

18. excella.com | @excellaco
1. CI/CD pipeline.
2. Blue/Green deployments.
3. Canary deployments.
4. Feature toggles.
5. Architecture patterns like circuit breaker and bulkhead.
6. Event-driven architecture.
Reducing impact in production.

19. excella.com | @excellaco
② Security, compliance, and audit.

20. excella.com | @excellaco
Using security, compliance, and audit controls dependent on
documentation and human review is “risk management
theater”.

21. excella.com | @excellaco
Typical result of the process…
“You can’t release until you fix
these.”

22. excella.com | @excellaco
Lengthens cycle times and
feedback loops.
Creates waste and inefficiency.
Involves decision-makers distant
from the actual work.
Can be circumvented.
Hurts situational awareness.
Counter-
productive
.

23. excella.com | @excellaco
Apply automation liberally.

24. excella.com | @excellaco
Use the exhaust of normal day-
to-day work.

25. excella.com | @excellaco
Like change history from source control.

26. excella.com | @excellaco
With the details.

27. excella.com | @excellaco
Integrate non-functional requirements from the beginning with automation.
Engage security, compliance, and audit early and often.
Be transparent and share information.
Shift left.

28. excella.com | @excellaco
Because we all care about
developing and operating
reliable, secure, high
performance systems at scale.
“I care. I care a lot.
It’s kinda my thing.”
-- Leslie Knope

29. excella.com | @excellaco
③ Agile.

30. excella.com | @excellaco
Agile is dead. Long live agility.

31. excella.com | @excellaco
If you’re changing from this…

32. excella.com | @excellaco
…to this...

33. excella.com | @excellaco
…you’re not getting all the value.
Doing the
work right. + Doing the
right work.

34. excella.com | @excellaco
Experimentation and iteration.
DESIRED
SOLUTION
ITERATIONSSTART RELEASE
INITIAL TARGET
RELEASE
RELEASE

35. excella.com | @excellaco
Agility.

36. excella.com | @excellaco
Agility as a mindset – not a methodology.

37. excella.com | @excellaco
④ Risk.

38. excella.com | @excellaco
The risk of change is less than the risk of TWWADI*.
* The Way We’ve Always Done It.

39. excella.com | @excellaco
16.2% of projects were deemed successful by being completed on time and
budget, with all the promised functionality.
52.7%, were over cost, over time, and/or lacking promised functionality.
31.1% failed, which means they were abandoned or cancelled.

40. excella.com | @excellaco

41. excella.com | @excellaco
Perspective.
Risk of not
accomplishing the
business objective in
the quickest, most
cost-effective way
>>
Risk of not meeting
cost, schedule, and
scope objectives

42. excella.com | @excellaco
Shift from project to product.
Project Oriented Product Oriented
Budgeting
Funding of milestones predefined at project scoping. New
discretionary budget means the creation of a new project.
Funding of value streams adjusted based on business results.
New budget allocation based on demand.
Timeframes
Term of the project (e.g., one year). Defined end date. Not
focused on the maintenance/ health after the project ends.
Life cycle of the product (multiple years) includes ongoing
health/maintenance activities.
Success
Cost center approach. Measured to being on time and on
budget. Capitalization of development results in large
projects.
Profit center approach. Measured in business objectives and
outcomes met (e.g., revenue). Focus on incremental value
delivery and regular checkpoints.
Prioritization
Program and portfolio management, project plan-driven, with
a focus on requirements delivery. Projects often drive
waterfall orientation.
Roadmap and hypothesis testing-driven, with a focus on
feature and business value delivery. Products drive Agile
orientation.
Delivery
IT is a black box. Project management offices create complex
mapping and obscurity.
Direct mapping to what the business wants that enables
transparency.
From “Moving from Project to Product”, 2018 DevOps Enterprise Forum.

43. excella.com | @excellaco

44. excella.com | @excellaco
The FAANGs are doing all this to make a buck…

45. excella.com | @excellaco
Why are we accepting less for our nation?
• [The Government mission matters. A lot. Pictures here.]

46. excella.com | @excellaco
More resources.

47. excella.com | @excellaco
Jeff Gallimore
jeff.gallimore@excella.com
@jgallimore
http://itsanicelife.com
https://www.linkedin.com/in/jgallimore

48. excella.com | @excellaco

49. excella.com | @excellaco