Talk presented at the 2017 opening session of UNED's Masters of Computer Engineering. Agenda 1. The Need for Speed (aka DevOps) 2. Failure = Quality in Complex Systems 3. Survival of High-Performing Cultures

1. Combining Speed of Delivery
and Quality in Complex Systems
Manuel Pais | DevOps & Delivery Consultant
@manupaisable | manuelpais.net

2. About me
Manuel Pais
MS Software Eng
@manupaisable
manuelpais.net
manuel.pais@gmail.com
DevOps and Delivery Consultant
Focused on teams and flow

3. releasabilitybook.com
Co-author:
Team Guide to
Software Releasability
by Chris O’Dell & Manuel Pais
About me

4. About me

5. Agenda
1. The Need for Speed (aka DevOps)
2. Failure = Quality in Complex Systems
3. Survival of High-Performing Cultures

6. 2001: Agile Manifesto
2017201420082003
Java / Web
Developer

7. Agile… Scrum

8. Agile… Scrum

9. Scrum, Scrum, Scrum

10. Scrum, Scrum, Scrum

11. Scrum, Scrum, Scrum

12. Scrum, Scrum, Scrum

13. Scrum, Scrum, Scrum

14. Wall of confusion

15. QA
Lead
2009: DevOps
2017201420082003
Java / Web
Developer
QA
Engineer
Build
Manager

16. “DevOps brought to the attention
that two worlds, typically apart in a
company, need to collaborate and
that actually gives you a
competitive edge”
–Patrick Debois
infoq.com/interviews/debois-devops

19. Agile… Scrum… DevOps

20. Agile… Scrum… DevOps

21. Agile… Scrum… DevOps

22. “Agile System Administration was
too long and too narrow… ”
–Patrick Debois
infoq.com/interviews/debois-devops

24. 2010: Continuous Delivery
QA
Lead
2017201420082003
Java / Web
Developer
Build
Manager
DevOps
Consultant
QA
Engineer

25. “ability to get changes of all types,
into production, or into the hands
of users, safely and quickly in a
sustainable way”
–Jez Humble
continuousdelivery.com

26. cdchecklist.info

28. Agenda
1. The Need for Speed (aka DevOps)
2. Failure = Quality in Complex Systems
3. Survival of High-Performing Cultures

34. Systems of systems
Complex run time dependencies
Vulnerable build time dependencies
Failure is endemic

38. “The zero-error fallacy”
Researchers at MIT have shown that:
a) the more incidents an airline has, the lower the
passenger mortality risk
b) construction sites with relatively more incidents in a
given year have fewer worker deaths than those with
zero incidents.
https://techbeacon.com/zero-error-fallacy-what-really-counts-devops-teams

39. “The zero-error fallacy”
Researchers at MIT have shown that:
a) the more incidents an airline has, the lower the
passenger mortality risk
b) construction sites with relatively more incidents in a
given year have fewer worker deaths than those with
zero incidents.
https://techbeacon.com/zero-error-fallacy-what-really-counts-devops-teams

41. Learning from Failure
Greatest illusion is that the difference between excellent
and crappy operations is the number of errors or failures

42. Learning from Failure
What makes a difference is the presence of positive
capacities—in people, in teams, in the organization.
Greatest illusion is that the difference between excellent
and crappy operations is the number of errors or failures

43. Learning from Failure
What makes a difference is the presence of positive
capacities—in people, in teams, in the organization.
Greatest illusion is that the difference between excellent
and crappy operations is the number of errors or failures
A safety culture is one in which the boss actually invites
bad news, and may even reward it.

44. time between failures
time to repair

45. time between failures
time to repair

48. Development vs Maintenance

49. Development vs Maintenance
TEAM builds, deploys, runs, monitors and fixes
+ Ops provides platform

50. DONE = deployed

51. DONE = deployed
DONE = monitored in production

52. DONE = deployed
DONE = monitored NOT in production

53. Incident reviews
http://www.slideshare.net/jhand2/its-not-your-fault-blameless-post-mortems

56. Chaos engineering
https://medium.com/netflix-techblog/chaos-engineering-upgraded-878d341f15fa

57. Wrong incentives
Rewarding zero defects / fixing defects

58. Wrong incentives
Focus on simple/single metric
Rewarding zero defects / fixing defects

59. Wrong incentives
Focus on simple/single metric
Rewarding zero defects / fixing defects
Different IT teams with different goals

61. Right incentives
Reward along business objectives

62. Right incentives
Combination of metrics (e.g. lead time + time to repair)
Reward along business objectives

63. Right incentives
Combination of metrics (e.g. lead time + time to repair)
Reward along business objectives
All IT teams share same objectives

64. High Performers Are More Agile
30x 200x
more frequent
deployments
faster lead times
than their peers
Source: Puppet Labs 2015 State Of DevOps: https://puppetlabs.com/2015-devops-report

65. High Performers Are More Reliable
60x 168x
the change
success rate
faster mean time
to recover (MTTR)
Source: Puppet Labs 2015 State Of DevOps: https://puppetlabs.com/2015-devops-report

66. Agenda
1. The Need for Speed (aka DevOps)
2. Failure = Quality in Complex Systems
3. Survival of High-Performing Cultures

67. source: http://continuousdelivery.com/implementing/culture
Culture Types

68. Blame Culture
Resistance to Change
Lack of Collaboration

69. Blame Culture
Resistance to Change
Lack of Collaboration

70. Blame Culture
Resistance to Change
Lack of Collaboration

73. Blameless
Continuous Learning
High Collaboration

74. Blameless
Continuous Learning
High Collaboration

75. Blameless
Continuous Learning
High Collaboration

77. →
→
___
http://itrevolution.com/speeding-scaling-devops-enterprise

79. Conclusion

80. Conclusion
People

81. Conclusion
PeopleProcess

82. Conclusion
PeopleProcessTools

83. References

84. Thank you!
Manuel Pais
MS Software Eng
@manupaisable
manuelpais.net
manuel.pais@gmail.com
DevOps and Delivery Consultant
Focused on teams and flow