SlideShare a Scribd company logo
1 of 32
Download to read offline
03.10.2019
Roman Pickl
Getting Out of Quicksand, With
DevOps!
2© Elektrobit (EB) 2018
• Technical Project Manager @ Elektrobit
• Former CTO @ Fluidtime (now a Kapsch company)
• Loves CI/CD/DevOps
• Here to learn
Roman Pickl (@rompic)
How did I end up here?
Getting Out of Quicksand, With DevOps!
3© Elektrobit (EB) 2018
https://youtu.be/BeNdum6u6eg?t=2081
Andrew Shafer, DevOpsDays Cairo 2018
Getting Out of Quicksand, With DevOps!
4© Elektrobit (EB) 2018
Tom DeMarco, 2001, Slack: Getting Past Burnout, Busywork, and the Myth of Total Efficiency
Efficiency vs. Effectiveness
Getting Out of Quicksand, With DevOps!
Efficiency vs. Effectiveness
By Xwizard35 - Own work, CC BY-SA 4.0,
https://commons.wikimedia.org/w/index.php?curid=58355021
5© Elektrobit (EB) 2018
Symptoms of overload
• Decreased team morale
• Working long hours / when sick
• More frequent illness
• Unhealthy tasks queue
• Imbalanced metrics
• …
How the team currently looks like
Getting Out of Quicksand, With DevOps!
100 90130
6© Elektrobit (EB) 2018
It hurts …
Getting Out of Quicksand, With DevOps!
… again and again
7© Elektrobit (EB) 2018
Source unknown
No thanks, too busy
Getting Out of Quicksand, With DevOps!
8© Elektrobit (EB) 2018
What your manager thinks should be done
Getting Out of Quicksand, With DevOps!
100 100 100 20 20
From another
team with a
project on
hold
9© Elektrobit (EB) 2018
https://en.wikipedia.org/wiki/Kingman%27s_formula
https://less.works/less/principles/queueing_theory.html
https://observablehq.com/@troymagennis/how-does-utilization-impact-lead-time-of-work?collection=@troymagennis/agile-software-development - Troy Magennis
• Different types of queues (e.g. G/G/1)
• Waiting Time:
– Service time
– Utilization
– Variation in process and arrival
• Kingman Equation / approximation
• Little’s Law:
– Avg. Lead Time = Avg. WiP / Avg. Throughput
 Focus on throughput rather than utilization
Queuing theory
Wait a moment? How high workload kills efficiency!
0
2
4
6
8
10
12
14
16
18
20
5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95%
Waittime
Wait Time = (%Busy)/(% Idle)
11© Elektrobit (EB) 2018
Source: https://www.eaglecreek.com/blog/how-escape-quicksand
Docker: Ops unleashed by Mario Kleinsasser referring to Quicksand: https://www.youtube.com/watch?v=Ocmx7nifh2I&feature=youtu.be&t=400
Feels like quicksand
Wait a moment? How high workload kills efficiency!
The more you fight it, the more it pulls you in.
12© Elektrobit (EB) 2018
“slack is the missing ingredient required for all change”
Invest roughly a day per week to
- Spur innovation
- Rethink
- Practice new ways
- Master new skills
- Improve efficiency.
Scrum: sustainable pace
3Ms 15% time
Google’s 20% time
Hard limit on toil at Google SRE: 50%
#2 SRE principle: SREs must have time to make tomorrow
better than today
What you think should be done: hire more people
Wait a moment? How high workload kills efficiency!
80 80 80 80
13© Elektrobit (EB) 2018
What actually happens
Getting Out of Quicksand, With DevOps!
YOU
„People under time pressure don‘t think faster“ - Tim Lister
14© Elektrobit (EB) 2018
KC Green
It sometimes takes a crisis
Getting Out of Quicksand, With DevOps!
15© Elektrobit (EB) 2018
2018 Accelerate State of DevOps Report: https://cloudplatformonline.com/2018-state-of-devops.html
Quantify the work: Activity Accounting
Getting Out of Quicksand, With DevOps!
16© Elektrobit (EB) 2018
Hydrargyrum
Flow time
– Lead time
– Cycle time
Frequency of deployment
Mean time to restore services
WiP is the biggest deterrent to flow
What is the biggest bottleneck?
Can we fix the bottleneck?
Ask why (5 times),
Measure
Getting Out of Quicksand, With DevOps!
17© Elektrobit (EB) 2018
https://xkcd.com/303/
XKCD - Compiling
Getting Out of Quicksand, With DevOps!
18© Elektrobit (EB) 2018
https://www.youtube.com/watch?v=5S90x6CCGJ0&feature=youtu.be&t=2696 - Carmen DeArdo
Getting Out of Quicksand, With DevOps!
19© Elektrobit (EB) 2018
A Practical Approach to Large-Scale Agile Development – Gruver, Young, Fulghum
Principles of Lean Product Managment by Jez Humble - https://youtu.be/cH6bnQzJojo?t=1083
"A Practical Approach to Large Scale Agile Development" - Gary Gruver at Spark 2013
https://www.youtube.com/watch?v=2QGYEwghRSM
• Tendency to do more planning / estimation and more control
• Accept uncertainty and appreciate flexibility
• Decreasing marginal utility of each hour spend planning
• Every hour spent planning is an hour not delivering (as
resources often overlap)
• Lightweight capacity planning
• Just-in-time approach
• Prioritize e.g. by cost of delay
Lightweight Capacity Management
Getting Out of Quicksand, With DevOps!
20© Elektrobit (EB) 2018
• Reduce WiP
• Reduce batch size
• Reduce variability / increase predictability
• Do less
• Self service platform
• Eliminate before you try to automate
Getting Out of Quicksand, With DevOps!
Do less (at the same time)
- = +
Less is More
21© Elektrobit (EB) 2018
Picture: https://moduscreate.com/blog/automation-what-why-when/
• Make time to create automation
• Documenting the process and making it repeatable is the first
step
• Automate, eliminate or engineer out the drivers that are not
key to the value proposition
– New work – hard to automate
– Triage work – automate for consistency
– Repetitive work – automate
• Fix the biggest bottleneck first
Automate
Getting Out of Quicksand, With DevOps!
22© Elektrobit (EB) 2018
The Practice of Cloud System Administration
Getting Out of Quicksand, With DevOps!
Left-Over Principle
DifficultEasy
Frequent
Rare
Acquire
Document
Manual
Automate
Hire
Create Tools
Eliminate
23© Elektrobit (EB) 2018
Demarco T, Slack: Getting Past Burnout, Busywork and the Myth of Total Efficiency, page 108ff
Forsgren N., Humble J., Gene K. Accelerate p. 12 ff
Accelerate State of DevOps Report 2018 p. 28ff
A word of caution
Getting Out of Quicksand, With DevOps!
The paradox of automation
24© Elektrobit (EB) 2018
“Improving your software delivery effectiveness will improve your ability to work in small
batches and incorporate customer feedback along the way”
“Lean product management practices positively impact software delivery performance,
stimulate a generative culture, and decrease burnout”
“[…] software delivery performance drives Lean product management practices”
“[…] it becomes a reciprocal model or, colloquially, a virtuous cycle” – Accelerate
Virtuous circle
Getting Out of Quicksand, With DevOps!
25© Elektrobit (EB) 2018
http://www-personal.umich.edu/~mrother/Homepage.html
Improvement Kata
Getting Out of Quicksand, With DevOps!
26© Elektrobit (EB) 2018
The Practice of Cloud System Administration
Getting Out of Quicksand, With DevOps!
Prioritization of Improvement
High ImpactLow Impact
Easy
Hard
Priority
1
Priority
2
Priority
4
Priority
3
27© Elektrobit (EB) 2018 27
Based on Virtuous Circle of CI/CD The Site Reliability Workbook
Virtuous Circle
Getting Out of Quicksand, With DevOps!
More release / test
automation /
improvements
Faster releases / more
time for new work
More frequent releases /
smaller batches / less WiP
Fewer changes per
release / lower lead time
Fewer bugs per release /
faster feedback/ higher
confidence / lower
variability and rework
More time to automate /
test / lower utilization /
decreased burnout /
higher job satisfaction
28© Elektrobit (EB) 2018
Spent 1 day a week to improve “something”
Created self service pipelines
Used Information radiators
Clear priorities and weekly goals
How it ended up working:
Getting Out of Quicksand, With DevOps!
80 80 80
29© Elektrobit (EB) 2018
Crisis situations are opportunities for change
Often it takes fresh eyes or at least courage to see the problems
Get out of the quicksand, the sky is the limit
Quantify the work and set goals
Measure
Less is more
Automate the right things
Establish a continuous improvement process
Start a virtuous circle
Conclusion
Getting Out of Quicksand, With DevOps!
30© Elektrobit (EB) 2018
Read these if you want to know more
Getting Out of Quicksand, With DevOps!
34© Elektrobit (EB) 2018
We don‘t let our servers get to 100%
capacity utilization, so let‘s not do that
to ourselves
Dominica DeGrandis – Making work visible
Getting Out of Quicksand, With DevOps!
35© Elektrobit (EB) 2018
• Daniel Vacanti: Little‘s (F)law: https://vimeo.com/52683659
• Håkan Forss, Queueing theory in software development - ALEBathtub 2011-06-30, https://de.slideshare.net/HkanForss/queueing-theory-in-
software-development-ale-bathtub-2011-0630
• GOTO 2015 • Why Scaling Agile Doesn't Work • Jez Humble - https://www.youtube.com/watch?v=2zYxWEZ0gYg&feature=youtu.be&t=3003 (it
often takes a disaster for lasting change)
• John Cutler – Too Busy to Improve https://anchor.fm/john-cutler/episodes/Too-Busy-To-Improve-for-Roman-e2ap3i
• 2018 Accelerate State of DevOps Report: https://cloudplatformonline.com/2018-state-of-devops.html
• Dominica DeGrandis - Making Work Visible: How to Unmask Capacity Killing WIP - https://www.youtube.com/watch?v=KR7Y8IUgyyA
• Keynote: Continuous Delivery Sounds Great By Jez Humble @ Agile India 2017 https://www.youtube.com/watch?v=SjVV3xuYKJs
• A Practical Approach to Large Scale Agile Development"- Gary Gruver at Spark 2013 https://www.youtube.com/watch?v=2QGYEwghRSM
• High-Performance Team, Management, and Leadership Behaviors and practices https://devops-
research.com/assets/transformation_practices.pdf
• John Cutler - This feels like going faster vs. this actually makes us faster https://twitter.com/johncutlefish/status/1029757026895720449
• John Cutler – Slack vs. Blocked https://medium.com/@johnpcutler/slack-vs-blocked-16c294938b1
• Mike Rother - http://www-personal.umich.edu/~mrother/Homepage.html
• Troy Magennis - https://observablehq.com/@troymagennis/how-does-utilization-impact-lead-time-of-work?collection=@troymagennis/agile-
software-development
Further references & information
Getting Out of Quicksand, With DevOps!
Roman Pickl (@rompic)
roman.pickl@elektrobit.com
www.elektrobit.com
Paldies! Don’t wait and get in touch

More Related Content

Recently uploaded

JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)
Max Lee
 

Recently uploaded (20)

KLARNA - Language Models and Knowledge Graphs: A Systems Approach
KLARNA -  Language Models and Knowledge Graphs: A Systems ApproachKLARNA -  Language Models and Knowledge Graphs: A Systems Approach
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
 
Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...
Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...
Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...
 
AI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning Framework
 
APVP,apvp apvp High quality supplier safe spot transport, 98% purity
APVP,apvp apvp High quality supplier safe spot transport, 98% purityAPVP,apvp apvp High quality supplier safe spot transport, 98% purity
APVP,apvp apvp High quality supplier safe spot transport, 98% purity
 
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product UpdatesGraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
 
Microsoft 365 Copilot; An AI tool changing the world of work _PDF.pdf
Microsoft 365 Copilot; An AI tool changing the world of work _PDF.pdfMicrosoft 365 Copilot; An AI tool changing the world of work _PDF.pdf
Microsoft 365 Copilot; An AI tool changing the world of work _PDF.pdf
 
JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)
 
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
 
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
 
AI Hackathon.pptx
AI                        Hackathon.pptxAI                        Hackathon.pptx
AI Hackathon.pptx
 
how-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdfhow-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdf
 
A Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationA Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data Migration
 
IT Software Development Resume, Vaibhav jha 2024
IT Software Development Resume, Vaibhav jha 2024IT Software Development Resume, Vaibhav jha 2024
IT Software Development Resume, Vaibhav jha 2024
 
Agnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in KrakówAgnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in Kraków
 
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdf
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdfImplementing KPIs and Right Metrics for Agile Delivery Teams.pdf
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdf
 
INGKA DIGITAL: Linked Metadata by Design
INGKA DIGITAL: Linked Metadata by DesignINGKA DIGITAL: Linked Metadata by Design
INGKA DIGITAL: Linked Metadata by Design
 
AI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in MichelangeloAI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in Michelangelo
 
OpenChain @ LF Japan Executive Briefing - May 2024
OpenChain @ LF Japan Executive Briefing - May 2024OpenChain @ LF Japan Executive Briefing - May 2024
OpenChain @ LF Japan Executive Briefing - May 2024
 
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi.pdf
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi.pdfStrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi.pdf
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi.pdf
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
 

Featured

How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 

Featured (20)

2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 

Getting out of quicksand with devops - DevOps unicorns Riga 2019

  • 1. 03.10.2019 Roman Pickl Getting Out of Quicksand, With DevOps!
  • 2. 2© Elektrobit (EB) 2018 • Technical Project Manager @ Elektrobit • Former CTO @ Fluidtime (now a Kapsch company) • Loves CI/CD/DevOps • Here to learn Roman Pickl (@rompic) How did I end up here? Getting Out of Quicksand, With DevOps!
  • 3. 3© Elektrobit (EB) 2018 https://youtu.be/BeNdum6u6eg?t=2081 Andrew Shafer, DevOpsDays Cairo 2018 Getting Out of Quicksand, With DevOps!
  • 4. 4© Elektrobit (EB) 2018 Tom DeMarco, 2001, Slack: Getting Past Burnout, Busywork, and the Myth of Total Efficiency Efficiency vs. Effectiveness Getting Out of Quicksand, With DevOps! Efficiency vs. Effectiveness By Xwizard35 - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=58355021
  • 5. 5© Elektrobit (EB) 2018 Symptoms of overload • Decreased team morale • Working long hours / when sick • More frequent illness • Unhealthy tasks queue • Imbalanced metrics • … How the team currently looks like Getting Out of Quicksand, With DevOps! 100 90130
  • 6. 6© Elektrobit (EB) 2018 It hurts … Getting Out of Quicksand, With DevOps! … again and again
  • 7. 7© Elektrobit (EB) 2018 Source unknown No thanks, too busy Getting Out of Quicksand, With DevOps!
  • 8. 8© Elektrobit (EB) 2018 What your manager thinks should be done Getting Out of Quicksand, With DevOps! 100 100 100 20 20 From another team with a project on hold
  • 9. 9© Elektrobit (EB) 2018 https://en.wikipedia.org/wiki/Kingman%27s_formula https://less.works/less/principles/queueing_theory.html https://observablehq.com/@troymagennis/how-does-utilization-impact-lead-time-of-work?collection=@troymagennis/agile-software-development - Troy Magennis • Different types of queues (e.g. G/G/1) • Waiting Time: – Service time – Utilization – Variation in process and arrival • Kingman Equation / approximation • Little’s Law: – Avg. Lead Time = Avg. WiP / Avg. Throughput  Focus on throughput rather than utilization Queuing theory Wait a moment? How high workload kills efficiency! 0 2 4 6 8 10 12 14 16 18 20 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% Waittime Wait Time = (%Busy)/(% Idle)
  • 10. 11© Elektrobit (EB) 2018 Source: https://www.eaglecreek.com/blog/how-escape-quicksand Docker: Ops unleashed by Mario Kleinsasser referring to Quicksand: https://www.youtube.com/watch?v=Ocmx7nifh2I&feature=youtu.be&t=400 Feels like quicksand Wait a moment? How high workload kills efficiency! The more you fight it, the more it pulls you in.
  • 11. 12© Elektrobit (EB) 2018 “slack is the missing ingredient required for all change” Invest roughly a day per week to - Spur innovation - Rethink - Practice new ways - Master new skills - Improve efficiency. Scrum: sustainable pace 3Ms 15% time Google’s 20% time Hard limit on toil at Google SRE: 50% #2 SRE principle: SREs must have time to make tomorrow better than today What you think should be done: hire more people Wait a moment? How high workload kills efficiency! 80 80 80 80
  • 12. 13© Elektrobit (EB) 2018 What actually happens Getting Out of Quicksand, With DevOps! YOU „People under time pressure don‘t think faster“ - Tim Lister
  • 13. 14© Elektrobit (EB) 2018 KC Green It sometimes takes a crisis Getting Out of Quicksand, With DevOps!
  • 14. 15© Elektrobit (EB) 2018 2018 Accelerate State of DevOps Report: https://cloudplatformonline.com/2018-state-of-devops.html Quantify the work: Activity Accounting Getting Out of Quicksand, With DevOps!
  • 15. 16© Elektrobit (EB) 2018 Hydrargyrum Flow time – Lead time – Cycle time Frequency of deployment Mean time to restore services WiP is the biggest deterrent to flow What is the biggest bottleneck? Can we fix the bottleneck? Ask why (5 times), Measure Getting Out of Quicksand, With DevOps!
  • 16. 17© Elektrobit (EB) 2018 https://xkcd.com/303/ XKCD - Compiling Getting Out of Quicksand, With DevOps!
  • 17. 18© Elektrobit (EB) 2018 https://www.youtube.com/watch?v=5S90x6CCGJ0&feature=youtu.be&t=2696 - Carmen DeArdo Getting Out of Quicksand, With DevOps!
  • 18. 19© Elektrobit (EB) 2018 A Practical Approach to Large-Scale Agile Development – Gruver, Young, Fulghum Principles of Lean Product Managment by Jez Humble - https://youtu.be/cH6bnQzJojo?t=1083 "A Practical Approach to Large Scale Agile Development" - Gary Gruver at Spark 2013 https://www.youtube.com/watch?v=2QGYEwghRSM • Tendency to do more planning / estimation and more control • Accept uncertainty and appreciate flexibility • Decreasing marginal utility of each hour spend planning • Every hour spent planning is an hour not delivering (as resources often overlap) • Lightweight capacity planning • Just-in-time approach • Prioritize e.g. by cost of delay Lightweight Capacity Management Getting Out of Quicksand, With DevOps!
  • 19. 20© Elektrobit (EB) 2018 • Reduce WiP • Reduce batch size • Reduce variability / increase predictability • Do less • Self service platform • Eliminate before you try to automate Getting Out of Quicksand, With DevOps! Do less (at the same time) - = + Less is More
  • 20. 21© Elektrobit (EB) 2018 Picture: https://moduscreate.com/blog/automation-what-why-when/ • Make time to create automation • Documenting the process and making it repeatable is the first step • Automate, eliminate or engineer out the drivers that are not key to the value proposition – New work – hard to automate – Triage work – automate for consistency – Repetitive work – automate • Fix the biggest bottleneck first Automate Getting Out of Quicksand, With DevOps!
  • 21. 22© Elektrobit (EB) 2018 The Practice of Cloud System Administration Getting Out of Quicksand, With DevOps! Left-Over Principle DifficultEasy Frequent Rare Acquire Document Manual Automate Hire Create Tools Eliminate
  • 22. 23© Elektrobit (EB) 2018 Demarco T, Slack: Getting Past Burnout, Busywork and the Myth of Total Efficiency, page 108ff Forsgren N., Humble J., Gene K. Accelerate p. 12 ff Accelerate State of DevOps Report 2018 p. 28ff A word of caution Getting Out of Quicksand, With DevOps! The paradox of automation
  • 23. 24© Elektrobit (EB) 2018 “Improving your software delivery effectiveness will improve your ability to work in small batches and incorporate customer feedback along the way” “Lean product management practices positively impact software delivery performance, stimulate a generative culture, and decrease burnout” “[…] software delivery performance drives Lean product management practices” “[…] it becomes a reciprocal model or, colloquially, a virtuous cycle” – Accelerate Virtuous circle Getting Out of Quicksand, With DevOps!
  • 24. 25© Elektrobit (EB) 2018 http://www-personal.umich.edu/~mrother/Homepage.html Improvement Kata Getting Out of Quicksand, With DevOps!
  • 25. 26© Elektrobit (EB) 2018 The Practice of Cloud System Administration Getting Out of Quicksand, With DevOps! Prioritization of Improvement High ImpactLow Impact Easy Hard Priority 1 Priority 2 Priority 4 Priority 3
  • 26. 27© Elektrobit (EB) 2018 27 Based on Virtuous Circle of CI/CD The Site Reliability Workbook Virtuous Circle Getting Out of Quicksand, With DevOps! More release / test automation / improvements Faster releases / more time for new work More frequent releases / smaller batches / less WiP Fewer changes per release / lower lead time Fewer bugs per release / faster feedback/ higher confidence / lower variability and rework More time to automate / test / lower utilization / decreased burnout / higher job satisfaction
  • 27. 28© Elektrobit (EB) 2018 Spent 1 day a week to improve “something” Created self service pipelines Used Information radiators Clear priorities and weekly goals How it ended up working: Getting Out of Quicksand, With DevOps! 80 80 80
  • 28. 29© Elektrobit (EB) 2018 Crisis situations are opportunities for change Often it takes fresh eyes or at least courage to see the problems Get out of the quicksand, the sky is the limit Quantify the work and set goals Measure Less is more Automate the right things Establish a continuous improvement process Start a virtuous circle Conclusion Getting Out of Quicksand, With DevOps!
  • 29. 30© Elektrobit (EB) 2018 Read these if you want to know more Getting Out of Quicksand, With DevOps!
  • 30. 34© Elektrobit (EB) 2018 We don‘t let our servers get to 100% capacity utilization, so let‘s not do that to ourselves Dominica DeGrandis – Making work visible Getting Out of Quicksand, With DevOps!
  • 31. 35© Elektrobit (EB) 2018 • Daniel Vacanti: Little‘s (F)law: https://vimeo.com/52683659 • Håkan Forss, Queueing theory in software development - ALEBathtub 2011-06-30, https://de.slideshare.net/HkanForss/queueing-theory-in- software-development-ale-bathtub-2011-0630 • GOTO 2015 • Why Scaling Agile Doesn't Work • Jez Humble - https://www.youtube.com/watch?v=2zYxWEZ0gYg&feature=youtu.be&t=3003 (it often takes a disaster for lasting change) • John Cutler – Too Busy to Improve https://anchor.fm/john-cutler/episodes/Too-Busy-To-Improve-for-Roman-e2ap3i • 2018 Accelerate State of DevOps Report: https://cloudplatformonline.com/2018-state-of-devops.html • Dominica DeGrandis - Making Work Visible: How to Unmask Capacity Killing WIP - https://www.youtube.com/watch?v=KR7Y8IUgyyA • Keynote: Continuous Delivery Sounds Great By Jez Humble @ Agile India 2017 https://www.youtube.com/watch?v=SjVV3xuYKJs • A Practical Approach to Large Scale Agile Development"- Gary Gruver at Spark 2013 https://www.youtube.com/watch?v=2QGYEwghRSM • High-Performance Team, Management, and Leadership Behaviors and practices https://devops- research.com/assets/transformation_practices.pdf • John Cutler - This feels like going faster vs. this actually makes us faster https://twitter.com/johncutlefish/status/1029757026895720449 • John Cutler – Slack vs. Blocked https://medium.com/@johnpcutler/slack-vs-blocked-16c294938b1 • Mike Rother - http://www-personal.umich.edu/~mrother/Homepage.html • Troy Magennis - https://observablehq.com/@troymagennis/how-does-utilization-impact-lead-time-of-work?collection=@troymagennis/agile- software-development Further references & information Getting Out of Quicksand, With DevOps!