Arty Starr, profile picture
Uploaded byArty Starr
PDF, PPTX621 views

Top 5 Reasons Why Improvement Efforts Fail

Janelle Klein discusses the challenges faced in software development and the common mistakes leading to project failures, primarily focusing on misidentifying the root causes of issues. She emphasizes the importance of understanding the actual problems instead of rushing to implement solutions, and highlights human factors as a significant source of development pain. The document concludes with recommendations for improving decision-making processes and fostering a culture of responsibility among team members.

Embed presentation

Download as PDF, PPTX
1 / 133
2 / 133
3 / 133
4 / 133
5 / 133
6 / 133
7 / 133
8 / 133
9 / 133
10 / 133
11 / 133
12 / 133
13 / 133
14 / 133
15 / 133
16 / 133
17 / 133
18 / 133
19 / 133
20 / 133
21 / 133
22 / 133
23 / 133
24 / 133
25 / 133
26 / 133
27 / 133
28 / 133
29 / 133
30 / 133
31 / 133
32 / 133
33 / 133
34 / 133
35 / 133
36 / 133
37 / 133
38 / 133
39 / 133
40 / 133
41 / 133
42 / 133
43 / 133
44 / 133
45 / 133
46 / 133
47 / 133
48 / 133
49 / 133
50 / 133
51 / 133
52 / 133
53 / 133
54 / 133
55 / 133
56 / 133
57 / 133
58 / 133
59 / 133
60 / 133
61 / 133
62 / 133
63 / 133
64 / 133
65 / 133
66 / 133
67 / 133
68 / 133
69 / 133
70 / 133
71 / 133
72 / 133
73 / 133
74 / 133
75 / 133
76 / 133
77 / 133
78 / 133
79 / 133
80 / 133
81 / 133
82 / 133
83 / 133
84 / 133
85 / 133
86 / 133
87 / 133
88 / 133
89 / 133
90 / 133
91 / 133
92 / 133
93 / 133
94 / 133
95 / 133
96 / 133
97 / 133
98 / 133
99 / 133
100 / 133
101 / 133
102 / 133
103 / 133
104 / 133
105 / 133
106 / 133
107 / 133
108 / 133
109 / 133
110 / 133
111 / 133
112 / 133
113 / 133
114 / 133
115 / 133
116 / 133
117 / 133
118 / 133
119 / 133
120 / 133
121 / 133
122 / 133
123 / 133
124 / 133
125 / 133
126 / 133
127 / 133
128 / 133
129 / 133
130 / 133
131 / 133
132 / 133
133 / 133
Janelle Klein openmastery.org @janellekz Improvement Efforts FAIL Top 5 Reasons Why
, Developer, Consultant, CTO @ Specialized in Statistical Process Control (SPC) and Supply Chain Optimization from Lean Manufacturing (data geek) Continuous Delivery infrastructure, automation strategy & technical mentorship Janelle Klein Who Am I? How to Measure the PAIN in Software Development Janelle Klein Author of “Idea Flow” leanpub.com/ideaflow Founder of newiron.com
What is this talk about? Mistakes
We were trying to do all the “right” things. About 8 Years Ago…
We were building a factory automation system...
We shipped to production...
We shipped to production... (again)
We couldn’t reproduce the problem.
We shipped to production... (AGAIN) The rollback failed!
Project FAILURE
The Situation 10 year old software project, 1.5M LOC, 24/7 uptime, programmable statistical processing engine Brought down production three releases in a row 12 developers on the team, disciplined with best practices, constantly working on improvements
The Retrospective “What are we going to do?!” Our biggest problem “Well, we know we’ve got a quality problem right?”
The Retrospective “What are we going to do?!” Our biggest problem “The problem is we don’t have enough test automation!”
So the Test Automation Began… Our regression testing took 3-4 weeks… Let’s automate the tests!
20% Percent Automated: So the Test Automation Began…
50% Maintenance started getting PAINFUL… Let’s refactor the tests! Percent Automated: So the Test Automation Began…
80% It was still really PAINFUL… “Well, at least our regression cycle is faster, right?” Our regression cycle still took 3-4 weeks! Percent Automated: So the Test Automation Began…
ApplicationTests We noticed a Pattern in Our Test Suite
ApplicationTests We noticed a Pattern in Our Test Suite Most of the bugs were found during exploratory testing.
The First Mistake Our biggest problem “Well, we know we’ve got a quality problem right?” “The problem is we don’t have enough test automation!” What’s the mistake we made?
What does “better” really mean? What qualifies as an “improvement”?
“Better” following best practices (solution-focused) Start with the Solution (problem-focused) “Better” solving the problems Start with the Problem What does “better” really mean?
The Retrospective Our biggest problem “The problem is we don’t have enough test automation!” Solution-Focused “Problems”
“What problem am I trying to solve?” Mistake #1: Starting with the Solution
ApplicationTests Most of the bugs were found during exploratory testing. Our Test Suite
The Retrospective “What are we supposed to do?!” Our biggest problem “The problem is all the technical debt is causing us to make mistakes.”
The Retrospective “What are we supposed to do?!” Our biggest problem “Let’s brainstorm a list of all the problems!”
Brainstorm a List Filled up Tech-Debt Backlog
The Retrospective “What are we supposed to do?!” Our biggest problem “Well, if we don’t understand a problem, we should collect data.”
The Retrospective Our biggest problem “What data would help us understand the problem?” “What are we supposed to do?!”
Technical Debt Mistakes I thought the problem was Technical Debt
? Most of our mistakes were in the most well-written parts of the code. Mistakes
We made significantly more mistakes in code that we didn’t write ourselves. Lower Familiarity More Mistakes= There had to be more to the story...
Complex( So*ware( PAIN This is what I knew... What made development feel painful?
Unexpected Behavior Problem Resolved Tracking Painful Interaction with the Code (Friction) Troubleshooting Progress 5 hours and 18 minutes of troubleshooting... PAINFUL
The amount of PAIN was caused by… Likeliness(of(( Unexpected( Behavior( Cost(to(Troubleshoot(and(Repair( High(Frequency( Low(Impact( Low(Frequency( Low(Impact( Low(Frequency( High(Impact( PAIN(
What Causes Unexpected Behavior (likeliness)? What Makes Troubleshooting Time-Consuming (impact)? Semantic Mistakes Stale Memory Mistakes Association Mistakes Bad Input Assumption Tedious Change Mistakes Copy-Edit Mistakes Transposition Mistakes Failed Refactor Mistakes False Alarm Non-Deterministic Behavior Ambiguous Clues Lots of Code Changes Noisy Output Cryptic Output Long Execution Time Environment Cleanup Test Data Creation Using Debugger Most of the pain was caused by human factors. What causes PAIN?
What Causes Unexpected Behavior (likeliness)? What Makes Troubleshooting Time-Consuming (impact)? Non-Deterministic Behavior Ambiguous Clues Lots of Code Changes Noisy Output Cryptic Output Long Execution Time Environment Cleanup Test Data Creation Using Debugger What causes PAIN? Most of the pain was caused by human factors. Semantic Mistakes Stale Memory Mistakes Association Mistakes Bad Input Assumption Tedious Change Mistakes Copy-Edit Mistakes Transposition Mistakes Failed Refactor Mistakes False Alarm
What Causes Unexpected Behavior (likeliness)? What Makes Troubleshooting Time-Consuming (impact)? Non-Deterministic Behavior Ambiguous Clues Lots of Code Changes Noisy Output Cryptic Output Long Execution Time Environment Cleanup Test Data Creation Using Debugger What causes PAIN? Semantic Mistakes Stale Memory Mistakes Association Mistakes Bad Input Assumption Tedious Change Mistakes Copy-Edit Mistakes Transposition Mistakes Failed Refactor Mistakes False Alarm Most of the pain was caused by human factors.
What Causes Unexpected Behavior (likeliness)? What Makes Troubleshooting Time-Consuming (impact)? Non-Deterministic Behavior Ambiguous Clues Lots of Code Changes Noisy Output Cryptic Output Long Execution Time Environment Cleanup Test Data Creation Using Debugger What causes PAIN? PAIN is a consequence of how we interact with the code. Semantic Mistakes Stale Memory Mistakes Association Mistakes Bad Input Assumption Tedious Change Mistakes Copy-Edit Mistakes Transposition Mistakes Failed Refactor Mistakes False Alarm
PAIN occurs during the process of understanding and extending the software Complex( So*ware( PAIN Not the Code. Optimize “Idea Flow”
ApplicationTests Most of the bugs were found during exploratory testing. Our Test Suite
ApplicationTests The bugs weren’t in the code, the bugs were in our Conceptual Models Our Test Suite
The Fix: We built a Data Dictionary The bugs weren’t in the code, the bugs were in our Conceptual Models
The “Coat of Armor” Principle Software Coat of Armor More tests result in better bug protection It’s not that simple!
“Alarm Pain” Six hours of troubleshooting and repairing tests when nothing is broken.
“Green Washing” — Kerry Kimbrough
“Green Washing” — Kerry Kimbrough If we don’t understand the system, we fix the tests incorrectly.
The “Waxy Coating” Principle Optimize the signal to noise ratio. Software (before) Software (after) Tests are like a waxy coating poured over the code.
Cost & Risk are a Function of Increased Difficulty Cost & Risk Difficulty of Work The Difficulty of Doing Our Jobs Human Limitations
Likelihood)of)) Unexpected) Behavior) Cost)to)Troubleshoot)and)Repair) High)Frequency) Low)Impact) Low)Frequency) Low)Impact) Low)Frequency) High)Impact) PAIN) “Better” = Reduce the Risk Reduce Likeliness = Mistake Proofing Reduce Recovery Cost = Mistake Tolerance
The Second Mistake Our biggest problem “The problem is all the technical debt that’s causing us to make mistakes.” What’s the mistake we made?
Complex( So*ware( PAIN Mistake #2: Assuming the problems are inside the code. What pain are we experiencing when we interact with the code?
Our biggest problem “Let’s brainstorm a list of all the problems!” The Third Mistake What’s the mistake we made?
“What’s the best opportunity for improvement? “The awful email template engine code!” Our biggest problem The Retrospective
“Fill in missing unit tests!” Our biggest problem The Retrospective “What’s the best opportunity for improvement?
“We should clean up the database code!” Our biggest problem The Retrospective “What’s the best opportunity for improvement?
“Let’s improve maintainability of our test framework!” Our biggest problem The Retrospective “What’s the best opportunity for improvement?
Just because a problem comes to mind, doesn’t mean it’s an important problem to solve. Our biggest problem The Retrospective “What’s the best opportunity for improvement?
Our biggest problem What do I feel the most intensely about? Daniel Kahneman Thinking Fast and Slow The Retrospective “What’s the best opportunity for improvement?
“The awful email template engine code!” Recency Bias Our biggest problem The Retrospective “What’s the best opportunity for improvement?
Guilt Bias “Fill in missing unit tests!” Our biggest problem The Retrospective “What’s the best opportunity for improvement?
Known Solution Bias Our biggest problem The Retrospective “What’s the best opportunity for improvement? “We should clean up the database code!”
Sunk Cost Bias “Let’s improve maintainability of our test framework!” Our biggest problem The Retrospective “What’s the best opportunity for improvement?
Mistake #3: Assuming the biggest problems will come to mind. How do I know the right problems are on the list? What do I feel the most intensely about? Daniel Kahneman Thinking Fast and Slow
The Deadline Effect Rushing before the deadline increases risk.
The Retrospective “What are we supposed to do?!” Our biggest problem “We should stop rushing before the deadline.” “Yes. Fixed deadline. Variable scope.”
Urgency Leads to High-Risk Decisions 7:01 Iterative Validation with Unit Tests 7:010:00 14:230:00 Skipping Tests and Validating at the End We gamble to save time: If I make no mistakes I save ~2 hours. If I make several mistakes I lose ~8 hours.
Time% Pressure% Compromise% Safety% for% Speed% Increase% Number%&% Severity%of% Hazards% % More%Pain% and%Higher% Task%Effort% Constant' Urgency' Cycle of Chaos High-Risk Decision Habits Problems
Time% Pressure% Compromise% Safety% for% Speed% Increase% Number%&% Severity%of% Hazards% % More%Pain% and%Higher% Task%Effort% Constant' Urgency' Cycle of Chaos High-Risk Decision Habits Problems
Time% Pressure% Compromise% Safety% for% Speed% Increase% Number%&% Severity%of% Hazards% % More%Pain% and%Higher% Task%Effort% Constant' Urgency' Cycle of Chaos High-Risk Decision Habits Problems
Time% Pressure% Compromise% Safety% for% Speed% Increase% Number%&% Severity%of% Hazards% % More%Pain% and%Higher% Task%Effort% Constant' Urgency' Cycle of Chaos High-Risk Decision Habits Problems
Time% Pressure% Compromise% Safety% for% Speed% Increase% Number%&% Severity%of% Hazards% % More%Pain% and%Higher% Task%Effort% Constant' Urgency' Cycle of Chaos High-Risk Decision Habits Problems
Likelihood)of)) Unexpected) Behavior) Cost)to)Troubleshoot)and)Repair) High)Frequency) Low)Impact) Low)Frequency) Low)Impact) Low)Frequency) High)Impact) PAIN) PAIN = Frequent Severe Explosions
Fewer% Problems%to% Fix% Stop%% and% Think% Mi8gate% the% Risk% Increased% Produc8vity% and% Innova8on% Safety' Cycle of Safety Low-Risk Decision Habits
Fewer% Problems%to% Fix% Stop%% and% Think% Mi8gate% the% Risk% Increased% Produc8vity% and% Innova8on% Safety' Cycle of Safety Low-Risk Decision Habits
Fewer% Problems%to% Fix% Stop%% and% Think% Mi8gate% the% Risk% Increased% Produc8vity% and% Innova8on% Safety' Cycle of Safety Low-Risk Decision Habits
Fewer% Problems%to% Fix% Stop%% and% Think% Mi8gate% the% Risk% Increased% Produc8vity% and% Innova8on% Safety' Cycle of Safety Low-Risk Decision Habits
Fewer% Problems%to% Fix% Stop%% and% Think% Mi8gate% the% Risk% Increased% Produc8vity% and% Innova8on% Safety' Cycle of Safety Low-Risk Decision Habits
Product( Requirements( So1ware(Task( Possible(Decision(Paths( So1ware( Software is a Reflection of our Decisions
Product( Requirements( So1ware(Task( High%Risk% Decision%Habits% So1ware( (and(all(our(problems)( ( RESET Software is a Reflection of our Decisions
Product( Requirements( So1ware(Task( High%Risk% Decision%Habits% So1ware( (and(all(our(problems)( ( Technical Debt is just a Symptom of the Problem Software is a Reflection of our Decisions
The Deadline Effect Rushing before the deadline increases risk.
Time-Bound Releases RISK-Bound Releases
We deleted our automated test suite… The automation wasn’t solving the problems!
What are the specific risks in this release? How can we mitigate the risks? Handful of manual tests Custom Tools +
Release Size (weeks) Time Defect Rates RISK-Bound Releases Continuous Delivery is about learning how to effectively manage risk
We built TONS of automation. The only real difference: We didn’t design a single solution without a specific problem in mind.
Our biggest problem “We should stop rushing before the deadline.” “Yes. Fixed deadline. Variable scope.” The Forth Mistake What’s the mistake we made?
Mistake #4: Over-simplification and hand-waving. How are you adapting your decision habits? Time% Pressure% Compromise% Safety% for% Speed% Increase% Number%&% Severity%of% Hazards% % More%Pain% and%Higher% Task%Effort% Constant' Urgency' Fewer% Problems%to% Fix% Stop%% and% Think% Mi8gate% the% Risk% Increased% Produc8vity% and% Innova8on% Safety'
Then I got into consulting… The Software Rewrite Cycle Start% Over% Unmaintainable% So0ware%
We Start with the Best of Intentions High Quality Code Low Technical Debt Easy to Maintain Good Code Coverage
Then This Happens!
PAIN The Classic Story of Project Failure Problems get deferred Builds start breaking Releases get chaotic Productivity slows to a crawl Developers begging for time It’s never enough Project Meltdown
Our “Solution” “We should just quit our jobs.” “Yeah, it’s hopeless.” What are we supposed to do?
What if we could measure our PAIN? 1. Test Data Generation 2. Merging Problems 3. Repairing Tests 1000 hours/month The Biggest Problem: ~700 hours/month generating test data
“Better” “Better” What if we could get managers and developers all pulling the same direction? Managers Developers
The Biggest Cause of FAILURE in our Industry: Next Talk: Stop Getting Crushed By Business Pressure
Mistake #5: Feeling helpless instead of taking responsibility. Leadership is not a title, it’s a choice.
Summary
The First Mistake Our biggest problem “Well, we know we’ve got a quality problem right?” “The problem is we don’t have enough test automation!” What’s the mistake we made?
“What problem am I trying to solve?” Mistake #1: Starting with the Solution
The Second Mistake Our biggest problem “The problem is all the technical debt that’s causing us to make mistakes.” What’s the mistake we made?
Complex( So*ware( PAIN Mistake #2: Assuming the problems are inside the code. What pain are we experiencing when we interact with the code?
Our biggest problem “Let’s brainstorm a list of all the problems!” The Third Mistake What’s the mistake we made?
Mistake #3: Assuming the biggest problems will come to mind. How do I know the right problems are on the list? What do I feel the most intensely about? Daniel Kahneman Thinking Fast and Slow
Our biggest problem “We should stop rushing before the deadline.” “Yes. Fixed deadline. Variable scope.” The Forth Mistake What’s the mistake we made?
Mistake #4: Over-simplification and hand-waving. Time% Pressure% Compromise% Safety% for% Speed% Increase% Number%&% Severity%of% Hazards% % More%Pain% and%Higher% Task%Effort% Constant' Urgency' Fewer% Problems%to% Fix% Stop%% and% Think% Mi8gate% the% Risk% Increased% Produc8vity% and% Innova8on% Safety' How are you adapting your decision habits?
The Fifth Mistake “We should just quit our jobs.” “Yeah, it’s hopeless.” What are we supposed to do?
Mistake #5: Feeling helpless instead of taking responsibility. Leadership is not a title, it’s a choice.
So what should we do instead?
Input: Decision Constraints Target: Optimize the Rate of Idea Flow short-term looplong-term loop 1. Visibility 2. Clarity 3. Awareness F ocus! Output: “Friction” in Idea Flow Improve Quality of Decisions Data-Driven Software Mastery
Input: Decision Constraints Target: Optimize the Rate of Idea Flow short-term looplong-term loop 1. Visibility 2. Clarity 3. Awareness F ocus! Output: “Friction” in Idea Flow Target - The direction of “better” Target: Optimize the Rate of Idea Flow Data-Driven Software Mastery
Input: Decision Constraints Target: Optimize the Rate of Idea Flow short-term looplong-term loop 1. Visibility 2. Clarity 3. Awareness F ocus! Output: “Friction” in Idea Flow Input - The constraints that limit our short-term choices… Data-Driven Software Mastery
Input: Decision Constraints Target: Optimize the Rate of Idea Flow short-term looplong-term loop 1. Visibility 2. Clarity 3. Awareness F ocus! Output: “Friction” in Idea Flow Output - The pain signal we’re trying to improve Data-Driven Software Mastery
Input: Decision Constraints Target: Optimize the Rate of Idea Flow short-term looplong-term loop 1. Visibility 2. Clarity 3. Awareness F ocus! Output: “Friction” in Idea Flow Focus on the biggest pain… F ocus! Data-Driven Software Mastery
Input: Decision Constraints Target: Optimize the Rate of Idea Flow short-term looplong-term loop 1. Visibility 2. Clarity 3. Awareness F ocus! Output: “Friction” in Idea Flow 1. Visibility - Identify the specific patterns 1. Visibility Data-Driven Software Mastery
Input: Decision Constraints Target: Optimize the Rate of Idea Flow short-term looplong-term loop 1. Visibility 2. Clarity 3. Awareness F ocus! Output: “Friction” in Idea Flow 2. Clarity - Understand cause and effect 2. Clarity Data-Driven Software Mastery
Input: Decision Constraints Target: Optimize the Rate of Idea Flow short-term looplong-term loop 1. Visibility 2. Clarity 3. Awareness F ocus! Output: “Friction” in Idea Flow 3. Awareness 3. Awareness - Stop and think to adjust habits Data-Driven Software Mastery
Input: Decision Constraints Target: Optimize the Rate of Idea Flow short-term looplong-term loop 1. Visibility 2. Clarity 3. Awareness F ocus! Output: “Friction” in Idea Flow 4. Run Experiments to Learn What Works Data-Driven Software Mastery
If any part of the feedback loop is broken:
My team spent tons of time working on improvements that didn’t make much difference. We had tons of automation, but the automation didn’t catch our bugs.
My team spent tons of time working on improvements that didn’t make much difference. We had well-modularized code, but it was still extremely time-consuming to troubleshoot defects.
The hard part isn’t solving the problems it’s identifying the right problems to solve. “What are the specific problems that are causing the team’s pain?”
Retrospective: “Mastery Circle” Circle Leader Circle Member Focus: What’s the problem to solve? Observe: Ask questions about the facts Conclude: Breakdown causes into patterns Optimize: Discuss strategies for improvement Learn: Run experiments to learn what works Observation Questions Make a F.O.C.O.L. Point!
What’s Next?
Emergent Practice Everything I’m showing today:
Stop Getting Crushed By Business Pressure Next Talk:
Talk #3: Let’s Make the PAIN Visible! #OpenDX An Open Standard for Measuring PAIN (Specification for Data Collection) Developer Experience
Talk #4: Learn Your Way to AWESOME. Input: Decision Constraints Target: Optimize the Rate of Idea Flow short-term looplong-term loop 1. Visibility 2. Clarity 3. Awareness F ocus! Output: “Friction” in Idea Flow
LEARN YOUR WAY TO AWESOME. Free to Join Industry Peer Mentorship Network openmastery.org
Industry Peer Mentorship Network Companies Community Groups HQ in Austin Open Mastery Austin meetup.com/Open-Mastery-Austin
Janelle Klein openmastery.org @janellekz Check out openmastery.org for details. Read my Book. Think About It. FREE with MembershipBuy It How to Measure the PAIN in Software Development Janelle Klein

More Related Content

PDF
Stop Getting Crushed By Business Pressure
byArty Starr
 
PDF
The Ultimate Metric
byArty Starr
 
PDF
Why #OpenDX?
byArty Starr
 
PDF
Data-Driven Software Mastery @Open Mastery Austin
byArty Starr
 
PDF
Let's Make the PAIN Visible!
byArty Starr
 
PPTX
Learn Your Way to AWESOME.
byArty Starr
 
PDF
A Programmer's Guide to Humans
byArty Starr
 
PDF
The Ultimate Metric
byArty Starr
 
Stop Getting Crushed By Business Pressure
byArty Starr
 
The Ultimate Metric
byArty Starr
 
Why #OpenDX?
byArty Starr
 
Data-Driven Software Mastery @Open Mastery Austin
byArty Starr
 
Let's Make the PAIN Visible!
byArty Starr
 
Learn Your Way to AWESOME.
byArty Starr
 
A Programmer's Guide to Humans
byArty Starr
 
The Ultimate Metric
byArty Starr
 

What's hot

PDF
Identify Development Pains and Resolve Them with Idea Flow
byTechWell
 
PDF
Let's Make the PAIN Visible!
byArty Starr
 
PDF
Bringing Science to Software Development
byArty Starr
 
PDF
Technical excellence - practices matter
byLeland Newsom CSP-SM, SPC5, SDP
 
PDF
The Rationale for Continuous Delivery by Dave Farley
byBosnia Agile
 
PDF
What We Learned from Three Years of Sciencing the Crap Out of DevOps
bySeniorStoryteller
 
PPTX
Testing for cognitive bias in ai systems
byPeter Varhol
 
PDF
How To (Not) Open Source - Javazone, Oslo 2014
bygdusbabek
 
PDF
The Rationale for Continuous Delivery
byPerforce
 
PDF
RecSysOps: Best Practices for Operating a Large-Scale Recommender System
byEhsan38
 
PDF
Dealing with Estimation, Uncertainty, Risk, and Commitment
byTechWell
 
PDF
[HCMC STC Jan 2015] Choosing The Best Of The Plan-Driven And Agile Developmen...
byHo Chi Minh City Software Testing Club
 
PPTX
Troublefree troubleshooting ian campbell sps jhb 2019
byIan Campbell
 
PDF
Agile bodensee - Agile Testing: Bug prevention vs. bug detection
byMichael Palotas
 
PPTX
Tools Won't Fix Your Broken DevOps
byNicole Forsgren
 
PDF
Good project from scratch - from developer's point of view
byPaweł Lewtak
 
PPTX
SecureWorld: Security is Dead, Rugged DevOps 1f
byGene Kim
 
PDF
A Rapid Introduction to Rapid Software Testing
byTechWell
 
PPTX
Theia H4D Stanford 2018
byStanford University
 
PDF
Am I a Brilliant Jerk?
byC4Media
 
Identify Development Pains and Resolve Them with Idea Flow
byTechWell
 
Let's Make the PAIN Visible!
byArty Starr
 
Bringing Science to Software Development
byArty Starr
 
Technical excellence - practices matter
byLeland Newsom CSP-SM, SPC5, SDP
 
The Rationale for Continuous Delivery by Dave Farley
byBosnia Agile
 
What We Learned from Three Years of Sciencing the Crap Out of DevOps
bySeniorStoryteller
 
Testing for cognitive bias in ai systems
byPeter Varhol
 
How To (Not) Open Source - Javazone, Oslo 2014
bygdusbabek
 
The Rationale for Continuous Delivery
byPerforce
 
RecSysOps: Best Practices for Operating a Large-Scale Recommender System
byEhsan38
 
Dealing with Estimation, Uncertainty, Risk, and Commitment
byTechWell
 
[HCMC STC Jan 2015] Choosing The Best Of The Plan-Driven And Agile Developmen...
byHo Chi Minh City Software Testing Club
 
Troublefree troubleshooting ian campbell sps jhb 2019
byIan Campbell
 
Agile bodensee - Agile Testing: Bug prevention vs. bug detection
byMichael Palotas
 
Tools Won't Fix Your Broken DevOps
byNicole Forsgren
 
Good project from scratch - from developer's point of view
byPaweł Lewtak
 
SecureWorld: Security is Dead, Rugged DevOps 1f
byGene Kim
 
A Rapid Introduction to Rapid Software Testing
byTechWell
 
Theia H4D Stanford 2018
byStanford University
 
Am I a Brilliant Jerk?
byC4Media
 

Viewers also liked

PPTX
CS5032 Lecture 5: Human Error 1
byJohn Rooksby
 
PPTX
Poka Yoka
bygoutam kumar
 
PPTX
5S, Kaizen, PokaYoke
byPrateek Singh Bapna
 
PPT
POKA-YOKE - A Lean Strategy to Mistake Proofing
byTimothy Wooi
 
PPT
10 Mistakes Brands & Causes Make When Creating Content
byBen Stroup
 
PPT
Poka yoke
byAshish Gupta
 
PDF
Poka yoke (mistake proofing)
byAnimesh Khamesra
 
PPTX
Poka Yoke Final Ppt
byJAGJITSINGH25
 
PDF
#Microposts16 - Comparing Social Media and Traditional Surveys around the Bos...
byCody Buntain
 
PPTX
BRN Symposium 03/06/06 Conclusions : The Microbiome in respiratory medicine
bybrnmomentum
 
PPTX
Contaminación ambiental
byclimanticacolmadrid
 
PPT
SEARCH MASTER
byPushpalatha Reddy
 
DOCX
Medical billing and coding specialist performance appraisal
bycodyfred747
 
PDF
Undang-Undang Nomor 22 Tahun 2009
byMuhammad Sirajuddin
 
DOC
RANJIT CV NEW
byranjit singh
 
PDF
Plumbing Suggestion That Will Save Your Family Some Cash
byinvincibleepic880
 
DOCX
Configuracion ,estilos y opciones
byshendry jaramillo
 
DOC
The Top Attractions in Chicago
by49ThingstoDo
 
PPTX
Top 8 associate manager resume samples
byjomwri
 
CS5032 Lecture 5: Human Error 1
byJohn Rooksby
 
Poka Yoka
bygoutam kumar
 
5S, Kaizen, PokaYoke
byPrateek Singh Bapna
 
POKA-YOKE - A Lean Strategy to Mistake Proofing
byTimothy Wooi
 
10 Mistakes Brands & Causes Make When Creating Content
byBen Stroup
 
Poka yoke
byAshish Gupta
 
Poka yoke (mistake proofing)
byAnimesh Khamesra
 
Poka Yoke Final Ppt
byJAGJITSINGH25
 
#Microposts16 - Comparing Social Media and Traditional Surveys around the Bos...
byCody Buntain
 
BRN Symposium 03/06/06 Conclusions : The Microbiome in respiratory medicine
bybrnmomentum
 
Contaminación ambiental
byclimanticacolmadrid
 
SEARCH MASTER
byPushpalatha Reddy
 
Medical billing and coding specialist performance appraisal
bycodyfred747
 
Undang-Undang Nomor 22 Tahun 2009
byMuhammad Sirajuddin
 
RANJIT CV NEW
byranjit singh
 
Plumbing Suggestion That Will Save Your Family Some Cash
byinvincibleepic880
 
Configuracion ,estilos y opciones
byshendry jaramillo
 
The Top Attractions in Chicago
by49ThingstoDo
 
Top 8 associate manager resume samples
byjomwri
 

Similar to Top 5 Reasons Why Improvement Efforts Fail

KEY
Acceptance testfurureinmind
byLeanDog
 
PPTX
Reviewing a Developer Experience
byArty Starr
 
PPTX
John Fodeh - Spend Wisely, Test Well
byTEST Huddle
 
PPT
Better Software Classic Testing Mistakes
bynazeer pasha
 
PPTX
QA overview
byRasha Bazbaz
 
PPTX
TDD All the Things!
byLiz Keogh
 
PPTX
Fundamentals of testing
byANDRI HAIRIYADI, S.Kom.
 
PDF
The Art of Better
byArty Starr
 
PDF
A Case for automated Tests
byAnja Pilz
 
PPT
Blackboxtesting 02 An Example Test Series
bynazeer pasha
 
PPTX
The Art of Testing Less without Sacrificing Quality @ ICSE 2015
byKim Herzig
 
PPT
Testing Framework
bynazeer pasha
 
PPTX
Fundamentals of Testing Chap 1 ISTQB Certification
byzubair khan shinwari
 
PDF
GDG Cloud Southlake #24: Arty Starr: Enabling Powerful Software Insights by V...
byJames Anderson
 
PPTX
Ch01-whyTest.pptx
bySteveIrwin25
 
PPT
Prvt file test
bydummyuser1analytics
 
PDF
Check upload1
byNitish Bhardwaj
 
PPTX
Testing Intelligence
byLalit Bhamare
 
PDF
Alexandra Schieren: It's Just...
byAnna Royzman
 
PPTX
Ch01-whyTest.pptx
byTaofeekAHammed
 
Acceptance testfurureinmind
byLeanDog
 
Reviewing a Developer Experience
byArty Starr
 
John Fodeh - Spend Wisely, Test Well
byTEST Huddle
 
Better Software Classic Testing Mistakes
bynazeer pasha
 
QA overview
byRasha Bazbaz
 
TDD All the Things!
byLiz Keogh
 
Fundamentals of testing
byANDRI HAIRIYADI, S.Kom.
 
The Art of Better
byArty Starr
 
A Case for automated Tests
byAnja Pilz
 
Blackboxtesting 02 An Example Test Series
bynazeer pasha
 
The Art of Testing Less without Sacrificing Quality @ ICSE 2015
byKim Herzig
 
Testing Framework
bynazeer pasha
 
Fundamentals of Testing Chap 1 ISTQB Certification
byzubair khan shinwari
 
GDG Cloud Southlake #24: Arty Starr: Enabling Powerful Software Insights by V...
byJames Anderson
 
Ch01-whyTest.pptx
bySteveIrwin25
 
Prvt file test
bydummyuser1analytics
 
Check upload1
byNitish Bhardwaj
 
Testing Intelligence
byLalit Bhamare
 
Alexandra Schieren: It's Just...
byAnna Royzman
 
Ch01-whyTest.pptx
byTaofeekAHammed
 

Recently uploaded

PDF
Metrics, Logs, Traces, and Mayhem_ Introducing an observability adventure gam...
byImma Valls Bernaus
 
PDF
Azure DevOps Services: Complete Guide to DevOps Tools, Automation & End-to-En...
byjohncarterjn
 
PDF
Project Tracking in Software Project Management
bySahanaBarveen1
 
PDF
Taxi App UIUX Design for Seamless Ride Booking
byV3CUBE TECHNOLABS
 
PDF
Monitoring your MySQL Server's Health Trends
byIgor Donchovski
 
PPTX
Salesforce Spring '26 Release presentation - Melbourne Salesforce User Group
byAnne N
 
PPTX
SAP S4 HANA Conversion and Migration .pptx
bysabeenakouser
 
PDF
apidays Amsterdam 2025 | Making AI work for you as an API Product Manager
byapidays
 
PDF
Attendance Dashboard – Time & Attendance Analytics
byHajir Pro
 
PDF
2026 Safety Roadmap: Systems, Skills, and Scale for EHS Leaders
byTECH EHS Solution
 
PDF
Track Phone Location on iPhone (2026)_ What Actually Works, What’s a Scam, an...
byOlive Dimitrescu
 
PPTX
HackYourBrain_JFokusConference_03022026.pptx
bySimonedeGijt
 
PPTX
BUSY Software Business Management Solution.pptx
byMoving Cloud
 
PDF
Image and video processing: From Mars to Hollywood with a stop at the hospita...
byHarinath Palavalli
 
PDF
On Demand Grocery Delivery App Development
byV3CUBE TECHNOLABS
 
PDF
apidays Amsterdam 2025 | APIs with a Purpose
byapidays
 
PPTX
Edison MuleSoft Meetup : Vibe Coding | MuleSoft Meetups
bySravan Lingam
 
PDF
Artificial Intelligence Planning (Harinath Palavalli)
byHarinath Palavalli
 
PDF
Massage Booking App Development Company
byV3CUBE TECHNOLABS
 
PPTX
Device Repair and Maintenance Management
byAxisTechnolabs
 
Metrics, Logs, Traces, and Mayhem_ Introducing an observability adventure gam...
byImma Valls Bernaus
 
Azure DevOps Services: Complete Guide to DevOps Tools, Automation & End-to-En...
byjohncarterjn
 
Project Tracking in Software Project Management
bySahanaBarveen1
 
Taxi App UIUX Design for Seamless Ride Booking
byV3CUBE TECHNOLABS
 
Monitoring your MySQL Server's Health Trends
byIgor Donchovski
 
Salesforce Spring '26 Release presentation - Melbourne Salesforce User Group
byAnne N
 
SAP S4 HANA Conversion and Migration .pptx
bysabeenakouser
 
apidays Amsterdam 2025 | Making AI work for you as an API Product Manager
byapidays
 
Attendance Dashboard – Time & Attendance Analytics
byHajir Pro
 
2026 Safety Roadmap: Systems, Skills, and Scale for EHS Leaders
byTECH EHS Solution
 
Track Phone Location on iPhone (2026)_ What Actually Works, What’s a Scam, an...
byOlive Dimitrescu
 
HackYourBrain_JFokusConference_03022026.pptx
bySimonedeGijt
 
BUSY Software Business Management Solution.pptx
byMoving Cloud
 
Image and video processing: From Mars to Hollywood with a stop at the hospita...
byHarinath Palavalli
 
On Demand Grocery Delivery App Development
byV3CUBE TECHNOLABS
 
apidays Amsterdam 2025 | APIs with a Purpose
byapidays
 
Edison MuleSoft Meetup : Vibe Coding | MuleSoft Meetups
bySravan Lingam
 
Artificial Intelligence Planning (Harinath Palavalli)
byHarinath Palavalli
 
Massage Booking App Development Company
byV3CUBE TECHNOLABS
 
Device Repair and Maintenance Management
byAxisTechnolabs
 

Top 5 Reasons Why Improvement Efforts Fail

Editor's Notes

  • #2 The really awesome thing about doing technical assessment and mentorship for a living, is for the last 5 years, I’ve been working on a research project on codifying the art of software development into a teachable craft
  • #4 [Read]
  • #5 We were trying to do all the right things. We had CI, unit testing, design reviews, code reviews. All that stuff you’re supposed to do.
  • #6 We were building this factory automation system that was responsible for detecting manufacturing problems then shutting down the tool responsible. I’d been on the project about 6 months, we were working through the final testing of a major release, tied a bow on it, shipped to production.
  • #7 Later that night we were on this conference call with IT. And I hear this guy just screaming in the background. Apparently, we had shut down every tool in the factory. So we rolled back the release and tried to figure out what happened. There was a configuration change that didn’t quite make it to production. We all felt terrible, but there wasn’t much we could do at this point. So we fixed the problem, and shipped to production... again.
  • #8 Back on the conference call with IT. And guess what... the same thing happened. What were we supposed to say… oops? So once again, we rolled back the release.
  • #9 We couldn’t reproduce the problem. We spent months trying to figure it out, and we were just completely stumped. We tried everything we could think of. Meanwhile, our development team was pretty much idle so management just told them to go ahead with the next release. Everyone was just working like nothing was wrong, but we couldn’t ship anything. We had a whole nother release in the queue before we finally figured it out. Guess what was wrong? We were scared to death to try again, but we didn’t really have a choice. So we cross our fingers, and shipped to production again.
  • #10 Back on the conference call with IT. We were all watching these real time activity charts and holding our breath. Finally everything seemed to be ok. I was so relieved that things would finally be back to normal again. And then about 3am, my phone rang. It was my team lead calling... he asked me about some code that I’d wrote and I knew exactly what happened. I’d made some improvements to the code that caused a memory leak. And my changes ground the system to a screeching halt. This time, it was my fault. So once again, we rolledback the release. This time the rollback failed.
  • #11 Fully ramped semiconductor factory. 50k wafer starts a day. Completely offline. My fault. I felt so horrible. I was in my bosses office, just sobbing.
  • #12 I thought the main obstacle was all the technical debt building up in the code base that was causing us to make mistakes. and if we made changes in the code that had more technical debt, we’d be more likely to make mistakes. So I got this idea to build a tool that could detect high-risk changes, and tell us where we needed to do more testing -- but what I found wasn’t what I found wasn’t what I expected at all.
  • #13 Our bugs were mostly in the code written by the senior engineers on the team where the design actually got the most scrutiny. It’s not like we didn’t have any awful crufty code -- but that’s not where the bugs were. The correlation I did find in the data was this...
  • #14 [read] And while that made some sense, I couldn’t help but think, there had to be more to the story...
  • #15 When I had to work with complex code, it was really painful. [read]
  • #16 So I started keeping track of all my painful interaction with the code and visualizing it on a timeline like this. The pain started [] when I ran into some unexpected behavior and ended [] when I had the problem resolved. So that was 5 hours and 18 minutes of troubleshooting, I think everyone would agree that’s pretty painful.
  • #17 The amount of pain was driven by two factors...
  • #18 So I started breaking down the problems into categories. And when I did this, I realized that most of the pain was actually caused by human factors.
  • #19 This is when I have an idea in my head about how the code is supposed to work, but it doesn’t work that way anymore.
  • #20 This is when your running an experiment, and there’s multiple possibilities for how a behavior can occur, and you make a bad assumption, and down the rabbit hole you go. These aren’t really problems with the code itself, [read]
  • #21 These aren’t really problems with the code itself… [read]
  • #22 The pain isn’t something inside the code, pain occurs during the process of interacting with the code. So I started optimizing for… and I did that, with the help of a data driven feedback loop.
  • #23 On our project, we ended up [read] For almost a year! [read]
  • #24 [read] Then we started asking []
  • #25 [read] That’s when everything changed [] We were finally able to turn the project around. And I learned one of the most valuable lessons in my career. [read]
  • #27 The pain was caused by problems building up in the code.
  • #28 A typical improvement effort usually starts with brainstorming a list [slow] We think about the things that bugged us recently, how we’re not following best practices, or the code that just makes us feel shameful. [] -- Then all that goes into our technical debt backlog, and we chip away at improvements for months. But just because a problem comes to mind, doesn’t mean it’s an important problem to solve When we’re brainstorming, [] we can easily miss our biggest problems then [our improvements don’t make...]. [] Don’t do this.
  • #29 From the outside it looks like we’re trying to drive a car without a steering wheel. We line up the car’s trajectory based on our ideals, then close our eyes and floor the gas pedal.
  • #30 Really wanted to help.
  • #31 Really wanted to help.
  • #32 ©2014 New Iron Group
  • #33 ©2014 New Iron Group
  • #34 ©2014 New Iron Group
  • #35 ©2014 New Iron Group
  • #36 ©2014 New Iron Group
  • #37 ©2014 New Iron Group
  • #38 ©2014 New Iron Group
  • #39 ©2014 New Iron Group
  • #41 The other thing I think we really need to question is [read] All of these things are really a means to an end. If you’ve ever spent hours and hours troubleshooting a bug in that one line of magic code, it really makes you wonder [read]. [read] What are we aiming for as an industry? So let me summarize what Idea Flow brings to the table.
  • #42 Since best practices are solution-focused, we’re always start with the hammer and looking for the nail. Test automation is our favorite hammer. Instead we need to be characterizing all the different types of nails,
  • #43 Since best practices are solution-focused, we’re always start with the hammer and looking for the nail. Test automation is our favorite hammer. Instead we need to be characterizing all the different types of nails,
  • #46 Really wanted to help.
  • #47 Really wanted to help.
  • #53 Find out what the substitution thing is called.
  • #54 Find out what the substitution thing is called.
  • #59 We optimize for execution time, even when the time spent on human cycles can completely dwarf the execution time. Why do you think that is?
  • #60 When it comes to solving these really complex problems, our intuition is just wrong. It leads us astray.
  • #62 The pain isn’t something inside the code, pain occurs during the process of interacting with the code. The problems I focused on fundamentally changed.
  • #63 Avoiding Pain
  • #65 Rework Risk is driven by the likelihood... Things like... bad assumptions about the architecture or design or bad assumptions about customer requirements. The longer we delay before making corrections, the greater the rework.
  • #66 This is from a project about 10 months old where we actively focused on reducing troubleshooting time. With our everyday problem-solving effort, we still spent about 10-20% of our time on friction.
  • #67 So in this first case study, there was a huge mess inherited by a really great team, it was a 12 year old project, where all the original developers had left. This is what it looks like when 90% of your time figuring out what to do, and 10% of your time actually doing stuff. The lack of familiarity has an enormous impact on how much friction we experience.
  • #68 So there were tons of problems, and the team wasn’t sure what to focus on, so they set a goal to raise unit test coverage by 5%. If you start adding up all the problems across the team [], these guys were spending about 700 hours per month generating test data to support whatever task they were working on. But oddly, in all the retrospective meetings, this problem didn’t even come up. It was just part of the work.
  • #69 This second case studies was with a massive rewrite effort. They had this big monolith application that they rewrote completely from scratch, with micro services, a continuous delivery pipeline, the whole nine years. And what really surprised me about this project, is that after only 18 months, they were already spending 40-60% of their development capacity troubleshooting problems.
  • #70 They had this design for the architecture, that looked good on paper, but then once they distributed the design across teams, and discovered the architecture had some flaws, they were stuck. The good ol’ Conway’s law effect, and they couldn’t seem to adapt. So I got involved with the team, just as they were getting into the thrashing stage, and starting to lose control.
  • #71 You could see this pattern of pain building up over time, that we always talk about, but have never been able to measure. So I don’t have quite enough data to make a chart like this, but these are some of the patterns you could see.
  • #72 First, learning is front-loaded while the team figures out what to do.
  • #73 Then there’s this rush before the deadline where validation ends up deferred.
  • #74 Then the pain builds, and you see the baseline friction level rising over time.
  • #75 Then finally chaos reigns, and the unpredictable work stops fitting in the timebox. So I’m measuring capacity hours over time, so even though all these releases are the same size, you can see how the team had to work twice as many hours to get the release out the door.
  • #76 Then management got resentful because nothing was actually getting better.
  • #77 Now I want to point out something. For all three projects, these tasks all took one to three days. Generally speaking, as the problems build, we can still break down the work into bite-sized chunks. but what we work on during that time dramatically changes. [read] even when the problems are severe. So if you thought about how much time you spend doing troubleshooting, learning and rework. What percentage of time do you think it would be? Which do you think is the biggest? What do you think the biggest causes are of troubleshooting time?
  • #78 So If I wanted to know what was causing the pain I needed to understand the things that caused these 2 factors. A lot of the problems had more to do with human factors than anything going on with the code. Stale Memory mistakes, Ambiguous Clues. But once I understood what was causing the pain, [read -- most of the problems were easy to avoid] For example...
  • #80 Really wanted to help.
  • #81 Really wanted to help.
  • #84 We’d always get a different set of bugs. What would you do?
  • #86 Troubleshooting Risk we’ve already talked about, it’s driven by the likelihood...
  • #87 So If I wanted to know what was causing the pain I needed to understand the things that caused these 2 factors. A lot of the problems had more to do with human factors than anything going on with the code. Stale Memory mistakes, Ambiguous Clues. But once I understood what was causing the pain, [read -- most of the problems were easy to avoid] For example...
  • #88 For the problem categories -- I use hashtags in the Idea Flow Maps, then add up the durations for each hashtag.
  • #89 For the problem categories -- I use hashtags in the Idea Flow Maps, then add up the durations for each hashtag.
  • #90 For the problem categories -- I use hashtags in the Idea Flow Maps, then add up the durations for each hashtag.
  • #91 [read] And while that made some sense, I couldn’t help but think, there had to be more to the story...
  • #92 [read] And while that made some sense, I couldn’t help but think, there had to be more to the story...
  • #93 We’d always get a different set of bugs. What would you do?
  • #95 Really wanted to help.
  • #96 Really wanted to help.
  • #98 So we’re all familiar with the haystack principle…
  • #99 Then I realized George was missing a key conceptual model.
  • #100 N
  • #101 Rework Risk is driven by the likelihood... Things like... bad assumptions about the architecture or design or bad assumptions about customer requirements. The longer we delay before making corrections, the greater the rework.
  • #102 Used thinking checklists to codify a decision-making process… let me show you what I mean.
  • #107 Focus on one decision principle until you have it down.
  • #108 It’s not that best practices are bad, or wrong, they’re just backwards.
  • #110 Really wanted to help.
  • #111 Really wanted to help.
  • #112 We start off with the best of intentions…  We're going to write high quality code that’s low in technical debt, easy to maintain and of course, has good code coverage.
  • #113 So I started working 60-70 hour weeks for about 6 months strait. And my team started working 60 hour weeks for 6 months strait. Then the releases started falling apart. Things were just constantly going wrong.
  • #114 Crazy deadlines, and I tried to explain to management that we needed to go slower, but they threatened to outsource the project if we didn’t get it done. This project was my baby.
  • #115 When we fall into urgency mode, we start compromising safety for speed. We make decisions that don’t seem like a big deal at the time, but they create a hazardous work environment. Instead of taking a little more time to put our toys away, we end up falling down the stairs and in the hospital.
  • #116 First, we ignore the risks, basically ignoring tango -- a lot of times because the risk isn’t obvious.
  • #117 First, we ignore the risks, basically ignoring tango -- a lot of times because the risk isn’t obvious.
  • #118 First, we ignore the risks, basically ignoring tango -- a lot of times because the risk isn’t obvious.
  • #119 First, we ignore the risks, basically ignoring tango -- a lot of times because the risk isn’t obvious.
  • #120 First, we ignore the risks, basically ignoring tango -- a lot of times because the risk isn’t obvious.
  • #121 Troubleshooting Risk we’ve already talked about, it’s driven by the likelihood...
  • #122 When we fall into urgency mode, we start compromising safety for speed. We make decisions that don’t seem like a big deal at the time, but they create a hazardous work environment. Instead of taking a little more time to put our toys away, we end up falling down the stairs and in the hospital.
  • #123 Next thing you know we’re working late nights and weekends, choking down red bull to stay awake... hacking out last minute fixes and hoping that nothing else breaks. Who’s done this before? We make jokes about programmers running on caffeine and pizza... but this problem is really serious. When our project is on the line, we give up a lot -- we’re skipping our kids recitals, missing our annivery dinners, we get sick, we gain weight stress deteriorates our health and can tear apart our relationships. Just because we’re not bleeding doesn’t mean we don’t get hurt by all this. We can’t run a sustainable business by compromising the safety of the people doing the work.
  • #124 I watched my project get crushed.
  • #125 [read] What do you guys think? What are the biggest obstacles?
  • #128 If our feedback loop is broken, we don’t respond.
  • #130 Troubleshooting Risk we’ve already talked about, it’s driven by the likelihood...
  • #131 Learning Risk is driven by the likelihood... Things like... lots of 3rd party libraries, complex frameworks, a really large code base, or high turn-over rate -- all these things can cause extra learning work.
  • #132 Rework Risk is driven by the likelihood... Things like... bad assumptions about the architecture or design or bad assumptions about customer requirements. The longer we delay before making corrections, the greater the rework.
  • #133 We make decisions that save a few hours that lead to side effects that cost several hours. When we try to go faster, we do things that increase the likelihood of mistakes and the cost to recover when things go wrong. We’ve been in this pattern for the last 2 years, and now we’re here.
  • #135 So If I wanted to know what was causing the pain I needed to understand the things that caused these 2 factors. A lot of the problems had more to do with human factors than anything going on with the code. Stale Memory mistakes, Ambiguous Clues. But once I understood what was causing the pain, [read -- most of the problems were easy to avoid] For example...
  • #136 We’ve been collecting lots of data and have identified our biggest problems. I think we can dramatically reduce risk with some focused effort. I'd like to propose a 3-month trial [] with one person working full-time on these problems. The team will make decisions on the improvement work, and I’ll share our progress and lessons learned with you each month. I know we can do this, but I need your help. Will you help me make this happen?
  • #138 Really wanted to help.
  • #147 From the outside it looks like we’re trying to drive a car without a steering wheel. We line up the car’s trajectory based on our ideals, then close our eyes and floor the gas pedal.
  • #148 [read] That’s when everything changed [] We were finally able to turn the project around. And I learned one of the most valuable lessons in my career. [read]
  • #149 Iterative clarify then implement “better”
  • #150 Iterative clarify then implement “better”
  • #151 If you want to join me, then read the book, and think about the ideas, see if this is something you want to be a part of. You can either buy the book, or if you start a reading group for Idea Flow, I’ll provide free e-books for all the attendees. Check out openmastery.org for details.
  • #152 And if you don’t want to take my word for it, you should read Idea Flow because Rene and Matt said so.