With the explosion of new frameworks, a mountain of automation, and our applications distributed across hundreds of services in the cloud, the level of complexity in software development is growing at an insane pace. With increased complexity comes increased costs and risks. When diagnosing unexpected behavior can take days, weeks, or sometimes months, all while our release is on the line, our projects plunge into chaos. In the invisible world of software development, how do we identify what's causing our pain? How do we escape the chaos? Janelle Klein presents a novel approach to measuring the chaos, identifying the causes, and systematically driving improvement with a data-driven feedback loop. Rather than measuring the problems in the code, Janelle suggests measuring the "friction in Idea Flow", the time it takes a developer to diagnose and resolve unexpected confusion, which disrupts the flow of progress during development. With visibility of the symptoms, we can identify the cause—whether it's bad architecture, collaboration problems, or technical debt. Janelle discusses how to measure Idea Flow, why it matters, and the implications for our teams, our organizations, and our industry.

1. K5
Keynote
6/8/2017 4:15:00 PM
K5 Identify Development Pains and
Resolve Them with Idea Flow
Presented by:
Janelle Klein
Open Mastery
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073
888-­‐268-­‐8770 ·∙ 904-­‐278-­‐0524 - info@techwell.com - https://www.techwell.com/

2. Janelle Klein
Open Mastery
Janelle Klein is a No Fluff Just Stuff speaker; author of Idea Flow; and founder of
Open Mastery, an industry collaborative tools foundation focused on data-driven
software mastery. She founded Open Mastery to rally the industry to work
together and learn to break down the walls of ignorance between managers and
developers that drive software projects into the ground. By making the pain
visible with Idea Flow, there is a universal definition of effective practice, a
language for sharing experiences, and an opportunity to learn together like never
before. Janelle's development background spans financial core processors,
factory automation, supply chain optimization, and statistical process control.
Her consulting has focused on continuous delivery infrastructure, database
automation, test automation strategies, and helping organizations identify and
solve their biggest problems.

3. Janelle Klein
openmastery.org @janellekz
Flow I went to college to be a song-writer…
REALITY CHECK
Career in Music?
@janellekz

4. “Ooh, x86 Assembly! That sounds fun!”
Let’s take a class together!
@janellekz
Realization:
I can build anything I can dream…
“Software development is the closest thing in this world to magic.”
— Denisse Osorio de Large

5. The World of Business
We Start with the Best of Intentions
High Quality Code
Low Technical Debt
Easy to Maintain
Good Code Coverage
@janellekz

6. Constant Urgency Four Stages of Chaos
Product Owner: “We’ve got more important things to do.”
Deferring(
Problems(
@janellekz

7. Deferring(
Problems(
Painful(
Releases(
Manager: “Good job everyone! Keep up that great work ethic!”
Four Stages of Chaos
@janellekz
Deferring(
Problems(
Painful(
Releases(
Thrashing)
Manager: “We need to go faster! Let’s hire more developers.”
Four Stages of Chaos
@janellekz

8. Deferring(
Problems(
Painful(
Releases(
Thrashing)
Project(
Meltdown(
Developer: “I give up. I don’t care anymore if the project fails.”
Four Stages of Chaos
@janellekz
RESET
Would you like to do it all over again?
@janellekz

9. Every few years we rewrite our software…
Start%
Over%
Unmaintainable%
So0ware%
Across the Industry
@janellekz
Our Solution…

10. RESET
“A description of the goal is not a strategy.”
-- Richard P. Rumelt
What’s wrong with our current strategy? Our “Strategy” for Success
High Quality Code
Low Technical Debt
Easy to Maintain
Good Code Coverage
@janellekz

11. RESET
“A good strategy is a specific and coherent response to—
and approach for overcoming—the obstacles to progress.”
-- Richard P. Rumelt
The problem is we don’t have a strategy... Obstacle #1:
Managers and Engineers might as well
be speaking different languages
$$$$ $$$$
PAIN PAIN
@janellekz

12. Obstacle #2:
We can’t see what we’re doing wrong…
@janellekz
Strategy to Overcome Obstacles
How to measure the PAIN
in Software Development
Janelle Klein
leanpub.com/ideaflow
Translate PAIN into the language of $$$$
@janellekz

13. SPC
Looks okay.
Alert!
Measurements
Tools
Statistical Process Control (SPC)
Semiconductor
@janellekz
Great Team
Disciplined with Best Practices
Constantly Working on Improvements+
Project FAILURE
Statistical Process Control (SPC)
Semiconductor
@janellekz

14. The Retrospective
Our biggest problem
“More automation…?”
What are we going to do?
We’ve got to fix
all this Technical Debt!
@janellekz
“Without data
you’re just
another person
with an opinion.”
- W. Edwards Deming
@janellekz

15. Technical Debt Mistakes
I thought the problem was
Technical Debt
@janellekz
SPC
Looks okay.
Alert!
Measurements
Tools
Config UI
code
Charting UI
Code
Statistical Process Control (SPC)
System Architecture
@janellekz

16. Beautiful
Looks okay.
Alert!
Measurements
Tools UGLY UGLY
Statistical Process Control (SPC)
System Architecture
@janellekz
We made significantly more mistakes
in code that we didn’t write ourselves.
Lower
Familiarity
More
Mistakes=
There had to be more to the story...
@janellekz

17. Complex(
So*ware(
This is what I knew...
What made development feel painful?
PAIN
@janellekz
Unexpected
Behavior
Problem
Resolved
Tracking Painful Experience with the Code
Troubleshooting
Progress
5 hours and 18 minutes of troubleshooting...
PAINFUL

18. The amount of PAIN was caused by…
Likeliness(of((
Unexpected(
Behavior(
Cost(to(Troubleshoot(and(Repair(
High(Frequency(
Low(Impact(
Low(Frequency(
Low(Impact(
Low(Frequency(
High(Impact(
PAIN(
@janellekz
What Causes Unexpected
Behavior (likeliness)?
What Makes Troubleshooting
Time-Consuming (impact)?
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
Most of the pain was caused by human factors.
What causes PAIN?

19. What Causes Unexpected
Behavior (likeliness)?
What Makes Troubleshooting
Time-Consuming (impact)?
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
What causes PAIN?
Most of the pain was caused by human factors.
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
What Causes Unexpected
Behavior (likeliness)?
What Makes Troubleshooting
Time-Consuming (impact)?
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
What causes PAIN?
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
Most of the pain was caused by human factors.

20. PAIN occurs during the process of
understanding and extending the software
Complex(
So*ware(
PAIN
Not the Code.
Optimize “Idea Flow”
@janellekz
Technical Debt
Technical Debt is a “Thing”
Blame:
Visible Object
“Gut feel” maps to an Object-Oriented paradigm

21. What Causes Unexpected
Behavior (likeliness)?
What Makes Troubleshooting
Time-Consuming (impact)?
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
Friction() in the Context of Idea Flow
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
“Gut feel” re-maps to a Flow-Oriented paradigm
My team spent tons of time working on
improvements that didn’t make much difference.
We had tons of automation, but the
automation didn’t catch our bugs.
@janellekz

22. My team spent tons of time working on
improvements that didn’t make much difference.
We had well-modularized code,
but it was still extremely time-consuming to troubleshoot defects.
@janellekz
The hard part isn’t solving the problems
it’s identifying the right problems to solve.
“What is the specific friction
that is causing the team’s pain?”

23. Realization:
Lean Agile
Metaphorical Mapping Error
We really screwed this up…
@janellekz
Result:
Object-Oriented Lean
Flow-Oriented Lean
Flow
@janellekz

24. Flow
Metaphorical Lens for observing the world
@janellekz
“We practice Flow”
We continuously optimize one or more Flow Systems using Flow Data Science,
and actively contribute to the Flow Community’s body of explicit know how
@janellekz

25. Flow Data Science
1. What is a Flow System?
@janellekz
A Flow System defines an explicit scope
“Socio-Technical System”
Software
Nancy Leveson
@janellekz

26. 2. How do we measure Friction()?
Flow Data Science
1. What is a Flow System?
@janellekz
How to measure the PAIN
in Software Development
Janelle Klein
“Idea Flow Mapping” is a visualization technique
that teaches us to recognize Flow
@janellekz

27. IdeaFlowDX Profiler
Intellij Plugin + Profiler (OSS)
Patterns over Time
Situational context: state of files, state of tests, etc
Track movements: type, navigate, execute, etc
@janellekz
Intellij Plugin + Profiler (OSS)
RAGE
IdeaFlowDX Profiler
duration
Patterns over Time
@janellekz

28. Developer is Productive
Developer is Learning
Intellij Plugin + Profiler (OSS)
IdeaFlowDX Profiler
@janellekz @janellekz
Intellij Plugin + Profiler (OSS)
IdeaFlowDX Profiler
Informed Answer: What’s the biggest problem to solve?

29. Why THESE specific metrics?
Theory of Productivity
@janellekz
Statistical Process Control (Manufacturing)
3. Feedback Loop
1. Optimization Target
2. Chaos Signal
Sensor Data
Rules
Engine
First Principle: Measure “Chaos” as a deviation from “Target”

30. What is “Chaos”?
Earthquake Patterns
@janellekz
What is a “Target”?
Plato’s “Ideal Form”
@janellekz

31. Quality Target
Lower Control Limit
Upper Control Limit
X"
Lower Variability => Better Control
Perfect Quality
Upper Control Limit
Lower Control Limit
Out of Control (OOC): Root Cause Analysis
SPC “Quake” Charts
@janellekz
3. Feedback Loop
1. Optimization Target
2. Chaos Signal
Sensor Data
Rules
Engine
Statistical Process Control (Software)
@janellekz

32. 3. Feedback Loop
1. Optimization Target
2. Pain Signal
Rules Engine
Statistical Process Control (Software)
Idea Flow
@janellekz
MTTU
Mean time to understand the unfamiliar
Mean time to resolve confusion
MTTR
MTTE
Mean time to build an experiment
Software “Quake” Metrics

33. Optimal Friction
Upper Control Limit
X"
“Out of Control”
20min
0m
30m
0m Perfect Flow
PAIN
Limit
42 min QUAKE
Lower Variability => Better Control
“Monitor the Quakes”
@janellekz
2. How do we measure Friction()?
Flow Data Science
1. What is a Flow System?
3. What does a project quake look like?
@janellekz

34. 18 months after a Micro-Services/Continuous Delivery rewrite
Troubleshooting
Progress
Learning
40-60% of dev capacity on Friction
0:00 28:15
12:230:00
Case Study: From Monolith to Microservices The Architecture Looked Good on Paper!
Team A Team B Team C
Complexity Moved Here
%$#?! %$#^?!
Fix TechDebt! Fix TechDebt! Fix TechDebt!
More Automated Tests!

35. Project Quake (Thrashing)
0%
100%
Release 1 Release 2 Release 3
Troubleshooting
Productivity
Learning
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
(extrapolated from samples)
0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
Figure out what to do
Learning is front-loaded
Troubleshooting
Productivity
Learning
Project Quake (Thrashing)

36. 0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
Rush Before the Deadline
Validation is Deferred
Project Quake (Thrashing)
Troubleshooting
Productivity
Learning
0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
Pain Builds
Baseline friction keeps rising
Project Quake (Thrashing)
Troubleshooting
Productivity
Learning

37. 0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
Chaos Reigns
Unpredictable work stops
fitting in the timebox
Troubleshooting
Productivity
Learning
Project Quake (Thrashing)
The cost of bad architecture
in the microservices world
are EXTREMELY HIGH
Visibility gives us a way to
manage non-local optimization
@janellekz

38. Application
I don’t
UNDERSTAND!
Quakes cascade through the supply chain
Usability
Problems
@janellekz
Application
I don’t
UNDERSTAND!
90% of our software
is 3rd party components
Usability
Problems
@janellekz

39. Application
Feedback
Feedback
Feedback
What if we shared our pain signals upstream?
I don’t
UNDERSTAND!
Usability
Problems
@janellekz
Application
Pressure
for better Idea Flow
Usability
Problems
Feedback
Feedback
Feedback
I don’t
UNDERSTAND!
What if we shared our pain signals upstream?

40. Everything is
AWESOME!
Application
Pressure
for better Idea Flow
Feedback
Feedback
Feedback
What if we shared our pain signals upstream?
Shared dumping ground of crappy software
I don’t
UNDERSTAND!
I don’t
UNDERSTAND!
I don’t
UNDERSTAND!
Non-Local optimization is
the challenge of our generation
@janellekz

41. 2. How do we measure Friction()?
Flow Data Science
1. What is a Flow System?
3. What does a project quake look like?
4. How do we reduce the quakes?
@janellekz
3. Feedback Loop
1. Optimization Target
2. Pain Signal
Rules Engine
Statistical Process Control (Software)
Idea Flow
”Continuous Improvement” = Optimize Quality of Decisions

42. Product(
Requirements( So1ware(Task(
Possible(Decision(Paths(
So1ware(
Software is a Reflection of our Decisions
@janellekz
Product(
Requirements( So1ware(Task(
High%Risk%
Decision%Habits%
So1ware(
(and(all(our(problems)(
(
RESET
Software is a Reflection of our Decisions
@janellekz

43. Product(
Requirements( So1ware(Task(
High%Risk%
Decision%Habits%
So1ware(
(and(all(our(problems)(
(
Technical Debt is really a Symptom of the Problem
Software is a Reflection of our Decisions
Decision Optimization
@janellekz
How do we optimize decisions?
Likeliness(of((
Unexpected(
Behavior(
Cost(to(Troubleshoot(and(Repair(
High(Frequency(
Low(Impact(
Low(Frequency(
Low(Impact(
Low(Frequency(
High(Impact(
PAIN(
Reduce Likeliness
=
Quake Proofing
Reduce Recovery Cost = Quake Tolerance
@janellekz

44. Time%
Pressure%
Compromise%
Safety%
for%
Speed%
Increase%
Number%&%
Severity%of%
Hazards%
%
More%Pain%
and%Higher%
Task%Effort%
Constant'
Urgency'
Cycle of Chaos
High-Risk Decision Habits
ProblemsQuakes
@janellekz
Time%
Pressure%
Compromise%
Safety%
for%
Speed%
Increase%
Number%&%
Severity%of%
Hazards%
%
More%Pain%
and%Higher%
Task%Effort%
Constant'
Urgency'
Cycle of Chaos
High-Risk Decision Habits
ProblemsQuakes
@janellekz

45. Time%
Pressure%
Compromise%
Safety%
for%
Speed%
Increase%
Number%&%
Severity%of%
Hazards%
%
More%Pain%
and%Higher%
Task%Effort%
Constant'
Urgency'
Cycle of Chaos
High-Risk Decision Habits
ProblemsQuakes
@janellekz
Time%
Pressure%
Compromise%
Safety%
for%
Speed%
Increase%
Number%&%
Severity%of%
Hazards%
%
More%Pain%
and%Higher%
Task%Effort%
Constant'
Urgency'
Cycle of Chaos
High-Risk Decision Habits
ProblemsQuakes
@janellekz

46. Time%
Pressure%
Compromise%
Safety%
for%
Speed%
Increase%
Number%&%
Severity%of%
Hazards%
%
More%Pain%
and%Higher%
Task%Effort%
Constant'
Urgency'
Cycle of Chaos
High-Risk Decision Habits
ProblemsQuakes
@janellekz
Fewer%
Problems%to%
Fix%
Stop%%
and%
Think%
Mi8gate%
the%
Risk%
Increased%
Produc8vity%
and%
Innova8on%
Safety'
Cycle of Safety
Low-Risk Decision Habits
Slack Time
QuakesQuakes
@janellekz

47. Fewer%
Problems%to%
Fix%
Stop%%
and%
Think%
Mi8gate%
the%
Risk%
Increased%
Produc8vity%
and%
Innova8on%
Safety'
Cycle of Safety
Low-Risk Decision Habits
Slack Time
QuakesQuakes
@janellekz
Fewer%
Problems%to%
Fix%
Stop%%
and%
Think%
Mi8gate%
the%
Risk%
Increased%
Produc8vity%
and%
Innova8on%
Safety'
Cycle of Safety
Low-Risk Decision Habits
Slack Time
QuakesQuakes
@janellekz

48. Fewer%
Problems%to%
Fix%
Stop%%
and%
Think%
Mi8gate%
the%
Risk%
Increased%
Produc8vity%
and%
Innova8on%
Safety'
Cycle of Safety
Low-Risk Decision Habits
Slack Time
QuakesQuakes
@janellekz
Fewer%
Problems%to%
Fix%
Stop%%
and%
Think%
Mi8gate%
the%
Risk%
Increased%
Produc8vity%
and%
Innova8on%
Safety'
Cycle of Safety
Low-Risk Decision Habits
Quakes
Slack Time
@janellekz

49. Quality Target
Lower Control Limit
Upper Control Limit
X"
Perfect Quality
Upper Control Limit
Lower Control Limit
OOC Alarm: Root Cause Analysis
SPC “Quake” Charts (Manufacturing)
Learn
@janellekz
#Symptom
#Symptom
#Symptom
#Cause
#Cause
#Fix
#Fix
#Fix
#Fix#Cause
#Cause
#Symptom
OOC Alarm: Root Cause Analysis
@janellekz

50. #Symptom
#Symptom
#Symptom
#Cause
#Cause
#Fix
#Fix
#Fix
#Fix#Cause
#Cause
#Symptom
OOC Alarm: Root Cause Analysis
@janellekz
Automate: Data-Driven Rules Engine
#Symptom
#Symptom
#Cause
#Cause
#Fix
#Fix
Quake
Monitor
Danger?
Trigger
Actions
Rules Engine
@janellekz

51. Temporal Semantic SPC
Generalized AI Platform
Flow paradigm applied to AI
Synthesis: Theory of Consciousness
@janellekz

52. Idea Flow
3. Feedback Loop
1. Optimization Target
2. Pain Signal
Rules Engine
Streaming
Monitor
Danger?
Trigger
Analytics Job
Real-Time Streaming SPC Platform
@janellekz
Temporal Semantic SPC
Trusted Buddy
Network
wifi
mic
speaker
headphones
“What if you checked the config?
That’s what broke last time I saw that error…”
Pair Programming Buddy

53. Temporal Semantic SPC
Trusted Buddy
Network
wifi
mic
speaker
headphones
“Jerry’s been stuck for an hour
on your EmailTemplate code…
maybe you can help?”
XP:
Tony’ Buddy Lvl. 7
Pair Programming Buddy Idea Flow “Story Frames”
Ticker Tape
Sensory
Receiver
/ideaflow
Patterns
I don’t
UNDERSTAND!
RAGE
@janellekz

54. Situational Deep Learning
“StoryWeb” Semantic Graphs
ESV Monitors and NLP Feedback
Patterns
Temporal Semantic SPC Platform
RAGE
Empathy
@janellekz
Imagine a world where you are not only
building AI into your software,
you’re building your software with AI
Next-Generation Meta-Programming
Empathy
@janellekz

55. AI that asks the right question
at the right time
Design Goal: Amplify the Humans
@janellekz
What does this mean for our future?
“Umm, you just said you’re building Skynet…?”
@janellekz

56. Our choices from this moment onward…
MATTER A LOT
What does this mean for our future?
@janellekz
Power = Electricity

57. x
Infinite = Creativity
World’s Greatest Magic Trick
@janellekz

58. How to measure the PAIN
in Software Development
Janelle Klein
Step 1. Read my Book
leanpub.com/ideaflow/c/MAGIC
FREE!
@janellekz
To Understand Architecture…
1.
2.
3. 4.
5.
6.
@janellekz

59. Step 2. We practice Flow
Pattern: Cycle of Chaos
@janellekz
Pattern: Cycle of Safety
Visibility => Empathy => Clarity => Alignment
ArtistsArtists
ArtistsArtists
@janellekz
Step 2. We practice Flow

60. CommunityWare Platform for Flow
Brian Sletten Miko Matsumura Kara RawsonJanelle Klein
Co-Founder R&D Team
@janellekz
Step 3. We keep going!
Flow
@janellekz

61. Long Term Roadmap
Software
Battle Mage
Mentorship AI for Software Craftsmanship
Infinite Support for Creative Economy
DreamerOS
Runtime
VR Dream
Simulator
@janellekz
Janelle Klein
janelle@openmastery.org @janellekz
Janelle Klein
Founder of Open Mastery
janelle@openmastery.org
Join Us:
openmastery.org
FREE!
MAGIC