SlideShare a Scribd company logo
@LMaccherone @TheAgileCraft
@LMaccherone @TheAgileCraft
Why don’t coaches go for it more often?
@LMaccherone @TheAgileCraft @LMaccherone @TheAgileCraft
We don't see things
the way they are.
We see things
the way we are.
~The Talmud
@LMaccherone @TheAgileCraft
@LMaccherone @TheAgileCraft
Bias eats good decisions
for breakfast, lunch, and
dinner
By understanding probabilistic
decision making, we learn to
trust and overcome bias
@LMaccherone @TheAgileCraft
Larry
Maccherone
@LMaccherone
@LMaccherone @TheAgileCraft
Every decision is a
forecast!
@LMaccherone @TheAgileCraft
You are forecasting that
your choice will have better
outcomes than the other
alternatives
@LMaccherone @TheAgileCraft
Group  decisions
@LMaccherone @TheAgileCraft
Argument  is  about  who is  right.
Decision  making  is  about  what is  right.
@LMaccherone @TheAgileCraft
1.  Different  Models
2.  Different  Values
3.  Different  Risk  Tolerance
Why do people disagree?
favor different alternatives
Fear-based decision making
@LMaccherone @TheAgileCraft
Models  and  Values
§ Models  calculate  probability  in  terms  of  proxy  variables
§ Values  translate  those  probabilities  into  money
§ Different  models  example:
§ Joe  forecasts  that  alternative  A  will  make  the  most  money
§ Sally  forecasts  that  alternative  B  will  make  the  most  money
§ Different  values  example:
§ Betty  favors  the  alternative  with  higher  quality
§ George  favors  the  alternative  that  will  get  to  market  faster
@LMaccherone @TheAgileCraft
So…
quality  of  decision  depends  upon:
1.  alternatives  considered,  and
2.  models  used  to  forecast  the
outcome  of  those  alternatives.
Probabilistic  models  are  superior
@LMaccherone @TheAgileCraft
@LMaccherone @TheAgileCraft
For  a  given  alternative,  let:
Pg =  Probability  of  good  thing  happening
Vg =  “Value”  of  good  thing  happening
Then:
Value  of  the  alternative  =  Pg × Vg
@LMaccherone @TheAgileCraft
An  
lean/agile  product  
management  example
@LMaccherone @TheAgileCraft
$8M
Best  case  
(25%)
$1M
Likely  case  
(50%)
$1M
Worst  case  
(25%)
1
$2M$2M$1M
2
Which  strategy  is  best…
…for  your  company?
PW × VW =  .25 × -­$1.00M  =  -­$0.25M
PL × VL =  .50 × $1.00M  =   $0.50M
PB × VB =  .25 × $8.00M  =    $2.00M  
-­-­-­-­-­-­-­-­-­-­-­
$2.25M
…for  your  career?
PW × VW =  .25 × $1.00M  =    $0.25M
PL × VL =  .50 × $2.00M  =   $1.00M
PB × VB =  .25 × $2.00M  =    $0.50M  
-­-­-­-­-­-­-­-­-­-­-­
$1.75M
@LMaccherone @TheAgileCraft
If  you  get  only  1 project  then  
strategy  2  is  better
75%  of  the  time
If  you  get  ∞ projects  then
strategy  1  is  better
100%  of  the  time
How  many  projects  do  you  need  for  
strategy  1  to  be  better  
more  often  than  not?
@LMaccherone @TheAgileCraft
@LMaccherone @TheAgileCraft
Play with it yourself at:
http://jsfiddle.net/lmaccherone/j3wh61r7/
@LMaccherone @TheAgileCraft
@LMaccherone @TheAgileCraft
Emotion  and  bias  plays  a  part
@LMaccherone @TheAgileCraft
Did  any  of  you  get  emotional  
about  the  $1M  loss?
Did  any  of  you  want  to  
question  the  $8M  number?
We’ve totally…
…eliminated fear from the equation
…changed the nature of the conversation
@LMaccherone @TheAgileCraft
Argument  is  about  who is  right.
Decision  making  is  about  what is  right.
@LMaccherone @TheAgileCraft
Getting
probability
input you
can trust
@LMaccherone @TheAgileCraft
Trained/Calibrated
Untrained/Uncalibrated
Statistical  Error
“Ideal” Confidence
30%
40%
50%
60%
70%
80%
90%
100%
50% 60% 80% 90% 100%
25
75 71 65 58
21
17
68 152
65
45
21
70%
Assessed  Chance  Of  Being  Correct
Percent  Correct
99 #  of  Responses
We  are  inaccurate  when  assessing  probabilities
Copyright  HDR  2007  
dwhubbard@hubbardresearch.com
But,  training  can  “calibrate” people  so  that  of  all  the  times  they  
say  they  are  X%  confident,  they  will  be  right  X%  of  the  time
@LMaccherone @TheAgileCraft
Equivalent  Bet  calibration
What  year  did  Newton  published  the  Universal  Laws  of  
Gravitation?
Pick  year  range  that  you  are  90%  certain  it  would  fall  within.
Win  $1,000:
1. It  is  within  your  range;;  or
2. You  spin  this  wheel  and  it  lands  green  
Adjust  your  range  until  1  and  2  seem  equal.  
Even  pretending to  bet  money  works.
90%
10%
@LMaccherone @TheAgileCraft
An
agile  delivery  date  forecast
example  
@LMaccherone @TheAgileCraft
Monte Carlo Forecasting
What it looks like
Live demo: http://lumenize.com(use Chrome)
@LMaccherone @TheAgileCraft
Seek  to
change  the  nature  of  
the  conversation
@LMaccherone @TheAgileCraft
Getting even more sophisticated
1. Only use slopes after it stabilizes. Discard the first N.
(Lumenize has v-optimal algorithm for finding this inflection point)
2. Weight later slopes more heavily.
3. Markov chain pattern reproduction. Accomplishes 1
and 2 above automatically.
4. Simulate the movement of each individual work item
through the system. Can find bottlenecks and help
optimize your role balance.
Troy Magennis has the expertise and tools for this.
@LMaccherone @TheAgileCraft
Using measurement in
an agile environment
@LMaccherone @TheAgileCraft
@LMaccherone @TheAgileCraft
…  but  for  those  brave  enough  to  journey  
into  the  dangerous  world  of  
agile  measurement  
there  are  great  riches  to  be  had.
The  trick  is  to  slay  the  dragons.
@LMaccherone @TheAgileCraft
The  Dragons  of  Agile  Measurement
If  you  do  metrics  wrong,  you  will  harm  your  agile  transformation
1. Dragon:  Measurement  as  a  lever
Slayer:  Measurement  as  feedback
2. Dragon:  Unbalanced  metrics
Slayer:  1  each  for  Do  it  
fast/right/on-­time,  and  Keep  doing  it
3. Dragon:  Metrics  can  replace  
thinking
Slayer:  Metrics  compliment  thinking
4. Dragon:  Expensive  metrics
Slayer:  1st work  with  the  data  you  
are  already  passively  gathering
5. Dragon:  Using  a  convenient  
metric
Slayer:  Outcomes  ß Decisions  ß
Insight  ß Metric  (ODIM)
6. Dragon:  Bad  analysis
Slayer:  Simple  stats  and  
simulation
7. Dragon:  Single  outcome  forecasts  
Slayer:  Forecasts  w/  probability
8. Dragon:  Human  emotion  and  bias
Slayer:  Tricks  to  avoid  your  own  
biases  and  overcome  those  of  
others
@LMaccherone @TheAgileCraft
Manipulating
Others
Dragon #1
Using metrics as a
lever to drive
someone else’s
behavior
@LMaccherone @TheAgileCraft
Self
Improvement
Dragon slayer #1
Using metrics to
reflect on your own
performance
@LMaccherone @TheAgileCraft
Dragon  #5
Using  a  convenient  metric
aka  “Lamp  post  metrics”
@LMaccherone @TheAgileCraft
@LMaccherone @TheAgileCraft
Good  players?
Monta Ellis
9th  highest  scorer  
(8th  last  season)
Carmelo  Anthony  (Melo)
8th  highest  scorer
(3rd  last  season)
@LMaccherone @TheAgileCraft
Dragon  slayer  #5
ODIM
O U T C O M E
D E C I S I O N
I N S I G H T
M E A S U R E
THINK
EFFECT
like Vic Basili’s
Goal-Question-Metric (GQM)
but without
ISO/IEC 15939 baggage
@LMaccherone @TheAgileCraft
The  Dragons  of  Agile  Measurement
If  you  do  metrics  wrong,  you  will  harm  your  agile  transformation
1. Dragon:  Measurement  as  a  lever
Slayer:  Measurement  as  feedback
2. Dragon:  Unbalanced  metrics
Slayer:  1  each  for  Do  it  
fast/right/on-­time,  and  Keep  doing  it
3. Dragon:  Metrics  can  replace  
thinking
Slayer:  Metrics  compliment  thinking
4. Dragon:  Expensive  metrics
Slayer:  1st work  with  the  data  you  
are  already  passively  gathering
5. Dragon:  Using  a  convenient  
metric
Slayer:  Outcomes  ß Decisions  ß
Insight  ß Metric  (ODIM)
6. Dragon:  Bad  analysis
Slayer:  Simple  stats  and  
simulation
7. Dragon:  Single  outcome  forecasts  
Slayer:  Forecasts  w/  probability
8. Dragon:  Human  emotion  and  bias
Slayer:  Tricks  to  avoid  your  own  
biases  and  overcome  those  of  
others
@LMaccherone @TheAgileCraft
Top  10  criteria  for  great  visualization
1. Answers  the  question,  
"Compared  with  what?”    
(SO  What?)
2. Shows  causality,  or  is  at  least  
informed  by  it.  
(NOW  WHAT?)
3. Tells  a  story  with  whatever  it  
takes.
4. Is  credible.  
5. Has  business  value or  impact  in  
its  social  context.
6. Shows  
differences
easily.
7. Allows  you  to  see  the  forest  
AND  the  trees.  
8. Informs  along  multiple  
dimensions.  
9. Leaves  in  the  numbers  where  
possible.
10. Leaves  out  glitter.
Credits:
• Edward Tufte
• Stephen Few
• Gestalt
(School of Psychology)
@LMaccherone @TheAgileCraft
Now  what?  
• Questions?
• Day-­long  seminar  on  agile  metrics
• Workshop  to  design  your  own  
metrics  regimen
• AgileCraft  Demo  -­ LJ  Alefantis
• Contact  me  on  LinkedIn
https://linkedin.com/in/larrymaccherone
@LMaccherone @TheAgileCraft
“They” say…
Nobody knows what’s gonna happen
next: not on a freeway, not in an
airplane, not inside our own bodies
and certainly not on a racetrack with
40 other infantile egomaniacs.
– Days of Thunder
Trying to predict the future is like
trying to drive down a country road
at night with no lights while looking
out the back window.
– Peter Drucker
Never make predictions, especially
about the future.
– Casey Stengel
@LMaccherone @TheAgileCraft
When  you  come  to  a  
fork  in  the  road…  
take  it!
~Yogi  Berra
@LMaccherone @TheAgileCraft
Now  what?  
• Questions?
• Day-­long  seminar  on  agile  metrics
• Workshop  to  design  your  own  
metrics  regimen
• AgileCraft  Demo  -­ LJ  Alefantis
• Contact  me  on  LinkedIn
https://linkedin.com/in/larrymaccherone

More Related Content

Similar to Larry Maccherone: "Probabilistic Decision Making"

Impact of Agile Quantified - Late 2014 Edition
Impact of Agile Quantified - Late 2014 EditionImpact of Agile Quantified - Late 2014 Edition
Impact of Agile Quantified - Late 2014 Edition
Larry Maccherone
 
Influx/Days 2017 San Francisco | Baron Schwartz
Influx/Days 2017 San Francisco | Baron SchwartzInflux/Days 2017 San Francisco | Baron Schwartz
Influx/Days 2017 San Francisco | Baron Schwartz
InfluxData
 
Casual gaming metrics applied to social gaming
Casual gaming metrics applied to social gamingCasual gaming metrics applied to social gaming
Casual gaming metrics applied to social gaming
MediaShifters
 
Your Agile Leadership Journey: Leading People, Managing Paradoxes
Your Agile Leadership Journey: Leading People, Managing ParadoxesYour Agile Leadership Journey: Leading People, Managing Paradoxes
Your Agile Leadership Journey: Leading People, Managing Paradoxes
Paul Boos
 
Introduction to DevOps with Lego and Chocolate simulation game.
Introduction to DevOps with Lego and Chocolate simulation game.Introduction to DevOps with Lego and Chocolate simulation game.
Introduction to DevOps with Lego and Chocolate simulation game.
Dana Pylayeva
 
Embrace Uncertainty
Embrace UncertaintyEmbrace Uncertainty
Embrace Uncertainty
abcd82
 
Agile Metrics: Make Better Decisions with Data
Agile Metrics: Make Better Decisions with DataAgile Metrics: Make Better Decisions with Data
Agile Metrics: Make Better Decisions with Data
TechWell
 
Your Agile Leadership Journey: Leading People-Managing Paradoxes - Agile Char...
Your Agile Leadership Journey: Leading People-Managing Paradoxes - Agile Char...Your Agile Leadership Journey: Leading People-Managing Paradoxes - Agile Char...
Your Agile Leadership Journey: Leading People-Managing Paradoxes - Agile Char...
Paul Boos
 
Applied Data Science for monetization: pitfalls, common misconceptions, and n...
Applied Data Science for monetization: pitfalls, common misconceptions, and n...Applied Data Science for monetization: pitfalls, common misconceptions, and n...
Applied Data Science for monetization: pitfalls, common misconceptions, and n...
DevGAMM Conference
 
Beyond A/B: Tips and Tech for Optimizing to the Open
Beyond A/B: Tips and Tech for Optimizing to the OpenBeyond A/B: Tips and Tech for Optimizing to the Open
Beyond A/B: Tips and Tech for Optimizing to the Open
LiveIntent
 
Game of Sprints
Game of SprintsGame of Sprints
Game of Sprints
Kevlin Henney
 
Engineering your culture / Oren Ellenbogen
Engineering your culture  / Oren EllenbogenEngineering your culture  / Oren Ellenbogen
Engineering your culture / Oren Ellenbogen
geektimecoil
 
@RISK Unchained Webinar
@RISK Unchained Webinar@RISK Unchained Webinar
@RISK Unchained WebinarAndrew Sich
 
Emily Greer at GDC 2018: Data-Driven or Data-Blinded?
Emily Greer at GDC 2018: Data-Driven or Data-Blinded?Emily Greer at GDC 2018: Data-Driven or Data-Blinded?
Emily Greer at GDC 2018: Data-Driven or Data-Blinded?
Kongregate
 
Focus fast bigd15_roger_belveal_2015-09-19
Focus fast bigd15_roger_belveal_2015-09-19Focus fast bigd15_roger_belveal_2015-09-19
Focus fast bigd15_roger_belveal_2015-09-19
Roger Belveal
 
We are not building chairs
We are not building chairsWe are not building chairs
We are not building chairs
Chris Lema
 
The Art of Speaking Data.
The Art of Speaking Data.The Art of Speaking Data.
The Art of Speaking Data.
David Wellman
 
Hushcon 2016 Keynote: Test for Echo
Hushcon 2016 Keynote: Test for EchoHushcon 2016 Keynote: Test for Echo
Hushcon 2016 Keynote: Test for Echo
Deja vu Security
 
Knowing Ranking Factors won't be enough!
Knowing Ranking Factors won't be enough!Knowing Ranking Factors won't be enough!
Knowing Ranking Factors won't be enough!
Mark Orr
 
What they didn't know they needed
What they didn't know they needed What they didn't know they needed
What they didn't know they needed
Megan Grocki
 

Similar to Larry Maccherone: "Probabilistic Decision Making" (20)

Impact of Agile Quantified - Late 2014 Edition
Impact of Agile Quantified - Late 2014 EditionImpact of Agile Quantified - Late 2014 Edition
Impact of Agile Quantified - Late 2014 Edition
 
Influx/Days 2017 San Francisco | Baron Schwartz
Influx/Days 2017 San Francisco | Baron SchwartzInflux/Days 2017 San Francisco | Baron Schwartz
Influx/Days 2017 San Francisco | Baron Schwartz
 
Casual gaming metrics applied to social gaming
Casual gaming metrics applied to social gamingCasual gaming metrics applied to social gaming
Casual gaming metrics applied to social gaming
 
Your Agile Leadership Journey: Leading People, Managing Paradoxes
Your Agile Leadership Journey: Leading People, Managing ParadoxesYour Agile Leadership Journey: Leading People, Managing Paradoxes
Your Agile Leadership Journey: Leading People, Managing Paradoxes
 
Introduction to DevOps with Lego and Chocolate simulation game.
Introduction to DevOps with Lego and Chocolate simulation game.Introduction to DevOps with Lego and Chocolate simulation game.
Introduction to DevOps with Lego and Chocolate simulation game.
 
Embrace Uncertainty
Embrace UncertaintyEmbrace Uncertainty
Embrace Uncertainty
 
Agile Metrics: Make Better Decisions with Data
Agile Metrics: Make Better Decisions with DataAgile Metrics: Make Better Decisions with Data
Agile Metrics: Make Better Decisions with Data
 
Your Agile Leadership Journey: Leading People-Managing Paradoxes - Agile Char...
Your Agile Leadership Journey: Leading People-Managing Paradoxes - Agile Char...Your Agile Leadership Journey: Leading People-Managing Paradoxes - Agile Char...
Your Agile Leadership Journey: Leading People-Managing Paradoxes - Agile Char...
 
Applied Data Science for monetization: pitfalls, common misconceptions, and n...
Applied Data Science for monetization: pitfalls, common misconceptions, and n...Applied Data Science for monetization: pitfalls, common misconceptions, and n...
Applied Data Science for monetization: pitfalls, common misconceptions, and n...
 
Beyond A/B: Tips and Tech for Optimizing to the Open
Beyond A/B: Tips and Tech for Optimizing to the OpenBeyond A/B: Tips and Tech for Optimizing to the Open
Beyond A/B: Tips and Tech for Optimizing to the Open
 
Game of Sprints
Game of SprintsGame of Sprints
Game of Sprints
 
Engineering your culture / Oren Ellenbogen
Engineering your culture  / Oren EllenbogenEngineering your culture  / Oren Ellenbogen
Engineering your culture / Oren Ellenbogen
 
@RISK Unchained Webinar
@RISK Unchained Webinar@RISK Unchained Webinar
@RISK Unchained Webinar
 
Emily Greer at GDC 2018: Data-Driven or Data-Blinded?
Emily Greer at GDC 2018: Data-Driven or Data-Blinded?Emily Greer at GDC 2018: Data-Driven or Data-Blinded?
Emily Greer at GDC 2018: Data-Driven or Data-Blinded?
 
Focus fast bigd15_roger_belveal_2015-09-19
Focus fast bigd15_roger_belveal_2015-09-19Focus fast bigd15_roger_belveal_2015-09-19
Focus fast bigd15_roger_belveal_2015-09-19
 
We are not building chairs
We are not building chairsWe are not building chairs
We are not building chairs
 
The Art of Speaking Data.
The Art of Speaking Data.The Art of Speaking Data.
The Art of Speaking Data.
 
Hushcon 2016 Keynote: Test for Echo
Hushcon 2016 Keynote: Test for EchoHushcon 2016 Keynote: Test for Echo
Hushcon 2016 Keynote: Test for Echo
 
Knowing Ranking Factors won't be enough!
Knowing Ranking Factors won't be enough!Knowing Ranking Factors won't be enough!
Knowing Ranking Factors won't be enough!
 
What they didn't know they needed
What they didn't know they needed What they didn't know they needed
What they didn't know they needed
 

Recently uploaded

Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Inflectra
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
Product School
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
Product School
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
Alison B. Lowndes
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
Product School
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
Dorra BARTAGUIZ
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 

Recently uploaded (20)

Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 

Larry Maccherone: "Probabilistic Decision Making"

  • 2. @LMaccherone @TheAgileCraft Why don’t coaches go for it more often?
  • 3. @LMaccherone @TheAgileCraft @LMaccherone @TheAgileCraft We don't see things the way they are. We see things the way we are. ~The Talmud
  • 5. @LMaccherone @TheAgileCraft Bias eats good decisions for breakfast, lunch, and dinner By understanding probabilistic decision making, we learn to trust and overcome bias
  • 8. @LMaccherone @TheAgileCraft You are forecasting that your choice will have better outcomes than the other alternatives
  • 10. @LMaccherone @TheAgileCraft Argument  is  about  who is  right. Decision  making  is  about  what is  right.
  • 11. @LMaccherone @TheAgileCraft 1.  Different  Models 2.  Different  Values 3.  Different  Risk  Tolerance Why do people disagree? favor different alternatives Fear-based decision making
  • 12. @LMaccherone @TheAgileCraft Models  and  Values § Models  calculate  probability  in  terms  of  proxy  variables § Values  translate  those  probabilities  into  money § Different  models  example: § Joe  forecasts  that  alternative  A  will  make  the  most  money § Sally  forecasts  that  alternative  B  will  make  the  most  money § Different  values  example: § Betty  favors  the  alternative  with  higher  quality § George  favors  the  alternative  that  will  get  to  market  faster
  • 13. @LMaccherone @TheAgileCraft So… quality  of  decision  depends  upon: 1.  alternatives  considered,  and 2.  models  used  to  forecast  the outcome  of  those  alternatives. Probabilistic  models  are  superior
  • 15. @LMaccherone @TheAgileCraft For  a  given  alternative,  let: Pg =  Probability  of  good  thing  happening Vg =  “Value”  of  good  thing  happening Then: Value  of  the  alternative  =  Pg × Vg
  • 16. @LMaccherone @TheAgileCraft An   lean/agile  product   management  example
  • 17. @LMaccherone @TheAgileCraft $8M Best  case   (25%) $1M Likely  case   (50%) $1M Worst  case   (25%) 1 $2M$2M$1M 2 Which  strategy  is  best… …for  your  company? PW × VW =  .25 × -­$1.00M  =  -­$0.25M PL × VL =  .50 × $1.00M  =   $0.50M PB × VB =  .25 × $8.00M  =    $2.00M   -­-­-­-­-­-­-­-­-­-­-­ $2.25M …for  your  career? PW × VW =  .25 × $1.00M  =    $0.25M PL × VL =  .50 × $2.00M  =   $1.00M PB × VB =  .25 × $2.00M  =    $0.50M   -­-­-­-­-­-­-­-­-­-­-­ $1.75M
  • 18. @LMaccherone @TheAgileCraft If  you  get  only  1 project  then   strategy  2  is  better 75%  of  the  time If  you  get  ∞ projects  then strategy  1  is  better 100%  of  the  time How  many  projects  do  you  need  for   strategy  1  to  be  better   more  often  than  not?
  • 20. @LMaccherone @TheAgileCraft Play with it yourself at: http://jsfiddle.net/lmaccherone/j3wh61r7/
  • 22. @LMaccherone @TheAgileCraft Emotion  and  bias  plays  a  part
  • 23. @LMaccherone @TheAgileCraft Did  any  of  you  get  emotional   about  the  $1M  loss? Did  any  of  you  want  to   question  the  $8M  number? We’ve totally… …eliminated fear from the equation …changed the nature of the conversation
  • 24. @LMaccherone @TheAgileCraft Argument  is  about  who is  right. Decision  making  is  about  what is  right.
  • 26. @LMaccherone @TheAgileCraft Trained/Calibrated Untrained/Uncalibrated Statistical  Error “Ideal” Confidence 30% 40% 50% 60% 70% 80% 90% 100% 50% 60% 80% 90% 100% 25 75 71 65 58 21 17 68 152 65 45 21 70% Assessed  Chance  Of  Being  Correct Percent  Correct 99 #  of  Responses We  are  inaccurate  when  assessing  probabilities Copyright  HDR  2007   dwhubbard@hubbardresearch.com But,  training  can  “calibrate” people  so  that  of  all  the  times  they   say  they  are  X%  confident,  they  will  be  right  X%  of  the  time
  • 27. @LMaccherone @TheAgileCraft Equivalent  Bet  calibration What  year  did  Newton  published  the  Universal  Laws  of   Gravitation? Pick  year  range  that  you  are  90%  certain  it  would  fall  within. Win  $1,000: 1. It  is  within  your  range;;  or 2. You  spin  this  wheel  and  it  lands  green   Adjust  your  range  until  1  and  2  seem  equal.   Even  pretending to  bet  money  works. 90% 10%
  • 28. @LMaccherone @TheAgileCraft An agile  delivery  date  forecast example  
  • 29. @LMaccherone @TheAgileCraft Monte Carlo Forecasting What it looks like Live demo: http://lumenize.com(use Chrome)
  • 30. @LMaccherone @TheAgileCraft Seek  to change  the  nature  of   the  conversation
  • 31. @LMaccherone @TheAgileCraft Getting even more sophisticated 1. Only use slopes after it stabilizes. Discard the first N. (Lumenize has v-optimal algorithm for finding this inflection point) 2. Weight later slopes more heavily. 3. Markov chain pattern reproduction. Accomplishes 1 and 2 above automatically. 4. Simulate the movement of each individual work item through the system. Can find bottlenecks and help optimize your role balance. Troy Magennis has the expertise and tools for this.
  • 34. @LMaccherone @TheAgileCraft …  but  for  those  brave  enough  to  journey   into  the  dangerous  world  of   agile  measurement   there  are  great  riches  to  be  had. The  trick  is  to  slay  the  dragons.
  • 35. @LMaccherone @TheAgileCraft The  Dragons  of  Agile  Measurement If  you  do  metrics  wrong,  you  will  harm  your  agile  transformation 1. Dragon:  Measurement  as  a  lever Slayer:  Measurement  as  feedback 2. Dragon:  Unbalanced  metrics Slayer:  1  each  for  Do  it   fast/right/on-­time,  and  Keep  doing  it 3. Dragon:  Metrics  can  replace   thinking Slayer:  Metrics  compliment  thinking 4. Dragon:  Expensive  metrics Slayer:  1st work  with  the  data  you   are  already  passively  gathering 5. Dragon:  Using  a  convenient   metric Slayer:  Outcomes  ß Decisions  ß Insight  ß Metric  (ODIM) 6. Dragon:  Bad  analysis Slayer:  Simple  stats  and   simulation 7. Dragon:  Single  outcome  forecasts   Slayer:  Forecasts  w/  probability 8. Dragon:  Human  emotion  and  bias Slayer:  Tricks  to  avoid  your  own   biases  and  overcome  those  of   others
  • 36. @LMaccherone @TheAgileCraft Manipulating Others Dragon #1 Using metrics as a lever to drive someone else’s behavior
  • 37. @LMaccherone @TheAgileCraft Self Improvement Dragon slayer #1 Using metrics to reflect on your own performance
  • 38. @LMaccherone @TheAgileCraft Dragon  #5 Using  a  convenient  metric aka  “Lamp  post  metrics”
  • 40. @LMaccherone @TheAgileCraft Good  players? Monta Ellis 9th  highest  scorer   (8th  last  season) Carmelo  Anthony  (Melo) 8th  highest  scorer (3rd  last  season)
  • 41. @LMaccherone @TheAgileCraft Dragon  slayer  #5 ODIM O U T C O M E D E C I S I O N I N S I G H T M E A S U R E THINK EFFECT like Vic Basili’s Goal-Question-Metric (GQM) but without ISO/IEC 15939 baggage
  • 42. @LMaccherone @TheAgileCraft The  Dragons  of  Agile  Measurement If  you  do  metrics  wrong,  you  will  harm  your  agile  transformation 1. Dragon:  Measurement  as  a  lever Slayer:  Measurement  as  feedback 2. Dragon:  Unbalanced  metrics Slayer:  1  each  for  Do  it   fast/right/on-­time,  and  Keep  doing  it 3. Dragon:  Metrics  can  replace   thinking Slayer:  Metrics  compliment  thinking 4. Dragon:  Expensive  metrics Slayer:  1st work  with  the  data  you   are  already  passively  gathering 5. Dragon:  Using  a  convenient   metric Slayer:  Outcomes  ß Decisions  ß Insight  ß Metric  (ODIM) 6. Dragon:  Bad  analysis Slayer:  Simple  stats  and   simulation 7. Dragon:  Single  outcome  forecasts   Slayer:  Forecasts  w/  probability 8. Dragon:  Human  emotion  and  bias Slayer:  Tricks  to  avoid  your  own   biases  and  overcome  those  of   others
  • 43. @LMaccherone @TheAgileCraft Top  10  criteria  for  great  visualization 1. Answers  the  question,   "Compared  with  what?”     (SO  What?) 2. Shows  causality,  or  is  at  least   informed  by  it.   (NOW  WHAT?) 3. Tells  a  story  with  whatever  it   takes. 4. Is  credible.   5. Has  business  value or  impact  in   its  social  context. 6. Shows   differences easily. 7. Allows  you  to  see  the  forest   AND  the  trees.   8. Informs  along  multiple   dimensions.   9. Leaves  in  the  numbers  where   possible. 10. Leaves  out  glitter. Credits: • Edward Tufte • Stephen Few • Gestalt (School of Psychology)
  • 44. @LMaccherone @TheAgileCraft Now  what?   • Questions? • Day-­long  seminar  on  agile  metrics • Workshop  to  design  your  own   metrics  regimen • AgileCraft  Demo  -­ LJ  Alefantis • Contact  me  on  LinkedIn https://linkedin.com/in/larrymaccherone
  • 45. @LMaccherone @TheAgileCraft “They” say… Nobody knows what’s gonna happen next: not on a freeway, not in an airplane, not inside our own bodies and certainly not on a racetrack with 40 other infantile egomaniacs. – Days of Thunder Trying to predict the future is like trying to drive down a country road at night with no lights while looking out the back window. – Peter Drucker Never make predictions, especially about the future. – Casey Stengel
  • 46. @LMaccherone @TheAgileCraft When  you  come  to  a   fork  in  the  road…   take  it! ~Yogi  Berra
  • 47. @LMaccherone @TheAgileCraft Now  what?   • Questions? • Day-­long  seminar  on  agile  metrics • Workshop  to  design  your  own   metrics  regimen • AgileCraft  Demo  -­ LJ  Alefantis • Contact  me  on  LinkedIn https://linkedin.com/in/larrymaccherone