Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Improving decision-making
through evidence-based
management
University of Bath taster event – June 15, 2015 - London
Our mission for today:
By the end of this workshop you will be more concerned
about the way you typically make decisions a...
20 min EBDsc: What is it and why do you need it?
20 min Professional experience & judgment
10 min Organizational data
20 m...
Exercise
Think about a decision you have been involved in
making. This decision should be one which:
 Was reasonably impo...
Exercise
Discuss with your neighbor (2 min)
What exactly was the problem (or opportunity)?
How many alternative decision o...
Evidence based decision-making:
What is it?
Evidence-based decision making
Central Premise:
Decisions should be based on a
combination of critical thinking and
the ‘b...
Evidence?
outcome of scientific research,
organizational facts & figures,
benchmarking, best practices,
personal experience
All managers and leaders
base their decisions on
‘evidence’
But…many managers and
leaders pay little or no
attention to the quality of the
evidence they base their
decisions on
Trust me, 20 years of
management experience
SO ...
Teach managers/leaders
how to critically evaluate the
validity, and generalizability of
the evidence and help them
find ‘t...
Evidence based decision
Professional
experience and
judgment
Organizational data,
facts and figures
Stakeholders’ values
a...
Evidence based practice:
Where does it come from?
McMaster University Medical School, Canada
Medicine: Founding fathers
David Sackett Gordon Guyatt
How it all started
1. Ask: translate a practical issue into an answerable question
2. Acquire: systematically search for and retrieve the evi...
Evidence-Based Practice
1991 Medicine
1998 Education
2000 Social care, public policy
Nursing, Criminal justice,
Policing, ...
Evidence-Based Practice
Evidence-Based Practice
Evidence-Based Practice
Evidence-Based Practice
Evidence-based decision-making
=
the use of critically appraised
evidence from multiple sources to
increase the likelihood...
“Where the evidence is strong, we should act on it.
Where the evidence is suggestive, we should consider it.
Where the evi...
Evidence-Based Decision-Making
Why do we need it?
1. Incompetent people benefit more from feedback than
highly competent people.
2. Task conflict improves work group perfor...
How evidence-based are we?
 959 (US) + 626 (Dutch) HR professionals
 35 statements, based on an extensive body of eviden...
Outcome: not better than random chance
Evidence-based decision making
Professional
experience and
judgment
Organizational data,
facts and figures
Stakeholders’ v...
Thinking critical about
professional experience and judgment
Discuss with your neighbor
Why is a physician’s clinical experience,
as a rule, more trustworthy than
a manager’s professi...
Developing expertise
1. A sufficiently regular, predictable environment
2. Opportunities to learn regularities through pro...
Bounded rationality
Bounded rationality / prospect theory
System 1
 Fast
 Intuitive, associative
 heuristics & biases
System 2
 Slow (lazy...
System 1: short cuts
Shepard’s tables
System 1: short cuts
System 1: dominant, cause necessary to survive
95%
 Seeing order in randomness
 Overconfidence bias
 Halo effect
 False consensus effect
 Group think
 Self serving att...
1. Pattern recognition
2. Confirmation-bias
3. Groupthink
Cognitive errors that mess up decision making
We are predisposed to see order, pattern and causal
relations in the world.
Patternicity: The tendency to find meaningful ...
We are pattern seeking primates: association learning
Bias 1: pattern recognition
Points of impact of V-1 bombs in London
Points of impact of V-1 bombs in London
 A Type I error or a false positive, is
believing a pattern is real when it is not
(finding a non existent pattern)
 A T...
 A Type I error or a false positive: believe that the
rustle in the grass is a dangerous predator when it is
just the win...
 A Type II error or a false negative: believe that the
rustle in the grass is just the wind when it is a
dangerous predat...
 A Type I error or a false positive: believe that the
rustle in the grass is a dangerous predator when it is
just the win...
superstitious
rituals
superstitious
rituals
more stress = more prone to type 1 errors
Error 1: pattern recognition
Error 1: pattern recognition
1. Pattern recognition
2. Confirmation-bias
3. Groupthink
Cognitive errors
We are predisposed to selectively
search for or interpret information in
ways that confirms our existing beliefs,
expectat...
Example
You may believe that astrology actually
works. As a result of confirmation bias
you’ll remember only those instanc...
Confirmation bias
Pattern recognition
Error 2: confirmation bias
McKinsey (1997 case study / 2001 book)
McKinsey: case study
War on Talent
1. Pattern recognition
2. Confirmation-bias
3. Groupthink
Errors
Groupthink:
Groupthink is a psychological
phenomenon that occurs within a
group of people, in which the desire
for harmony...
Error 3: Groupthink
Error 3: Groupthink
Lean Management / Lean Six Sigma
Self steering / autonomous teams
Agile working / New World of Working
Talent management
E...
“I’ve been studying judgment for 45 years, and I’m no better
than when I started. I make extreme predictions. I’m over-
co...
EBDsc = Safeguard
20 min EBDsc: What is it and why do you need it?
20 min Professional experience & judgment
10 min Organizational data
20 m...
Evidence based decision
Professional
experience and
judgment
Organizational data,
facts and figures
Stakeholders’ values
a...
Evidence-based decision making
Professional
experience and
judgment
Stakeholders’ values
and concerns
Scientific
research
...
1. financial data (cash flow, solvability)
2. business outcomes (ROI, market share)
3. customer/client impact (customer sa...
Organizational facts and figures
Examples
Trends
People operate with beliefs
& biases. To the extent you
can reduce both and replace
them with data, you gain a
clear compe...
Organizational facts and figures
Organizational facts and figures
Can your organization correlate/regress
level of education
years of experience
productivity?
customer satisfaction?
failur...
In the next weeks,
before you make a decision, ask yourself:
 What exactly is the problem?
 What is the evidence availab...
 Identify the many other factors, rather than evidence, that shape
managerial decisions such as cognitive biases, politic...
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Bath University taster event: Evidence-Based Decision Making
Upcoming SlideShare
Loading in …5
×

Bath University taster event: Evidence-Based Decision Making

Improving decision-making through evidence-based management
University of Bath School of Management
London, June 16th, 2015

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

Bath University taster event: Evidence-Based Decision Making

  1. 1. Improving decision-making through evidence-based management University of Bath taster event – June 15, 2015 - London
  2. 2. Our mission for today: By the end of this workshop you will be more concerned about the way you typically make decisions as a manager/leader/consultant, so you firmly resolve that, within the next year, you will increase the ‘evidence- base’ of your decisions by at least 25%. You can do this, of course, by signing up for the executive course ‘Evidence-based decision-making’ at Bath University School of Management.
  3. 3. 20 min EBDsc: What is it and why do you need it? 20 min Professional experience & judgment 10 min Organizational data 20 min Scientific evidence: mini case Agenda
  4. 4. Exercise Think about a decision you have been involved in making. This decision should be one which:  Was reasonably important for your organization  Involved spending significant resources  Involved several or more people  Was made over a period of time (ie. weeks or months)
  5. 5. Exercise Discuss with your neighbor (2 min) What exactly was the problem (or opportunity)? How many alternative decision options were considered? How much evidence was used, and from which sources (scientific, organizational, experience, crystal ball?) Was any attempt made to explicitly evaluate its quality or trustworthiness?
  6. 6. Evidence based decision-making: What is it?
  7. 7. Evidence-based decision making Central Premise: Decisions should be based on a combination of critical thinking and the ‘best available evidence‘.
  8. 8. Evidence? outcome of scientific research, organizational facts & figures, benchmarking, best practices, personal experience
  9. 9. All managers and leaders base their decisions on ‘evidence’
  10. 10. But…many managers and leaders pay little or no attention to the quality of the evidence they base their decisions on
  11. 11. Trust me, 20 years of management experience
  12. 12. SO ...
  13. 13. Teach managers/leaders how to critically evaluate the validity, and generalizability of the evidence and help them find ‘the best available’ evidence
  14. 14. Evidence based decision Professional experience and judgment Organizational data, facts and figures Stakeholders’ values and concerns Scientific research outcomes Ask Acquire Appraise Aggregate Apply Assess diagnosis intervention
  15. 15. Evidence based practice: Where does it come from?
  16. 16. McMaster University Medical School, Canada Medicine: Founding fathers David Sackett Gordon Guyatt
  17. 17. How it all started
  18. 18. 1. Ask: translate a practical issue into an answerable question 2. Acquire: systematically search for and retrieve the evidence 3. Appraise: critically judge the trustworthiness of the evidence 4. Apply: incorporate the evidence into the decision-making process 5. Assess: evaluate the outcome of the decision taken 5 steps of EBmed
  19. 19. Evidence-Based Practice 1991 Medicine 1998 Education 2000 Social care, public policy Nursing, Criminal justice, Policing, Architecture, Conservation 2010 Management
  20. 20. Evidence-Based Practice
  21. 21. Evidence-Based Practice
  22. 22. Evidence-Based Practice
  23. 23. Evidence-Based Practice
  24. 24. Evidence-based decision-making = the use of critically appraised evidence from multiple sources to increase the likelihood of a favourable outcome Focus on the decision making process Think in terms of probability
  25. 25. “Where the evidence is strong, we should act on it. Where the evidence is suggestive, we should consider it. Where the evidence is weak, we should build the knowledge to support better decisions in the future.” Jeffrey Zients, acting director of the Office of Management and Budget and President Obama’s Economic Advisor
  26. 26. Evidence-Based Decision-Making Why do we need it?
  27. 27. 1. Incompetent people benefit more from feedback than highly competent people. 2. Task conflict improves work group performance while relational conflict harms it. 3. Encouraging employees to participate in decision making is more effective for improving organizational performance than setting performance goals. True or false?
  28. 28. How evidence-based are we?  959 (US) + 626 (Dutch) HR professionals  35 statements, based on an extensive body of evidence  true / false / uncertain HR Professionals' beliefs about effective human resource practices: correspondence between research and practice, (Rynes et al, 2002, Sanders et al 2008)
  29. 29. Outcome: not better than random chance
  30. 30. Evidence-based decision making Professional experience and judgment Organizational data, facts and figures Stakeholders’ values and concerns Scientific research outcomes Ask Acquire Appraise Aggregate Apply Assess
  31. 31. Thinking critical about professional experience and judgment
  32. 32. Discuss with your neighbor Why is a physician’s clinical experience, as a rule, more trustworthy than a manager’s professional experience? 60595857565453525150494847464544434241403938373635343332313029282726252423222120191817161514131211109876543210
  33. 33. Developing expertise 1. A sufficiently regular, predictable environment 2. Opportunities to learn regularities through prolonged practice and feedback The management domain is not highly favorable to expertise!
  34. 34. Bounded rationality
  35. 35. Bounded rationality / prospect theory System 1  Fast  Intuitive, associative  heuristics & biases System 2  Slow (lazy)  Deliberate,  Reasoning  Rational
  36. 36. System 1: short cuts
  37. 37. Shepard’s tables System 1: short cuts
  38. 38. System 1: dominant, cause necessary to survive 95%
  39. 39.  Seeing order in randomness  Overconfidence bias  Halo effect  False consensus effect  Group think  Self serving attribution bias  Sunk cost fallacy  Cognitive dissonance reduction System 1: cognitive errors  Confirmation bias  Authority bias  Small numbers fallacy  In-group bias  Recall bias  Anchoring bias  Availability bias
  40. 40. 1. Pattern recognition 2. Confirmation-bias 3. Groupthink Cognitive errors that mess up decision making
  41. 41. We are predisposed to see order, pattern and causal relations in the world. Patternicity: The tendency to find meaningful patterns in both meaningful and meaningless noise. Error 1: pattern recognition
  42. 42. We are pattern seeking primates: association learning Bias 1: pattern recognition
  43. 43. Points of impact of V-1 bombs in London
  44. 44. Points of impact of V-1 bombs in London
  45. 45.  A Type I error or a false positive, is believing a pattern is real when it is not (finding a non existent pattern)  A Type II error or a false negative, is not believing a pattern is real when it is (not recognizing a real pattern) Dr. Michael Shermer (Director of the Skeptics Society) Error 1: pattern recognition
  46. 46.  A Type I error or a false positive: believe that the rustle in the grass is a dangerous predator when it is just the wind (low cost) Error 1: pattern recognition
  47. 47.  A Type II error or a false negative: believe that the rustle in the grass is just the wind when it is a dangerous predator (high cost) Error 1: pattern recognition
  48. 48.  A Type I error or a false positive: believe that the rustle in the grass is a dangerous predator when it is just the wind (low cost)  A Type II error or a false negative: believe that the rustle in the grass is just the wind when it is a dangerous predator (high cost) Error 1: pattern recognition
  49. 49. superstitious rituals superstitious rituals more stress = more prone to type 1 errors Error 1: pattern recognition
  50. 50. Error 1: pattern recognition
  51. 51. 1. Pattern recognition 2. Confirmation-bias 3. Groupthink Cognitive errors
  52. 52. We are predisposed to selectively search for or interpret information in ways that confirms our existing beliefs, expectations and assumptions, and ignore information to the contrary. In other words, we “see what we want to see” 2. Confirmation bias
  53. 53. Example You may believe that astrology actually works. As a result of confirmation bias you’ll remember only those instances when when the prediction in the astrology column came true and forget the majority of the cases when the prediction was very wrong. As a result you will continue to believe astrology has some base in reality 2. Confirmation bias
  54. 54. Confirmation bias Pattern recognition Error 2: confirmation bias
  55. 55. McKinsey (1997 case study / 2001 book)
  56. 56. McKinsey: case study
  57. 57. War on Talent
  58. 58. 1. Pattern recognition 2. Confirmation-bias 3. Groupthink Errors
  59. 59. Groupthink: Groupthink is a psychological phenomenon that occurs within a group of people, in which the desire for harmony or conformity in the group results in an incorrect or irrational decision Error 3: Groupthink
  60. 60. Error 3: Groupthink
  61. 61. Error 3: Groupthink
  62. 62. Lean Management / Lean Six Sigma Self steering / autonomous teams Agile working / New World of Working Talent management Employee engagement Groupthink?
  63. 63. “I’ve been studying judgment for 45 years, and I’m no better than when I started. I make extreme predictions. I’m over- confident. I fall for every one of the biases.” Bounded rationality
  64. 64. EBDsc = Safeguard
  65. 65. 20 min EBDsc: What is it and why do you need it? 20 min Professional experience & judgment 10 min Organizational data 20 min Scientific evidence: mini case Agenda
  66. 66. Evidence based decision Professional experience and judgment Organizational data, facts and figures Stakeholders’ values and concerns Scientific research outcomes Ask Acquire Appraise Aggregate Apply Assess diagnosis intervention
  67. 67. Evidence-based decision making Professional experience and judgment Stakeholders’ values and concerns Scientific research outcomes Ask Acquire Appraise Aggregate Apply Assess Organizational data, facts and figures
  68. 68. 1. financial data (cash flow, solvability) 2. business outcomes (ROI, market share) 3. customer/client impact (customer satisfaction) 4. performance indicators (occupancy rate, failure frequency) 5. HR metrics (absenteeism, employee engagement) 6. marketing intelligence (brand awareness, customer feedback) 7. ‘soft’ data (organizational culture, trust in senior management, leadership style, commitment) 8. data from benchmarking Types organizational evidence
  69. 69. Organizational facts and figures
  70. 70. Examples
  71. 71. Trends
  72. 72. People operate with beliefs & biases. To the extent you can reduce both and replace them with data, you gain a clear competitive advantage Laszlo Bock (CHRO Google) Organizational data
  73. 73. Organizational facts and figures
  74. 74. Organizational facts and figures
  75. 75. Can your organization correlate/regress level of education years of experience productivity? customer satisfaction? failure frequency? employee satisfaction? employee turnover? absenteeism? +
  76. 76. In the next weeks, before you make a decision, ask yourself:  What exactly is the problem?  What is the evidence available?  Where does it come from (multiple sources?)  How trustworthy is it?
  77. 77.  Identify the many other factors, rather than evidence, that shape managerial decisions such as cognitive biases, politics, power, management fads and the ideas of management gurus  Distinguish between the four different types of evidence used in evidence-based management  Identify differences between your own and your organisation’s decision- making and an evidence-based management approach to decision- making  Understand practical ways in which your and your organisation’s decision- making can become more evidence-based and the potential benefits and costs that may result.  Learn how to critically evaluate the trustworthiness of evidence (including assumed best practices, organisational data, findings from research etc.) At the end of the course you will be able to

×