Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Getting started with
Evidence-Based Management
Dublin, April 28th, 2016
Evidence-Based HR: is it ‘a thing’?
25 min EBMgt: What is it and why you you need it?
20 min A practical example
Agenda
Evidence based management:
What is it?
Evidence-based management
Central Premise:
Decisions should be based on
the ‘best available evidence‘.
Evidence?
information, facts or data
supporting (or contradicting)
a claim, assumption or hypothesis
Evidence?
outcome of scientific research,
organizational facts & figures,
benchmarking, best practices,
personal experience
All managers and leaders base
their decisions on ‘evidence’
But…many managers pay
little or no attention to
the quality of the evidence
they base their decisions on
and use
too few s...
Trust me, 20 years of
management experience
Sources of evidence
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders
values a...
Evidence based practice:
Where does it come from?
McMaster University Medical School, Canada
Medicine: Founding fathers
David Sackett Gordon Guyatt
How it all started
1. Ask: translate a practical issue into an answerable question
2. Acquire: systematically search for and retrieve the evi...
Evidence-Based Practice
1991 Medicine
1998 Education
2000 Social care, public policy
Nursing, Criminal justice,
Policing, ...
Evidence-Based Practice
Evidence-Based Practice
Evidence-based … whatever
=
the use of evidence from multiple
sources to increase the likelihood of a
favourable outcome
F...
In general: managers don’t like EBMgt
 Undermines formal authority
 They feel it constrains freedom to make
managerial decisions
 Speed valued and rewarded m...
32
Evidence-Based Decision-Making
Why do we need it?
Advice: lie babies down to sleep on their belly
(unanimous support through to the 1990s)
Example: medicine
Nr of cot deaths (Holland)
Collateralized Debt Obligations > AAA
p = 0.12 (about 1 chance in 850) default in 5 years
Example: finance
Forecasted Actual
Forecasted and actual 5-year default rates for
AAA-rated CDO tranches
0.12%
28%
Scared straight
Example: policy / prevention
Example: HR management
1. Incompetent people benefit more from feedback than
highly competent people.
2. Task conflict improves work group perfor...
How evidence-based are HR managers?
 959 (US) + 626 (Dutch) HR professionals
 35 statements, based on an extensive body ...
Outcome: not better than random chance
Relying on only 1 source: bad idea!
problem solution
Practitioners
professional expertise
Organization
internal data
Stake...
Discuss with your neighbor (1 min)
Over a 5 year period,
why is an orthopedic surgeon's
experience, as a rule, more trustw...
Developing expertise
1. A sufficiently regular, predictable environment
2. Numerous opportunities to practice
3. Receive a...
Learning from feedback is hard!
Bounded rationality
How your brain works
System 1
 Fast
 Intuitive, associative
 heuristics & biases
 emotional
System 2
 Lazy
 Slow
 D...
System 1: short cuts
System 1 or system 2?
10 seconds
System 1 or system 2?
 A bat and a ball cost $1.10 in total.
 The bat costs $1 more than the ball
 How much does the ba...
System 1: necessary to survive
95%
 Pattern recognition
 Overconfidence bias
 Halo effect
 False consensus effect
 Group think
 Self serving attributio...
 Pattern recognition
 Overconfidence bias
 Halo effect
 False consensus effect
 Group think
 Self serving attributio...
“I’ve been studying judgment for 45 years, and I’m no better
than when I started. I make extreme predictions. I’m over-
co...
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
emp...
The performance of
knowledge workers
A Practical Example
 550 beds
 3300 employees
 210 medical specialists
 225,000 admissions
 Top Clinical & Teaching hospital
Organization
2015: 7.2 2016: 6.3
How can we increase job satisfaction
and employee engagement?
Dear HR department,
Evidence-based approach, step 1: ASK
problem solution
Practitioners
professional expertise
Organization
internal data
Stak...
 What is the problem?
 Why is this a problem: what are its organizational
consequences?
 How big: what is its impact on...
problem & underlying cause
hidden assumptions
causal mechanism
Some terminology
A happy & engaged employee
is a productive employee
Fundamental assumption
 What is the evidence for this assumption?
 Where multiple sources consulted?
 How trustworthy is the evidence?
Step 2:...
Let’s have a look
Professional
experience and
judgment
Organizational data,
facts and figures
Stakeholders’ values
and con...
GREAT! NOW WHAT?
Outcome
Evidence-based managers,
please
Evidence-based managers,
please
Step 1: ASK
Translate a practical issue into an
answerable question
Population? Knowledge workers!
Whether nurses, lawyers, engineers, managers, or staff members,
nowadays most workers in or...
Question
“Which of the factors that are related to
the performance of knowledge workers
are most widely studied and what i...
Step 2: ACQUIRE
Search for the best available scientific evidence
 ABI, BSP, PsycINFO
 Scholarly journals, peer reviewed
 1980 – 2013
 English
 performance, productivity, knowledge wo...
step 3: APPRAISE & AGGREGATE
Effect size?
Largest effect
1. Social cohesion .5 / .7
2. Perceived supervisory support .5
3. Information sharing / TM .5
4. Vision / g...
step 3b: CROSS VALIDATE
Step 4: APPLY
Three examples
social cohesion supervisory
support
information
sharing
Social cohesion
Social cohesion
… a shared liking or team attraction
that includes bonds of friendship, caring,
closeness, and enjoyment o...
Social cohesion
Measuring social cohesion
Perceived supervisory support
…how employees feel the supervisor
helps them in times of need, praises
them for a job well done or recognizes
them for ex...
Perceived supervisory support
Measuring perc. sup. support
Information sharing
Information sharing?
…refers to how teams pool and access their
knowledge and expertise – which positively
affects decisio...
Information sharing
Measuring information sharing
Outcome
The departments with the lowest performance
scored under average on most factors
Reactions
Who
knew?
Evidence-based … whatever
=
the use of evidence from multiple
sources to increase the likelihood of a
favourable outcome
F...
Multiple sources of evidence
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders...
Postgraduate Course
Postgraduate Course
> 80 Fellows
CEBMa: what we do
 Promote (seminars, papers, blogs, tweets)
 Educate (universities & business schools)
 Train & coach ...
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Getting Started With Evidence-Based HR
Upcoming SlideShare
Loading in …5
×

4

Share

Download to read offline

Getting Started With Evidence-Based HR

Download to read offline

Big data, evidence-based, predictive analytics, today these terms are all over the place. Is this just another fad or an irreversible trend? An increasing group of HR leaders relies on science, critical thinking and data analyses to make decisions.

Evidence-based HR, however, is still perceived by many as too time-consuming, narrow or impractical. Meanwhile, evidence-based practice is becoming mainstream in many other disciplines (like medicine). This is the momentum for pioneering HR leaders to seize the opportunity and make a difference with evidence. As part of an inclusive approach, valuing different perspectives.

We will enter into the dialogue about the why, the what, and most of all the how of evidence-based HR. How to get started and how to blend it with softer, less tangible HR practices? A pragmatic introduction, with realistic ambitions and openness towards other approaches.

Getting Started With Evidence-Based HR

  1. 1. Getting started with Evidence-Based Management Dublin, April 28th, 2016
  2. 2. Evidence-Based HR: is it ‘a thing’?
  3. 3. 25 min EBMgt: What is it and why you you need it? 20 min A practical example Agenda
  4. 4. Evidence based management: What is it?
  5. 5. Evidence-based management Central Premise: Decisions should be based on the ‘best available evidence‘.
  6. 6. Evidence? information, facts or data supporting (or contradicting) a claim, assumption or hypothesis
  7. 7. Evidence? outcome of scientific research, organizational facts & figures, benchmarking, best practices, personal experience
  8. 8. All managers and leaders base their decisions on ‘evidence’
  9. 9. But…many managers pay little or no attention to the quality of the evidence they base their decisions on and use too few sources of evidence
  10. 10. Trust me, 20 years of management experience
  11. 11. Sources of evidence problem solution Practitioners professional expertise Organization internal data Stakeholders values and concerns Scientific literature empirical studies Ask Acquire Appraise Aggregate Apply Assess
  12. 12. Evidence based practice: Where does it come from?
  13. 13. McMaster University Medical School, Canada Medicine: Founding fathers David Sackett Gordon Guyatt
  14. 14. How it all started
  15. 15. 1. Ask: translate a practical issue into an answerable question 2. Acquire: systematically search for and retrieve the evidence 3. Appraise: critically judge the trustworthiness of the evidence 4. Apply: incorporate the evidence into the decision-making process 5. Assess: evaluate the outcome of the decision taken 5 steps of EBmed
  16. 16. Evidence-Based Practice 1991 Medicine 1998 Education 2000 Social care, public policy Nursing, Criminal justice, Policing, Architecture, Conservation 2010 Management
  17. 17. Evidence-Based Practice
  18. 18. Evidence-Based Practice
  19. 19. Evidence-based … whatever = the use of evidence from multiple sources to increase the likelihood of a favourable outcome Focus on the decision making process Think in terms of probability
  20. 20. In general: managers don’t like EBMgt
  21. 21.  Undermines formal authority  They feel it constrains freedom to make managerial decisions  Speed valued and rewarded more than accuracy  Feel they cannot use their own experience and judgment (not true)  Managers not necessarily rewarded for doing what works (organizations rarely evaluate)  THEY LOVE FADS & QUICK FIXES Why don’t managers like EBMgt?
  22. 22. 32
  23. 23. Evidence-Based Decision-Making Why do we need it?
  24. 24. Advice: lie babies down to sleep on their belly (unanimous support through to the 1990s) Example: medicine
  25. 25. Nr of cot deaths (Holland)
  26. 26. Collateralized Debt Obligations > AAA p = 0.12 (about 1 chance in 850) default in 5 years Example: finance
  27. 27. Forecasted Actual Forecasted and actual 5-year default rates for AAA-rated CDO tranches 0.12% 28%
  28. 28. Scared straight Example: policy / prevention
  29. 29. Example: HR management
  30. 30. 1. Incompetent people benefit more from feedback than highly competent people. 2. Task conflict improves work group performance while relational conflict harms it. 3. Encouraging employees to participate in decision making is more effective for improving organizational performance than setting performance goals. Likely or unlikely?
  31. 31. How evidence-based are HR managers?  959 (US) + 626 (Dutch) HR professionals  35 statements, based on an extensive body of evidence  true / false / uncertain HR Professionals' beliefs about effective human resource practices: correspondence between research and practice, (Rynes et al, 2002, Sanders et al 2008)
  32. 32. Outcome: not better than random chance
  33. 33. Relying on only 1 source: bad idea! problem solution Practitioners professional expertise Organization internal data Stakeholders values and concerns Scientific literature empirical studies Ask Acquire Appraise Aggregate Apply Assess
  34. 34. Discuss with your neighbor (1 min) Over a 5 year period, why is an orthopedic surgeon's experience, as a rule, more trustworthy than an change manager’s experience? 0
  35. 35. Developing expertise 1. A sufficiently regular, predictable environment 2. Numerous opportunities to practice 3. Receive accurate (objective) feedback The management domain is not highly favorable to expertise!
  36. 36. Learning from feedback is hard!
  37. 37. Bounded rationality
  38. 38. How your brain works System 1  Fast  Intuitive, associative  heuristics & biases  emotional System 2  Lazy  Slow  Deliberate  Rational
  39. 39. System 1: short cuts
  40. 40. System 1 or system 2? 10 seconds
  41. 41. System 1 or system 2?  A bat and a ball cost $1.10 in total.  The bat costs $1 more than the ball  How much does the ball cost? 0
  42. 42. System 1: necessary to survive 95%
  43. 43.  Pattern recognition  Overconfidence bias  Halo effect  False consensus effect  Group think  Self serving attribution bias  Sunk cost fallacy  Cognitive dissonance reduction System 1: prone to cognitive errors  Confirmation bias  Authority bias  Small numbers fallacy  In-group bias  Recall bias  Anchoring bias  Availability bias
  44. 44.  Pattern recognition  Overconfidence bias  Halo effect  False consensus effect  Group think  Self serving attribution bias  Sunk cost fallacy  Cognitive dissonance reduction System 1: prone to cognitive errors  Confirmation bias  Authority bias  Small numbers fallacy  In-group bias  Recall bias  Anchoring bias  Availability bias
  45. 45. “I’ve been studying judgment for 45 years, and I’m no better than when I started. I make extreme predictions. I’m over- confident. I fall for every one of the biases.”
  46. 46. Practitioners professional expertise Organization internal data Stakeholders values and concerns Scientific literature empirical studies Ask Acquire Appraise Aggregate Apply Assess Four sources of evidence (not only 1)
  47. 47. The performance of knowledge workers A Practical Example
  48. 48.  550 beds  3300 employees  210 medical specialists  225,000 admissions  Top Clinical & Teaching hospital Organization
  49. 49. 2015: 7.2 2016: 6.3
  50. 50. How can we increase job satisfaction and employee engagement? Dear HR department,
  51. 51. Evidence-based approach, step 1: ASK problem solution Practitioners professional expertise Organization internal data Stakeholders values and concerns Scientific literature empirical studies Ask Acquire Appraise Aggregate Apply Assess
  52. 52.  What is the problem?  Why is this a problem: what are its organizational consequences?  How big: what is its impact on the organization when nothing is done?  Why does this problem exist, what is the assumed major cause?  What is the assumed causal mechanism? How does the cause lead to the problem and its consequences? Step 1: What is the problem?
  53. 53. problem & underlying cause hidden assumptions causal mechanism Some terminology
  54. 54. A happy & engaged employee is a productive employee Fundamental assumption
  55. 55.  What is the evidence for this assumption?  Where multiple sources consulted?  How trustworthy is the evidence? Step 2: What is the evidence?
  56. 56. Let’s have a look Professional experience and judgment Organizational data, facts and figures Stakeholders’ values and concerns Scientific research outcomes Ask Acquire Appraise Apply Assess problem solution
  57. 57. GREAT! NOW WHAT? Outcome
  58. 58. Evidence-based managers, please
  59. 59. Evidence-based managers, please
  60. 60. Step 1: ASK Translate a practical issue into an answerable question
  61. 61. Population? Knowledge workers! Whether nurses, lawyers, engineers, managers, or staff members, nowadays most workers in organizations are highly dependent on information and communication technology and are involved in work that involves a high level of cognitive activity.
  62. 62. Question “Which of the factors that are related to the performance of knowledge workers are most widely studied and what is known of their effect?”
  63. 63. Step 2: ACQUIRE Search for the best available scientific evidence
  64. 64.  ABI, BSP, PsycINFO  Scholarly journals, peer reviewed  1980 – 2013  English  performance, productivity, knowledge work* ACQUIRE
  65. 65. step 3: APPRAISE & AGGREGATE
  66. 66. Effect size?
  67. 67. Largest effect 1. Social cohesion .5 / .7 2. Perceived supervisory support .5 3. Information sharing / TM .5 4. Vision / goal clarity .5 5. Trust .3 / .6
  68. 68. step 3b: CROSS VALIDATE
  69. 69. Step 4: APPLY
  70. 70. Three examples social cohesion supervisory support information sharing
  71. 71. Social cohesion
  72. 72. Social cohesion … a shared liking or team attraction that includes bonds of friendship, caring, closeness, and enjoyment of each other’s company.
  73. 73. Social cohesion
  74. 74. Measuring social cohesion
  75. 75. Perceived supervisory support
  76. 76. …how employees feel the supervisor helps them in times of need, praises them for a job well done or recognizes them for extra effort. Perceived supervisory support
  77. 77. Perceived supervisory support
  78. 78. Measuring perc. sup. support
  79. 79. Information sharing
  80. 80. Information sharing? …refers to how teams pool and access their knowledge and expertise – which positively affects decision making and team processes. This has led to the idea of a team ‘Transactive Memory System’ (TMS), which can be thought of as a collective memory in a collective mind - enabling a team to think and act together
  81. 81. Information sharing
  82. 82. Measuring information sharing
  83. 83. Outcome The departments with the lowest performance scored under average on most factors
  84. 84. Reactions Who knew?
  85. 85. Evidence-based … whatever = the use of evidence from multiple sources to increase the likelihood of a favourable outcome Focus on the decision making process Think in terms of probability
  86. 86. Multiple sources of evidence problem solution Practitioners professional expertise Organization internal data Stakeholders values and concerns Scientific literature empirical studies Ask Acquire Appraise Aggregate Apply Assess
  87. 87. Postgraduate Course
  88. 88. Postgraduate Course
  89. 89. > 80 Fellows
  90. 90. CEBMa: what we do  Promote (seminars, papers, blogs, tweets)  Educate (universities & business schools)  Train & coach (companies > projects)  Support / REAs (companies)  Support / 2nd opinion (BS detector)
  • navyliu

    Jul. 25, 2018
  • damies

    May. 1, 2016
  • stephendale

    Feb. 22, 2016
  • riyadhonline

    Feb. 20, 2016

Big data, evidence-based, predictive analytics, today these terms are all over the place. Is this just another fad or an irreversible trend? An increasing group of HR leaders relies on science, critical thinking and data analyses to make decisions. Evidence-based HR, however, is still perceived by many as too time-consuming, narrow or impractical. Meanwhile, evidence-based practice is becoming mainstream in many other disciplines (like medicine). This is the momentum for pioneering HR leaders to seize the opportunity and make a difference with evidence. As part of an inclusive approach, valuing different perspectives. We will enter into the dialogue about the why, the what, and most of all the how of evidence-based HR. How to get started and how to blend it with softer, less tangible HR practices? A pragmatic introduction, with realistic ambitions and openness towards other approaches.

Views

Total views

3,974

On Slideshare

0

From embeds

0

Number of embeds

29

Actions

Downloads

142

Shares

0

Comments

0

Likes

4

×