Big data, evidence-based, predictive analytics, today these terms are all over the place. Is this just another fad or an irreversible trend? An increasing group of HR leaders relies on science, critical thinking and data analyses to make decisions. Evidence-based HR, however, is still perceived by many as too time-consuming, narrow or impractical. Meanwhile, evidence-based practice is becoming mainstream in many other disciplines (like medicine). This is the momentum for pioneering HR leaders to seize the opportunity and make a difference with evidence. As part of an inclusive approach, valuing different perspectives. We will enter into the dialogue about the why, the what, and most of all the how of evidence-based HR. How to get started and how to blend it with softer, less tangible HR practices? A pragmatic introduction, with realistic ambitions and openness towards other approaches.

1. Getting started with
Evidence-Based Management
Dublin, April 28th, 2016

2. Evidence-Based HR: is it ‘a thing’?

9. 25 min EBMgt: What is it and why you you need it?
20 min A practical example
Agenda

10. Evidence based management:
What is it?

12. Evidence-based management
Central Premise:
Decisions should be based on
the ‘best available evidence‘.

13. Evidence?
information, facts or data
supporting (or contradicting)
a claim, assumption or hypothesis

14. Evidence?
outcome of scientific research,
organizational facts & figures,
benchmarking, best practices,
personal experience

15. All managers and leaders base
their decisions on ‘evidence’

16. But…many managers pay
little or no attention to
the quality of the evidence
they base their decisions on
and use
too few sources of evidence

17. Trust me, 20 years of
management experience

18. Sources of evidence
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess

19. Evidence based practice:
Where does it come from?

20. McMaster University Medical School, Canada
Medicine: Founding fathers
David Sackett Gordon Guyatt

21. How it all started

22. 1. Ask: translate a practical issue into an answerable question
2. Acquire: systematically search for and retrieve the evidence
3. Appraise: critically judge the trustworthiness of the evidence
4. Apply: incorporate the evidence into the decision-making process
5. Assess: evaluate the outcome of the decision taken
5 steps of EBmed

23. Evidence-Based Practice
1991 Medicine
1998 Education
2000 Social care, public policy
Nursing, Criminal justice,
Policing, Architecture, Conservation
2010 Management

26. Evidence-Based Practice

27. Evidence-Based Practice

28. Evidence-based … whatever
=
the use of evidence from multiple
sources to increase the likelihood of a
favourable outcome
Focus on the decision making process
Think in terms of probability

29. In general: managers don’t like EBMgt

30.  Undermines formal authority
 They feel it constrains freedom to make
managerial decisions
 Speed valued and rewarded more than accuracy
 Feel they cannot use their own experience and
judgment (not true)
 Managers not necessarily rewarded for doing
what works (organizations rarely evaluate)
 THEY LOVE FADS & QUICK FIXES
Why don’t managers like EBMgt?

32. 32

33. Evidence-Based Decision-Making
Why do we need it?

34. Advice: lie babies down to sleep on their belly
(unanimous support through to the 1990s)
Example: medicine

35. Nr of cot deaths (Holland)

36. Collateralized Debt Obligations > AAA
p = 0.12 (about 1 chance in 850) default in 5 years
Example: finance

37. Forecasted Actual
Forecasted and actual 5-year default rates for
AAA-rated CDO tranches
0.12%
28%

38. Scared straight
Example: policy / prevention

40. Example: HR management

41. 1. Incompetent people benefit more from feedback than
highly competent people.
2. Task conflict improves work group performance while
relational conflict harms it.
3. Encouraging employees to participate in decision
making is more effective for improving organizational
performance than setting performance goals.
Likely or unlikely?

42. How evidence-based are HR managers?
 959 (US) + 626 (Dutch) HR professionals
 35 statements, based on an extensive body of evidence
 true / false / uncertain
HR Professionals' beliefs about effective human resource practices: correspondence
between research and practice, (Rynes et al, 2002, Sanders et al 2008)

43. Outcome: not better than random chance

44. Relying on only 1 source: bad idea!
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess

45. Discuss with your neighbor (1 min)
Over a 5 year period,
why is an orthopedic surgeon's
experience, as a rule, more trustworthy
than an change manager’s experience?
0

46. Developing expertise
1. A sufficiently regular, predictable environment
2. Numerous opportunities to practice
3. Receive accurate (objective) feedback
The management domain is not highly
favorable to expertise!

47. Learning from feedback is hard!

48. Bounded rationality

49. How your brain works
System 1
 Fast
 Intuitive, associative
 heuristics & biases
 emotional
System 2
 Lazy
 Slow
 Deliberate
 Rational

51. System 1: short cuts

52. System 1 or system 2?
10 seconds

53. System 1 or system 2?
 A bat and a ball cost $1.10 in total.
 The bat costs $1 more than the ball
 How much does the ball cost?
0

54. System 1: necessary to survive
95%

55.  Pattern recognition
 Overconfidence bias
 Halo effect
 False consensus effect
 Group think
 Self serving attribution bias
 Sunk cost fallacy
 Cognitive dissonance reduction
System 1: prone to cognitive errors
 Confirmation bias
 Authority bias
 Small numbers fallacy
 In-group bias
 Recall bias
 Anchoring bias
 Availability bias

56.  Pattern recognition
 Overconfidence bias
 Halo effect
 False consensus effect
 Group think
 Self serving attribution bias
 Sunk cost fallacy
 Cognitive dissonance reduction
System 1: prone to cognitive errors
 Confirmation bias
 Authority bias
 Small numbers fallacy
 In-group bias
 Recall bias
 Anchoring bias
 Availability bias

57. “I’ve been studying judgment for 45 years, and I’m no better
than when I started. I make extreme predictions. I’m over-
confident. I fall for every one of the biases.”

58. Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess
Four sources of evidence (not only 1)

59. The performance of
knowledge workers
A Practical Example

60.  550 beds
 3300 employees
 210 medical specialists
 225,000 admissions
 Top Clinical & Teaching hospital
Organization

61. 2015: 7.2 2016: 6.3

62. How can we increase job satisfaction
and employee engagement?
Dear HR department,

64. Evidence-based approach, step 1: ASK
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess

65.  What is the problem?
 Why is this a problem: what are its organizational
consequences?
 How big: what is its impact on the organization when
nothing is done?
 Why does this problem exist, what is the assumed
major cause?
 What is the assumed causal mechanism? How does
the cause lead to the problem and its consequences?
Step 1: What is the problem?

66. problem & underlying cause
hidden assumptions
causal mechanism
Some terminology

67. A happy & engaged employee
is a productive employee
Fundamental assumption

68.  What is the evidence for this assumption?
 Where multiple sources consulted?
 How trustworthy is the evidence?
Step 2: What is the evidence?

69. Let’s have a look
Professional
experience and
judgment
Organizational data,
facts and figures
Stakeholders’ values
and concerns
Scientific
research
outcomes
Ask
Acquire
Appraise
Apply
Assess
problem solution

75. GREAT! NOW WHAT?
Outcome

76. Evidence-based managers,
please

77. Evidence-based managers,
please

78. Step 1: ASK
Translate a practical issue into an
answerable question

79. Population? Knowledge workers!
Whether nurses, lawyers, engineers, managers, or staff members,
nowadays most workers in organizations are highly dependent on
information and communication technology and are involved in
work that involves a high level of cognitive activity.

80. Question
“Which of the factors that are related to
the performance of knowledge workers
are most widely studied and what is
known of their effect?”

81. Step 2: ACQUIRE
Search for the best available scientific evidence

82.  ABI, BSP, PsycINFO
 Scholarly journals, peer reviewed
 1980 – 2013
 English
 performance, productivity, knowledge work*
ACQUIRE

84. step 3: APPRAISE & AGGREGATE

85. Effect size?

86. Largest effect
1. Social cohesion .5 / .7
2. Perceived supervisory support .5
3. Information sharing / TM .5
4. Vision / goal clarity .5
5. Trust .3 / .6

87. step 3b: CROSS VALIDATE

88. Step 4: APPLY

89. Three examples
social cohesion supervisory
support
information
sharing

90. Social cohesion

91. Social cohesion
… a shared liking or team attraction
that includes bonds of friendship, caring,
closeness, and enjoyment of each
other’s company.

92. Social cohesion

93. Measuring social cohesion

94. Perceived supervisory support

95. …how employees feel the supervisor
helps them in times of need, praises
them for a job well done or recognizes
them for extra effort.
Perceived supervisory support

96. Perceived supervisory support

97. Measuring perc. sup. support

98. Information sharing

99. Information sharing?
…refers to how teams pool and access their
knowledge and expertise – which positively
affects decision making and team processes.
This has led to the idea of a team ‘Transactive
Memory System’ (TMS), which can be thought of
as a collective memory in a collective mind -
enabling a team to think and act together

100. Information sharing

101. Measuring information sharing

102. Outcome
The departments with the lowest performance
scored under average on most factors

103. Reactions
Who
knew?

104. Evidence-based … whatever
=
the use of evidence from multiple
sources to increase the likelihood of a
favourable outcome
Focus on the decision making process
Think in terms of probability

105. Multiple sources of evidence
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess

107. Postgraduate Course

108. Postgraduate Course

109. > 80 Fellows

110. CEBMa: what we do
 Promote (seminars, papers, blogs, tweets)
 Educate (universities & business schools)
 Train & coach (companies > projects)
 Support / REAs (companies)
 Support / 2nd opinion (BS detector)