1.
Getting started with
Evidence-Based Management
Dublin, April 28th, 2016

2.
Evidence-Based HR: is it ‘a thing’?

3.
25 min EBMgt: What is it and why you you need it?
20 min A practical example
Agenda

4.
Evidence based management:
What is it?

5.
Evidence-based management
Central Premise:
Decisions should be based on
the ‘best available evidence‘.

6.
Evidence?
information, facts or data
supporting (or contradicting)
a claim, assumption or hypothesis

7.
Evidence?
outcome of scientific research,
organizational facts & figures,
benchmarking, best practices,
personal experience

8.
All managers and leaders base
their decisions on ‘evidence’

9.
But…many managers pay
little or no attention to
the quality of the evidence
they base their decisions on
and use
too few sources of evidence

10.
Trust me, 20 years of
management experience

11.
Sources of evidence
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess

12.
Evidence based practice:
Where does it come from?

13.
McMaster University Medical School, Canada
Medicine: Founding fathers
David Sackett Gordon Guyatt

14.
How it all started

15.
1. Ask: translate a practical issue into an answerable question
2. Acquire: systematically search for and retrieve the evidence
3. Appraise: critically judge the trustworthiness of the evidence
4. Apply: incorporate the evidence into the decision-making process
5. Assess: evaluate the outcome of the decision taken
5 steps of EBmed

16.
Evidence-Based Practice
1991 Medicine
1998 Education
2000 Social care, public policy
Nursing, Criminal justice,
Policing, Architecture, Conservation
2010 Management

17.
Evidence-Based Practice

18.
Evidence-Based Practice

19.
Evidence-based … whatever
=
the use of evidence from multiple
sources to increase the likelihood of a
favourable outcome
Focus on the decision making process
Think in terms of probability

20.
In general: managers don’t like EBMgt

21.
 Undermines formal authority
 They feel it constrains freedom to make
managerial decisions
 Speed valued and rewarded more than accuracy
 Feel they cannot use their own experience and
judgment (not true)
 Managers not necessarily rewarded for doing
what works (organizations rarely evaluate)
 THEY LOVE FADS & QUICK FIXES
Why don’t managers like EBMgt?

22.
32

23.
Evidence-Based Decision-Making
Why do we need it?

24.
Advice: lie babies down to sleep on their belly
(unanimous support through to the 1990s)
Example: medicine

25.
Nr of cot deaths (Holland)

26.
Collateralized Debt Obligations > AAA
p = 0.12 (about 1 chance in 850) default in 5 years
Example: finance

27.
Forecasted Actual
Forecasted and actual 5-year default rates for
AAA-rated CDO tranches
0.12%
28%

28.
Scared straight
Example: policy / prevention

29.
Example: HR management

30.
1. Incompetent people benefit more from feedback than
highly competent people.
2. Task conflict improves work group performance while
relational conflict harms it.
3. Encouraging employees to participate in decision
making is more effective for improving organizational
performance than setting performance goals.
Likely or unlikely?

31.
How evidence-based are HR managers?
 959 (US) + 626 (Dutch) HR professionals
 35 statements, based on an extensive body of evidence
 true / false / uncertain
HR Professionals' beliefs about effective human resource practices: correspondence
between research and practice, (Rynes et al, 2002, Sanders et al 2008)

32.
Outcome: not better than random chance

33.
Relying on only 1 source: bad idea!
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess

34.
Discuss with your neighbor (1 min)
Over a 5 year period,
why is an orthopedic surgeon's
experience, as a rule, more trustworthy
than an change manager’s experience?
0

35.
Developing expertise
1. A sufficiently regular, predictable environment
2. Numerous opportunities to practice
3. Receive accurate (objective) feedback
The management domain is not highly
favorable to expertise!

36.
Learning from feedback is hard!

37.
Bounded rationality

38.
How your brain works
System 1
 Fast
 Intuitive, associative
 heuristics & biases
 emotional
System 2
 Lazy
 Slow
 Deliberate
 Rational

39.
System 1: short cuts

40.
System 1 or system 2?
10 seconds

41.
System 1 or system 2?
 A bat and a ball cost $1.10 in total.
 The bat costs $1 more than the ball
 How much does the ball cost?
0

42.
System 1: necessary to survive
95%

43.
 Pattern recognition
 Overconfidence bias
 Halo effect
 False consensus effect
 Group think
 Self serving attribution bias
 Sunk cost fallacy
 Cognitive dissonance reduction
System 1: prone to cognitive errors
 Confirmation bias
 Authority bias
 Small numbers fallacy
 In-group bias
 Recall bias
 Anchoring bias
 Availability bias

44.
 Pattern recognition
 Overconfidence bias
 Halo effect
 False consensus effect
 Group think
 Self serving attribution bias
 Sunk cost fallacy
 Cognitive dissonance reduction
System 1: prone to cognitive errors
 Confirmation bias
 Authority bias
 Small numbers fallacy
 In-group bias
 Recall bias
 Anchoring bias
 Availability bias

45.
“I’ve been studying judgment for 45 years, and I’m no better
than when I started. I make extreme predictions. I’m over-
confident. I fall for every one of the biases.”

46.
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess
Four sources of evidence (not only 1)

47.
The performance of
knowledge workers
A Practical Example

48.
 550 beds
 3300 employees
 210 medical specialists
 225,000 admissions
 Top Clinical & Teaching hospital
Organization

49.
2015: 7.2 2016: 6.3

50.
How can we increase job satisfaction
and employee engagement?
Dear HR department,

51.
Evidence-based approach, step 1: ASK
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess

52.
 What is the problem?
 Why is this a problem: what are its organizational
consequences?
 How big: what is its impact on the organization when
nothing is done?
 Why does this problem exist, what is the assumed
major cause?
 What is the assumed causal mechanism? How does
the cause lead to the problem and its consequences?
Step 1: What is the problem?

53.
problem & underlying cause
hidden assumptions
causal mechanism
Some terminology

54.
A happy & engaged employee
is a productive employee
Fundamental assumption

55.
 What is the evidence for this assumption?
 Where multiple sources consulted?
 How trustworthy is the evidence?
Step 2: What is the evidence?

56.
Let’s have a look
Professional
experience and
judgment
Organizational data,
facts and figures
Stakeholders’ values
and concerns
Scientific
research
outcomes
Ask
Acquire
Appraise
Apply
Assess
problem solution

57.
GREAT! NOW WHAT?
Outcome

58.
Evidence-based managers,
please

59.
Evidence-based managers,
please

60.
Step 1: ASK
Translate a practical issue into an
answerable question

61.
Population? Knowledge workers!
Whether nurses, lawyers, engineers, managers, or staff members,
nowadays most workers in organizations are highly dependent on
information and communication technology and are involved in
work that involves a high level of cognitive activity.

62.
Question
“Which of the factors that are related to
the performance of knowledge workers
are most widely studied and what is
known of their effect?”

63.
Step 2: ACQUIRE
Search for the best available scientific evidence

64.
 ABI, BSP, PsycINFO
 Scholarly journals, peer reviewed
 1980 – 2013
 English
 performance, productivity, knowledge work*
ACQUIRE

65.
step 3: APPRAISE & AGGREGATE

66.
Effect size?

67.
Largest effect
1. Social cohesion .5 / .7
2. Perceived supervisory support .5
3. Information sharing / TM .5
4. Vision / goal clarity .5
5. Trust .3 / .6

68.
step 3b: CROSS VALIDATE

69.
Step 4: APPLY

70.
Three examples
social cohesion supervisory
support
information
sharing

71.
Social cohesion

72.
Social cohesion
… a shared liking or team attraction
that includes bonds of friendship, caring,
closeness, and enjoyment of each
other’s company.

73.
Social cohesion

74.
Measuring social cohesion

75.
Perceived supervisory support

76.
…how employees feel the supervisor
helps them in times of need, praises
them for a job well done or recognizes
them for extra effort.
Perceived supervisory support

77.
Perceived supervisory support

78.
Measuring perc. sup. support

79.
Information sharing

80.
Information sharing?
…refers to how teams pool and access their
knowledge and expertise – which positively
affects decision making and team processes.
This has led to the idea of a team ‘Transactive
Memory System’ (TMS), which can be thought of
as a collective memory in a collective mind -
enabling a team to think and act together

81.
Information sharing

82.
Measuring information sharing

83.
Outcome
The departments with the lowest performance
scored under average on most factors

84.
Reactions
Who
knew?

85.
Evidence-based … whatever
=
the use of evidence from multiple
sources to increase the likelihood of a
favourable outcome
Focus on the decision making process
Think in terms of probability

86.
Multiple sources of evidence
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess

87.
Postgraduate Course

88.
Postgraduate Course

89.
> 80 Fellows

90.
CEBMa: what we do
 Promote (seminars, papers, blogs, tweets)
 Educate (universities & business schools)
 Train & coach (companies > projects)
 Support / REAs (companies)
 Support / 2nd opinion (BS detector)