This document discusses evidence-based practice for hospital managers. It describes a pilot program that trained managers in evidence-based decision making. The program evaluated current decision processes, examined completed projects, and made recommendations for new projects using an evidence-based perspective. Preliminary results found managers focused more on procedures than evidence and relied on anecdotal sources. Analyzing decisions retrospectively using evidence identified gaps managers were unaware of. The program showed how an evidence-based approach leads to different and potentially better decisions informed by scientific research rather than individual experience alone. Lessons included it being a new approach that recalibrates power dynamics and the importance of senior management support for successful implementation of evidence-based practice in management.

1. Evidence-Based Practice
AUPHA Annual Meeting, June 20, 2013, Monterey
If doctors can do it managers can do it?

2. Evidence?
outcome of scientific research,
organizational facts & data,
benchmarking, best practices,
collective experience, personal
experience, intuition

3. All managers base their
decisions on ‘evidence’

4. However ...

5. Many managers pay little
or no attention to the quality
of the evidence they base
their decisions on

6. Trust me, 20 years of
management experience

7. Teach managers how to
critically evaluate the validity,
and generalizability of the
evidence and help them find
‘the best available’ evidence

8. Evidence-based decision

9. What is the added value of evidence-
based practice for managers within
the field of hospital care?
Proof of concept

10. Teaching Hospital
- 6 managers
University Hospital
- 4 managers
Evidence based pilot

11. Phase 1: Training managers in the principles of EBP
Phase 2: Examination of the current decision making
processes that managers are using
Phase 3: Evaluation of 4 completed projects from an
EB perspective (retrospective)
Phase 4: Making EB recommendations for 4 new
projects (prospective)
Phase 5: Evaluation
Evidence based pilot

12. Some preliminary results

13. Decision making process

14. Decision making process
 Focus on procedures instead of evidence
 Internal politics and power struggles
 No critical appraisal of the evidence at hand
 Relying on anecdotal evidence (workshops, best
practices, popular management books, consultants)
 One option (sometimes two)
 Bias: Outcome, Halo, Confirmation, etc.

15. Post mortem analysis

16. Evidence-based perspective
NOT: Did we made the right decision?
BUT: Is there evidence from scientific research to
support (or call into question) the approach
taken?

17. Post mortem: leadership training

18. leadership training: dm process
 No problem definition
 No organizational evidence consulted
 Selection of training companies based on
experience, recommendation or reputation
 No explicit selection criteria / procedure
 ‘Best’ presentation has won: one size fits all

19. leadership training: scientific evidence
 15 meta analyses, 5 relevant
 37 (‘systematic’) reviews, 2 relevant
 Lots of relevant primary studies

20. leadership training: scientific evidence
 Long history (30 yrs): moderate effect sizes
 Senior & middle managers tend to benefit more than
managers at the supervisory level
 Effect on ‘poor’ leaders is limited.
 Leadership trainings that focus on interpersonal / social
skills show higher effect sizes than those based on a
specific leadership ‘model’

21. Reactions
Who
knew?
Denial
Anger
Bargaining
Acceptance

22. Prospective /
EB recommendations

23. Questions / projects
 360 degree feedback
 Financial incentives
 Lean Six Sigma
 Hand Hygiene
 Goal setting
 Value Based Health Care
 Downsizing

24. Evidence-based perspective
NOT: What works?
BUT: What are, given the target group, the problem
and the context involved, the main factors
determining the success or failure of the project
that need to be taken into account?

25. Prospective: Multi Source Feedback

26. Multi Source Feedback: background
 IFMS: based on multi source feedback
 Regulating bodies and insurance companies
(KPI’s – prices/ revenue)
 Based on CANMEDS, no standard method
 New market: consulting firms

27. Process
 Scoping session: inventory of the aspects relevant
to the question
 Session with leading academic
 Search in relevant databases
 Critical Appraisal
 Summary / research synthesis
 Recommendations / guidelines

28. Multi Source Feedback: scientific evidence
 223 primary studies on MSF, 42 relevant
 6 meta analyses or systematic reviews on
MSF, 3 relevant
 18 meta analyses or systematic reviews on
‘feedback’ or ‘performance appraisal’,
5 relevant

29. Content of the
feedback (neg vs pos)
Way of the delivery
of the feedback
Interpretation of the
feedbackPersonality of the
ratee
Feedback orientation
of the ratee
Type and
number of ratersSelection of
raters
Rater reliablity
(patients, nurses,
colleagues)
Type of
response
scale
Development vs
perfomance appraisal
Organizational
culture
Perceived procedural
justice
Multi Source Feedback: main factors

30. Reactions
Who
knew?
Wow!
Great!
Good stuff!
Relevant!

32. Lessons learned I
 New approach
 Recalibrates the power dynamics (accountability!)
 The profit is in the process
 Different (better?) decisions were made
 Doctors love it!

33. Lessons learned II
 Hard for individual managers
 It starts with the senior management team
 It’s all about accountability
 Support system
 EBP > Planning & Control

34. One day, maybe …
Chief Evidence Officer