Evidence-Based Practice in Health Care Management


Published on

Evidence-Based Practice in Health Care Management
Presentation for the American College of Physician Executives
April 27, 2013, New York

Published in: Education, Health & Medicine
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Evidence-Based Practice in Health Care Management

  1. 1. Evidence-Based ManagementA New Approach to Teachingthe Practice of ManagementACPE Annual Meeting, April 26 30, 2013, New YorkTony Kovner, Michiel Bosman, Eric Barends
  2. 2. EBMgt: My Experiences EBMgt as a Process Why Don‟t Managers use it? Learning from Medicine Contributions of Organizational Behavior Experience in Teaching The Politics of EBMgt
  3. 3. EBMgmt: Definition “…the systematic, evidence-informedpractice of management, incorporatingscientific knowledge in the content andprocess of making decisions.”(Rousseau 2012)
  4. 4. EBMgt as a process Framing the question behind the decision Finding the sources of information Assessing the accuracy of information Assessing the applicability of information Assessing the actionability of information Determining if the information is adequate(Hsu and others 2009)
  5. 5. Why don’t Managers use EBMgt? Never heard of it. They are using it. Cost now more probable than benefits in thefuture Managers have to be persuaded and trained The process doesn‟t clearly lead to one-bestway to intervene Politics impacts interests
  6. 6. Learning from EB-Medicine Interventions that achieve positivepredictable results Hindrances that blocked implementation ofEBM same now for EBMgt Financial as well as political impacts
  7. 7. Contributions of Organizational Behavior Hiring Talent-relying on structured interviews(Rousseau) Challenging Performance goals impacting onperformance (Latham and Locke) Managers who set a vision outperform othermanagers (Kirkpattrick)
  8. 8. Teaching EBMgt NYU / Wagner, Capstone course Projects Hourly nurse rounding process Redesigning the nurses‟ clinical ladder Improving the supply distribution process Identifying causes of emergency department waiting
  9. 9. The Politics of EBMgmt Evidence is not sufficient to change people‟sbehavior The quality of the argument and story-telling bypersons presenting the evidence is what persuadesstakeholders How persuade the manager that this is in his owninterest?
  10. 10. Michiel Bosman MD MMM FACPEMD, University of AmsterdamMMM, CMUExec PhD (2015), OSU Spears SchoolCollaborator, Center for EBMgmtSerial Entrepreneur
  11. 11.  “there is a large research-user gap” “practitioners do not read academic journals” “the findings of research into what is an effective interventionare not being translated into actual practice” “the relevance, quality and applicability of research isquestionable” “practice is being driven more by fads and fashions thanresearch” “many practices are doing more harm than good”What field is this?
  12. 12. McMaster University Medical School, CanadaMedicine: Founding fathersDavid Sackett Gordon Guyatt
  13. 13. How it all started
  14. 14.  More than 1 million articles in 40,000 medical journals peryear (= 1995; now probably more than 2 million). For aspecialist to keep up this means reading 25 articles everyday (for a primary care physician more than 100!)Problem: too much ‘evidence’
  15. 15. Problem: too much evidence HRM: 1,350 articles in 2010 (ABI/INFORM). For an HRmanager to keep up this means reading 3 to 4 articlesevery day (for a „general‟ manager more than 50!)
  16. 16. Problem: too many half truthsBTW: most of the research is seriously flawed or irrelevantfor practice. And some claim so is most management advice.
  17. 17. The 5 steps of Evidence-Based Practice1. Formulate a focused question (Ask)2. Search for the best available evidence (Acquire)3. Critically appraise the evidence (Appraise)4. Integrate the evidence with your professionalexpertise and apply (Apply)5. Monitor the outcome (Assess)
  18. 18. 1. Incompetent people benefit more from feedback thanhighly competent people.2. Task conflict improves work group performance whilerelational conflict harms it.3. Encouraging employees to participate in decisionmaking is more effective for improving organizationalperformance than setting performance goals.True or false?
  19. 19. How evidence-based are managers? 959 (US) + 626 (Dutch) HR professionals 35 statements, based on an extensive body of evidence true / false / uncertainOn average: 35% - 57% correctHR Professionals beliefs about effective human resource practices: correspondencebetween research and practice, (Rynes et al, 2002, Sanders et al 2008)
  20. 20. Evidence-Based Practice1991 Medicine1998 Education1999 Social care, public policy2000 Nursing2005 Criminal justice2010 Marketing2013 Healthcare Management?
  21. 21. Evidence-Based Consulting Close the research/practice gap Professor + Consultant EB Mgmt Workshops Teach the 5 step EB Practice process Answer business questions CAT: Critically Appraised Topic
  22. 22. Closing the research/practice gap Executive Doctorate in Management University of Maryland College Oklahoma State University Development of tools to measure EBMgmt Attitudes (EBMAS) andskills/competencies (Fresno)
  23. 23. Teaching Evidence Based Practiceto managers
  24. 24. Evidence?intuition, expertise, personalexperience, collectiveexperience, organizational facts &data, bestpractices, benchmarking, outcome of scientific research
  25. 25. All managers base theirdecisions on „evidence‟
  26. 26. BUT ...
  27. 27. Managers give littleor no consideration to thequality of the evidencethey base their decisions on
  28. 28. SO ...
  29. 29. Teach managers how to findand and critically appraisethe evidence before theymake their decision
  30. 30. The 5 steps of EBP1. Formulate a focused question (Ask)2. Search for the best available evidence (Acquire)3. Critically appraise the evidence (Appraise)4. Integrate the evidence with your professionalexpertise and apply (Apply)5. Monitor the outcome (Assess)
  31. 31. 0. Creating awarenessWhy do we need it?
  32. 32. Trust me, 20 years ofmanagement experience
  33. 33. Errors and Biases of Human Judgment
  34. 34.  Seeing order in randomness Mental corner cutting Misinterpretation of incomplete data Halo effect False consensus effect Attribution error Group think Self serving bias Sunk cost fallacy Cognitive dissonance reduction Confirmation bias Outcome bias Authority bias Small numbers fallacy Recall bias Anchoring bias Inaccurate covariation detectionErrors and Biases of Human Judgment
  35. 35. 1. Formulate a focused question
  36. 36. Asking the right question? Does team-building work? Does the introduction of self-steering teams work? Does lean management improve the performance ofour hospital? Is 360 degree feedback for doctors effective?
  37. 37.  What is a „team‟? What kind of team? In what contexts/ settings? What counts as „team-building‟? What does „work‟ mean?Focused question? Does team-building work?
  38. 38. P =PopulationI = Intervention or success factorC = ComparisonO = OutcomeC = ContextAnswerable question: PICOC
  39. 39. P =PopulationI = Intervention or successfactorC = ComparisonO = OutcomeC = ContextFocused question: PICOC Employee productivity? Patient satisfaction? Return on investment? Market share? Organizational commitment?
  40. 40. 2. Finding the best available evidence
  41. 41. Best availableexperiential evidenceBest availableinternal evidenceOrganizational valuesand stakeholders’concernsBest availableexternal evidenceEvidence-baseddecisionEvidence-based decision
  42. 42. Scientific databases: management ABI/INFORM Business Source Elite PubMed PsycINFO Web of Knowledge ERIC
  43. 43. 3. Critical appraisal of studiesMaking sense of evidence
  44. 44. Critical appraisal Construct validity(lean six sigma = value stream mapping, root causeanalysis, goal setting, participative decision making) Internal validity(does it work?) External validity(will it also work for my employees / organization?)
  45. 45. Levels of internal validity
  46. 46. Best research design?Best available
  47. 47. Critical appraisal
  48. 48. Step 4: Turning evidence intopractice
  49. 49. Applicable? organizational characteristics cultural & political aspects financial aspects /cost-effectiveness / ROI priorities change readiness / resistance to change implementation capacity timing
  50. 50. TEST?
  51. 51. CAT: Critically Appraised Topic
  52. 52. CAT-walk
  53. 53. Evidence-based practice:If doctors can do it managers can do it!