Evidence-Based ManagementHelping Managers to Make Better DecisionsDenise M. RousseauH.J. Heinz II University Professor of ...
EBMgt is the practice of making organizationaldecisions based upon conscientious use of1. Science-based principles & knowl...
 Better Decisions by Using Practices that Work (andavoiding those that don’t!) Defensible Decisions that Stand Up to Scr...
What does EBMgt Look Like?
Evidence-based Piloting?
 Chesley Sullenberger, USAIR pilot, has been avisiting scholar at UC Berkeley’s Collaborativefor Catastrophic Risk Manage...
 Has written and analyzed aviation accidentreports for over 20 years2. Reliance on Reliable and ValidOrganizational Facts
 Used Decision Aids to Support Good Decision: As Sullyconsidered what decision to make that day, he had hiscopilot review...
 The last person to leave the plane, ChesleySullenberger twice walked the plane’s aisle tocheck all passengers were off ...
In Sullenberger’s Own Words…“One way of looking at this might be thatfor 42 years, Ive been making small,regular deposits ...
How Is Sullenberger’s ExampleRelevant to Your Own LeadershipExperience?
EBMgt is a means toimprove decision quality.It’s a career, not a course.
 Evidence-based practice movements abound inmedicine, education, and public policy Management research frompsychology, e...
EBMgt Overcomes Limits of Unaided Decisions Bounded Rationality The Small NumbersProblem of IndividualExperience Prone ...
1. Get Evidence into the Conversation2. Use Relevant Scientific Evidence3. Use Reliable and Valid Business Facts4. Become ...
#1 Get evidence into the conversationRegularly ask “what’s the evidence…?”Illustration- Discuss with your seatmates…What’s...
 Evidence is not the same as „proof‟ or „hard facts‟ ... can be- so strong that no one doubts its correctness, or- so we...
 Evidence of effect (do!) Evidence of no effect (don‟t!) No evidence of effect (research!)Don’t confuse
#2 Use Relevant Scientific Evidence Focus on Action Principles Where Science is Clear Rely on Science-based Sources Exa...
Scientific EvidenceBest Scientific Evidence is based on large N (sample size ofpeople/organizations) well-controlled stu...
Peer Reviewed Journals
#3 Use Reliable and Valid Business FactsBest Business Facts are large numbers sampled relative to population(not single o...
Illustration--Discuss with your seatmates… What indicators does your organization mostcommonly use to make important deci...
 # Medication errors in Unit 1 were 200% greater in 2011 than Unit 2’s. Is patientsafety worse in Unit 1? Depends on numb...
#4 Become “Decision Aware” Identify different kinds of decisions learnersface? What kinds of different approaches areused...
“Decision Awareness” Promotes Decision Quality To manage decisions, know what decisions must bemade. Map out decisions t...
Five Good EBMgt HabitsAwareness Calls Attention to Decision Process.Proper Processes Improve Decision Quality What is the...
Using Evidence Well Requires YourOwn Critical Judgment
#5 Reflect on Decision’s EthicalImplications Who are stakeholders for this decision? Possible effects? How might the de...
Scientific Principles for Effective Teaching Set learning goals (2-5) Pre-test: where does learner stand on learning goa...
Evidence based management:closing the gap between research and practiceTurning Evidence into Practice& Practice into Evide...
J. Ehrlinger, K. Johnson, M. Banner, D. Dunning, J. Kruger. (2008) Why the unskilled are unaware: Furtherexplorations of (...
Upcoming SlideShare
Loading in …5
×

Denise Rousseau's Generic EBMgt Class 1

1,701 views

Published on

Denise Rousseau's Generic EBMgt Class 1: What It Means to be an Evidence-Based Manager

Published in: Education, Health & Medicine
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,701
On SlideShare
0
From Embeds
0
Number of Embeds
18
Actions
Shares
0
Downloads
41
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • How can we supplement your experience?
  • How can we supplement your experience?
  • How can we supplement your experience?
  • How can we supplement your experience?
  • How can we supplement your experience?
  • How can we supplement your experience?
  • How can we supplement your experience?
  • How can we supplement your experience?
  • How can we supplement your experience?
  • How can we supplement your experience?
  • Denise Rousseau's Generic EBMgt Class 1

    1. 1. Evidence-Based ManagementHelping Managers to Make Better DecisionsDenise M. RousseauH.J. Heinz II University Professor of Organizational BehaviorCarnegie Mellon Universitydenise@cmu.eduWhat It Means to be an Evidence-Based Manager
    2. 2. EBMgt is the practice of making organizationaldecisions based upon conscientious use of1. Science-based principles & knowledge2. Valid & relevant organizational and business facts3. Critical thinking aided by decision supports4. Ethical considerations (i.e., effects on stakeholders)What is Evidence-Based Management?
    3. 3.  Better Decisions by Using Practices that Work (andavoiding those that don’t!) Defensible Decisions that Stand Up to Scrutiny (usingbest evidence and best process) Developing Expertise throughout a Career(experience can be a poor teacher--bad habits!)20 years of valid experience is different than 1 year of experiencerepeated 20 times!Why Should We Care about EBMgt?
    4. 4. What does EBMgt Look Like?
    5. 5. Evidence-based Piloting?
    6. 6.  Chesley Sullenberger, USAIR pilot, has been avisiting scholar at UC Berkeley’s Collaborativefor Catastrophic Risk Management since 2007 Does research on how to make decisions tomaintain safety despite technologicalcomplexity and crisis conditions1. Use of Scientific Findings
    7. 7.  Has written and analyzed aviation accidentreports for over 20 years2. Reliance on Reliable and ValidOrganizational Facts
    8. 8.  Used Decision Aids to Support Good Decision: As Sullyconsidered what decision to make that day, he had hiscopilot review and follow all checklists on boardrelevant to crash landings Formal Education to Prime His Skills: Sully is a graduateof the U.S. Air Force Academy and holds mastersdegrees from both Purdue University in IndustrialPsychology and the University of Northern Colorado inPublic Administration3. Mindful Decision Making:Becoming Decision Aware
    9. 9.  The last person to leave the plane, ChesleySullenberger twice walked the plane’s aisle tocheck all passengers were off Sully’s last act onboard was to grab thepassenger list. Used on-shore to verify rescueof all passengers and crew4. Ethics and Responsibility toStakeholders
    10. 10. In Sullenberger’s Own Words…“One way of looking at this might be thatfor 42 years, Ive been making small,regular deposits in this bank of experience,education and training.And on January 15, the balance was sufficientso that I could make a very large withdrawal.”
    11. 11. How Is Sullenberger’s ExampleRelevant to Your Own LeadershipExperience?
    12. 12. EBMgt is a means toimprove decision quality.It’s a career, not a course.
    13. 13.  Evidence-based practice movements abound inmedicine, education, and public policy Management research frompsychology, engineering, operations research (ETC.)yields 1000s of studies annually Internet (scholar.google.com) gives ready access Innovative companies now hiring “chief evidenceofficers” Public demands accountability (quality decisions thatare defensible)The Zeitgeist
    14. 14. EBMgt Overcomes Limits of Unaided Decisions Bounded Rationality The Small NumbersProblem of IndividualExperience Prone to See PatternsEven in Random Data Critical Thinking Decision Supports Research• Large Ns > individualexperience• Controls reduce biasThe “Human” Problem Evidence-Based Practice
    15. 15. 1. Get Evidence into the Conversation2. Use Relevant Scientific Evidence3. Use Reliable and Valid Business Facts4. Become “Decision Aware” and UseAppropriate Processes5. Reflect on Decision’s Ethical and StakeholderImplicationsFive Good EBMgt Habits
    16. 16. #1 Get evidence into the conversationRegularly ask “what’s the evidence…?”Illustration- Discuss with your seatmates…What’s a practice in your organization that you suspect mightnot be NOT evidence-based?Five Good EBMgt Habits
    17. 17.  Evidence is not the same as „proof‟ or „hard facts‟ ... can be- so strong that no one doubts its correctness, or- so weak that it is hardly convincing at allWhat is evidence?
    18. 18.  Evidence of effect (do!) Evidence of no effect (don‟t!) No evidence of effect (research!)Don’t confuse
    19. 19. #2 Use Relevant Scientific Evidence Focus on Action Principles Where Science is Clear Rely on Science-based Sources Example: Locke’s Handbook of Organizational Behavior(access electronic copy in blackboard) Peer-reviewed research, especially meta-analysesReduce dysfunctional variations in practiceBuild effective routines, procedures, checklistsFive Good EBMgt Habits
    20. 20. Scientific EvidenceBest Scientific Evidence is based on large N (sample size ofpeople/organizations) well-controlled studies with comparison groups&/or longitudinal data peer-reviewedFive Good EBMgt Habits
    21. 21. Peer Reviewed Journals
    22. 22. #3 Use Reliable and Valid Business FactsBest Business Facts are large numbers sampled relative to population(not single or isolated cases, e.g. sales/# sales calls) linked to context (season, location, #users, etc.) provide key indicators for business decisionsFive Good EBMgt Habits
    23. 23. Illustration--Discuss with your seatmates… What indicators does your organization mostcommonly use to make important decisions? Are these the “best business facts” you need tomake these decisions? What indicators would be more useful, if youcould get them??Five Good EBMgt Habits
    24. 24.  # Medication errors in Unit 1 were 200% greater in 2011 than Unit 2’s. Is patientsafety worse in Unit 1? Depends on number of unsafe incidents divided by #patients or # procedures—needs a control. Mike has w/10 subordinates & 20% turnover while Kim has 55 employees & 10%turnover. Is retention better in one? Hard to determine. Small N’s have greaterbias and are more variable. McDonald’s stores average 300+% turnover/year. Does Mickey D. have a problem?Depends on industry comparison and business strategy. Company A managers focus decisions on monthly cost, downtime and revenues.Company B managers focus on service quality, employee retention andprofitability by customer category. So what? B’s more diverse performancecriteria can promote attention to longer-term and growth-oriented outcomes.A’s narrower economic focus can promote shorter-term thinking.Help Learner How to Interpret Business Facts
    25. 25. #4 Become “Decision Aware” Identify different kinds of decisions learnersface? What kinds of different approaches areused to them? Why? How can you determine whether you made a“good decision” when you cannot know theoutcome? (The answer to this question is what isknown as “decision quality”)Five Good EBMgt Habits
    26. 26. “Decision Awareness” Promotes Decision Quality To manage decisions, know what decisions must bemade. Map out decisions that affect key outcomes. Who is responsible? (Are they prepared?) What information is required? (Will it be available whenneeded?)Five Good EBMgt Habits
    27. 27. Five Good EBMgt HabitsAwareness Calls Attention to Decision Process.Proper Processes Improve Decision Quality What is the process for making the decision? Different processes work better…- for routine decisions (create validated checklists and action plans)- for decisions with known unknowns (systematic sequence ofconsiderations)- for decisions with unknown unknowns (pilot-tests and trial/experiment) Decisions have an “aftermath” and a “pre-math” that agood manager actively manages. Is the decision well-managed? Help make it so.
    28. 28. Using Evidence Well Requires YourOwn Critical Judgment
    29. 29. #5 Reflect on Decision’s EthicalImplications Who are stakeholders for this decision? Possible effects? How might the decision be altered to optimize positivestakeholder effects and reduce negative?Five Good EBMgt Habits
    30. 30. Scientific Principles for Effective Teaching Set learning goals (2-5) Pre-test: where does learner stand on learning goal before course Build opportunities for practicing those learnings throughoutcourse (curriculum) Post-test: Measure progress on each learning goal and providefeedback Feedback & Redesign: Use feedback to make course moreeffective over timeFive Good EBMgt Habits
    31. 31. Evidence based management:closing the gap between research and practiceTurning Evidence into Practice& Practice into Evidence
    32. 32. J. Ehrlinger, K. Johnson, M. Banner, D. Dunning, J. Kruger. (2008) Why the unskilled are unaware: Furtherexplorations of (absent) self-insight among the incompetent. Organizational Behavior and Human DecisionProcesses, 105,(1) pg. 98E.A. Locke (ed.), Handbook of Principles of Organizational Behavior, 2nd edition, 2009. Malden, MA: Blackwell.D. M. Rousseau (2012) Oxford Handbook of Evidence-Based Management, New York.D.M. Rousseau, D.M. & E. Barends (2011) Becoming an evidence-based manager. Human ResourceManagement Journal, 21, 221-235.D.M. Rousseau, J. Manning & D. Denyer (2008) Evidence in Management and Organizational Science:Assembling the field’s full weight of scientific knowledge through reflective reviews. Annals of theAcademy of Management, 2, 475-515.R.C. Schank, D. Llyras & E. Soloway (2010) The future of decision making. New York: Palmgrave Macmillan.J.F. Yates. (2003). Decision management. San Francisco: Jossey-Bass.J.F. Yates & M.D. Tschirhart (2006). Decision making expertise. In K. A. Ericsson, N. Charness, P. J. Feltovich, &R. R. Hoffman. (Eds.). Cambridge handbook of expertise and expert performance (pp. 421-438). New York:Cambridge University Press.J.F. Yates, E.S. Veinott & A.L. Patalano (2003). Hard decisions, bad decisions: On decision quality and decisionaiding. In S. L. Schneider & J. C. Shanteau (Eds.), Emerging perspectives on judgment and decision research(pp. 13-63). New York: Cambridge University Press.Got Evidence? References

    ×