• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Presentation Teaching Evidence-Based Management NYU Wagner 2014
 

Presentation Teaching Evidence-Based Management NYU Wagner 2014

on

  • 127 views

Presentation workshop 'Teaching Evidence-Based Management" at NYU Wagner, May 6th, 2014

Presentation workshop 'Teaching Evidence-Based Management" at NYU Wagner, May 6th, 2014
Eric Barends, Denise Rousseau and Tony Kovner

Statistics

Views

Total Views
127
Views on SlideShare
127
Embed Views
0

Actions

Likes
0
Downloads
5
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Presentation Teaching Evidence-Based Management NYU Wagner 2014 Presentation Teaching Evidence-Based Management NYU Wagner 2014 Presentation Transcript

    • Evidence-Based Practice A New Approach of Teaching The Practice of Management Eric Barends, CEBMa – Denise Rousseau, CMU May 6th, 2014
    • Mission Today • Teaching Evidence-based practice (in 1 hour) • Disappointment • Inspiration Who are we and what is our mission (today)? Eric Barends • manager • teacher • director • CEBMa Denise Rousseau • researcher • teacher • professor • CMU / Heinz / Tepper
    • Postgraduate Course
    • Postgraduate Course
    • Postgraduate Course
    • Postgraduate Course Current developments Education Access to research databases Rapid Evidence Assessments Building a community
    • Postgraduate Course CEBMa Database of Evidence Summaries Online learning modules Accreditational bodies Future developments: practice
    • 1. Some background 2. Teaching: learning principles 3. Teaching: curriculum 4. Teaching: examples 5. Your questions Today
    • Evidence based management: What is it? 1. Some background
    • Evidence-based practice Central Premise: Decisions should be based on a combination of critical thinking and the ‘best available evidence‘.
    • Evidence? findings from scientific research, organizational facts & figures, benchmarking, best practices, professional experience 16
    • All managers base their decisions on ‘evidence’ 17
    • But…many managers pay little or no attention to the quality of the evidence they base their decisions on 18
    • Trust me, 20 years of management experience 19
    • SO ... 20
    • Teach managers how to critically evaluate the trustworthiness of evidence from multiple sources and help them find ‘the best available’ evidence
    • An example
    • An example
    • An example
    • Maslow, A.H. "A Theory of Human Motivation” An example
    • An example
    • Professional experience and judgment Organizational data, facts and figures Stakeholders’ values and concerns Scientific research findings Ask Acquire Appraise Aggregate Apply Assess An example
    • Professional experience and judgment
    • Maslow, A.H. (1943). "A Theory of Human Motivation," Psychological Review 50(4) Wahba, M. A., & Bridwell, L. G. (1976). Maslow reconsidered: A review of research on the need hierarchy theory. Organizational Behavior and Human Performance, 15(2) Scientific research findings
    • How evidence-based are we (managers)? “I’ve never thought I need more evidence before making a decision; I know what needs to be done, we get on with it and we get results.”
    • 1. Incompetent people benefit more from feedback than highly competent people. 2. Task conflict improves work group performance while relational conflict harms it. 3. Encouraging employees to participate in decision making is more effective for improving organizational performance than setting performance goals. True (likely) or false (not likely)?
    • How evidence-based are we?  959 (US) + 626 (Dutch) HR professionals  35 statements, based on an extensive body of evidence  true / false / uncertain HR Professionals' beliefs about effective human resource practices: correspondence between research and practice, (Rynes et al, 2002, Sanders et al 2008)
    • Outcome: not better than random chance
    • EBP: Teach managers how to critically evaluate the trustworthiness of evidence from multiple sources and help them find ‘the best available’ evidence
    • 2. Teaching EBP: Learning principles
    • Discuss with your neighbours: When it comes to teaching, what are important learning principles? 60595857565453525150494847464544434241403938373635343332313029282726252423222120191817161514131211109876543210
    • Learning principles 1. The brain needs time to get used to new things. Longer periods in between practice sessions lead to a longer overall retention. 2. Training program should include opportunities for practice, linked to real world situations . 3. The human brain seeks comfort in what it knows and is familiar with; addressing / confronting prior knowledge increases understanding (start from where the students are)
    • Learning principles 4. Higher order thinking only happens when people work on questions / problems / issues themselves. 5. Learning from failure is important for learning. Speculating and predicting before finding the correct answers helps people become adaptive learners / experts.
    • Teaching EBP =  Small groups  Problem based  Real life cases EBP starts with a practical question, not with an academic answer
    • 3. Teaching EBP: Curriculum
    •  Limitations of human judgment & common forms of cognitive bias  Retrieving & critically assessing experiential evidence  Retrieving & critically assessing organizational evidence (qualitative & quantitative)  Searching in research databases  Efficiently reading research articles  Critically appraising evidence from research  Weighing and aggregating evidence from multiple sources  Incorporating evidence into the decision making process  Different types of decisions and decision-making processes  Assessing the outcome of decisions made EBP: Curriculum
    • Professional experience and judgment Organizational data, facts and figures Stakeholders’ values and concerns Scientific research findings Ask Acquire Appraise Aggregate Apply Assess
    •  A critical and reflective attitude  Skills to distinguish trustworthy from less trustworthy evidence.  Thinking in terms of probabilities EBP: outcome
    • CAT: Critically Appraised Topic
    • CAT: Critically Appraised Topic A critically appraised topic (CAT) is a structured, short (2 – 5 pages max) summary of evidence on a topic of interest, focused around a practical problem or question..
    • CAT: structure 1) Background / context 2) Question (PICOC) 3) Search strategy 4) Results / evidence summary 5) Findings 6) Limitations 7) Recommendation
    • CAT: 3 hits 1. Small group (2-3), scientific evidence, list of topics 2. Individual, scientific evidence & theory, own topic 3. Individual, evidence from multiple sources, real life question / issue
    • CAT: examples  Is there a valid and reliable way to measure the productivity of knowledge workers?  To what extent will leadership training improve the effectiveness of leaders (e.g. managers, executives), what are the characteristics of effective leadership training programs?  What research evaluating the effects of 360-degree has been published in peer-reviewed scholarly journals in the past 10 years? Which of the variables that are expected to have an impact on the effectiveness of 360-degree feedback are most widely studied and what is known of their effect?
    • CAT-walk
    • 4. Teaching EBP: Examples
    •  Critical attitude  ASK  Critically appraisal: research findings  Organizational evidence 4. Teaching EBP: examples
    •  Critical attitude  ASK  Critically appraisal: research findings  Organizational evidence 4. Teaching EBP: examples
    • Discuss with your neighbours: What are the most common cognitive biases in management? (give an example) 60595857565453525150494847464544434241403938373635343332313029282726252423222120191817161514131211109876543210
    • We are predisposed to selectively search for or interpret information in ways that confirms our existing beliefs, expectations and assumptions, and ignore information to the contrary. In other words, we “see what we want to see” Confirmation bias
    • The amount of information increases faster than our ability to process it. Increase of information
    • McKinsey (1997 / 2001) Case study / best practice
    • War on Talent
    • 1. Pattern recognition 2. Confirmation-bias 3. Small numbers fallacy 4. Outcome bias 5. Halo effect 6. Authority-bias 7. Groupthink 8. Availability bias Biases
    • Assignment One of your best friends is trader on the stock exchange. He enthusiastically tells you he has analyzed a large number of financial and economic data and that he has discovered an interesting phenomenon: "The position of the Dow Jones index multiplied by the price of oil is two days ahead of the gold price!" In other words, if both the Dow Jones and the oil price go up, the price of gold will rise within a day.  Bias?  Critical questions?
    • We are predisposed to see order, pattern and causal relations in the world. Patternicity: The tendency to find meaningful patterns in both meaningful and meaningless noise. Bias: pattern recognition
    • Assignment Most people will prefer doctor B  Why? Bias?  Critical questions? doctor A doctor B Last operation: patient died on the operating table Last operation: patient recovered faster than expected
    • Outcome bias We are inclined to evaluate the quality of a decision (intervention, method) on the basis of its outcome. Bias: Outcome bias
    • The Asch Experiment
    • Nasa, Challenger
    • Groupthink: Groupthink is a psychological phenomenon that occurs within a group of people, in which the desire for harmony or conformity in the group results in an incorrect or irrational decision Bias: Groupthink
    • Bounded rationality System 1  Fast  Intuitive, associative  heuristics & biases System 2  Slow (lazy)  Deliberate, ‘reasoning’  Rational
    • Bounded rationality limbic system and brainstem (system 1) neo cortex (system 2)
    •  Critical attitude  ASK  Critically appraisal: research findings  Organizational evidence Teaching EBP: examples
    • Postgraduate Course 1. Problem identification 2. Surfacing assumptions 3. Logic model 4. Formulating a searchable question 5. Formulate sub-questions 5 steps
    • Postgraduate Course For which problem is ….. the solution?  For who(m)  Why  How big ?  How do we know (what is the evidence?) Step 1: What is the problem?
    • Postgraduate Course “Door to needle time”, 48 UK hospitals in Westmidland Our hospital The problem
    • Postgraduate Course Discuss with your neighbours possible causes for these differences that could not be solved by the implementation of Lean / Six Sigma 60595857565453525150494847464544434241403938373635343332313029282726252423222120191817161514131211109876543210
    • Postgraduate Course 1. Problem identification 2. Surfacing assumptions 3. Logic model 4. Formulating a searchable question 5. Formulate sub-questions 5 steps
    • Postgraduate Course Logic model
    • Postgraduate Course A logic model spells out the process by which a problem or intervention is expected to produce certain outcomes. In making expectations (its logic) explicit, a logic model helps identify the kind of evidence needed. Logic model
    •  Critical attitude  ASK  Critically appraisal: research findings  Organizational evidence 4. Teaching EBP: examples
    • Critical appraisal Always start with a practical question: - does it work? - how many employees …? - how do employees feel about …? How could we find out?
    • 87
    • Critical appraisal Amanda Burls: “I never tell them anything about randomization or blinding, I just ask: “How are you going to know? How would you test this if I would give you a half million dollars to test it?” And when they come up with a suggestion I say, “Alright, can you think of any reason you got the results showing it works, while in fact it doesn’t.” And they say, “Well, it could be this, it could be that” And then I say, “Ok, then redesign your study so it can’t be this,” and what they come up with are precisely those things: randomization and blinding.”
    • How trustworthy is this study?  Two studies, different designs  Studies with methodological flaws  Best available evidence (so what now?)  Online course! Critical appraisal
    •  Critical attitude  ASK  Critically appraisal: research findings  Organizational evidence 4. Teaching EBP: examples
    • 91
    • Organizational facts and figures
    • Organizational facts and figures
    • Organizational facts and figures
    • 95
    • Organizational facts and figures
    • Organizational facts and figures
    • Examples
    • 1. Added value 1. Types of organizational evidence 1. Starting point: logic model 2. Statistical aspects to consider 3. Barriers to overcome Organizational evidence
    • 5. Barriers to overcome 1. Small numbers problem 2. Measurement errors 3. Context 4. Tainted data 5. Politics