Evidence-Based Practice     Lunchbijeenkomst      14 maart 2013
Post GradHBO   Universiteit   (E)MBA                     Business                     Schools
Aanleiding   1. Wat moet een student in de praktijk straks kunnen:      - zelf onderzoek doen?      - onderzoek kunnen beo...
HBO standaard (2009)“In onze moderne samenleving is het cruciaal dat hbo-bachelors over een onderzoekend vermogen beschikk...
Evidence-Based   Practice ?
 Wat is het? Waar komt het vandaan? Hoe ziet het er uit in een opleiding? Hoe zit dat bij 4e jaars studenten?
Evidence based practice:      Wat is het?
Evidence-based practiceUitgangspunt bij evidence-based practice isdat beslissingen gebaseerd dienen te zijnop een combinat...
Evidence based practice    Met het begrip evidence wordt niet    meer bedoeld dan informatie.    Dit kan informatie zijn a...
Evidence based practice    In principe neemt iedere manager dus    beslissingen op basis van evidence.    De meeste manage...
Evidence-based practice: kritisch en wetenschappelijk denken van verschillende informatiebronnen gebruik maken de besch...
Evidence based practice          Best available                          Professional       scientific evidence           ...
Evidence based practice:Waar komt het vandaan?
What field is this? “there is a large research-user gap” “practitioners do not read academic journals” “the findings of...
Medicine: Founding fathers   David Sackett                 Gordon Guyatt      McMaster University Medical School, Canada
How it all started
Problem I: persistent convictions      if you’re                        breathe into a bag    hyperventilating
Problem I: persistent convictions elderly people who have      give them a drug that an irregular heartbeat are          r...
How 40,000 cardiologists can be wrong                In the early 1980s newly introduced                anti-arrhythmic dr...
David Sackett Half of what you learn in medical school will be  shown to be either dead wrong or out-of-date  within 5 ye...
Problem II: too much information More than 1 million articles in 40,000 medical journals per  year (= 1995; now probably ...
Problem II: too much information HRM: 1,350 articles in 2010 (ABI/INFORM). For an HR  manager to keep up this means readi...
The 5 steps EBP1. Formulate a focused question (Ask)2. Search for the best available evidence (Acquire)3. Critically appra...
Evidence-Based Practice   1991   Medicine   1998   Education   1999   Social care, public policy   2000   Nursing   2000  ...
Evidence based practice:Hoe ziet het er uit in de opleiding?
Evidence-based practiceFase 1: Het ontwikkelen van kritisch en wetenschappelijkdenken dat leidt tot een professioneel-krit...
Evidence based practice          Best available                          Professional       scientific evidence           ...
Waarom (wetenschappelijk) onderzoek?
Trust me, 20 years of experience
Bounded rationality
Richard Feynman“The first principle is that you must not foolyourself - and you are the easiest person tofool”.
Bounded rationality
Het feilbare brein                     System 1                      Snel, actie                      Intuitief, associa...
Systeem 1
Systeem 1: het feilbare brein   Seeing order in randomness   Mental corner cutting   Misinterpretation of incomplete da...
Waarom onderzoek?                    Het feilbare brein                    - Meningen                    - Assumpties (aan...
Fase 1: Kritisch & wetenschappelijk denken  1. Denkfouten  2. Informatiebronnen  3. Mythbusting  4. Assumpties
7 Denkfouten (die u beter aan anderen kunt overlaten) 1. Seeing order in randomness 2. Confirmation bias 3. Small numbers ...
Seeing order in randomness Een Type I fout of een vals positief:  denken dat er een patroon / verband is  terwijl dat er ...
Het feilbare brein: patern recognition    Een Type I fout of een vals positief: denken dat het     geritsel in de bosjes ...
Het feilbare brein: patern recognition    Een Type II fout of een vals negatief: denken dat het     geritsel in de bosjes...
Het feilbare brein: patern recognition  Het probleem met patroon herkenning:  Het kritisch beoordelen of er sprake is van ...
Het feilbare brein: patern recognition    Een Type I fout of een vals positief: denken dat het     geritsel in de bosjes ...
Het feilbare brein: patern recognition   Ook ervaren mensen en experts zien patronen          en verbanden waar ze niet zi...
Oct 2005    Peptic ulcer – an infectious disease!This years Nobel Prize in Physiology or Medicine goes to Barry Marshall a...
Het feilbare brein: patern recognition Ook slimme mensen houden er verkeerde ideeën op na, niet omdat ze dom of eigenwijs ...
Evidence-based practiceFase 1: Het ontwikkelen van kritisch en wetenschappelijkdenken dat leidt tot een professioneel-krit...
5-step approach           EBMgt is a 5-step approach1. Formulate an answerable question (PICOC)2. Search for the best avai...
Formulate a focused question
Focused question? Does team-building work? What are the costs and benefits of self-steering teams? What are the success...
Foreground question?  Does team-building work?     What is a „team‟?     What kind of teams?     In what contexts/sett...
Answerable question: PICOC   P = Population   I = Intervention or success factor   C = Comparison   O = Outcome   C = Cont...
2. Finding the best available evidence
Searching evidence      What do we search?
What do we search?            Current Information            Overview of a subject            General background          ...
Peer reviewed journals
Searching evidence      Where do we search?
Searching for evidence
Databases             ABI/INFORM             Business Source Elite             PsycINFO             Web of Knowledge  ...
Searching for evidence
Searching evidence       How do we search?        Search Strategy
Search strategy        Two types of search strategies     Snowball method   Building blocks method
Search strategy
Exercise: Search for evidence                       Search in ABI/Inform:1. How many articles has Stephen Covey published ...
Exercise: Search for evidence               Search in ABI/Inform or BSE: Search for peer reviewed research articles to an...
Evidence-based practiceFase 1: Het ontwikkelen van kritisch en wetenschappelijkdenken dat leidt tot een professioneel-krit...
Research designs  Which design for which question?
What is the best design?      Randomized controlled study?      Grounded theory approach?      Cohort / panel study?   ...
What is the BEST car?
What is the best design? quants vs quallies, positivists vs post structuralist, etc
Effect   vs   Non-effect
Types of questions         Does it work?         Does it work better than ....?         Does it have an effect on ....?Eff...
Types of questions: non-effectNeeds:         What do people want or need?Attitude:      What do people think or feel?Exper...
Internal validity
Internal validity   internal validity = indicates to what extent the   results of the research may be biased and is thus  ...
Causal relations      We are pattern seeking primates:      we are predisposed to see order      and causal relations in t...
Causality      Considerations for research:  1. Are the "cause" and the "effect” related?     effect size  1. Does the "ca...
Bias & Confounding Research shows: Shoe size > quality of handwriting Smoking youngsters > better lung function
Levels of internal validity
Which design for which question?Explanation
Different types of research questionsrequire different types of research designs,but ...
But 1: feasibility           Best research design?
But 1: feasibility
Step 3: Critical appraisal of studies
Intermezzo    How to read a research article?
Critical appraisal: quick and dirty  Is the study design appropriate to the stated  aims?  Are the measurements likely to ...
Levels of internal validity 1. Were there enough subjects in the study? 2. Was a control group used? 3. Were the subjects ...
Appraisal  Critical appraisal questionnaires            www.cebma.org/ebp-tools
CAT: Critically Appraised Topic
CAT: Critically Appraised Topic  A critically appraised topic (or CAT) is a structured, short (3  pages max) summary of ev...
CAT: structure   1) Background / context   2) Question (PICOC)   3) Search strategy   4) Results / evidence summary   5) F...
CAT-walk
Presentation NVAO Evidence Based Practice
Presentation NVAO Evidence Based Practice
Presentation NVAO Evidence Based Practice
Presentation NVAO Evidence Based Practice
Upcoming SlideShare
Loading in …5
×

Presentation NVAO Evidence Based Practice

634 views
508 views

Published on

Presentatie over Evidence-Based Practice,
Lunchbijeenkomst NVAO, 14 maart 2013

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
634
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
8
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Presentation NVAO Evidence Based Practice

  1. 1. Evidence-Based Practice Lunchbijeenkomst 14 maart 2013
  2. 2. Post GradHBO Universiteit (E)MBA Business Schools
  3. 3. Aanleiding 1. Wat moet een student in de praktijk straks kunnen: - zelf onderzoek doen? - onderzoek kunnen beoordelen + toepassen? 2. Wat betekent dat voor het curriculum? 3. Wat betekent dat voor de accreditatie?
  4. 4. HBO standaard (2009)“In onze moderne samenleving is het cruciaal dat hbo-bachelors over een onderzoekend vermogen beschikkendat leidt tot reflectie, tot evidence-based practice, en totinnovatie.”
  5. 5. Evidence-Based Practice ?
  6. 6.  Wat is het? Waar komt het vandaan? Hoe ziet het er uit in een opleiding? Hoe zit dat bij 4e jaars studenten?
  7. 7. Evidence based practice: Wat is het?
  8. 8. Evidence-based practiceUitgangspunt bij evidence-based practice isdat beslissingen gebaseerd dienen te zijnop een combinatie van wetenschappelijkdenken en de best beschikbare evidence.
  9. 9. Evidence based practice Met het begrip evidence wordt niet meer bedoeld dan informatie. Dit kan informatie zijn afkomstig uit wetenschappelijk onderzoek, maar ook interne bedrijfsinformatie en zelfs persoonlijke ervaring geldt als evidence’.
  10. 10. Evidence based practice In principe neemt iedere manager dus beslissingen op basis van evidence. De meeste managers besteden echter nauwelijks aandacht aan de kwaliteit van de evidence waarop ze hun beslissingen baseren.
  11. 11. Evidence-based practice: kritisch en wetenschappelijk denken van verschillende informatiebronnen gebruik maken de beschikbare evidence kritisch tegen het licht houden denken in termen van waarschijnlijkheid in plaats van golden bullets.
  12. 12. Evidence based practice Best available Professional scientific evidence expertise and judgement Evidence-based decision Best available Stakeholders’ organizational values and concerns evidence
  13. 13. Evidence based practice:Waar komt het vandaan?
  14. 14. What field is this? “there is a large research-user gap” “practitioners do not read academic journals” “the findings of research into what is an effective intervention are not being translated into actual practice” “academics not practitioners are driving the research agenda” “the relevance, quality and applicability of research is questionable” “practice is being driven more by fads and fashions than research” “many practices are doing more harm than good”
  15. 15. Medicine: Founding fathers David Sackett Gordon Guyatt McMaster University Medical School, Canada
  16. 16. How it all started
  17. 17. Problem I: persistent convictions if you’re breathe into a bag hyperventilating
  18. 18. Problem I: persistent convictions elderly people who have give them a drug that an irregular heartbeat are reduces the much more likely to die of number of coronary disease irregular beats
  19. 19. How 40,000 cardiologists can be wrong In the early 1980s newly introduced anti-arrhythmic drugs were found to be highly successful at suppressing arrhythmias. Not until a RCT was performed was it realized that, although these drugs suppressed arrhythmias, they actually increased mortality. By the time the results of this trial were published, at least 100,000 such patients had been taking these drugs.
  20. 20. David Sackett Half of what you learn in medical school will be shown to be either dead wrong or out-of-date within 5 years of your graduation; the trouble is that nobody can tell you which half. The most important thing to learn is how to learn on your own: search for the evidence! (Remember that your teachers are as full of bullshit as your parents)
  21. 21. Problem II: too much information More than 1 million articles in 40,000 medical journals per year (= 1995; now probably more than 2 million). For a specialist to keep up this means reading 25 articles every day (for a GP more than 100!)
  22. 22. Problem II: too much information HRM: 1,350 articles in 2010 (ABI/INFORM). For an HR manager to keep up this means reading 3 to 4 articles every day (for a „general‟ manager more than 50!) BTW: most of the research is seriously flawed or irrelevant for practice
  23. 23. The 5 steps EBP1. Formulate a focused question (Ask)2. Search for the best available evidence (Acquire)3. Critically appraise the evidence (Appraise)4. Integrate the evidence with your professional expertise and apply (Apply)5. Monitor the outcome (Assess)
  24. 24. Evidence-Based Practice 1991 Medicine 1998 Education 1999 Social care, public policy 2000 Nursing 2000 Criminal justice ???? Management?
  25. 25. Evidence based practice:Hoe ziet het er uit in de opleiding?
  26. 26. Evidence-based practiceFase 1: Het ontwikkelen van kritisch en wetenschappelijkdenken dat leidt tot een professioneel-kritische attitude metbetrekking tot organisatievraagstukkenFase 2: Het kunnen formuleren van een expliciete vraag enop basis van deze vraag kunnen zoeken in online databasesnaar uitkomst van relevant wetenschappelijk onderzoek.Fase 3: Het kritisch kunnen beoordelen vanwetenschappelijke en organizational evidence (criticalappraisal)Fase 4: Uitkomst van wetenschappelijk onderzoek kunnentoepassen in de eigen beroepspraktijk (application of science)
  27. 27. Evidence based practice Best available Professional scientific evidence expertise and judgment Evidence-based decision Best available Stakeholders’ organizational values and concerns evidence
  28. 28. Waarom (wetenschappelijk) onderzoek?
  29. 29. Trust me, 20 years of experience
  30. 30. Bounded rationality
  31. 31. Richard Feynman“The first principle is that you must not foolyourself - and you are the easiest person tofool”.
  32. 32. Bounded rationality
  33. 33. Het feilbare brein System 1  Snel, actie  Intuitief, associatief  shortcuts & biasses System 2  Langzaam (lui!)  Rationeel  Nadenken
  34. 34. Systeem 1
  35. 35. Systeem 1: het feilbare brein Seeing order in randomness Mental corner cutting Misinterpretation of incomplete data Halo effect False consensus effect Group think  Confirmation bias Self serving bias  Authority bias Sunk cost fallacy  Small numbers fallacy Cognitive dissonance reduction  In-group bias  Recall bias  Anchoring bias  Inaccurate covariation detection  Distortions due to plausibility
  36. 36. Waarom onderzoek? Het feilbare brein - Meningen - Assumpties (aannames) - Overtuigingen BIAS - Persoonlijke ervaringen Wetenschappelijk onderzoek - Feiten - Evidence: bewijs / aanwijzingen
  37. 37. Fase 1: Kritisch & wetenschappelijk denken 1. Denkfouten 2. Informatiebronnen 3. Mythbusting 4. Assumpties
  38. 38. 7 Denkfouten (die u beter aan anderen kunt overlaten) 1. Seeing order in randomness 2. Confirmation bias 3. Small numbers fallacy 4. Outcome bias 5. Halo effect 6. Authority bias 7. Group think
  39. 39. Seeing order in randomness Een Type I fout of een vals positief: denken dat er een patroon / verband is terwijl dat er in het echt niet is. Een Type II fout of een vals negatief: denken dat er geen patroon / verband is terwijl dat er in het echt wel is Dr. Michael Shermer (Director of the Skeptics Society)
  40. 40. Het feilbare brein: patern recognition  Een Type I fout of een vals positief: denken dat het geritsel in de bosjes een gevaarlijk roofdier is, terwijl het gewoon de wind is (goedkoop foutje)
  41. 41. Het feilbare brein: patern recognition  Een Type II fout of een vals negatief: denken dat het geritsel in de bosjes gewoon de wind is, terwijl het een gevaarlijk roofdier is (duur foutje)
  42. 42. Het feilbare brein: patern recognition Het probleem met patroon herkenning: Het kritisch beoordelen of er sprake is van een Type I of een Type II fout is best moeilijk, (vooral in „split second life and death‟ situaties), dus de default positie is om aan te nemen dat alle patronen echt zijn.
  43. 43. Het feilbare brein: patern recognition  Een Type I fout of een vals positief: denken dat het geritsel in de bosjes een gevaarlijk roofdier is, terwijl het gewoon de wind is (goedkoop foutje)  Een Type II fout of een vals negatief: denken dat het geritsel in de bosjes gewoon de wind is, terwijl het een gevaarlijk roofdier is (duur foutje)
  44. 44. Het feilbare brein: patern recognition Ook ervaren mensen en experts zien patronen en verbanden waar ze niet zijn. stress & lifestyle peptic ulcer
  45. 45. Oct 2005 Peptic ulcer – an infectious disease!This years Nobel Prize in Physiology or Medicine goes to Barry Marshall and RobinWarren, who with tenacity and a prepared mind challenged prevailing dogmas. Byusing technologies generally available (fibre endoscopy, silver staining ofhistological sections and culture techniques for microaerophilic bacteria), theymade an irrefutable case that the bacterium Helicobacter pylori is causing disease.By culturing the bacteria they made them amenable to scientific study. In 1982, when this bacterium was discovered by Marshall and Warren, stress and lifestyle were considered the major causes of peptic ulcer disease. It is nowfirmly established that Helicobacter pyloricauses more then 90% of duodenal ulcers.The link between Helicobacter pyloriinfection and peptic ulcer disease has beenestablished through studies of humanvolunteers, antibiotic treatment studies andepidemiological studies.
  46. 46. Het feilbare brein: patern recognition Ook slimme mensen houden er verkeerde ideeën op na, niet omdat ze dom of eigenwijs zijn, maar omdat het de meest logische conclusie is op basis van hun eigen ervaringen. (systeem 1 doet altijd mee!)
  47. 47. Evidence-based practiceFase 1: Het ontwikkelen van kritisch en wetenschappelijkdenken dat leidt tot een professioneel-kritische attitude metbetrekking tot organisatievraagstukkenFase 2: Het kunnen formuleren van een expliciete vraag enop basis van deze vraag kunnen zoeken in online databasesnaar uitkomst van relevant wetenschappelijk onderzoek.Fase 3: Het kritisch kunnen beoordelen vanwetenschappelijke en organizational evidence (criticalappraisal)Fase 4: Uitkomst van wetenschappelijk onderzoek kunnentoepassen in de eigen beroepspraktijk (application of science)
  48. 48. 5-step approach EBMgt is a 5-step approach1. Formulate an answerable question (PICOC)2. Search for the best available evidence3. Critically appraise the quality of the found evidence4. Integrate the evidence with managerial expertise and organizational concerns and apply5. Monitor and evaluate the results
  49. 49. Formulate a focused question
  50. 50. Focused question? Does team-building work? What are the costs and benefits of self-steering teams? What are the success factors for culture change? Does management development improve the performance of managers? Does employee participation prevent resistance to change? How do employees feel about 360 degree feedback?
  51. 51. Foreground question?  Does team-building work?  What is a „team‟?  What kind of teams?  In what contexts/settings?  What counts as „team-building‟?  What does „work‟ mean?  What outcomes are relevant?  Over what time periods?
  52. 52. Answerable question: PICOC P = Population I = Intervention or success factor C = Comparison O = Outcome C = Context
  53. 53. 2. Finding the best available evidence
  54. 54. Searching evidence What do we search?
  55. 55. What do we search? Current Information Overview of a subject General background Academic Information Statistical Information Theories about a subject Company information
  56. 56. Peer reviewed journals
  57. 57. Searching evidence Where do we search?
  58. 58. Searching for evidence
  59. 59. Databases  ABI/INFORM  Business Source Elite  PsycINFO  Web of Knowledge  ERIC  Google Scholar
  60. 60. Searching for evidence
  61. 61. Searching evidence How do we search? Search Strategy
  62. 62. Search strategy Two types of search strategies Snowball method Building blocks method
  63. 63. Search strategy
  64. 64. Exercise: Search for evidence Search in ABI/Inform:1. How many articles has Stephen Covey published in peer reviewed journals?2. How many of these articles are based on scientific research?3. Are there articles (by other authors) that are critical of Covey’s 7 Habits?4. How many of these critical articles are based on scientific research?
  65. 65. Exercise: Search for evidence Search in ABI/Inform or BSE: Search for peer reviewed research articles to answer the following question: What is the long term effect of a hostile take-over on the financial performance of the acquired organization? Use the following search terms: “hostile takeovers”, “financial performance”, “long term” How many studies did you find?
  66. 66. Evidence-based practiceFase 1: Het ontwikkelen van kritisch en wetenschappelijkdenken dat leidt tot een professioneel-kritische attitude metbetrekking tot organisatievraagstukkenFase 2: Het kunnen formuleren van een expliciete vraag enop basis van deze vraag kunnen zoeken in online databasesnaar uitkomst van relevant wetenschappelijk onderzoek.Fase 3: Het kritisch kunnen beoordelen vanwetenschappelijke en organizational evidence (criticalappraisal)Fase 4: Uitkomst van wetenschappelijk onderzoek kunnentoepassen in de eigen beroepspraktijk (application of science)
  67. 67. Research designs Which design for which question?
  68. 68. What is the best design?  Randomized controlled study?  Grounded theory approach?  Cohort / panel study?  Qualitative field research?  Longitudinal study?  Post-test only study?  Survey?  Action research?  Case study?
  69. 69. What is the BEST car?
  70. 70. What is the best design? quants vs quallies, positivists vs post structuralist, etc
  71. 71. Effect vs Non-effect
  72. 72. Types of questions Does it work? Does it work better than ....? Does it have an effect on ....?Effect What is the success factor for ....? What is required to make it work ...? Will it do more good than harm?
  73. 73. Types of questions: non-effectNeeds: What do people want or need?Attitude: What do people think or feel?Experience: What are peoples’ experiences?Prevalence: How many / often do people / organizations ...?Procedure: How can we implement ...?Process: How does it work?Explanation: Why does it work?Economics: How much does it cost?
  74. 74. Internal validity
  75. 75. Internal validity internal validity = indicates to what extent the results of the research may be biased and is thus a comment on the degree to which alternative explanations for the outcome found are possible.
  76. 76. Causal relations We are pattern seeking primates: we are predisposed to see order and causal relations in the world
  77. 77. Causality Considerations for research: 1. Are the "cause" and the "effect” related? effect size 1. Does the "cause" precede the "effect" in time? before and after measurement 2. Are there no plausible alternative explanations for the observed effect? randomization, blinding, control group, measurements
  78. 78. Bias & Confounding Research shows: Shoe size > quality of handwriting Smoking youngsters > better lung function
  79. 79. Levels of internal validity
  80. 80. Which design for which question?Explanation
  81. 81. Different types of research questionsrequire different types of research designs,but ...
  82. 82. But 1: feasibility Best research design?
  83. 83. But 1: feasibility
  84. 84. Step 3: Critical appraisal of studies
  85. 85. Intermezzo How to read a research article?
  86. 86. Critical appraisal: quick and dirty Is the study design appropriate to the stated aims? Are the measurements likely to be valid and reliable? Was there a relevant effect size? Is the outcome (population, type of organization) generalizable to your situation?
  87. 87. Levels of internal validity 1. Were there enough subjects in the study? 2. Was a control group used? 3. Were the subjects randomly assigned? 4. Was a pretest used? 5. Was the study started prior to the intervention or event? 6. Was the outcome measured in an objective and reliable way? 6x yes = very high (A) 5x yes = high (A) 4-3x yes = limited (B) 2x yes = low (C) 1-0x yes = very low (D)
  88. 88. Appraisal Critical appraisal questionnaires www.cebma.org/ebp-tools
  89. 89. CAT: Critically Appraised Topic
  90. 90. CAT: Critically Appraised Topic A critically appraised topic (or CAT) is a structured, short (3 pages max) summary of evidence on a topic of interest, usually focused around a practical problem or question. A CAT is like a “quick and dirty” version of a systematic review, summarizing the best available research evidence on a topic. Usually more than one study is included in a CAT.
  91. 91. CAT: structure 1) Background / context 2) Question (PICOC) 3) Search strategy 4) Results / evidence summary 5) Findings 6) Limitations 7) Recommendation
  92. 92. CAT-walk

×