Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

IPSSW Pre-Conference Workshop 7 (2016, Glasgow)

178 views

Published on

Thanks for attending the Workshop 7 - Improving Rigor & Scholarship in Sim-based Research. Here is the Powerpoint.

Published in: Science
  • Be the first to comment

  • Be the first to like this

IPSSW Pre-Conference Workshop 7 (2016, Glasgow)

  1. 1. Simulation Research: Improving Rigor, Impact, and Methodology May 8, 2016 Glasgow, UK IPSSW Pre-Conference Workshop
  2. 2. International Network for Simulation-based Pediatric Innovation, Research and Education Introduction / Faculty • Ralph MacKinnon • Todd Chang • Vinay Nadkarni • David Kessler
  3. 3. Disclosures • No relevant financial conflicts of interest • INSPIRE receives funding/support from International Network for Simulation-based Pediatric Innovation, Research and Education
  4. 4. International Network for Simulation-based Pediatric Innovation, Research and Education Mission We aim to improve the delivery of medical care to acutely ill children by answering important research questions pertaining to resuscitation, technical skills, behavioral skills, debriefing and simulation-based education
  5. 5. International Network for Simulation-based Pediatric Innovation, Research and Education
  6. 6. International Network for Simulation-based Pediatric Innovation, Research and Education Who are you? • Where do you work with Simulation? • What is your simulation experience? • What is your research experience?
  7. 7. International Network for Simulation-based Pediatric Innovation, Research and Education Learning Objectives • Define the 2 types of simulation-based research • Evaluate characteristics of a simulation-based research question or protocol that are conducive to quality research and output • Identify 6 specific challenges in simulation-based research that threaten rigor and quality of research output
  8. 8. Objectives 1. Develop a study design using simulation as the subject of research 2. Develop a study design using simulation as an investigative methodology 3. Use Reporting Guidelines to improve study methodology International Network for Simulation-based Pediatric Innovation, Research and Education
  9. 9. Schedule International Network for Simulation-based Pediatric Innovation, Research and Education Didactic Discussion Small group work Break Closing Discussion & Networking
  10. 10. Objectives 1. Develop a study design using simulation as the subject of research 2. Develop a study design using simulation as an investigative methodology 3. Use Reporting Guidelines to improve study methodology International Network for Simulation-based Pediatric Innovation, Research and Education
  11. 11. International Network for Simulation-based Pediatric Innovation, Research and Education 2 Types of Simulation Research Research subject (e.g. Team training) Research method (e.g. Test new defibrillator)
  12. 12. International Network for Simulation-based Pediatric Innovation, Research and Education 1: Simulation as the research subject • Is simulation an effective training methodology? • How do we maximize effects on patient outcomes? 2: Simulation as the research method • Leverages standardization of simulation • Questions that might not be safe, feasible, ethical, timely in traditional research
  13. 13. Simulation as the subject of research International Network for Simulation-based Pediatric Innovation, Research and Education
  14. 14. International Network for Simulation-based Pediatric Innovation, Research and Education INSPIRE Research Themes Simulation as the subject Debriefing Develop/assess/implement effective techniques for debriefing real/sim events IPE, Teamwork, Communication Develop/assess/implement effective techniques for team training Procedural, Psychomotor Skills Develop/assess/implement effective techniques for skills development retention
  15. 15. Simulation as the subject • Research examining whether or not specific features of simulation experiences are effective International Network for Simulation-based Pediatric Innovation, Research and Education
  16. 16. International Network for Simulation-based Pediatric Innovation, Research and Education Simulation as the subject
  17. 17. #1 start with strong conceptual model
  18. 18. What we know… • 609 studies • 23% RCT’s • 12% multicenter studies • 5% reported patient and/or healthcare outcomes • Sim is an effective educational intervention compared to no intervention International Network for Simulation-based Pediatric Innovation, Research and Education
  19. 19. Pediatric Research • 57 studies • 1/3 RCT • compare sim to no intervention • Large pooled effective size • Knowledge • Skills in simulation • Time to completion • < 10% on patient outcome International Network for Simulation-based Pediatric Innovation, Research and Education
  20. 20. 2 Lessons International Network for Simulation-based Pediatric Innovation, Research and Education 1. Avoid the null comparison • (Don’t compare to no intervention)
  21. 21. International Network for Simulation-based Pediatric Innovation, Research and Education
  22. 22. Varying Instructional Design 1. Feedback *** 2. Repetitive Practice *** 3. Distributed Practice 4. Curriculum Integration 5. Clinical variation 6. Range of difficulty 7. Individualized learning 8. Multiple learning strategies 9. Defined Outcomes 10. Valid simulator International Network for Simulation-based Pediatric Innovation, Research and Education
  23. 23. What examples of Instructional Design Features have you investigated? Discussion 1
  24. 24. 2 Lessons International Network for Simulation-based Pediatric Innovation, Research and Education 2. Measure patient outcomes • (Or at least provider behavior)
  25. 25. Knowledge/skills Attitudes/perceptions Learners’ satisfaction/reaction Yadley 2011, BEME 2005, Miller 1990, Kirkpatrick 1967
  26. 26. Δ practice Δ behavior Knowledge/skills Attitudes/perceptions Learners’ satisfaction/reaction Yadley 2011, BEME 2005, Miller 1990, Kirkpatrick 1967
  27. 27. Popul- ation Patient Δ practice Δ behavior Knowledge/skills Attitudes/perceptions Learners’ satisfaction/reaction Yadley 2011, BEME 2005, Miller 1990, Kirkpatrick 1967
  28. 28. Framing your Research Outcomes • Goal of medical education • To transmit knowledge, impart skills, and inculcate the values of medicine that will improve health outcomes Cooke, Irby, O’Brien. Carnegie Foundation Report: Educating Physicians: A call for reform of medical school and residency. 2010 International Network for Simulation-based Pediatric Innovation, Research and Education
  29. 29. Selecting outcomes • Important to patient • Measurable • Dependent on provider skills/behaviors • Quality • Satisfaction, comfort • Procedure related ** • Success/failure • Complications (Erb’s Palsy, Line Infections) • Survival • Cardiac resuscitation, obstetrics, NICU International Network for Simulation-based Pediatric Innovation, Research and Education
  30. 30. INSPIRE Examples International Network for Simulation-based Pediatric Innovation, Research and Education
  31. 31. Infant Lumbar Punctures • Essential pediatric procedure • Decreasing experience • Novice success rate 35% • Bench top simulator • Low cost • Easy to use Auerbach et al Pediatric Emergency Care 2013, Gaies Pediatrics 2007 International Network for Simulation-based Pediatric Innovation, Research and Education
  32. 32. Infant Lumbar Punctures • Instructional Design principles: • Distributed Practice (JITT) • Repeated Practice • Defined Outcomes • Outcomes selected: • Clinical Lumbar Puncture success International Network for Simulation-based Pediatric Innovation, Research and Education
  33. 33. Research Question • Population • Intervention • Comparison • Outcome International Network for Simulation-based Pediatric Innovation, Research and Education
  34. 34. Research Question • P – 1st year residents (‘interns’) • I – Mastery Learning LP Training + Just-in-Time Training on LP Simulator just prior to clinical LP • C – Mastery Learning LP Training only • O – Clinical LP success rate International Network for Simulation-based Pediatric Innovation, Research and Education
  35. 35. Auerbach et al, Pediatric Emergency Care 2013 Overall Performance Score International Network for Simulation-based Pediatric Innovation, Research and Education
  36. 36. Let’s design a Research Project: Simulation as the subject Small Group 1
  37. 37. Objectives 1. Develop a study design using simulation as the subject of research 2. Develop a study design using simulation as an investigative methodology 3. Use Reporting Guidelines to improve study methodology International Network for Simulation-based Pediatric Innovation, Research and Education
  38. 38. Simulation as an investigative methodology International Network for Simulation-based Pediatric Innovation, Research and Education
  39. 39. International Network for Simulation-based Pediatric Innovation, Research and Education INSPIRE Research Themes Simulation as the method Technology Develop/assess/implement novel technologies designed to improve processes of care and pediatric patient outcomes Acute Care and Resuscitation Develop/assess/implement novel techniques for improving care of pediatric patients Human Factors Assess the role of human factors when providing care to pediatric patients Patient Safety Explore the key variables that influence patient safety and assess strategies to mitigate
  40. 40. Simulation as the method • The simulated environment is used as an experimental model to study factors affecting human and systems performance in healthcare. International Network for Simulation-based Pediatric Innovation, Research and Education
  41. 41. International Network for Simulation-based Pediatric Innovation, Research and Education Why use simulated instead of real scenarios • Standardization of “noise” • Patient- age, clinical status, parent actions • Individuals- experience, stress, fatigue • Team- structure, numbers • Environment- structure, distractions, noise • Technology- equipment availability, size, safety • Ask questions that might not be safe, feasible, ethical, timely in traditional research
  42. 42. Threats to internal validity International Network for Simulation-based Pediatric Innovation, Research and Education
  43. 43. Threats to validity • Educator training • Simulator Selection • Scenario Design • Confederates • Realism (physical, conceptual and emotional) • Debriefing • Video Capture and Review • Study Outcomes Cheng et al, Pediatrics, 2014
  44. 44. • ADD SLIDES ON EDUCATIONAL DESIGNS- e.g. solomon 4, cluster randomized etc…(martin’s paper, selin workshop) • Ethics- rct not always possible. (perfume example)
  45. 45. What examples of Threats to Validity have you encountered? Discussion 2
  46. 46. INSPIRE Examples International Network for Simulation-based Pediatric Innovation, Research and Education
  47. 47. Technology • A QCPR simulator was used to show that Feedback Devices improved CPR performance International Network for Simulation-based Pediatric Innovation, Research and Education
  48. 48. Systems/Human Factors • Individuals • Teams • Environments • Technological Factors • Systems Factors • Patient Factors International Network for Simulation-based Pediatric Innovation, Research and Education
  49. 49. SEIPS 2.0. Ergonomics . 2013 November ; 56(11) Holden, Carayon et al International Network for Simulation-based Pediatric Innovation, Research and Education
  50. 50. SEIPS 2.0. Ergonomics . 2013 November ; 56(11) Holden, Carayon et al International Network for Simulation-based Pediatric Innovation, Research and Education
  51. 51. New Environments International Network for Simulation-based Pediatric Innovation, Research and Education
  52. 52. International Network for Simulation-based Pediatric Innovation, Research and Education Safety / Quality • In-situ simulation • Mobile unit brought into work environment • Use real life equipment in “context” • Work with real life team • Identify latent safety threats • Provider level - insufficient staffing/training • Systems level - missing/broken equipment
  53. 53. International Network for Simulation-based Pediatric Innovation, Research and Education Direct comparison: same patient at different hospital
  54. 54. Crash Testing, not Normal Testing • Identify and remediate threats to safety • Active • Latent: unrecognized errors in design or organization International Network for Simulation-based Pediatric Innovation, Research and Education
  55. 55. Let’s design a Research Project: Simulation as the method Small Group 2
  56. 56. Objectives 1. Develop a study design using simulation as the subject of research 2. Develop a study design using simulation as an investigative methodology 3. Use Reporting Guidelines to improve study methodology International Network for Simulation-based Pediatric Innovation, Research and Education
  57. 57. Reporting Guidelines • Standardized reporting will enhance quality of studies and assist journal reviewer/editors • Explanation and elaboration document will support implementation of reporting guidelines • Reporting guidelines currently under review • Eventual adoption for author guidelines International Network for Simulation-based Pediatric Innovation, Research and Education
  58. 58. Reporting Guidelines • Extensions to prospective trial design: • CONSORT (randomized-control) • STROBE (observation cohort) International Network for Simulation-based Pediatric Innovation, Research and Education
  59. 59. How do we use Reporting Guidelines? International Network for Simulation-based Pediatric Innovation, Research and Education
  60. 60. Let’s review our Research Project: Using Reporting Guidelines Small Group 3
  61. 61. Objectives 1. Develop a study design using simulation as the subject of research 2. Develop a study design using simulation as an investigative methodology 3. Use Reporting Guidelines to improve study methodology International Network for Simulation-based Pediatric Innovation, Research and Education
  62. 62. What changes will you be making? International Network for Simulation-based Pediatric Innovation, Research and Education
  63. 63. What changes will you be making? International Network for Simulation-based Pediatric Innovation, Research and Education
  64. 64. International Network for Simulation-based Pediatric Innovation, Research and Education Questions?
  65. 65. www.inspireSim.com International Network for Simulation-based Pediatric Innovation, Research and Education

×