Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Memory Science & Medical Education


Published on

Ben and Frank were just at Dartmouth Medical School taking about the science behind Gunner's approach. Check out the presentation and let us know if you like free Pizza (i.e. if you'd like us to visit your school to talk about USMLE prep, match, etc.)! :)

Published in: Health & Medicine
  • Be the first to comment

Memory Science & Medical Education

  1. 1. Memory Science in Medical Education Frank Lau, MD Co-Founder & Chief Medical Officer
  2. 2. The Challenge for Medical Students “I feel like every time I review something or learn something new, I forget something else.” – C.C., 2nd year medical student, University of Michigan Medical School
  3. 3. The Challenge Persists for Residents   Resuscitation, July 2009   70 Johns Hopkins pediatrics residents   66% failed to start timely CPR (basic life support)   95% of the residents had received this level of training   83% made errors during defibrillation (shocking the patient)   80% of the residents had been trained in defibrillation   More years of training did not improve physician performance Medical knowledge is routinely forgotten, even by the best and the brightest.
  4. 4. The Challenge Persists for Attending Physicians   Journal of Trauma, June 1996   60 practicing trauma physicians took the standard Advanced Trauma Life Support course, followed by an examination   National course for managing traumas (car accidents, gunshots, stab wounds, etc)   Immediate testing: 86% average score   6 months later, 50% failed   13 years after this study, no novel approaches to learning and retaining medical knowledge Basic, critical knowledge is routinely forgotten, even by experienced specialists.
  5. 5. Who Should Care?   Medical Students   Physicians-in-training (residents)   Educators   Foreign medical graduates   Practicing physicians   Patients   Healthcare policy makers   Patient safety advocates
  6. 6. The 2 Phases of Knowledge Acquisition Mastering the material Maintaining that knowledge
  7. 7. Mastering the Material   We think we’re good at this   We did well on the MCAT, we pass our exams   But these are graded on a curve!   Patient outcomes don’t work on a curve   In reality, we do poorly at judging how well we’ve mastered specific topics
  8. 8. Mastery: The Problem 32% of the reviewed material is immediately forgotten.
  9. 9. Mastery: The Problem
  10. 10. Maintaining Your Knowledge   We rarely focus on knowledge maintenance   Johns Hopkins study, the ATLS study   Affects patient outcomes   For Step 1, the review process involves re-learning a lot of material that you once knew but have since forgotten
  11. 11. Intro to Gunner Training   The first Knowledge Management platform   Created to help students master and maintain their Step 1 knowledge base   Personalized: users choose which subjects to study  can correlate with school's curriculum   Dynamic: custom review schedule changes according to student's forward learning   Mobile:   iPhone/mobile browser compatible   No software installation required
  12. 12. Intro to Gunner Training   Launched April 2009:   Hundreds of users   Thousands of data points   We use this data and user feedback to continually validate and improve Gunner Training
  13. 13. The GT Way 1.  Learn the material from flashcards •  High-yield, interactive 2.  Immediate testing on that material •  Open-ended study questions 3.  Spaced Learning algorithms create a personalized review schedule for that material •  Incorporates Step 1 style multiple choice questions •  Short intervals help you master that material •  Longer intervals maintain your knowledge 4.  Exam simulation •  Eases anxiety •  Increases test-taking stamina
  14. 14. The GT Way 1.  Learn the material from flashcards •  High-yield, interactive 2.  Immediate testing on that material •  Open-ended study questions Apply 19 3.  Spaced Learning algorithms create a principles of personalized review schedule for that material efficient, •  Incorporates Step 1 style multiple choice questions effective •  Short intervals help you master that material learning •  Longer intervals maintain your knowledge 4.  Exam simulation •  Eases anxiety •  Increases test-taking stamina
  15. 15. GT: Flashcard Learning Principles 1.  Contiguity Effects. Ideas that are associated should be presented contiguously in space and time   One flashcard per topic, rather than one-fact-per-flashcard 2.  Dual Code and Multimedia Effects. Visual + multimedia form richer representations vs. a single medium   Hundreds of relevant images: gram stains, CT scans, X-rays, clinical photos 3.  Coherence Effect. Materials should explicitly link related ideas
  16. 16. GT: Flashcard Learning Principles 4.  Segmentation Principle. A complex lesson should be broken down into manageable subparts   Two levels of bulletpoints 5.  Manageable Cognitive Load. The information presented to the learner should not overload working memory   Second bulletpoints are initially hidden 6.  Cognitive Flexibility. This improves w/ multiple viewpoints linking facts and deep conceptual principles   Material presented in several locations   Ex: TB is covered under Microbiology, Pulmonology, Immunology 7.  Anchored Learning. Materials are anchored in real world problems that you care about   Clinical correlation
  17. 17. Contiguity, Dual code/Multimedia, & Coherence Effects
  18. 18. Segmentation & Manageable Cognitive Load
  19. 19. Cognitive Flexibility
  20. 20. Anchored Learning
  21. 21. GT: Study Question Principles 8.  Imperfect Metacognition. We rarely have an accurate knowledge of our cognition  need assistance with calibrating comprehension, learning, and memory   Immediate testing solves this 9.  Generation Effect. Learning is enhanced when learners produce answers (vs. recognizing answers)   This is why GT employs open-ended study questions 10.  Testing Effect. Testing enhances learning, especially when the tests are aligned with important content
  22. 22. GT: Study Question Principles 11.  Feedback Effects. Students benefit from immediate feedback on their performance 12.  Negative Suggestion Effects. Learning wrong information can be reduced when feedback is immediate 13.  Deep questions. More benefit from answering questions that elicit explanations (e.g., why, why not, how, what-if)
  23. 23. Imperfect Metacognition, Generation Effect & Testing Effect
  24. 24. Feedback Effects & Negative Suggestion Effects
  25. 25. GT: Review Schedule Principles 14.  Goldilocks Principle. Assignments should be at the right difficulty level 15.  Self-regulated Learning. Most students need to self- regulate their learning 16.  Exam Expectations. Students benefit more from repeated testing 17.  Spaced Learning.
  26. 26. Goldilocks, Self-regulated Learning, and Exam Expectations
  27. 27. GT: MCQ Principles 18.  Desirable difficulties. Challenges make learning and retrieval effortful  improve retention   Multi-part questions, on par with Step 1 question difficulty 20.  Cognitive Disequilibrium. Deep reasoning and learning is stimulated by problems that create cognitive disequilibrium, such as obstacles to goals, contradictions, conflict, and anomalies   “2-jump” & “3-jump” questions
  28. 28. MCQs in Review Schedule
  29. 29. MCQs in Review Schedule
  30. 30. Spaced Learning in Gunner Training Personalized Review Schedule Generation
  31. 31. Spaced Learning   In 1913, Ebbinghaus discovered that the best time to review something was just before it was forgotten   Minimizes the amount of time spent learning   Minimizes the number of times the same material must be relearned
  32. 32. Spaced Learning: Example   Let’s say it takes 2 minutes to memorize the 15 bones of the wrist   Reviewing this material if it’s not forgotten takes 30 seconds   Re-learning this material if it’s forgotten takes another 2 minutes   Without review, this information is forgotten in 4 days   There is a test at the end of Day 6
  33. 33. Spaced Learning: Example   Option 1: Review it every day   Day 0: 2 minutes learning   Day 1: 30 second review   Day 2: 30 second review   Day 3: 30 second review   Day 4: 30 second review   Day 5: 30 second review   Day 6: 30 second review  Take the test   Total time: 5 minutes   Pros: In this approach, the knowledge is never forgotten   Cons: Large burden of review  few people will adhere to this review schedule
  34. 34. Spaced Learning: Example   Option 2: Cram for the exam   Day 0: 2 minutes learning   Day 1: no time spent   Day 2: no time spent   Day 3: no time spent   Day 4: no time spent (knowledge forgotten)   Day 5: no time spent   Day 6: 2 minutes re-learning  Take the test   Total time: 4 minutes   Pros: It’s “easy” – learn it once and re-learn it just before the test   Cons: The knowledge is forgotten   Cramming produces high stress   Doesn’t work in clinical settings (no predetermined “test” date)   Less effective as the volume of information goes up
  35. 35. Spaced Learning: Example   Option 3: Spaced Learning   Day 0: 2 minutes learning   Day 1: no time spent   Day 2: no time spent   Day 3: no time spent   Day 4: 30 second review   Day 5: no time spent   Day 6: Take the test   Total time: 2 minutes, 30 seconds   Compared to daily review (5 mins): 50% less time spent   Compared to cramming (4 mins): 38.5% less time spent   Pros: In this approach, information is never forgotten and time spent studying is minimized
  36. 36. Spaced Learning Works in Medicine   3 randomized, controlled trials at Harvard Medical School demonstrate that Spaced Learning improves retention of medical knowledge   RCTs are the gold standard experimental methodology   Published in:   American Journal of Surgery, January 2009   Journal of General Internal Medicine, January 2008   Medical Education, January 2007
  37. 37. Basic Research on Spaced Learning   The molecular mechanism behind Spaced Learning is an active area of research   Cell, October 2009. Cold Spring Harbor & Mount Sinai School of Medicine   The Phosphatase SHP2 Regulates the Spacing Effect for Long-Term Memory Induction   Journal of Neuroscience, August 2009. McGill University   PKC Differentially Translocates during Spaced and Massed Training in Alypsia   Neuron, December 2006. Baylor College of Medicine   Drosophila Mushroom Body Neurons Form a Branch-Specific, Long-Term Cellular Memory Trace after Spaced Olfactory Conditioning There is rock-solid science behind Spaced Learning.
  38. 38. Gunner Training Works!
  39. 39. Gunner Training’s Effectiveness * 19.3% (p<0.000001)** * All data where user completed 4 reviews within the maximum time (115 days) ** Heteroscedastic, one-tailed t-test. Same results with paired, 2-tailed t-test.
  40. 40. Gunner Training’s Effectiveness Avg time to 4th review: 26.4 days Max time to 4th review: 115 days   Two major successes:   1) Knowledge isn’t lost during >100 days since initial review   2) 19.3% gain in retained knowledge
  41. 41. Summary   GT is the first Knowledge Management specifically platform for medical students   The application of 19 Learning Principles yields a highly efficient and effective program   Users who follow our algorithms gain an average of 19.3% in ability to recall knowledge after 4 reviews   Users don’t forget what they have already learned
  42. 42. Priced for Students
  43. 43. User Comments   “I absolutely loved this site. I   “I'll definitely let the 2nd years didn't get far into the program in my school know about because of time constraints, but your program. I've already I really wish I knew about it recommended the program to before starting 2nd year. It the Dean of Student Affairs at would've been amazing to use my school.” it while studying for the – J.W., 2nd year course. I used to study with medical student RecallPlus and Anki during the course but found it so time consuming to make the cards, so you can say Gunner Training is just perfect.” – H.A., 2nd year medical student
  44. 44. Anonymous Comments   It's surprising how much I've learned   “On my exam, there were questions in such a short time span with what that I actually thought, ‘I know that feels like not a lot of effort. I've maybe flashcard exactly.’ This is, however, spent 30 minutes to an hour every after I had continuously gotten the nit other day (sometimes every 2nd or picky factoid wrong on several 3rd day) on it and it's gone quickly. occasions at Gunner Training. ” I've been tracking my progress with Caboose, on the analytics it provides. After 2 weeks of studying and a few Student Doctor Network repetitions my recall has gone up significantly (the amount of 5's has increased by almost 20% - this meaning that when I do my scheduled review questions, the number of answers that I get right without having to think about the question is increasing). ” AggieSean, on Student Doctor Network
  45. 45.