Investigating a blended model of virtual and live simulated patient
Investigating and optimizing a blended model of virtual and live patient simulated cases Mary Power, Centre for Teaching Excellence firstname.lastname@example.org Lisa Craig, School of Pharmacy email@example.com with Pharmacy students: Peter Pagacz and Carol Khalil
Situated in downtown Kitchener, First graduating class 20117 km from rest of UW campus ~ 115 students/year
Professional Practice CurriculumProfessional Practice (PP) Courses/Labs are the only place where counseling skills are reinforced. with standardized patients and peer “patients”. – 1st year: PP1 and PP2 lecture and PP1 lab – 2nd year: PP3 lecture and PP2 lab – 3rd year: PP3 lab (spring** more than one year after PP2 lab**) – 4th year: PP4 course (spring)
Co-op• The first co-op term occurs at the end of first year following (PP1 and 2 lecture and PP1 lab)• 2nd year starts with the PP3 course and PP2 lab followed by another co-op term• Students return for an academic term without any “skills” labs, then another co-op term before they return to PP 3 lab.• It was felt that the co-op terms would be a sufficient way to ensure that the “skills” were reinforced.
What happened?Patient information gathering skills were lost BUT……– A key core competency is the ability to safely and effectively conduct a comprehensive and accurate patient history.
Why cases? % Retention Bloom’s Taxonomy (6 weeks) Knowledge Lectures 4-8 Comprehension Reading 6-10 Application Visuals 12-18 Demonstrations Analysis 15-65 Hands On Independ. Study CASE Synthesis Coop Group Work 60-80STUDIES Evaluation Teaching Others 80-90 Cone of Learning adapted from http://sciencecases.lib.buffalo.edu/
What does the literature say?• simulations such as virtual and standardized patients shown to be beneficial for the development of patient information gathering skills (Chaikoolvatana& Goodyer, 2003) (Schittek Janda et al., 2004) (Yudkowsky, Downing, & Ommert, 2006) (Haist et al., 2008).• preliminary research also suggests that a blended model, using virtual and standardized patients, may be optimal (Raij et al., 2007). however little is known about the optimal use of these modalities.
What do we see as the value of virtual patients?• solidify in students’ minds the essential information gathering questions• immediate feedback students learn as they go• students can re-visit reinforce the knowledge• reusable therefore financially beneficial
ObjectivesDetermine if virtual patients will help our studentsimprove their patient information gathering skills andwhat an optimal blend of VP and SP would beTo improve end of term OSCE (Objective Standardized Clinical Evaluation) scoresUltimately to best prepare our students for the OSCEthat is a requirement for licensure
Experimental Design1. Baseline information gathering skills assessed (in PP1) pre-OSCE2. 1st 5 weeks - students interact with cases in various blends
Experimental Design3. 6th week information gathering skills assessed (post-OSCE) assess change in rubric score4. 2nd 5 weeks in lab- students interacted with reverse case blend scenario5. final OSCE given and assessed compared with previous years)6. survey students to ascertain perceived value of virtual patient cases
Experimental Design (Phase 2)7. Trial the optimal blend with students the following year8. Determine optimum temporal placement of virtual and standardized patients9. Survey students to ascertain perceived value of virtual patient cases
Did VP# affect student learning? # VPs Pre-OS CE Post-OS CE Avg gain (avg) (avg) (+sd)Five (n=17) 48% 52% 4% (+36%Four (n =16) 54% 50% -4% (+ 22%)Three (n=12) 48% 60% 12% (+22%) Two (n=18) 44% 55% 11% (+28%) One (n=10) 44% 54% 10% (+23%) No significant difference
“Do you agree that virtual patients help inretention/reinforce application/help learning….” (n = 25) 16 14 12 10 8 6 4 2 0 retention reinforce application help learning strongly agree/agree neutral disagree/strongly disagree 6 respondents from 5 VP group 5 disagree/strongly disagree for all three categories and one disagreed for all but retention question (agree)
Do you agree that the virtual patient cases enhance your learning?“I liked them because they helped me to learn the questions that I needed to ask real patients.”“While I definitely see the value in the use of Virtual Patients in Professional Practice Lab, I think it really important to recognize that they will NEVER replace standardized patients. They are great for "pre-practice", but at the end of the day, the standardized patients is much more similar to what we will be dealing with in real-life.”“Having 5 virtual patients during the first 5 weeks definitely hindered the progression of my line of questioning. I was basically clicking the questions I thought were right and they were already there for me so I didnt have to do any thinking.”
What do you see as the most valuable use of the virtual patients? 0 5 10 15 20 25 30 35 review/study aid reinforce material learned in class/lab counseling practice in lab for testing not useful at all
Phase 2: Implementation of “Optimal” Blend • Decided upon a blend of 3 virtual patients and 3 standardized patients • Virtual patients were provided on-line ahead of class for students to work through individually • In tutorial guided discussion of case (concentrating on information gathering) • Standardized patients following week in lab
“Do you agree that virtual patients help inretention/reinforce application/help learning….” (n = 31) 28 24 20 16 12 8 4 0 retention reinforce applications help learning strongly agree/agree neutral disagree/strongly disagree
What do you see as the most valuable use of the virtual patients? 0 5 10 15 20 25 review/study aid reinforce material learned in class/labcounseling practice in lab for testing not useful at all
Do you agree that the virtual patient cases enhance your learning?“The virtual cases are similar to OSCEs that is great. The patients have a problem and you have to ask the right questions to get to the answer.”“I liked how some of the patients were more difficult to get information out of. It was more realistic.”“The fact the the virtual patient interviews provided multiple choice options to choose from in terms of questions for the information gathering made it simpler for us to know what to ask because the options were in front of us.”“The level of information needed for them was definitely above that of a first year student which made it quite difficult and often I just ended up clicking buttons until it was over”
SummaryVirtual patients do have value! Students like virtual patients but not as replacement to SPContext very important - if students feel they are losingout on some face-to-face opportunity satisfaction isdecreasedCases worked best with guided discussion follow-up
Going forward: Create a bank of virtual patient cases Work on integrating cases with online or F2F guideddiscussion more fully into the curriculum Ongoing monitoring of end-of-term OSCEs
An exciting outcome of the VP story…… Peter Pagacz, Class of 2013 • Scotiabank Pharmacy Entrepreneurship Award • University of Waterloo’s Nicol Entrepreneurial Award Competition • Nigel Stokes E-Launch Scholarship, Enterprise Co-op @ MBET Centre
Acknowledgements This project has been funded by the Learning Initiatives Fund (LIF) of the University of Waterloo. Thanks also to Nancy Waite, Associate Director of the School of Pharmacy for her ongoing support and Lisa McLean, PP Course Demonstrator for logistical support References• Chaikoolvatana, A., & Goodyer, L. (2003). Evaluation of a multimedia case-history simulation program for pharmacy students. American Journal of Pharmaceutical Education, 67(1)• Schittek Janda, M., Mattheos, N., Nattestad, A., Wagner, A., Nebel, D., Farbom, C., et al. (2004). Simulation of patient encounters using a virtual patient in periodontology instruction of dental students: Design, usability, and learning effect in history-taking skills. European Journal of Dental Education, 8(3), 111-119.• Yudkowsky, R., Downing, S. M., & Ommert, D. (2006). Prior experiences associated with residents scores on a communication and interpersonal skill OSCE. Patient Education & Counseling, 62(3), 368-373.• Haist, S. A., Lineberry, M. J., Griffith, C. H., Hoellein, A. R., Talente, G. M., & Wilson, J. F. (2008). Sexual history inquiry and HIV counseling: Improving clinical skills and medical knowledge through an interactive workshop utilizing standardized patients. Advances in Health Sciences Education, 13(4), 427-434.• Raij, A. B., Johnsen, K., Dickerson, R. F., Lok, B. C., Cohen, M. S., Duerson, M., et al. (2007). Comparing interpersonal interactions with a virtual human to those with a real human. IEEE Transactions on Visualization & Computer Graphics, 13(3), 443-457.