1. VIRTUAL FAILURE:
Diagnostic Reasoning Analysis of Computer Case Simulation Does NOT
Correlate with Academic Performance in Medical Students
Jeannette Guerrasio, Ethan Cumbler, Roger Turcotte, Mel Anderson, Eva Aagaard
University of Colorado Anschutz Medical Campus
BACKGROUND RESULTS
 Increasing widespread use of computerized virtual Table 1: Third Year Medical Student Performance on Assessment Methods
Outpt clerkship = Ambulatory Adult Care, Inpt Clerkship = Hospitalized Adult Care
patients Grades: Fail, Pass, High Pass, Honors
 Virtual patients have not been validated as an Assessment Method Mean SD Min Max
assessment method. Outpt Clerkship Grade High pass n/a Pass Honors
Inpt Clerkship Grade High pass n/a Fail Honors
PURPOSE Outpt Clerkship Exam 76.1 5.57 59 89
Inpt Clerkship Exam 64.3 6.47 50 79
 To determine if performance on a computer Step I 228 20.7 168 264
simulated case correlated with other aspects of Step II 237 17.3 181 275
medical student performance. CorrectDx on Simulator 2.41 1.33 0 5
 To determine if an computer case simulator can
be used as a method of student evaluations in the Table 2: Pearson’s Correlation of Computer Simulation Diagnostic Performance with
clinical clerkships. Other Standard Assessment Methods
Outpt clerkship = Ambulatory Adult Care, Inpt Clerkship = Hospitalized Adult Care
METHODS Assessment Method 95% CI p value
Recruitment Outpt Clerkship Grade 0.99-1.03 0.42
145 third year medical students at the University of
Inpt Clerkship Grade 0.99-1.02 0.47
Colorado completed one computer simulated case
Outpt Clerkship Exam 0.99-1.03 0.44
Computer Clinical Case Simulator Inpt Clerkship Exam 0.99-1.02 0.47
DxR Clinical Software
Presentation: 47 yo male with jaundice and STEP I 0.99-1.00 0.38
hematemesis Step II CK 0.99-1.01 0.80
Correct diagnoses as determined by expert panel:
•Variceal bleed
•Cirrhosis
•Encephalopathy KEY LESSONS
With additional points for We found NO correlation between performance on standard medical
•Anemia student assessment methods and the ability to diagnose a patient using
•Coagulopathy computer simulation.
•Autoimmune hepatitis This computer simulated case has limited value in the evaluation of
medical student performance.
Assessment Method
Using Pearson’s Correlation, we compared
diagnosis score on the Computer Clinical Case LIMITATIONS
Simulator with: Single institution, single medical student class
•Inpatient Medicine Clerkship Grade Only one case was used
•Inpatient Medicine Clerkship Exam Score May be limited by the computer simulation program used.
•Outpatient Medicine Clerkship Grade
•Outpatient Medicine Clerkship Exam Score REFERENCES:
Clauser BE, Schuwirth LWT. The Use of Computers in Assessment. In: Norman GR, van derVleuten C, Newble DI, eds. International Handbook of Research in
•USMLE Step I Medical Education.Vol 2. Kluwer Academic Publishers; 2002:757-791.
Cook DA, Erwin PJ, Triola MM. Computerized virtual patients in health professions education: a systematic review and meta-analysis. Acad Med.
•USMLE Step II 2010;85(10):1589-1602.