Multimedia-based ScriptConcordance Testing as a methodof testing Clinical Decision Making Skills Todd Chang, MD firstname.lastname@example.org
Background• Assessing Clinical Decision Making Skills (CDMS) is difficult but necessary• Script Concordance Testing (SCT) – Has been validated as a method of assessing trainees in many subspecialties – Is currently being studied for infant lumbar punctures (LP) to assess infant LP management
Background• Text-based questions and team-based simulations may not isolate individual’s CDMS – Multimedia questions have lower scores than text- based Holtzman KZ, Swanson DB, Ouyang W, Hussie K, Allbee K. Use of Multimedia on the Step 1 and Step 2 Clinical Knowledge Components of USMLE: A Controlled Trial of the Impact on Item Characteristics. Acad Med 2009; 84(10s): s90-s93
SCTA 1-month-old male has a rectal temperature of 40.3 Celsius. There is mild rhinorrhea.Does the following change your likelihood to perform an LP: He is RSV+ -2 Much less likely -1 Less likely 0 No change +1 More likely +2 Much more likely
PICO Question• Population – In pediatric residents, subspecialty fellows, and attendings• Intervention – Does the use of Multimedia depictions of clinical scenarios• Comparison – Compared with text-based depictions• Outcome – Affect SCT scores negatively?
Approach• Create 2 versions of every SCT question: – text-based case scenario – multimedia-based (screen-based simulation) case scenario Q1. An alert 2-week-old infant is RSV+ and has a respiratory rate of 70 and subcostal retractions Q1. 2-week-old RSV+
Approach• Optimize a 15-question SCT set to test 2 CDMS: – Infant Lumbar Puncture – Infant/Pediatric/Adolescent Intubation• Randomize multimedia vs. text-based & administer questions• Evaluate score differences against training status, (sub)specialty status, and self-reported experience
3 Questions• How do we create the optimal multimedia element – VR vs. true patients?• Should randomization be per question or per student?• Are there more optimal methods of validating the SCT question set? Todd Chang, MD email@example.com