Presented by Brad Marcum and Margaret Sidle Vertical- Medical This session will show how one small medical school used data generated from ExamSoft reports to inform assumptive reports, specifically to inform the programmatic goal of improving licensure pass rates. At the end of the session, participants will be able to: 1) Participants will have a better understanding of what steps to follow to engage faculty in using data to improve their teaching. 2) Participants will be better able to identify what courses will predict how students perform on their board exams. 3) Participants will be able to identify test questions that address the learning outcomes for better board scores.

1. 1ExamSoft Assessment Conference | #ASSESS2015
ExamSoft Assessment Conference
#ASSESS2015

2. 2ExamSoft Assessment Conference | #ASSESS2015
Building A Better Test Taker
• Brad Marcum, Director,
• Academic Data Services, KYCOM
• Meg Sidle, Director, Institutional Research
• and Effectiveness, UPIKE

3. 3ExamSoft Assessment Conference | #ASSESS2015
A Little Background
•Medical School Mission
•Computer-based testing
began in Fall 2012
•Provided immediate
formative assessment
results to improve student
learning within individual
courses

4. 4ExamSoft Assessment Conference | #ASSESS2015
Unaddressed Issue
First-time pass rates of COMLEX
Level 1 remained low
Since 2004, continued to examine
relationships with
admissions criteria
individual course performance
Not making a difference

5. 5ExamSoft Assessment Conference | #ASSESS2015
A Fresh Approach
• Comprehensive Osteopathic Medical Self-
Assessment Examination (COMSAE)
• Strong correlation (r = .80) between KYCOM
student performance on this assessment and
their actual COMLEX Level 1 scores
• Find predictors for performance on the
COMSAE:
Regression indicated that seven predictors
explained 57.2% of the variance
(R2 =.59, F(7,177)=36.07, ρ<.01).

6. 6ExamSoft Assessment Conference | #ASSESS2015
Common Faculty Reaction to Statistics

7. 7ExamSoft Assessment Conference | #ASSESS2015
Summative Assessment
• See if data from the computer-based testing
on the item content from the four courses in
the model could prove useful
• Major issues with categories
• Thirty-eight (38) specific topics from these
courses were significantly related to the
students’ COMSAE scores. (ρ<.01)
Not necessarily strong relationships

8. 8ExamSoft Assessment Conference | #ASSESS2015
Where Do We Go From Here
• Faculty need to question why these particular
topics were significant out of the more that
150 topics in the four courses
• Faculty should examine the range of topics
and specific lessons that they taught in the
respective courses on these topics
• Faculty development workshops how to write
criteria

9. 9ExamSoft Assessment Conference | #ASSESS2015
THANK YOU!
• Brad Marcum, Director,
• Academic Data Services, KYCOM
• Meg Sidle, Director, Institutional Research
• and Effectiveness, UPIKE