This document analyzes different methods for preparing for the USMLE Step 1 exam. The author conducted surveys of medical students at UT Health Science Center (UTHSC) about their study methods and Step 1 scores. Regression analysis found that using High Yield review materials had a positive impact on scores, while group tutoring sessions and checking out books from the student academic support services had negative impacts. Surveys of students on an online forum found some differences in common study resources compared to UTHSC students. The author concludes that specific study resources may have little effect and recommends schools provide alternative prep support and consider curricular reforms. Future research with multiple schools and better controls is needed.

1. +
Analysis of Step 1 Prep
Methods
Jenny Loome
Summer Educational Research Fellowship

2. +
What is Step 1?
 Step 1 of United States Medical Licensing Examination
 End of M2 year
 High stakes
 Intended only for licensing purposes
 Used for selecting residents

3. +
Research Questions
 What are best practices for Step 1 prep?
 Focusing on extracurricular study
 How are worldwide students preparing for Step 1?
 How are UT students preparing for Step 1?
 How effective are common preparations methods?

4. +
Background
 Anxiety causes indiscriminate use of many anti-anxiety
resources (Encandela et al. 2014)
 Can’t take student perceptions as fact
 Students use huge range of test-prep resources
 Lack of recent third-party studies
 Old evidence of ineffective or detrimental commercial resources
(Thadani et al. 2000, Werner et al. 2003)
 Old evidence of effective free/curriculum-based resources (Thadani
et al. 2000)

5. +
UTHSC Survey
 Students completed surveys after Step 1 2012-2013
 Test prep resources
 Practice tests
 Step 1 scores

6. +
Survey Analysis
Repeated regressions until p≤0.05 for all variables
Eliminated one variable at a time
Multivariate linear regressions on the variables of p<0.200.
Controlled for NBME score and year
Univariate analyses v. Step 1 score and ∆score.
U Mann Whitney for binary variables
Linear regressions for continuous
variables
Chose variables
Present both years and relevant to test prep

7. +
Results from UT
Significant Variables
Model
Unstandardized
Coefficients
Standardized
Coefficients
t Sig.B Std. Error Beta
1 (Constant)
-21333.807 4308.576 -4.951 .000
Year
10.657 2.140 .278 4.981 .000
NBME score
.598 .044 .770
13.68
8
.000
Group tutor sessions
-18.315 6.539 -.155 -2.801 .006
Check out SASS
books
-4.305 2.132 -.108 -2.019 .046
High Yield
7.513 3.611 .112 2.081 .040
a. Dependent Variable: Step 1 score

8. +
Results from UT
Insignificant variables:
 First Aid
 DIT
 USMLERx
 UWorld Q Bank
 Kaplan Q Bank
 ½-length practice exams
 Full-length practice exams
 Any practice exams
 Goljan Rapid Review
 Flashcards
 Microbiology Made Ridiculously
Simple
 Step-up to Step 1
 Anatomy review
 Making a schedule with and
educational specialist
 Meeting with specialist about test
anxiety
 Number of resources

9. +
Online Source Analysis
 On Dr-Ed listserv, recorded reports of institutional
test-prep methods
 Anecdotal evidence
 On the Student Doctor Network (Official 2012/13
Step 1 Experiences and Scores), recorded voluntary
reports of study methods
 Used t-tests to compare study resource frequencies on
SDN and UTHSC

10. +
DR-ED Results
 Two schools provided free access to online question banks
 Six schools use mandatory custom NBMEs in their curricula.
 Four schools provide their students access to CBSEs for free.
 Nine schools use mandatory CBSEs as evaluations in their
curricula.
 CBSEs and NBMEs are often used to screen for students at
risk of failing Step 1 and provide them interventions.

11. +
SDN Results
 First Aid
 DIT
 USMLE Rx
 Kaplan Question Bank
 Practice exams
 Goljan
 Flashcards
 Step-up to USMLE Step 1
 USMLE World Question Bank
 Microbiology Made Ridiculously
Simple
 Number of resources
Different from UTHSC Similar to UTHSC

12. +
SDN Results
Comparing Frequencies at UTHSC and SDN
Source N Mean Std. Deviation
Std. Error
Mean
P value
1st_Aid SDN
85 .918 .2765 .0300
.018
UTHSC
132 .992 .0870 .0076
DIT SDN
85 .176 .3835 .0416
.000
UTHSC
132 .432 .4972 .0433
USMLERx SDN
85 .188 .3932 .0427
.003
UTHSC
132 .045 .2091 .0182
World_QB SDN
85 .988 .1085 .0118
.377
UTHSC
132 .970 .1721 .0150
Kap_QB SDN
85 .494 .5029 .0546
.000
UTHSC
132 .121 .3276 .0285

13. +
SDN Results
Comparing Frequencies at UTHSC and SDN
Source N Mean Std. Deviation
Std. Error
Mean
P value
Total practice
exams
SDN
85 .73 .447 .048
.001
UTHSC
132 .91 .289 .025
Goljan RR SDN
85 .506 .5029 .0546
.023
UTHSC
132 .348 .4783 .0416
Flashcards SDN
85 .153 .3621 .0393
.043
UTHSC
132 .265 .4431 .0386
MMRS SDN
85 .094 .2937 .0319
.437
UTHSC
132 .129 .3362 .0293
Step-up SDN
85 .000 .0000 .0000
.014
UTHSC
132 .045 .2091 .0182
Number of
Resources
SDN
85 4.25 1.214 .132
.986
UTHSC
132 4.25 1.162 .101

14. +
Findings
 Only High Yield had a significant positive impact at UT
 Using more resources did not result in higher scores
 Possible explanations:
 Specific resources do little to affect scores
 Extracurricular study during designated time has little effect
 Med schools report very little Step 1 prep
 Students report full-time Step 1 prep

15. +
Improvements
 Schools provide alternative Step 1 prep
 Peer-led review (Alcamo et al. 2010)
 Encourage use of class-related instead of commercial resources
(Thadani et al. 2010)
 Provide anti-anxiety planning (Encandela et al. 2014)
 Curricular reform
 Studies have varying results (Cuddy et al. 2013, Williams et al.
2011)

16. +
Future Research
 Survey of student test prep methods
 Multiple schools
 Designed for this study
 Consistent across years
 Include MCAT scores
 Better control groups

17. +
Works cited
Alcamo AM, Davids AR, Way DP, Lynn DJ, Vandre DD. The Impact of a Peer-
Designed and -Led USMLE Step 1 Review Course: Improvement in Preparation and
Scores. Academic Medicine. 2010;85(10):S45-S48.
Cuddy MM, Swanson DB, Drake RL, Pawlina W. Changes in anatomy instruction and
USMLE performance: Empirical evidence on the absence of a relationship.
Anatomical Sciences Education. 2013;6(1):3-10.
Encandela J, Gibson C, Angoff N, Leydon G, Green M. Characteristics of test anxiety
among medical students and congruence of strategies to address it. Medical
Education Online. 2014;19.
Thadani RA, Swanson DB, Galbraith RM. A Preliminary Analysis of Different
Approaches to Preparing for the USMLE Step 1. Academic Medicine.
2000;75(10):S40-S42.
Werner LS, Bull BS. The effect of three commercial coaching courses on Step One
USMLE performance. Medical Education. 2003;37(6):527-531.
Williams PCMD, Epps ACP, McCammon SM. The Strategic Impact of a Changing
Curriculum and Learning Environment on Medical Students' Academic Performance.
Journal of the National Medical Association. 2011;103(9/10):802-810.

18. +
Questions