Presentation at AMEE 2011. The abstract can be found here:

1. Engagement with an e-learning
tool during an Emergency
Department induction
programme:
Can it predict junior doctor
performance?
Dr. Damian Roland
NIHR Doctoral Research Fellow

2. A question…

3. Medical students' compliance with
simple administrative tasks and
success in final examinations
Care Free Hospitals NHS Trust
Identification Badge
Medical Student: T.Ardy
Placement: Paediatrics
Start date: 1st August 2011
Photograph Date: 4th October 2011
Neil Wright, M S Tanner BMJ 2002;324:
1554-1555
doi:10.1136/bmj.324.7353.1554

4. Related Research
 Disorganized junior doctors fail the MRCP
(UK)
Stanley, A. Medical Teacher, Vol. 28, No. 1, 2006, pp. e40–e42
 Use of unsupervised online quizzes as
formative assessment in a medical
physiology course: effects of incentives on
student participation and performance
 Kibble, J. Adv PhysiolEduc 31: 253–260, 2007;
doi:10.1152/advan.00027.2007.
 The use of formative online quizzes to
enhance class preparation and scores on
summative exams
 Dobson, J. Adv PhysiolEduc 32: 297–302, 2008;
doi:10.1152/advan.90162.2008.

6. The REMIT Project
Refining Evaluation Methodologies
for IntervenTions that change
practice

7. Junior Doctors Induction
 36 New Starters in
an Emergency
Department
 “Un-enforced”
Mandatory
requirements
 Various Metrics
applied

8. Junior Doctors Induction
 Performance in
initial Paediatric
MCQ
 Use of “Spotting
the Sick Child”
 Number of Patients
seen in first two
months
 Consultant spot-
check assessment:

9. Junior Doctors Induction
Darzi
Performing well above
expected standard
GSP
Good solid provider
HDU
High Dependency – concern
expressed and fed-back to
Clinical/Educational
Supervisor
ITU
Intensive Care – intervention
required

10. Key Findings
 Supervisor
Assessment
9Darzi
21 GSP
5 HDU
1 Didn’t start
 Range of Scores 131-
171
 Spotting the Sick
Child Use
24 No log-in
4 No log-in longer
than 2 minutes
8 Log-in consistent
with some activity

11. Patients seen in 1st
Two months
All Patients 0-15 Years 0-5 Years
Number seen
[Min (IQ Range)
Max]
115 (179-239) 366 17 (28-56) 117 6 (16-34) 63
Discharge Rate 30.8% - 55.7% 36.8%-91.7% 35.7%-96.2%
0
50
100
150
200
250
300
350
400
All Patients
0
20
40
60
80
100
120
140
Age 0-15
0
10
20
30
40
50
60
70
Age 0-5

12. Risk Factors
 Doctors who didn’t undertake MCQ
Relative Risk of being HDU/ITU:
8 (CI 2.4 – 27)
 Doctors who undertook <2mins SSC
No doctor who was HDU/ITU
completed more than 2 mins

13. Discussion
Trends continue towards engagement
being linked in some way to
performance
 What is a gold standard of
“engagement”?
 What is the best measure of
“performance”?
 Can we separate “performance” and
“professionalism”?

14. Thanks
 Dr Ffion Davies
 Dr. Holger Wahl
 Dr. Monica Lakhanpaul
 Nic Blackwell
 Prof. Tim Coats
 NIHR (the views expressed in this
presentation may not be reflective of the
NIHR)
My wife who has let me come to this
event