This study examined the relationship between medical students' engagement with an ePortfolio, as measured by a novel engagement score, and their end-of-year summative assessment performance. The researchers assessed 90 randomly selected undergraduate ePortfolios and compared engagement scores to end-of-year assessment results. No correlation was found between ePortfolio engagement score and overall summative assessment performance or individual assessment components. The study suggests ePortfolios should focus on formative assessment and developing lifelong learning skills rather than being used as a surrogate for knowledge and skills acquisition.

1. Dr
Laura-­‐Jane
Smith,
Dr
Rosie
Belcher,
Dr
Deborah
Gill
Academic
Centre
for
Medical
Educa?on,
UCL
Medical
School,
London,
United
Kingdom
In
this
study
there
was
no
correla2on
between
engagement
with
an
ePor7olio
as
measured
by
a
novel
engagement
score,
and
end
of
year
summa2ve
assessment
performance.
A
focus
on
maximising
the
benefit
students
can
get
from
ePor7olio
use
in
rela2on
to
forma2ve
assessment
and
lifelong
learning
skills
is
advisable.
Take
home
message
Does
ePor7olio
engagement
predict
performance
in
undergraduate
medicine?
ePorDolios
are
a
core
component
of
postgraduate
medical
educa?on
and
training.
Increasingly,
undergraduates
mirror
postgraduate
learners
in
the
use
of
ePorDolios1.
Assessment
of
porDolios
is
usually
a
combina?on
of
grading
individual
components
and
a
measure
of
engagement
with
the
ePorDolio.
There
remains
some
debate
about
engagement
and
aMendance
as
predictors
of
success
in
undergraduate
and
postgraduate
medical
educa?on2.
Anecdotally
it
has
been
claimed
that
engagement
with
an
ePorDolio
is
a
predictor
for
other
outcomes
such
as
performance
in
end
of
year
wriMen
and
OSCE
examina?ons.
Background Summary
of
work
Students
were
required
to
complete
16
workplace-­‐based
assessments
(WPBAs),
aMend
and
record
4
mee?ngs
with
their
personal
tutor
and
upload
a
number
of
documents
to
their
personal
library
(both
wriMen
and
peer
assessments).
In
addi?on
students
were
invited
to
use
other
elements
of
the
ePorDolio
such
as
the
reflec?ve
logs
and
personal
development
plans,
but
these
were
not
required
or
assessed.
We
created
a
matrix
to
score
ePorDolio
engagement,
including
measures
of
students’
use
of
compulsory
and
non-­‐compulsory
elements
of
the
ePorDolio,
and
the
students’
organisa?on
of
contents
(Table
1).
Summary
of
results
As
ePorDolio
use
increases
in
undergraduate
medical
educa?on,
cau?on
must
be
exercised
to
ensure
that
they
are
not
used
as
surrogates
of
other
outcomes
without
a
robust
evidence
base.
We
plan
to
extend
the
study
to
the
three
collabora?ng
schools,
to
further
inves?gate
links
between
porDolio
engagement
and
assessment
outcomes.
Engagement
with
a
porDolio
is
required
for
postgraduate
learners
to
progress
in
training
and
retain
a
licence
to
prac?ce.
Rather
than
a
surrogate
for
knowledge
and
skills
acquisi?on,
porDolios
should
be
conceptualised
as
assessing
other
facets
of
professional
prac?ce
not
captured
by
exis?ng
assessment
methods.
Seen
in
this
light
they
offer
an
opportunity
to
support
and
develop
essen?al
lifelong
learning
skills,
freed
from
the
o[en
rigid
constraints
of
summa?ve
assessment
methodologies.
Conclusions
We
assessed
90
randomly
selected
undergraduate
ePorDolios
(selected
using
a
random
number
generator)
and
compared
engagement
scores
to
end
of
year
summa?ve
assessment
scores.
We
used
SPSSv20
for
descrip?ve
sta?s?cs
and
to
calculate
the
Pearson
coefficient.
Mean
engagement
score
was
5.4
+/-­‐
1.75.
No
correla?on
was
found
between
engagement
score
(range
0
–
10)
and
overall
summa?ve
assessment
results
(r=0.11,
p=0.31),
or
with
individual
components
of
the
engagement
matrix.
For
example
when
WPBA
number
(range
9
–
40)
was
taken
as
a
con?nuous
variable
there
was
also
no
correla?on
(r=0.11,
p=0.32).
Authors
0 1 2
Workplace
based
assessments
<16
16
-­‐
20
21
–
30
Personal
Library
content*
Not
all
required
elements
present
All
required
elements
present
Personal
Library
content
No
extra
material
Extra
material
Personal
Library
organisa2on
Not
organised
into
folders
and/or
file
names
unclear
Some
organisa?on
into
folders
Completely
organised
into
folders
Reflec2ve
logs/
Personal
Development
Plan
Not
used
Used
Photo
upload
/
personal
profile
updated
No
Yes
Personal
tutor
reports
None
1
–
2
3
–
4
References
1. Buckley,
Sharon
et
al.
“The
educa?onal
effects
of
porDolios
on
undergraduate
student
learning:
a
Best
Evidence
Medical
Educa?on
(BEME)
systema?c
review.
BEME
Guide
No.11.”
Medical
Teacher
(2009);
31(4):
282-­‐98
2.
Hyde,
Richard
M.,
and
D.J.
Flournoy.
“A
case
against
mandatory
lecture
aMendance.”
Journal
of
Medical
Educa?on
(1986);
61(3)
:
175-­‐176
Table
1.
Engagement
scoring
matrix
UCL
is
one
of
four
medical
schools
working
collabora?vely
with
NHS
Educa?on
for
Scotland
to
adapt
the
postgraduate
ePorDolio
currently
used
in
UK
Founda?on
Schools
for
use
in
undergraduate
medicine.
The
ePorDolio
collects
and
collates
large
amounts
of
electronic
data
on
students
about
their
engagement
with
the
programme,
and
the
first
year
of
adop?on
provided
an
opportunity
to
inves?gate
the
links
between
porDolio
engagement
and
assessment
success.
NHS
Educa?on
for
Scotland
UCL
Medical
School
Bristol
Medical
School
Glasgow
Medical
School
Brighton
and
Sussex
Medical
School
Maximum
score
was
14.
Students
comple?ng
the
compulsory
elements
with
no
addi?onal
elements
scored
4.
100
110
120
130
140
150
160
170
180
0
2
4
6
8
10
12
Summa2ve
end
of
year
examina2on
score
ePor7olio
engagement
score
Plot
of
ePor7olio
engagement
score
against
score
in
summa2ve
end
of
year
examina2on
score.
Plot
of
engagement
score
vs
examina2on
score