Analyzing exemplars

1. Analyzing exemplars to support
students’ understandings of
assessment
David Carless, CRADLE seminar,
Melbourne, September 17th, 2015
The University of Hong Kong

2. Overview
1. Exemplars: rationale and benefits
2. Challenges and how they might be
tackled
3. A case study of practice
4. Implementation suggestions
The University of Hong Kong

3. Definition
Exemplars are samples of student work used to
illustrate dimensions of quality
Usually assignments from a previous cohort
The University of Hong Kong

4. Why used?
Exemplars convey messages
that nothing else can (Sadler,
2002)
The University of Hong Kong

5. Concreteness
Concreteness of
exemplars
vs
Rubrics which
students often see
as vague and
abstract
The University of Hong Kong

6. Promote self-evaluation
Illustrate what good
work looks like
Benchmark for
student comparison
with own
performance
The University of Hong Kong

7. Evaluative expertise
The evolving ability to
apply criteria;
self-evaluate;
and make informed
judgments.
The University of Hong Kong

8. Analysis
Identify strengths, weaknesses & suggest how
samples could be improved
The University of Hong Kong

9. Benefits for students
Gain experience in making
judgments
Apply insights to own work
Notice discrepancies between
present level & target level
The University of Hong Kong

10. Challenges
The University of Hong Kong

11. Potential problems
Difficulties for some students in evaluating
exemplars accurately
The University of Hong Kong

12. Reducing challenge
May reduce student
creativity
May encourage
unproductive copying
The University of Hong Kong

13. Time issues
• Time in collecting them
• Time in seeking consent
for their use
• Time taken away from
instruction
The University of Hong Kong

14. Key inference
The quality of the dialogue about the exemplars
is a crucial factor determining student learning
The University of Hong Kong

15. Key issue
To what extent does the use of exemplars
facilitate positive learning outcomes?
The University of Hong Kong

16. ONGOING RESEARCH
The University of Hong Kong

17. Project 1
Ed D study, Phil Smyth
GT study of how teachers manage the process of
using exemplars
The University of Hong Kong

18. Project 2
Enhancing dialogic use of exemplars amongst 10
teachers in a Faculty of Education
The University of Hong Kong

19. The University of Hong Kong

20. Context of the case study
Participants:
• Trainee Science teachers
• 10 student teachers
• BEd/BSc double degree
• Year 3
Teacher-
researcher: Dr
Kennedy Chan
The University of Hong Kong

21. Exemplars used
• 2 high quality exemplars
– Same format (i.e., reflective essays) but different
content focus
The University of Hong Kong

22. Data collection
• Classroom observation
• Open-ended student survey
• 2 focus group interviews
• Interview with Kennedy
• Student artefacts: ‘exit slips’, assignments etc.
The University of Hong Kong

23. Dilemmas
Students
constructing
notion of quality
VS
Telling students
about quality
 Time for students
to talk with
peers
VS
Time for
developing shared
understandings
 Giving students
hints to discover
quality
VS
Too much
guidance stifling
creativity
Using students’
voices
VS
Making
teacher’s voice
explicit

The University of Hong Kong

24. Dilemma #1: Constructing vs Telling
Discussing is better than
telling us the criteria.
Knowledge created by
students themselves leaves
deeper impression.
Telling students
the desired
quality
Allowing students
to construct the
meaning of quality
on their own
Maybe he didn’t want to
tell us the answer directly,
but I think it’s important
for him to clarify what a
good essay should be
like.
VS
The University of Hong Kong

25. Dilemma #2: Whose voices?
A good essay
should be our
ideas, not
Kennedy’s idea.
Making the
teacher’s voice
explicit
Using students’
voices
Analysis of
samples could be
made more
explicit
It’s difficult to strike a
balance between what
they say and my
voice. (Kennedy, Teacher
Interview)
VS
The University of Hong Kong

26. Dilemma #3: Time
If I was asked to analyse
samples alone, I may not
identify all the pros & cons
Time for
developing
shared
understandings
with the teacher
Time for students
to talk in peer
groups
I think we need to save
time for the whole
discussion; a summary is
quite important
VS
The University of Hong Kong

27. Dilemma #4: Guidance vs creativity
Samples should be of
high quality and
diverse in nature.
CATCH-22: If the
instructor tells
them, student’s
creativity may be
limited. If you don’t
tell them, creative
students usually
get low marks.
Giving too much
which stifles their
creativity
Giving students
enough hints to
discover the
quality
2 samples is good because
if you give too many
exemplars, students may
be afraid to use the same
ideas.
If a teacher gives
too much
guidance then
you tend to follow
that format.
VS
The University of Hong Kong

28. Managing these dilemmas
Constructing
VS
Telling

Time

Guidance VS
Creativity
Shared
voices

Strategies for
highlighting
the valued
quality (e.g.
creativity)
Strategies to
consolidate
student
insights
Strategies to
balance time
for dialogue
vs input
Strategies for
building
shared
understanding
The University of Hong Kong

29. CONCLUSION
The University of Hong Kong

30. Evolving connoisseurship
Enhancing students’ understanding of goals and
standards helps them to produce good work
The University of Hong Kong

31. Analogical reasoning
High-leverage strategy with potential to support
positive learning outcomes
The University of Hong Kong

32. Implementation strategies
Written
task
Students brainstorm
criteria for good task
response
No. of
samples? 2
Peer discussion
Teacher scaffolding
Student ownership
The University of Hong Kong