Clarifying learning goals & standards: rubrics & exemplars

Clarifying learning goals &
standards: rubrics & exemplars
David Carless
University of Hong Kong
At EdUHK, January, 20, 2017
The University of Hong Kong

Overview
1. Key concepts
2. Students’ views on rubrics & exemplars
3. Using rubrics for learning
4. Analyzing exemplars
5. Summary of key implications

KEY CONCEPTS

Competing assessment functions
Stimulating
productive student
learning
Satisfying
accountability
needs
Judging student
achievement

Learning-oriented assessment
 Prioritizing student learning as a key aim of all
assessment

Learning-oriented assessment
framework (Carless, 2015a)
Productive
assessment task
design
Student
engagement with
feedback
Understanding
quality academic
work

Importance of learning goals
Where am I going?
How to get there?

What are rubrics?
• A.k.a. ‘assessment criteria’, ‘grade descriptors,’
‘marking schemes’
• Rubrics combine criteria and standards in the
form of a grid to provide statements of the
performance needed to achieve different
standards (Carless, 2015, p. 132)

What are exemplars?
Exemplars are samples of student work used to
illustrate dimensions of quality
Usually assignments from a previous cohort

Two relevant chapters
Carless, D. (2017). Students’ Experiences of
Assessment for Learning. In D. Carless et al., (Eds.),
Scaling up Assessment for learning in Higher
Education. Singapore: Springer.
Jönsson, A. & Panadero, E. (2017). The Use and
Design of Rubrics to Support Assessment for
Learning. In D. Carless et al., (Eds.), Scaling up
Assessment for learning in Higher Education.
Singapore: Springer.

EVIDENCE FROM INTERVIEWS
WITH STUDENTS

Research process
Classroom observations
Interviews with teachers & students

Rubrics data (1)
“Criteria are rather vague to me …”
“I read the criteria descriptions some time ago
and I’ve already forgotten them”
“The criteria were similar to previous ones so I
didn’t study them carefully or consult them
when preparing the assignment”.

Rubrics data (2)
“The descriptions of assessment criteria are
almost the same for every subject with the same
key words like critical & analytic”
“The assessment criteria are a bit lifeless. I
wanted to know the teacher’s interpretation of
the criteria and his personal preference”.

Rubrics data (3)
“Lists of criteria are all the same, it’s all very
vague. Like for an ‘A’, it is excellent mastery and
correct application and then ‘B+’ would be
above average mastery and good application.
You don’t know what it means so you just do
what you can. Once you know how to get good
academic results, you don’t need to rely on
criteria any more because the criteria are always
the same.”

Exemplars data (1)
“Samples are more concrete”
“It is a reference … you can imitate its
presentation style”

Exemplars data (2)
“The exemplar served as a useful example of
what a good report should be”
“There is a risk of copying but many of us try to
add something more or use our own approach”

Inferences
Rubrics are only useful for learning if
students engage with them
Exemplars are attractive to students but
teacher implementation is complex

RUBRICS & STUDENT
ENGAGEMENT

Dual aspects of rubrics
1. A scoring tool for teachers
2. A potential learning tool for students as aid to
self-evaluation

Rubric options
• Analytic or holistic
• Generic or Task-specific
(see handout)

Writing rubrics
Standards should be represented with respect to
features of work, not impenetrable terms, such
as ‘good’ or ‘excellent’ (Ashwin, Boud et al.,
2015, p. 259)

Rubrics activity 1
Students in groups brainstorm their own criteria
for an assessment task

Rubrics activity 2
Students read the criteria for (say) a grade B and
then re-write them in their own words

Rubrics activity 3
Students use rubric to evaluate work of peer
(peer evaluation)
AND/OR
Students use rubric to evaluate exemplars

DISCUSSION 1: RUBRICS
• Share some of your thinking on using rubrics
to enhance student learning
• What unanswered issues might you raise in
relation to rubrics?

RATIONALE & USE OF
EXEMPLARS

Why used?
Exemplars convey messages
that nothing else can (Sadler,
2002)

Promote self-evaluation
Illustrate what good
work looks like
Benchmark for
student comparison
with own
performance

Analysis
Identify strengths, weaknesses & suggest how
samples could be improved

When used?
Useful for students at
transition stages
When assignments are
complex, innovative or
hard to explain

How used?
• Posted on LMS
• Optional workshop (Rust et al., 2003)
• Part of regular class
• Displayed or made available
• In part or as a whole

Benefits for students
Gain experience in making
judgments
Apply insights to own work
Notice discrepancies between
present level & target level

Copying model problem
May reduce student
creativity
May encourage
unproductive copying

Time issues
• Time in collecting them
• Time in seeking consent
for their use
• Time taken away from
instruction

It’s challenging!
Difficulties for some students in evaluating
exemplars accurately

Synthesis on exemplars
• Students like exemplars
• They aid learning
But
• How to implement effectively
• How to avoid ‘copying model answer
syndrome’

Key inference
The quality of the dialogue about the exemplars
is a crucial factor determining student learning

DISCUSSION 2: EXEMPLARS
• Why don’t teachers make more & better use
of exemplars?
• What unanswered issues, might you raise in
relation to exemplars?

RECENT EXEMPLARS
RESEARCH

TDG Project
Enhancing dialogic use of exemplars amongst 10
teachers in a Faculty of Education

Research question
What are the main features of a specific
exemplars dialogue and how is it orchestrated?

Context of the case study
Participants:
• Trainee Science teachers
• BEd/BSc double degree
• Year 3
Teacher-
researcher: Dr
Kennedy Chan

Data collection
• Classroom observation
• Open-ended student survey
• 2 focus group interviews
• Interview with teacher-researcher
• Teacher journal
• Student artefacts: ‘exit slips’, assignments etc.

Exemplars provided
2 high quality exemplars
– Same format (i.e., reflective essays) but different
content focus
– Design aimed at reducing ‘copying’

Management of dialogue
1. Prior to class students analyzed exemplars;
2. Students discussed exemplars in pairs;
3. Students elicited views from peers and made
mini-presentations;
4. Teacher-orchestrated dialogue;
5. Students submitted exit slip of reflections.

Dilemmas
Students
constructing
notion of quality
VS
Telling students
about quality
 Time for students
to talk with
peers
VS
Time for
developing shared
understandings

Exemplars as
guide
VS
Exemplars as
model
Using students’
voices
VS
Making
teacher’s voice
explicit


Good Exemplars dialogue
• Airs multiple & divergent viewpoints
• Shows linkages between peer talk & whole-
class discussion
• Evidences development of student views
• Makes explicit some key qualities of exemplars
(Carless & Chan, 2016)

CONCLUSION

Using exemplars & criteria
Assessment task
Students devise
or engage with
rubric & quality
Two samples read
before class
Peer discussion
Teacher-led
dialogue
Student
ownership of
insights

Engaging with quality
Student need to engage with what quality looks
like and develop capacities in making judgments

THANK YOU

References
Ashwin, P., Boud, D., et al., (2015). Reflective teaching in higher education. London: Bloomsbury.
Carless, D. (2015a). Exploring learning-oriented assessment processes. Higher Education,
69(6), 963-976.
Carless, D. (2015b). Excellence in University Assessment: learning from award-winning
teachers. London: Routledge.
Carless, D. (2017). Students’ experiences of Assessment for Learning. In D. Carless, S.
Bridges, C.K.W. Chan & R. Glofcheski (Eds.), Scaling up Assessment for learning in Higher
Education. Singapore: Springer.
Carless, D., Bridges, S., Chan, C.K.W. & R. Glofcheski, R. (Eds.), (2017). Scaling up Assessment
for learning in Higher Education. Dordrecht: Springer.
Carless, D. & K.K.H. Chan (2016). Managing dialogic use of exemplars. Assessment and
Evaluation in Higher Education, http://dx.doi.org/10.1080/02602938.2016.1211246
Hendry, G., Armstrong, S. & Bromberger, N. (2012). Implementing standards‐based assessment
effectively: Incorporating discussion of exemplars into classroom teaching. Assessment and
Evaluation in Higher Education, 37(2), 149-161.

References (cont.)
Jonsson, A. & Panadero, E. (2017). The use and design of rubrics to support
assessment for learning. In D. Carless, S. Bridges, C.K.W. Chan & R. Glofcheski
(Eds.), Scaling up Assessment for learning in Higher Education. Singapore:
Springer.
Rust, C., Price, M. & O’Donovan, B. (2003). Improving students’ learning by
developing their understanding of assessment criteria and processes.
Assessment and Evaluation in Higher Education, 28(2), 147-164.
Sadler, D. R. (2002). Ah! … So that’s ‘quality’. In P. Schwartz & G. Webb (Eds.),
Assessment: Case Studies, Experience and Practice from Higher Education
(p.130-136). London: Kogan Page.
Smith, C., Worsfold, K., Davies, L., Fisher, R. & McPhail, R. (2013). Assessment
literacy and student learning: The case for explicitly developing students
‘assessment literacy’. Assessment and Evaluation in Higher Education, 38(1),
44-60.

Clarifying learning goals & standards: rubrics & exemplars

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Clarifying learning goals & standards: rubrics & exemplars

Similar to Clarifying learning goals & standards: rubrics & exemplars (20)

More from David Carless

More from David Carless (20)

Recently uploaded

Recently uploaded (20)

Clarifying learning goals & standards: rubrics & exemplars

Editor's Notes