2. Definition
A scoring tool that lays out the specific expectations for an assessment
task (Stevens & Levi, 2005)
A set of clear explanations or criteria used to help teachers and students
focus on what is valued in a subject, topic, or activity (Russell, &
Airasian, 2012).
Scoring rubrics are descriptive schemes that are developed by teachers
or other evaluators to guide the analysis of the products or processes of
students’ efforts. (Brookhart,1999 as cited by Moskal,2000)
Components of a rubric:
Criteria/Indicator
aspects of an assessment task which the assessor takes into account
when making their judgment
May use different weightings for different criteria
Level of Attainment
often use grade level descriptors
Types of rubrics:
Descriptive (Analytic), Holistic
Why and when we use particular types of rubrics
SCORING RUBRIC
3. WHY USE RUBRICS?
For teachers:
• Prompt a criterion-referenced
assessment
• Provide students with
detailed and timely feedback
• Encourage critical thinking
• Facilitate communication
with others involved in
scoring
• Help to refine teaching
skills/learning activities
For students:
• Clarify the teacher’s
expectations of student
performance
• Provide informative
descriptions of expected
performance
• Help to monitor and
critique own work
4. Well defined
Clearly describe the expected level of student
performance for each criterion in a rubric
Avoid general evaluative words (poor, excellent, etc.)
Use specific objective terms (correctly identifies,
uses only basic vocabulary, chooses incorrect
formula… )
Context specific
Describe what teachers expect from student for a
given performance or work product on a particular
subject domain
Viable for instruction
CHARACTERISTICS OF GOOD RUBRICS
5. Finite and exhaustive
Every response must be scorable
Too many score levels is confusing for students and
causes disagreement among teacher scores
Ordered
Represent the different levels of learning targets as
defined by LP
Related to Common Core theme/strand
Coherent with the cognitive complexity of the
standards
CHARACTERISTICS OF GOOD RUBRICS
10. Task Description:
Criteria
Level 5 Overall description of Level 5
Level 4 Overall description of Level 4
Level 3 Overall description of Level 3
Level 2 Overall description of Level 2
Level 1 Overall description of Level 1
HOLISTIC RUBRICS
A single scale with all criteria to be included in the
evaluation being considered together
Based on an overall judgment of student work
11.
12.
13. Advantage:
Saves time in developing
and scoring.
Disadvantage:
Does not provide specific
feedback for
improvement.
17. Reflecting on the task & content
Learning outcomes of the unit and the particular assessment
What we want from the students, why we created this
assessment, what our expectations are
Listing the learning outcomes & expectations
Focus on the particular details of the task and what specific
learning objectives we expect to see in the completed task
Grouping & labeling the outcomes & criteria
Organize the results of reflections, group similar expectations
together to become the rubric Indicators
Application of a rubric format
Apply the templates & descriptions to the final form of the
rubrics
4 KEY STAGES IN CREATING RUBRICS
18. Relationship between the intended Learning
Progression (LP) and rubrics
Direct use of the LP as standard reference
Use of a general strategy (i.e. modified Bloom’s
taxonomy or SOLO taxonomy)
Assign the target outcome as mapped onto the Cognitive Rigor
Matrix as the maximum level of the rubric
Use the SOLO taxonomy as scoring
Phenomenographic sorting
STRATEGIES FOR RUBRIC DEVELOPMENT
19.
20.
21.
22.
23.
24.
25. 1. Sort student responses into performance level piles
(excellent, good, …, and poor), or in terms of the
levels of understanding of the responses
2. Describe similarities within a pile and differences
between
These similarities and differences can inform the different
level of rubrics
3. Do sorting in pairs
To reconfirm matches & mismatches
Allow dialogue to maximize the effectiveness of rubric
development
PHENOMENOGRAPHIC SORTING
26. Explain what the test will emphasize
Emphasis and expectations will be delineated in the
assessment criteria in the rubrics
Inform students how the assessment will be scored
Explain what each of the assessment criteria defined in the
rubrics means
Explain how the results will be used
Explain the importance of the test scores in the student’s
learning progression
USING RUBRICS WITH STUDENTS
27. Make sure that the wording of the rubrics is
understandable to students (simplify wording for
lower grades).
Works best with holistic rubrics (or with a combined
version of analytic descriptions).
Provide rubrics to students in advance of the
administration of the assessment.
Alternatively, students can be provided with a
general rubric and a small number of papers (names
removed). Students can score the papers in small
groups; groups are required to come to consensus on
the grade to be assigned.
HOW TO USE RUBRICS WITH STUDENTS
28. Nitko, A. J., & Brookhart, S. (2007). Educational assessment of students. Upper
Saddle River, NJ: Pearson Education, Inc.
McMillan, J. H. (2007). Classroom assessment. Principles and practice for
effective standard-based instruction (4th ed.). Boston: Pearson - Allyn & Bacon.
Oregon Department of Education. (2014, June). Assessment guidance.
Popham, W. J. (2014). Criterion-referenced measurement: A half-century wasted?
Paper presented at the Annual Meeting of National Council on Measurement in
Education, Philadephia, PA.
Popham, W. J. (2014). Classroom assessment: What teachers needs to know . San
Francisco, CA: Pearson
Russell, M. K., & Airasian, P. W. (2012). Classroom assessment: Concepts and
applications. New York, NY: McGraw -Hill.
Stevens, D. & Levi, A. (2005). Introduction to rubrics. As assessment tool to save
grading time, convey effective feedback, and promote student learning . Sterling:
Stylus Publishing, LLC
Wihardini, D. (2010). Assessment development II. Unpublished manuscript.
Research and Development Department, Binus Business School, Jakarta,
Indonesia.
Wilson, M. (2005). Constructing measures: An item response modeling approach.
Mahwah, NJ: Lawrence Erlbaum Associates.
BIBLIOGRAPHY
29. Rubrics PPT by the Oregon Department of Education and Berkeley Evaluation and
Assessment Research Center is licensed under a CC BY 4.0.
Y ou a re free t o:
S hare — copy and redistribute the material in any medium or format
Adapt — remix, transform, and build upon the material
U nder t he fo llowing t erms:
Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes
were made. You may do so in any reasonable manner, but not in any way that suggests the licensor
endorses you or your use.
N onCommercial — You may not use the material for commercial purposes .
S hareAlike — If you remix, transform, or build upon the material, you must distribute your
contributions under the same license as the original.
Oregon Department of Education welcomes editing of these resources and would
greatly appreciate being able to learn from the changes made. To share an edited
version of this resource, please contact Cristen McLean, cristen.mclean@state.or.us.
CREATIVE COMMONS LICENSE
Editor's Notes
This chapter will involve the characteristics of good rubrics, and some general strategies that can be used to produce them.
Scoring rubrics involve
The specific expectations of an assessment task
A set of clear expectations or criteria, that allows both teachers and students to focus on what is valued in a subject, topic, or activity
The components of a rubric may include
Aspects of a task which may be taken into account when making a judgment about performance on that task
Different weightings for different criteria
Levels of attainment (which may include grade level descriptors)
We’ll be talking in the coming pages about two types of rubrics:
Descriptive or analytic
Holistic
As well as when and why we use particular types of rubrics
Rubrics have a number of advantages for both teachers and students.
For teachers
They provide the criteria against which an assessment should be evaluated
They assist in providing a student with detailed and timely feedback about performance
They encourage the teacher’s critical thinking
They facilitate communication with colleagues using the same rubric for scoring
An, they provide feedback that can be used to refine teaching and learning activities
For students
They clarify the teacher’s expectations around assessment
They provide information about the expected performance
And, they can help students to monitor, and to critique, their own work based on the rubric levels
Good rubrics have several characteristics. These include
They are well defined
They clearly define the expected student performance for each level
They use specific objective terms involving behavior, rather than general evaluative terms (which each user of the rubric may define differently)
They are context specific
They define teacher expectations for students for given performances in particular subject domains
This makes them useful for instruction as well as assessment purposes.
Additional components include that
They are finite
No need to make a new scoring category for each new type of response – and every response must be scorable (even those that are off task, or flippant!)
They are exhaustive
There should be a place for every type of response – even irrelevant or flippant types of responses (I think of this as “a place for everything and everything in its place”)
The number of levels is important; we want to differentiate between important level differences, but too many score levels can confuse both students and teachers
They are (at least partially) ordered
The order should represent the different levels of learning targets as represented by the LP
They should be related to the common core theme or strand, and coherent with the cognitive complexity of the standards.
The first of the two types of rubrics we’ll discuss is an analytic (or descriptive) type.
Analytic rubrics allow scoring of a task on several different aspects (or criteria) of the task (for example, grammar and creativity in ELA, or graphical skills and algorithmic correctness in math)
Components include the general task description, descriptions of the criteria, and specific performance level descriptions for each of the criteria.
Such rubrics provide separate judgments for each criteria
Good for diagnostic purposes
Provides more information from larger tasks
However, such rubrics can be time consuming to make, and to use for scoring purposes.
The second type of rubric is a holistic rubric
The holistic type provides a single scale in which all criteria for evaluation are considered together, providing an overall judgment of student work as a whole
Such a rubric is quicker to make and use
However, it does not provide diagnostic feedback for the student as to which components need improvement. It can also be tricky to use if a student is much better at one subskill than another
Rubrics are most often used on performance-based tasks, such as extended response items, projects, presentations, or portfolios. They are of course not needed for forced-choice items (like multiple choice or true-false), and are rarely needed for short-answer items.
There are several key steps to designing a rubric for a particular assessment task
First, we need to think about the content, the task, and the students:
What did we intend to teach?
What is the purpose of the assessment?
What we want from our students?
Having done this, we need to list the particular details of the learning outcomes and objectives that we would like to see in an ideal response to a completed task.
We then organize the results, grouping similar expectations together if we are designing an analytic rubric, and determining which expectations are easier and more difficult
Finally, we use a rubric design template to produce the final form of the rubric
We’ll now be discussing several strategies that can be used to design rubrics for a particular task.
The first of these includes using the description of the Learning Progression levels (either within or between grades) as general descriptors to design the specific levels of the rubric.
The second is the use of a general rubric strategy such as Bloom’s taxonomy or a modification of it, or the Structure of the Learning Outcome taxonomy (which we’ll show in the next slide), to design the specific levels of the rubric
The third is a technique called phenomenographic sorting, which we’ll explore in more detail in a few minutes. This technique basically involves sorting a set of student outcomes according to quality, and using the result to describe the specific rubric levels.
Cognitive rigor encompasses the complexity of content, the cognitive engagement
with that content, and the scope of the planned learning activities. The Hess
Cognitive Rigor Matrix (CRM) is a tool that has significant potential to enhance
instructional and assessment planning and practices at the classroom level
(Hess, Carlock, Jones, & Walkup, 2009). The CRM superimposes two different
cognitive complexity measures – Bloom’s Taxonomy and Webb’s Depth of
Knowledge – to produce a means of analyzing the emphasis placed on curricular
materials, instructional focus, and classroom assessment
The “before” example below shows a rubric that measures a lot of things that don’t have anything to do with whether a student understands the lunar phases: things like creativity, attractiveness, and whether the task is completed on time. If the purpose of the assignment is to assess whether students have really learned the lunar phases, the rubric should focus primarily on whether the content in the model is accurate and effective.
The “before” example below shows a rubric that measures a lot of things that don’t have anything to do with whether a student understands the lunar phases: things like creativity, attractiveness, and whether the task is completed on time. If the purpose of the assignment is to assess whether students have really learned the lunar phases, the rubric should focus primarily on whether the content in the model is accurate and effective.
In the revised version below, all of the criteria is focused on the quality and accuracy of the information in the model and measures the desired scientific thinking from building the model in the first place.
With equal point breakdown
WEIGH THE CRITERIA APPROPRIATELY
As designers of rubrics, we can signal to students that certain criteria matter more than others. Just because a rubric has four criteria, doesn’t mean that each needs to be worth 25 percent of the score. With the weight of each criterion adjusted, the rubric itself guides students to focus on what is most important.
In the example below—which is the same revised rubric from above, but where the teacher wanted to include some accountability for mechanics—all four criteria are weighted exactly the same. This means a student who demonstrates a perfect understanding of the science behind lunar phases, but who struggles with spelling and punctuation, could end up with a C on the project. That would not be a true reflection of mastery.
With unequal point breakdown
Using deficit language
Using can do language
Phenomenographic sorting
involves sorting student responses to a particular item into piles of increasingly good performance (performances within a pile should be similar in level of quality, and piles should be ordered according to quality).
Then, the differences between the piles should be described.
These can be used to develop descriptions of different levels of the rubric
It is often most effective to do the sorting in pairs, and discussing the similarities and differences with a partner
Rubrics can also be used as a tool with students. This has a number of advantages
They can be used to explain what the test will emphasize, and what the expectations will be
They can be used to explain how the assessment will be scored, and what the meanings of the criteria are
Finally, they can be used to explain how the results will be used
How to go about using rubrics with students:
First, make sure that the language of the rubric can be understood by students (you may have to simplify them for younger children).
Holistic rubrics, or a compilation of the criteria for analytic rubrics, are easier for students to follow. They should provide characteristics of a good response, as opposed to information about correct answers
It’s a good idea to provide the rubrics as handouts to students in advance of administering the assessment, as they can use them to focus their preparation. For take-home projects, rubrics can be included in the project packet.
An alternative use for rubrics is to allow students to participate in a scoring exercise. This works as follows
Choose a small number (say, 5-7) of student responses (ideally from another year or class). Remove all identifying information.
Have the students work in small groups to score the papers. Require the groups to come to consensus about the scores to be assigned and the reasons why. Allow about half an hour for discussion.
If you like, have a whole group discussion about the results.
This technique works best with students 10 and older. The idea is to help students understand what teachers are looking for in responses.
That’s the end of our chapter on rubric development; if you’d like more information, here are some references for you.