Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

4th Annual Conference No Educator Left Behind: Equipping Adjunct Faculty with Knowledge and Skills Rubric’s Cube: Complimenting, Critiquing, and Challenging Student Work

Objectives
Gain an understanding of the role of grading rubrics in:
Clarifying teacher expectations
Evaluating student learning
Providing detailed feedback
Improving instruction
Performing outcome assessment
Evaluate the strengths and weaknesses of grading rubrics as a student assessment tool
Integrate the principles of primary trait analysis (PTA) into designing a rubric for an assignment or performance skill
Take the first step toward using a grading rubric for an assignment, project, or exam

Considerations For Effective Rubric Design
does its job (well written)
uses precise criteria and descriptors (well chosen)
is student-friendly (distinguish between clear and ambiguous grading criteria)

Why do we Grade?
Grading
The process used by faculty to assess student learning via assignments and exams, including:
Relating test items or assignments to learning objectives
Establishing criteria/standards
Helping students acquire needed knowledge/skills
Facilitating student motivation
Giving feedback about performance
Communicating about what has been learned
Using results to influence teaching and curriculum
( BE Walvoord, VJ Anderson, 1998. Effective grading: A tool for learning and assessment . Jossey-Bass: San Francisco)

Grading Challenge # 214: Student’s Perceptions

Myths About Grading
Grading:
Is a necessary EVIL
Must be objective
Should be easy
Should not take or detract from teaching
Uses knowledge of experts to evaluate novices
Is assessment only, doesn’t impact learning
With clear expectations, means you are “spoon-feeding”
Cannot be used in program assessment

Expert Input
Experts agree:
Rubrics are hard to design.
Rubrics are time-consuming to design.
“ A rubric is only as useful as it is good. Using a bad rubric is a waste of time…”
-- Michael Simkins in “Designing Great Rubrics”
Experts disagree:
how to design a “good” rubric
Bottom line: Is it working for you and for your students?

Principles of Effective Grading
Appreciate the complexity of grading
Substitute judgment for objectivity
Distribute time effectively
Be open to change
Listen & observe
Communicate & collaborate with students
Integrate grading with other key processes
Seize the teachable moment (feedback)
Make student learning the primary goal
Be a facilitator FIRST, gatekeeper LAST
Encourage learning-centered motivation
Emphasize student involvement
(Walvoord & Anderson, 1998)

Student Assessment Methods
Multiple choice examination
Short answer questions on tests
Essay questions
Annotated bibliographies
Literature reviews
Case Studies
Oral Examinations
Practical/Performance Examinations
Clinical Assessments
Journals
Portfolios
Lab Conclusions
(LC Jacobs, CI Chase, 1992. Developing and using tests effectively . Jossey-Bass: San Francisco) Student-Constructed Responses

Methods of Grading Student-Constructed Responses ( Blackinton, 2008) -Takes time to construct, may need to modify after 1 st run Explicit expectations, better feedback, greater inter-rater reliability, links to performance Grading Rubric : criterion referenced, describe performance expectations & weighting -Usually lacks descriptions -Lists + traits or behaviors, no negative Assignment directions match checklist, not difficult to prepare Checklist : list of criteria to include (introduction, research question…) -Potential for bias -Less opportunity for learning, vague Little work up front, recognizes faculty as ‘expert’, flexible Norm Referenced : categorize work into A, B, C, D Disadvantage Advantage Method

What is a Grading Rubric?
Method of “articulating expectations for an assignment by listing the criteria, or what counts, & describing levels of quality from excellent to poor” 1
Type of assessment that specifies gradations of quality from excellent to poor 2
A criterion-referenced method of grading using highly specific grading criteria that are linked to objectives
1 HG Andrade, Y Du (2005). Students perspectives on rubric-referenced assessment. Practical Assessment, Research & Evaluation , Vol 10 (3). 2 HG Andrade (2005). Teaching with rubrics: The good, the bad, and the ugly. CollegeTeaching , 53 (1)

What is a Grading Rubric? Gradation: excellent-poor Categories important to the teacher/class Weighted Points Dimensions also called criteria ( Blackinton, 2008) 0 pts 1 pts 2 pts Dimension 4 1 pt 2 pts 3 pts Dimension 3 1 pt 2.5 pts 4 pts Dimension 2 1 pt 2 pts 3 pts: describe Dimension 1 Performance Level 1 Performance Level 2 Performance Level 3

Sample Rubric Format ( Blackinton, 2008) 0 pts 1 pts 2 pts Level 4 1 pt 2 pts 3 pts Level 3 1 pt 2.5 pts 4 pts Level 2 1 pt 2 pts 3 pts: describe Level 1 Criteria 3 Criteria 2 Criteria 1

How do Students Perceive Rubrics?
Andrad & Yu, 2005
Investigated how students use grading rubrics
Focus group / qualitative design
Students reported that they used rubrics to
Help them determine faculty expectations
To plan an approach to assignment
Check/revise work before handing in
Help reflect on their learning-see strengths/weaknesses clearly
Perceived results of rubric use
Better, fairer grades
Improvements in quality across classes
Less anxiety

Using Primary Trait Analysis To Develop a Grading Rubric
Rationale
Analyze traits / characteristics of student learning and then clearly articulate them, leading to:
Assignment specific
Explicit criteria
Adds objectivity & consistency to holistic scoring
Lets students know in ADVANCE how will be graded
( Walvoord &Anderson, 1998) ( R Lloyd-Jones, C. Cooper & L. Odell (Eds), 1977. Primary Trait Scoring in Evaluating Writing: Describing, measuring, judging . Urbana IL: National Council of Teachers

Constructing a Primary Trait Analysis Scale 1
Choose assignment/test that matches course objective
Identify all traits that will count for scoring in assessment
Body Language vs Thoroughness vs Accuracy
Build scale for scoring performance, gradations of skill
Scale usually ranges from 2-5 points
Include what should be demonstrated and what should be avoided
Build a range that discriminates A from A- from B+
Run scale by colleague, graduate, teaching assistant
Weight items: content > spelling; accuracy > efficiency, etc
Evaluate performance against criteria
Try scale with sample & revise as needed
( BE Walvoord, VJ Anderson, 1998. Effective grading: A tool for learning and assessment . Jossey-Bass: San Francisco)

Rubric Tip #1
Don’t use generic or “canned” rubrics without careful consideration of their quality and appropriateness for your project.
These are your students, not someone else’s.
Your students have received your instruction.

Rubric Tip #2
Avoid dysfunctional detail.
“…in most instances, lengthy rubrics probably can be reduced to succinct…more useful versions for classroom instruction. Such abbreviated rubrics can still capture the key evaluative criteria needed to judge students’ responses. Lengthy rubrics, in contrast, will gather dust” (Benjamin 23).
--Includes wordiness, jargon, negativity

Rubric Tip #3
Limit the number of criteria
Well…
Don’t combine independent criteria.
“very clear” and “very organized” (may be clear but not organized or vice versa).

Rubric Tip #4
Use key, teachable “criteria” (What counts)
Don’t vaguely define levels of quality.
Concrete versus abstract
“ poorly organized” (Organization: sharply focused thesis, topic sentences clearly connected to thesis, logical ordering of paragraphs, conclusion ends with clincher)
“ inventive” “creative” “imaginative” UNLESS…
Key Question to ask yourself: What does it look like?

Rubric Tip #5
Use measurable criteria.
--Specify what quality or absence looks like
vs. comparatives (“not as thorough as”)
or value language (“excellent content”)

Rubric Tip #6
Aim for an even number of levels
Create continuum between least and most
Define poles and work inward
List skills and traits consistently across levels

Rubric Tip #7
Include students in creating or adapting rubrics
Consider using “I” in the descriptors
I followed precisely—consistently—inconsistently—MLA documentation format.
I did not follow MLA documentation format.
---Highlight the impact of the performance
--Was the paper persuasive or problem solved? (Note importance of PURPOSE)
--What are the traits of effective persuasion?
--Be sure that the descriptor is not the criterion and vice versa

Rubric Tip #8
Motivate students to use rubric.
Instructional rubric (“Buy one, get one…”)
“ At their very best, rubrics are also teaching tools that support student learning…” (Andrade 13).
Do they understand the criteria and descriptors? How do you know?
When do you give the rubric to your students?

Rubric Tip #9
Provide models of the different performance levels.

The Assignment Sheet
Don’t forget the importance of the assignment sheet
Connection to rubric (Use same language!)
The lawyers in your class
“ But the rubric doesn’t say that…”
Project/paper/presentation must meet all requirements of assignment
Due date and late penalty
Format requirements
Non-negotiables
Skills and reasonable expectations

Don’t Forget the Check-in Stage
Use your rubric as a formative assessment to give students feedback about how they are doing.
Isolate a particularly challenging aspect
Have student isolate an area of difficulty
Center revision instruction around rubric

Applying the Rubric
Distribute ( or post) rubric to students in advance
Have conversations about expectations
Ask students to attach rubric to assignment
Helps students pay attention!
Use rubric to grade
Match written comments to phrases in rubric
Revise after use
Make changes soon after grading for next time
Answer Curriculum Questions
Did the students learn? To what degree were objectives met?
If not, was it the teaching? The assignment? Background skills?
What, if any, changes should be made in the class or curriculum? Rubric?

Caution
Don’t let the rubric stand alone:
ALWAYS, ALWAYS provide specific
“ Comments” on your rubric and/or on the student product itself.

Advantages of Grading Rubrics
Save time in grading process
Makes process of grading reliable/fair
Clarifies expectations for students
Reinforces key concepts - help faculty relate to objectives
Students are participants as expectations are known
Student peer review
Works well if team-teaching
Share across courses or over curriculum
Basis for departmental/ program assessment
Which of these advantages pertain to your teaching in the health professions?
(Walvoord & Anderson, 1998)

CAUTION: Common Rubric Pitfalls
Rubric does not correspond with class or program outcomes
Example: Entire rubric focused on writing quality not content
Scale does not have enough gradations or levels
Not distinguishing the A’s from the B+’s
All traits are given equal weight regardless of complexity
Grammar = Content
Too broad, not enough content described
Words like ‘breadth’ and ‘depth’ used in lieu of specifics
Students still not sure, other grading faculty still not clear
Too long/too complicated
Faculty + students get lost in the rubric

A Rubric … … is a guide for the evaluation of student work that defines a facilitator’s expectations and identifies grading criteria point by point. … provides a clear set of criteria for judging students’ work by specifying factors on which the facilitator will grade the student thereby helping the facilitator define expectations ad prompting the student to focus on specific points. … takes extensive thought and planning to be effective and that facilitators need to be lucid in their explanations of grading standards and be sure of the assignment’s objective to create rubrics that are pedagogically sound.

Classroom Assessment Technique Application
Write down ONE thing you will commit to incorporate regarding the use of grading rubrics
Write down ONE unanswered question

https://blackboard.uc.edu	14
http://hbattung.blogspot.com	3
http://www.slideshare.net	1

Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

by eutsler on Dec 04, 2009

Accessibility

Upload Details

Usage Rights

3 Embeds 18

Statistics

$Rubric\’s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)$ Rubric\’s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009) Presentation Transcript