Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)


Published on

Presented at the No Educator Left Behind Conference 2009

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Checklists do not reflect developmental—indicates only presence or lack of a trait
  • This is the other extreme of too many task-specific rubrics. A project rubric should not be used to assess everything from a digital montage to a PowerPoint presentation on market economics. Yet, there are excellent resources available for you to adapt. Evaluate the resources available on the Web—don’t just use one because it is “free” and don’t think because it’s in a textbook that it is good! Find the middle ground—a template that you can adjust and tweak according to the specifications of a given task.
  • This includes educational jargon! Avoid sole adjective descriptors such as “inadequate” and avoid adjectives of “averageness”—below, above. The lowest score should describe what a novice, not “bad” performance looks like. Wordiness—often happens when groups devise—includes a little something for everyone
  • What’s important?
  • Not so much an issue of diction as describing the concrete behaviors and evidence of critical thinking Creativity= uses ideas from others (Developing), modifies ideas implemented by others (Basic), composition is self-generated (Proficient), composition is unique and imaginative(Advanced)--Myra
  • Actual traits that constitute good or poor persuasion, problem-solving. Be careful not to bury criteria—here is where some people find that their rubrics do not match their expectations—be sure that the descriptor is not a criterion and vice versa
  • 4 or 6 recommended Even recommended for delineating proficiency---Unless you want an equivocal position. Even number requires a decision between almost there and “barebones.” No implied levels.
  • You may also want students to self-assess and even use highlighters to document their claims.
  • In their hands at beginning. Use as revision tool. Give a quiz (Veronika!) Add self-assess column and defense piece or use as revision activity—highlight where it is
  • Have students list criteria for “What Counts.” Prompt them to think about any criteria they have missed and add them yourself. After class, combine criteria—create categories, making sure not to bury criteria that you want to emphasize.
  • Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB 2009)

    1. 1. 4th Annual Conference  No Educator Left Behind:  Equipping Adjunct Faculty with Knowledge and Skills   Rubric’s Cube: Complimenting, Critiquing, and Challenging Student Work
    2. 2. Objectives <ul><li>Gain an understanding of the role of grading rubrics in: </li></ul><ul><ul><ul><li>Clarifying teacher expectations </li></ul></ul></ul><ul><ul><ul><li>Evaluating student learning </li></ul></ul></ul><ul><ul><ul><li>Providing detailed feedback </li></ul></ul></ul><ul><ul><ul><li>Improving instruction </li></ul></ul></ul><ul><ul><ul><li>Performing outcome assessment </li></ul></ul></ul><ul><ul><li>Evaluate the strengths and weaknesses of grading rubrics as a student assessment tool </li></ul></ul><ul><ul><li>Integrate the principles of primary trait analysis (PTA) into designing a rubric for an assignment or performance skill </li></ul></ul><ul><ul><li>Take the first step toward using a grading rubric for an assignment, project, or exam </li></ul></ul>
    3. 3. Considerations For Effective Rubric Design <ul><li>does its job (well written) </li></ul><ul><li>uses precise criteria and descriptors (well chosen) </li></ul><ul><li>is student-friendly (distinguish between clear and ambiguous grading criteria) </li></ul>
    4. 4. Why do we Grade? <ul><li>Grading </li></ul><ul><ul><li>The process used by faculty to assess student learning via assignments and exams, including: </li></ul></ul><ul><ul><ul><li>Relating test items or assignments to learning objectives </li></ul></ul></ul><ul><ul><ul><li>Establishing criteria/standards </li></ul></ul></ul><ul><li>Helping students acquire needed knowledge/skills </li></ul><ul><ul><ul><li>Facilitating student motivation </li></ul></ul></ul><ul><ul><ul><li>Giving feedback about performance </li></ul></ul></ul><ul><ul><ul><li>Communicating about what has been learned </li></ul></ul></ul><ul><ul><ul><li>Using results to influence teaching and curriculum </li></ul></ul></ul><ul><ul><li>( BE Walvoord, VJ Anderson, 1998. Effective grading: A tool for learning and assessment . Jossey-Bass: San Francisco) </li></ul></ul>
    5. 5. Grading Challenge # 214: Student’s Perceptions
    6. 6. Myths About Grading <ul><li>Grading: </li></ul><ul><ul><li>Is a necessary EVIL </li></ul></ul><ul><ul><li>Must be objective </li></ul></ul><ul><ul><li>Should be easy </li></ul></ul><ul><ul><li>Should not take or detract from teaching </li></ul></ul><ul><ul><li>Uses knowledge of experts to evaluate novices </li></ul></ul><ul><ul><li>Is assessment only, doesn’t impact learning </li></ul></ul><ul><ul><li>With clear expectations, means you are “spoon-feeding” </li></ul></ul><ul><ul><li>Cannot be used in program assessment </li></ul></ul>
    7. 7. Expert Input <ul><li>Experts agree: </li></ul><ul><ul><li>Rubrics are hard to design. </li></ul></ul><ul><ul><li>Rubrics are time-consuming to design. </li></ul></ul><ul><ul><li>“ A rubric is only as useful as it is good. Using a bad rubric is a waste of time…” </li></ul></ul><ul><ul><li>-- Michael Simkins in “Designing Great Rubrics” </li></ul></ul><ul><ul><li>Experts disagree: </li></ul></ul><ul><ul><li>how to design a “good” rubric </li></ul></ul><ul><ul><li>Bottom line: Is it working for you and for your students? </li></ul></ul>
    8. 8. Principles of Effective Grading <ul><li>Appreciate the complexity of grading </li></ul><ul><li>Substitute judgment for objectivity </li></ul><ul><li>Distribute time effectively </li></ul><ul><li>Be open to change </li></ul><ul><li>Listen & observe </li></ul><ul><li>Communicate & collaborate with students </li></ul><ul><li>Integrate grading with other key processes </li></ul><ul><li>Seize the teachable moment (feedback) </li></ul><ul><li>Make student learning the primary goal </li></ul><ul><li>Be a facilitator FIRST, gatekeeper LAST </li></ul><ul><li>Encourage learning-centered motivation </li></ul><ul><li>Emphasize student involvement </li></ul><ul><ul><li>(Walvoord & Anderson, 1998) </li></ul></ul>
    9. 9. Student Assessment Methods <ul><li>Multiple choice examination </li></ul><ul><li>Short answer questions on tests </li></ul><ul><li>Essay questions </li></ul><ul><li>Annotated bibliographies </li></ul><ul><li>Literature reviews </li></ul><ul><li>Case Studies </li></ul><ul><li>Oral Examinations </li></ul><ul><li>Practical/Performance Examinations </li></ul><ul><li>Clinical Assessments </li></ul><ul><li>Journals </li></ul><ul><li>Portfolios </li></ul><ul><li>Lab Conclusions </li></ul>(LC Jacobs, CI Chase, 1992. Developing and using tests effectively . Jossey-Bass: San Francisco) Student-Constructed Responses
    10. 10. Methods of Grading Student-Constructed Responses ( Blackinton, 2008) -Takes time to construct, may need to modify after 1 st run Explicit expectations, better feedback, greater inter-rater reliability, links to performance Grading Rubric : criterion referenced, describe performance expectations & weighting -Usually lacks descriptions -Lists + traits or behaviors, no negative Assignment directions match checklist, not difficult to prepare Checklist : list of criteria to include (introduction, research question…) -Potential for bias -Less opportunity for learning, vague Little work up front, recognizes faculty as ‘expert’, flexible Norm Referenced : categorize work into A, B, C, D Disadvantage Advantage Method
    11. 11. What is a Grading Rubric? <ul><li>Method of “articulating expectations for an assignment by listing the criteria, or what counts, & describing levels of quality from excellent to poor” 1 </li></ul><ul><li>Type of assessment that specifies gradations of quality from excellent to poor 2 </li></ul><ul><li>A criterion-referenced method of grading using highly specific grading criteria that are linked to objectives </li></ul>1 HG Andrade, Y Du (2005). Students perspectives on rubric-referenced assessment. Practical Assessment, Research & Evaluation , Vol 10 (3). 2 HG Andrade (2005). Teaching with rubrics: The good, the bad, and the ugly. CollegeTeaching , 53 (1)
    12. 12. What is a Grading Rubric? Gradation: excellent-poor Categories important to the teacher/class Weighted Points Dimensions also called criteria ( Blackinton, 2008) 0 pts 1 pts 2 pts Dimension 4 1 pt 2 pts 3 pts Dimension 3 1 pt 2.5 pts 4 pts Dimension 2 1 pt 2 pts 3 pts: describe Dimension 1 Performance Level 1 Performance Level 2 Performance Level 3
    13. 13. Sample Rubric Format ( Blackinton, 2008) 0 pts 1 pts 2 pts Level 4 1 pt 2 pts 3 pts Level 3 1 pt 2.5 pts 4 pts Level 2 1 pt 2 pts 3 pts: describe Level 1 Criteria 3 Criteria 2 Criteria 1
    14. 14. How do Students Perceive Rubrics? <ul><li>Andrad & Yu, 2005 </li></ul><ul><ul><li>Investigated how students use grading rubrics </li></ul></ul><ul><ul><ul><li>Focus group / qualitative design </li></ul></ul></ul><ul><ul><li>Students reported that they used rubrics to </li></ul></ul><ul><ul><ul><li>Help them determine faculty expectations </li></ul></ul></ul><ul><ul><ul><li>To plan an approach to assignment </li></ul></ul></ul><ul><ul><ul><li>Check/revise work before handing in </li></ul></ul></ul><ul><ul><ul><li>Help reflect on their learning-see strengths/weaknesses clearly </li></ul></ul></ul><ul><ul><li>Perceived results of rubric use </li></ul></ul><ul><ul><ul><li>Better, fairer grades </li></ul></ul></ul><ul><ul><ul><li>Improvements in quality across classes </li></ul></ul></ul><ul><ul><ul><li>Less anxiety </li></ul></ul></ul>
    15. 15. Using Primary Trait Analysis To Develop a Grading Rubric <ul><li>Rationale </li></ul><ul><ul><li>Analyze traits / characteristics of student learning and then clearly articulate them, leading to: </li></ul></ul><ul><ul><ul><li>Assignment specific </li></ul></ul></ul><ul><ul><ul><li>Explicit criteria </li></ul></ul></ul><ul><ul><ul><li>Adds objectivity & consistency to holistic scoring </li></ul></ul></ul><ul><ul><ul><li>Lets students know in ADVANCE how will be graded </li></ul></ul></ul>( Walvoord &Anderson, 1998) ( R Lloyd-Jones, C. Cooper & L. Odell (Eds), 1977. Primary Trait Scoring in Evaluating Writing: Describing, measuring, judging . Urbana IL: National Council of Teachers
    16. 16. Constructing a Primary Trait Analysis Scale 1 <ul><ul><li>Choose assignment/test that matches course objective </li></ul></ul><ul><ul><li>Identify all traits that will count for scoring in assessment </li></ul></ul><ul><ul><ul><li>Body Language vs Thoroughness vs Accuracy </li></ul></ul></ul><ul><ul><li>Build scale for scoring performance, gradations of skill </li></ul></ul><ul><ul><ul><li>Scale usually ranges from 2-5 points </li></ul></ul></ul><ul><ul><ul><li>Include what should be demonstrated and what should be avoided </li></ul></ul></ul><ul><ul><ul><li>Build a range that discriminates A from A- from B+ </li></ul></ul></ul><ul><ul><ul><li>Run scale by colleague, graduate, teaching assistant </li></ul></ul></ul><ul><ul><ul><li>Weight items: content > spelling; accuracy > efficiency, etc </li></ul></ul></ul><ul><ul><li>Evaluate performance against criteria </li></ul></ul><ul><ul><ul><li>Try scale with sample & revise as needed </li></ul></ul></ul><ul><ul><ul><li>( BE Walvoord, VJ Anderson, 1998. Effective grading: A tool for learning and assessment . Jossey-Bass: San Francisco) </li></ul></ul></ul>
    17. 17. Rubric Tip #1 <ul><li>Don’t use generic or “canned” rubrics without careful consideration of their quality and appropriateness for your project. </li></ul><ul><ul><ul><li>These are your students, not someone else’s. </li></ul></ul></ul><ul><ul><ul><li>Your students have received your instruction. </li></ul></ul></ul>
    18. 18. Rubric Tip #2 <ul><li>Avoid dysfunctional detail. </li></ul><ul><ul><li>“…in most instances, lengthy rubrics probably can be reduced to succinct…more useful versions for classroom instruction. Such abbreviated rubrics can still capture the key evaluative criteria needed to judge students’ responses. Lengthy rubrics, in contrast, will gather dust” (Benjamin 23). </li></ul></ul><ul><li>--Includes wordiness, jargon, negativity </li></ul>
    19. 19. Rubric Tip #3 <ul><li>Limit the number of criteria </li></ul><ul><ul><li>Well… </li></ul></ul><ul><ul><li>Don’t combine independent criteria. </li></ul></ul><ul><ul><ul><li>“very clear” and “very organized” (may be clear but not organized or vice versa). </li></ul></ul></ul>
    20. 20. Rubric Tip #4 <ul><li>Use key, teachable “criteria” (What counts) </li></ul><ul><ul><li>Don’t vaguely define levels of quality. </li></ul></ul><ul><ul><li>Concrete versus abstract </li></ul></ul><ul><ul><ul><li>“ poorly organized” (Organization: sharply focused thesis, topic sentences clearly connected to thesis, logical ordering of paragraphs, conclusion ends with clincher) </li></ul></ul></ul><ul><ul><ul><li>“ inventive” “creative” “imaginative” UNLESS… </li></ul></ul></ul><ul><ul><ul><li>Key Question to ask yourself: What does it look like? </li></ul></ul></ul>
    21. 21. Rubric Tip #5 <ul><li>Use measurable criteria. </li></ul><ul><ul><ul><li>--Specify what quality or absence looks like </li></ul></ul></ul><ul><ul><ul><li>vs. comparatives (“not as thorough as”) </li></ul></ul></ul><ul><ul><ul><li> or value language (“excellent content”) </li></ul></ul></ul>
    22. 22. Rubric Tip #6 <ul><li>Aim for an even number of levels </li></ul><ul><ul><li>Create continuum between least and most </li></ul></ul><ul><ul><li>Define poles and work inward </li></ul></ul><ul><ul><li>List skills and traits consistently across levels </li></ul></ul>
    23. 23. Rubric Tip #7 <ul><li>Include students in creating or adapting rubrics </li></ul><ul><li>Consider using “I” in the descriptors </li></ul><ul><ul><ul><li>I followed precisely—consistently—inconsistently—MLA documentation format. </li></ul></ul></ul><ul><ul><ul><li>I did not follow MLA documentation format. </li></ul></ul></ul><ul><ul><ul><li>---Highlight the impact of the performance </li></ul></ul></ul><ul><ul><ul><li>--Was the paper persuasive or problem solved? (Note importance of PURPOSE) </li></ul></ul></ul><ul><ul><ul><li>--What are the traits of effective persuasion? </li></ul></ul></ul><ul><ul><ul><li>--Be sure that the descriptor is not the criterion and vice versa </li></ul></ul></ul>
    24. 24. Rubric Tip #8 <ul><li>Motivate students to use rubric. </li></ul><ul><li>Instructional rubric (“Buy one, get one…”) </li></ul><ul><li>“ At their very best, rubrics are also teaching tools that support student learning…” (Andrade 13). </li></ul><ul><li>Do they understand the criteria and descriptors? How do you know? </li></ul><ul><li>When do you give the rubric to your students? </li></ul>
    25. 25. Rubric Tip #9 <ul><li>Provide models of the different performance levels. </li></ul>
    26. 26. The Assignment Sheet <ul><li>Don’t forget the importance of the assignment sheet </li></ul><ul><li>Connection to rubric (Use same language!) </li></ul><ul><ul><li>The lawyers in your class </li></ul></ul><ul><li>“ But the rubric doesn’t say that…” </li></ul><ul><li>Project/paper/presentation must meet all requirements of assignment </li></ul><ul><ul><li>Due date and late penalty </li></ul></ul><ul><ul><li>Format requirements </li></ul></ul><ul><ul><li>Non-negotiables </li></ul></ul><ul><ul><ul><li>Skills and reasonable expectations </li></ul></ul></ul>
    27. 27. Don’t Forget the Check-in Stage <ul><li>Use your rubric as a formative assessment to give students feedback about how they are doing. </li></ul><ul><ul><li>Isolate a particularly challenging aspect </li></ul></ul><ul><ul><li>Have student isolate an area of difficulty </li></ul></ul><ul><ul><li>Center revision instruction around rubric </li></ul></ul>
    28. 28. Applying the Rubric <ul><li>Distribute ( or post) rubric to students in advance </li></ul><ul><ul><li>Have conversations about expectations </li></ul></ul><ul><li>Ask students to attach rubric to assignment </li></ul><ul><ul><li>Helps students pay attention! </li></ul></ul><ul><li>Use rubric to grade </li></ul><ul><ul><li>Match written comments to phrases in rubric </li></ul></ul><ul><li>Revise after use </li></ul><ul><ul><li>Make changes soon after grading for next time </li></ul></ul><ul><li>Answer Curriculum Questions </li></ul><ul><ul><li>Did the students learn? To what degree were objectives met? </li></ul></ul><ul><ul><li>If not, was it the teaching? The assignment? Background skills? </li></ul></ul><ul><ul><li>What, if any, changes should be made in the class or curriculum? Rubric? </li></ul></ul>
    29. 29. Caution <ul><ul><li>Don’t let the rubric stand alone: </li></ul></ul><ul><ul><li>ALWAYS, ALWAYS provide specific </li></ul></ul><ul><ul><li>“ Comments” on your rubric and/or on the student product itself. </li></ul></ul>
    30. 30. Advantages of Grading Rubrics <ul><li>Save time in grading process </li></ul><ul><li>Makes process of grading reliable/fair </li></ul><ul><li>Clarifies expectations for students </li></ul><ul><li>Reinforces key concepts - help faculty relate to objectives </li></ul><ul><li>Students are participants as expectations are known </li></ul><ul><li>Student peer review </li></ul><ul><li>Works well if team-teaching </li></ul><ul><li>Share across courses or over curriculum </li></ul><ul><li>Basis for departmental/ program assessment </li></ul><ul><li>Which of these advantages pertain to your teaching in the health professions? </li></ul><ul><ul><li>(Walvoord & Anderson, 1998) </li></ul></ul>
    31. 31. CAUTION: Common Rubric Pitfalls <ul><li>Rubric does not correspond with class or program outcomes </li></ul><ul><ul><li>Example: Entire rubric focused on writing quality not content </li></ul></ul><ul><li>Scale does not have enough gradations or levels </li></ul><ul><ul><li>Not distinguishing the A’s from the B+’s </li></ul></ul><ul><li>All traits are given equal weight regardless of complexity </li></ul><ul><ul><li>Grammar = Content </li></ul></ul><ul><li>Too broad, not enough content described </li></ul><ul><ul><li>Words like ‘breadth’ and ‘depth’ used in lieu of specifics </li></ul></ul><ul><ul><li>Students still not sure, other grading faculty still not clear </li></ul></ul><ul><li>Too long/too complicated </li></ul><ul><ul><li>Faculty + students get lost in the rubric </li></ul></ul>
    32. 32. A Rubric … … is a guide for the evaluation of student work that defines a facilitator’s expectations and identifies grading criteria point by point. … provides a clear set of criteria for judging students’ work by specifying factors on which the facilitator will grade the student thereby helping the facilitator define expectations ad prompting the student to focus on specific points. … takes extensive thought and planning to be effective and that facilitators need to be lucid in their explanations of grading standards and be sure of the assignment’s objective to create rubrics that are pedagogically sound.
    33. 33. Classroom Assessment Technique Application <ul><li>Write down ONE thing you will commit to incorporate regarding the use of grading rubrics </li></ul><ul><li>Write down ONE unanswered question </li></ul>