Creating RubricsA Guide to Development and            Use
Credits• Materials in this handout were compiled from the following web  sites.    –   http://www.carla.umn.edu/assessment...
Introduction• In this workshop we:    – Explore how to create tools for grading      complex assignments.    – Explore the...
Workshop Objectives• By the end of the workshop you should  be able to:    – Describe a rubric (what is it?)    – Describe...
What’s a rubric?• Analytic rubrics have    – Levels of performance    – Multiple criteria09/17/12                      5
The TLT defines rubrics as:• "Rubrics" explicitly state criteria for  assignments.• May lead to a grade or be part of  the...
Another definition                 (Jon Mueller. Professor of           Psychology, North Central College)• Assess student...
Example: Research Project           (Jon Mueller. Professor of Psychology, North Central College)  Criteria               ...
One-Sentence SummaryOn a 3x5 note card:Write a one-sentence summary that:   1. Defines a rubric.   2. Differentiates betwe...
Characteristics of Rubrics• Increase an assessments construct and  content validity• Increase an assessments reliability  ...
Characteristics Continued• Help learners develop their ability to  judge quality in their own and  others work• Provides s...
Characteristics Continued• Learners and teachers monitor  progress over a period of instruction• Reduces time spent gradin...
Which of these reasons are        important to you?• Importance of Reliability• Validity of the assessment• Reduction of b...
Types of Rubrics• Ask yourself:    – For a particular task, do you want to      be able to assess how well the students   ...
Holistic• a holistic rubric does not list  separate levels of performance for  each criterion• a holistic rubric assigns a...
Holistic Example  3 - Excellent Researcher  • included 10-12 sources  • no apparent historical inaccuracies  • can easily ...
Analytic Rubric  Criteria              3                 2                  1  Number of      X1     10-12             5-9...
When to choose an analytic            rubric• Want to assess each criterion  separately• Involve large number of criteria•...
Types of Rubrics Activity• In the Lesson Builder materials, page  6, you will find a sorting activity.09/17/12            ...
Designing an Analytic Rubric• Step 1. Re-examine learning objective to be  addressed by the task.• Step 2. Identify observ...
Design Analytic Continued• Step 4b. Write thorough narrative  description for excellent and poor work for  each individual...
Hint: Use Even Number of                Levels• Use an even number (4 or 6) of levels  of performance on the scale.• When ...
Hint: Arrange Levels High to               Low• High to low scale.• Students read first the description of  an exemplary p...
Sample High to Lows       4            3             2              1 Exemplary      Excellent     Acceptable    Unaccepta...
More Hints• Limited number of dimensions or criteria.    – The criteria are those components that are      most important ...
4                  3                     2                    1 Task              All               Most                  ...
Chocolate Chip Cookies• Break into pairs.• Compete the worksheet Creating a Rubric.• Create a rubric for judging the quali...
Using RubiStar• These are some simple steps for  getting started with Rubistar.• Access the site  http://rubistar.4teacher...
Rubistar Tips• If you are using Internet Explorer, you can  copy the rubric and paste it into a Word  document where you c...
Assignment• Create a rubric for an  objective/competency you have for a  class you teach.• Use the worksheet on creating a...
Review1. Describe a rubric (what is it).2. Describe the purpose of rubrics.3. Describe the difference between holistic    ...
Upcoming SlideShare
Loading in...5
×

Rubric pp

1,631

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,631
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
45
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Introduction and welcome Sign-in if appropriate Share something cool about yourself. Share expectations Look at my feet and guess example
  • Give credits.
  • Discuss points above…Page 3 LB, page 3 handout
  • Review objectives and relate to their expectations. Page 2 LB, page 2 handout
  • Page 4 LB, page 5 handout Explain how the printed handout works, where to find the materials in Blackboard, and slideshare.net/chedisky/slideshows Common rubrics are designed in a matrix. Many rubrics are designed as a matrix with Levels of performance across the columns (top) Criterion along the rows on the left.
  • Page 4 LB, page 5 handout TLT--Teaching, Learning, Technology Group which manages the Flashlight Projects developed this http://www.tltgroup.org/resources/Flashlight/Rubrics.htm
  • LB page 4, handout page 5 Jon Mueller http://jonathan.mueller.faculty.noctrl.edu/toolbox/rubrics.htm That is, a student's aptitude on a task is determined by matching the student's performance against a set of criteria to determine the degree to which the student's performance meets the criteria for the task. To measure student performance against a pre-determined set of criteria, a rubric, or scoring scale, is typically created which contains the essential criteria for the task and appropriate levels of performance for each criterion.
  • Page 4 LB, page 6 handout Modified from Jon Mueller--changed the order of the performance levels so goes from good to poor. Eliminated extra words. Discussion points: 1) performance levels across the top, 2) critical criteria in the 1st column, 3) weighting in column two, 4) goes from excellent (3) to poor (1). Forces the student to read the good before seeing the poor level.
  • PAGE 4 LB
  • Page 5 in LB, page 8 handout Well-designed rubrics increase an assessment's construct and content validity by aligning evaluation criteria to standards, curriculum, instruction, and assessment tasks. Well-designed rubrics increase an assessment's reliability by setting criteria that raters can apply consistently and objectively. Evaluating student work by established criteria reduces bias. Identifying the most salient criteria for evaluating a performance and writing descriptions of excellent performance can help teachers clarify goals and improve their teaching. Rubrics help learners set goals and assume responsibility for their learning—they know what comprises an optimal performance and can strive to achieve it. From: University of Minnesota’s Virtual Assessment Center http://www.carla.umn.edu/assessment/VAC/Evaluation/p_5.html
  • Page 5 in handout continued Rubrics used for self- and peer-assessment help learners develop their ability to judge quality in their own and others' work. Learners receive specific feedback about their areas of strength and weakness and about how to improve their performance. Learners can use rubrics to assess their own effort and performance, and make adjustments to work before submitting it for a grade.
  • Page 5 Lesson Builder Rubrics allow learners, teachers, and other stakeholders to monitor progress over a period of instruction. Time spent evaluating performance and providing feedback can be reduced. When students participate in designing rubrics, they are empowered to become self-directed learners. Rubrics help teachers move away from subjective grading by allowing them and others, including students themselves, to assess work based on consistent, often agreed upon, and objective criteria.
  • Page 5 in LB, page 10 in handout Can access in PPT in slideshare (slideshare.net/chedisky/slideshows) or in LB in Blackboard. Each link goes to a poll at PollDaddy. Students can link to this from the LB on page 5
  • Page 6 in LB, page 11 handout
  • Page 6 LB, handout page 11
  • Page 6 LB, page 12 handout
  • Page 6 LB
  • Page 6 LB, page 12 handout
  • Do activity in LB…sorting of holistic vs analytic characteristics.
  • Page 15 handout, Briefly cover the steps. (do not refer to LB at this point)
  • Page 15 handout Continue quick review of steps without reference to handout.
  • Page 8 LB, page 16 handout
  • Page 8 LB, page 17 handout
  • Page 8 LB, page 17 handout
  • Page 8 LB, page 17 handout
  • Page 8 LB, page 18 handout
  • Page 9 LB. Page 19 handout The handout includes the examples and form from the LB lesson. Students can open the pdf forms from LB to do this assignment. They can reference the handout if they want a hard copy. Be sure that they see that the form includes options for either holistic or analytic.
  • Page 10 LB, page 30 handout
  • Page 10 LB, page 30 handout
  • Page 10 LB. Can access pdf from by clicking on Creating a Rubric
  • Rewiew from page 12, page 33 handout (page 11 is resources)
  • Rubric pp

    1. 1. Creating RubricsA Guide to Development and Use
    2. 2. Credits• Materials in this handout were compiled from the following web sites. – http://www.carla.umn.edu/assessment/VAC/Evaluation/p_4.html – http://jonathan.mueller.faculty.noctrl.edu/toolbox/rubrics.htm – http://rubistar.4teachers.org – http://valenciacc.edu/learningevidence/rubrics.asp – http://faculty.sacredheart.edu/stiltnerb/students/rubrics.htm09/17/12 2
    3. 3. Introduction• In this workshop we: – Explore how to create tools for grading complex assignments. – Explore the purpose and characteristics of rubrics. – Use RubiStar as a tool for creating a rubric.• At the end of the session, – You will have drafted a rubric for an assignment. – Given and received feedback from a colleague on your rubric.09/17/12 3
    4. 4. Workshop Objectives• By the end of the workshop you should be able to: – Describe a rubric (what is it?) – Describe the purpose of rubrics. – Describe the difference between holistic and analytic rubrics. – List the characteristics of good rubrics. – Develop a rubric for judging the quality of a product (chocolate chip cookie). – Create/modify a rubric for an assignment or activity in a class you teach. – Identify strengths and weaknesses in a rubric.09/17/12 4
    5. 5. What’s a rubric?• Analytic rubrics have – Levels of performance – Multiple criteria09/17/12 5
    6. 6. The TLT defines rubrics as:• "Rubrics" explicitly state criteria for assignments.• May lead to a grade or be part of the grading process.• Are more specific, detailed, and disaggregated than a grade.• Show strengths and weaknesses in student work.09/17/12 6
    7. 7. Another definition (Jon Mueller. Professor of Psychology, North Central College)• Assess student performance along a task-specific set of criteria• Measures performance against a predetermined criteria• Includes essential criteria for the task• Has multiple levels of performance09/17/12 7
    8. 8. Example: Research Project (Jon Mueller. Professor of Psychology, North Central College) Criteria 3 2 1 Number of X1 10-12 5-9 1-4 Sources Historical X3 No apparent Few Lots of historical Accuracy inaccuracies inaccuracies inaccuracies Organization X1 Can easily tell Can tell with Cannot tell from which sources difficulty where which source info was drawn information information from came from came from Bibliography X1 All relevant Contains most Contains very information is relevant little information included information09/17/12 8
    9. 9. One-Sentence SummaryOn a 3x5 note card:Write a one-sentence summary that: 1. Defines a rubric. 2. Differentiates between performance levels and criteria. 3. Explains column weight.When directed by facilitator, share your sentences with a partner.Combine your two sentences into one that can be shared with the group.09/17/12 Lesson Builder Page 4 9
    10. 10. Characteristics of Rubrics• Increase an assessments construct and content validity• Increase an assessments reliability – set criteria that raters can apply consistently and objectively• Established criteria reduces bias• Can help teachers clarify goals and improve their teaching• Help learners set goals and assume responsibility for their learning09/17/12 10
    11. 11. Characteristics Continued• Help learners develop their ability to judge quality in their own and others work• Provides specific feedback about areas of strength and weakness• Learners can use rubrics to assess their own effort and performance before submitting it09/17/12 11
    12. 12. Characteristics Continued• Learners and teachers monitor progress over a period of instruction• Reduces time spent grading• Engaging students in the design empowers them• Moves away from subjective grading09/17/12 12
    13. 13. Which of these reasons are important to you?• Importance of Reliability• Validity of the assessment• Reduction of bias in grading• Clarifying goals for you as the teacher• Communicating expectations to students• Improve students ability to judge their own performance• Means for providing better feedback to students09/17/12 Page 5 Lesson Builder 13
    14. 14. Types of Rubrics• Ask yourself: – For a particular task, do you want to be able to assess how well the students perform on each criterion, or do you want to get a more global picture of the students performance on the entire task?09/17/12 14
    15. 15. Holistic• a holistic rubric does not list separate levels of performance for each criterion• a holistic rubric assigns a level of performance by assessing performance across multiple criteria as a whole.09/17/12 15
    16. 16. Holistic Example 3 - Excellent Researcher • included 10-12 sources • no apparent historical inaccuracies • can easily tell which sources information was drawn from • all relevant information is included 2 - Good Researcher • included 5-9 sources • few historical inaccuracies • can tell with difficulty where information came from • bibliography contains most relevant information 1 - Poor Researcher • included 1-4 sources • lots of historical inaccuracies • cannot tell from which source information came • bibliography contains very little information09/17/12 16
    17. 17. Analytic Rubric Criteria 3 2 1 Number of X1 10-12 5-9 1-4 Sources Historical X3 No apparent Few Lots of historical Accuracy inaccuracies inaccuracies inaccuracies Organization X1 Can easily tell Can tell with Cannot tell from which sources difficulty where which source info was drawn information information from came from came from Bibliography X1 All relevant Contains most Contains very information is relevant little information included information09/17/12 17
    18. 18. When to choose an analytic rubric• Want to assess each criterion separately• Involve large number of criteria• More variance across the criteria• Need to weight criteria differently09/17/12 18
    19. 19. Types of Rubrics Activity• In the Lesson Builder materials, page 6, you will find a sorting activity.09/17/12 19
    20. 20. Designing an Analytic Rubric• Step 1. Re-examine learning objective to be addressed by the task.• Step 2. Identify observable attributes you want to see (as well as those you don’t want to see) your students demonstrate in the product, process, or performance.• Step 3. Brainstorm characteristics of each attribute.09/17/12 20
    21. 21. Design Analytic Continued• Step 4b. Write thorough narrative description for excellent and poor work for each individual attribute.• Step 5b. Complete the rubric by describing other levels on the continuum that ranges from excellent to poor for each attribute.09/17/12 21
    22. 22. Hint: Use Even Number of Levels• Use an even number (4 or 6) of levels of performance on the scale.• When there are an odd number of levels, the middle level tends to become a catch-all category.• With an even number of levels, raters have to make a more precise judgment about a performance when its quality is not at the top or bottom of the scale.09/17/12 22
    23. 23. Hint: Arrange Levels High to Low• High to low scale.• Students read first the description of an exemplary performance in each criterion.09/17/12 23
    24. 24. Sample High to Lows 4 3 2 1 Exemplary Excellent Acceptable Unacceptable Exceeds Meets Progressing Not there yetexpectations expectations Superior Good Fair Needs work09/17/12 24
    25. 25. More Hints• Limited number of dimensions or criteria. – The criteria are those components that are most important to evaluate in the given task and instructional context. – A rubric with too many dimensions may be unworkable in classroom assessment.• Equal steps along the scale. – The difference between 4 and 3 should be equivalent to the difference between 3 - 2 and 2 - 1. – "Yes, and more", "Yes", "Yes, but", and "No"09/17/12 25
    26. 26. 4 3 2 1 Task All Most Some Very few or none Frequency Always Usually Some of the time Rarely or not at all Accuracy No errors Few errors Some errors Frequent errors Comprehen Always Almost always Gist and main Isolated bits are sibility comprehen comprehensible ideas are comprehensible sible comprehensible Content Fully Adequately Partially Minimally coverage developed, developed, developed, developed, fully adequately partially supported minimally supported supported supported Vocabulary Range Broad Adequate Limited Very limited Variety Highly Varied; occasionally Lacks variety; Memorized; varied; non- repetitive repetitive highly repetitive repetitive09/17/12 26
    27. 27. Chocolate Chip Cookies• Break into pairs.• Compete the worksheet Creating a Rubric.• Create a rubric for judging the quality of a chocolate chip cookie. – -Be sure to decide if you want to do a holistic or analytic rubric!!!! – Be prepared to defend your decision.• Use the handout on Rubric Examples to help you when you get stuck!• When you all finished, you will get to judge the cookies using the different rubrics.09/17/12 See Lesson Builder Page 9 for 27 Handouts
    28. 28. Using RubiStar• These are some simple steps for getting started with Rubistar.• Access the site http://rubistar.4teachers.org.• Take a moment to read about the different features.• When you are ready to get started, click on the tutorial and choose the option you prefer.09/17/12 28
    29. 29. Rubistar Tips• If you are using Internet Explorer, you can copy the rubric and paste it into a Word document where you can continue to edit.• When first getting started, you might want to choose an existing template.• You may find that you can create a rubric you like by combining existing rubrics into one of your own. – You can do this by opening two windows to Rubistar and copying and pasting between windows.09/17/12 29
    30. 30. Assignment• Create a rubric for an objective/competency you have for a class you teach.• Use the worksheet on creating a rubric to get you started.• Divide into teams.• Exchange rubrics and use the Aunt Olives Rubric on Rubrics to provide feedback to a colleague on their work.09/17/12 Lesson Builder Page 10 30
    31. 31. Review1. Describe a rubric (what is it).2. Describe the purpose of rubrics.3. Describe the difference between holistic and analytic rubrics.4. List the characteristics of good rubrics.5. Develop a rubric for judging the quality of a product (chocolate chip cookie).6. Create/modify a rubric for an assignment or activity in a class you teach.7. Identify strengths and weaknesses in a rubric.09/17/12 31
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×