Download to read offline
RUBRICS.pptx
- 1. RUBRIC DEVELOPMENT Why use rubrics? Characteristics ofgood rubrics Definition & types of rubric Strategies for rubric development SOLO taxonomy Phenomeno- graphic sorting Using rubrics with students
- 2. Definition Ascoring tool that lays out the specific expectations for an assessment task (Stevens & Levi, 2005) A set of clear explanations or criteria used to help teachers and students focus on what is valued in a subject, topic, or activity (Russell, & Airasian, 2012). Scoring rubrics are descriptive schemes that are developed by teachers or other evaluators to guide the analysis of the products or processes of students’ efforts. (Brookhart,1999 as cited by Moskal,2000) Components of a rubric: Criteria/Indicator aspects of an assessment task which the assessor takes into account when making their judgment May use different weightings for different criteria Level of Attainment often use grade level descriptors Types of rubrics: Descriptive (Analytic), Holistic Why and when we use particular types of rubrics SCORING RUBRIC
- 3. WHY USE RUBRICS? Forteachers: • Prompt a criterion-referenced assessment • Provide students with detailed and timely feedback • Encourage critical thinking • Facilitate communication with others involved in scoring • Help to refine teaching skills/learning activities For students: • Clarify the teacher’s expectations of student performance • Provide informative descriptions of expected performance • Help to monitor and critique own work
- 4. Well defined Clearly describe the expected level of student performance for each criterion in a rubric Avoid general evaluative words (poor, excellent, etc.) Use specific objective terms (correctly identifies, uses only basic vocabulary, chooses incorrect formula… ) Context specific Describe what teachers expect from student for a given performance or work product on a particular subject domain Viable for instruction CHARACTERISTICS OF GOOD RUBRICS
- 5. Finite andexhaustive Every response must be scorable Too many score levels is confusing for students and causes disagreement among teacher scores Ordered Represent the different levels of learning targets as defined by LP Related to Common Core theme/strand Coherent with the cognitive complexity of the standards CHARACTERISTICS OF GOOD RUBRICS
- 6. Components: (1) Task description (2) Assessment criteria (3)Performance levels Task Description: Level 1 Level 2 Level 3 Total Criteria 1 Criteria 2 Criteria 3 Criteria 4 Criteria 5 ANALYTIC (DESCRIPTIVE) RUBRICS
- 9. Advantage: Provides judgmenton each criterion. Disadvantage: Time consuming to make
- 10. Task Description: Criteria Level 5Overall description of Level 5 Level 4 Overall description of Level 4 Level 3 Overall description of Level 3 Level 2 Overall description of Level 2 Level 1 Overall description of Level 1 HOLISTIC RUBRICS A single scale with all criteria to be included in the evaluation being considered together Based on an overall judgment of student work
- 13. Advantage: Saves time indeveloping and scoring. Disadvantage: Does not provide specific feedback for improvement.
- 15. On performance-based tasks: extendedresponse items projects presentations portfolios WHEN TO USE RUBRICS
- 17. Reflecting onthe task & content Learning outcomes of the unit and the particular assessment What we want from the students, why we created this assessment, what our expectations are Listing the learning outcomes & expectations Focus on the particular details of the task and what specific learning objectives we expect to see in the completed task Grouping & labeling the outcomes & criteria Organize the results of reflections, group similar expectations together to become the rubric Indicators Application of a rubric format Apply the templates & descriptions to the final form of the rubrics 4 KEY STAGES IN CREATING RUBRICS
- 18. Relationship betweenthe intended Learning Progression (LP) and rubrics Direct use of the LP as standard reference Use of a general strategy (i.e. modified Bloom’s taxonomy or SOLO taxonomy) Assign the target outcome as mapped onto the Cognitive Rigor Matrix as the maximum level of the rubric Use the SOLO taxonomy as scoring Phenomenographic sorting STRATEGIES FOR RUBRIC DEVELOPMENT
- 25. 1. Sort studentresponses into performance level piles (excellent, good, …, and poor), or in terms of the levels of understanding of the responses 2. Describe similarities within a pile and differences between These similarities and differences can inform the different level of rubrics 3. Do sorting in pairs To reconfirm matches & mismatches Allow dialogue to maximize the effectiveness of rubric development PHENOMENOGRAPHIC SORTING
- 26. Explain whatthe test will emphasize Emphasis and expectations will be delineated in the assessment criteria in the rubrics Inform students how the assessment will be scored Explain what each of the assessment criteria defined in the rubrics means Explain how the results will be used Explain the importance of the test scores in the student’s learning progression USING RUBRICS WITH STUDENTS
- 27. Make surethat the wording of the rubrics is understandable to students (simplify wording for lower grades). Works best with holistic rubrics (or with a combined version of analytic descriptions). Provide rubrics to students in advance of the administration of the assessment. Alternatively, students can be provided with a general rubric and a small number of papers (names removed). Students can score the papers in small groups; groups are required to come to consensus on the grade to be assigned. HOW TO USE RUBRICS WITH STUDENTS
- 28. Nitko, A.J., & Brookhart, S. (2007). Educational assessment of students. Upper Saddle River, NJ: Pearson Education, Inc. McMillan, J. H. (2007). Classroom assessment. Principles and practice for effective standard-based instruction (4th ed.). Boston: Pearson - Allyn & Bacon. Oregon Department of Education. (2014, June). Assessment guidance. Popham, W. J. (2014). Criterion-referenced measurement: A half-century wasted? Paper presented at the Annual Meeting of National Council on Measurement in Education, Philadephia, PA. Popham, W. J. (2014). Classroom assessment: What teachers needs to know . San Francisco, CA: Pearson Russell, M. K., & Airasian, P. W. (2012). Classroom assessment: Concepts and applications. New York, NY: McGraw -Hill. Stevens, D. & Levi, A. (2005). Introduction to rubrics. As assessment tool to save grading time, convey effective feedback, and promote student learning . Sterling: Stylus Publishing, LLC Wihardini, D. (2010). Assessment development II. Unpublished manuscript. Research and Development Department, Binus Business School, Jakarta, Indonesia. Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, NJ: Lawrence Erlbaum Associates. BIBLIOGRAPHY
- 29. Rubrics PPT bythe Oregon Department of Education and Berkeley Evaluation and Assessment Research Center is licensed under a CC BY 4.0. Y ou a re free t o: S hare — copy and redistribute the material in any medium or format Adapt — remix, transform, and build upon the material U nder t he fo llowing t erms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. N onCommercial — You may not use the material for commercial purposes . S hareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original. Oregon Department of Education welcomes editing of these resources and would greatly appreciate being able to learn from the changes made. To share an edited version of this resource, please contact Cristen McLean, cristen.mclean@state.or.us. CREATIVE COMMONS LICENSE
Editor's Notes
- #2 This chapter will involve the characteristics of good rubrics, and some general strategies that can be used to produce them.
- #3 Scoring rubrics involve The specific expectations of an assessment task A set of clear expectations or criteria, that allows both teachers and students to focus on what is valued in a subject, topic, or activity The components of a rubric may include Aspects of a task which may be taken into account when making a judgment about performance on that task Different weightings for different criteria Levels of attainment (which may include grade level descriptors) We’ll be talking in the coming pages about two types of rubrics: Descriptive or analytic Holistic As well as when and why we use particular types of rubrics
- #4 Rubrics have a number of advantages for both teachers and students. For teachers They provide the criteria against which an assessment should be evaluated They assist in providing a student with detailed and timely feedback about performance They encourage the teacher’s critical thinking They facilitate communication with colleagues using the same rubric for scoring An, they provide feedback that can be used to refine teaching and learning activities For students They clarify the teacher’s expectations around assessment They provide information about the expected performance And, they can help students to monitor, and to critique, their own work based on the rubric levels
- #5 Good rubrics have several characteristics. These include They are well defined They clearly define the expected student performance for each level They use specific objective terms involving behavior, rather than general evaluative terms (which each user of the rubric may define differently) They are context specific They define teacher expectations for students for given performances in particular subject domains This makes them useful for instruction as well as assessment purposes.
- #6 Additional components include that They are finite No need to make a new scoring category for each new type of response – and every response must be scorable (even those that are off task, or flippant!) They are exhaustive There should be a place for every type of response – even irrelevant or flippant types of responses (I think of this as “a place for everything and everything in its place”) The number of levels is important; we want to differentiate between important level differences, but too many score levels can confuse both students and teachers They are (at least partially) ordered The order should represent the different levels of learning targets as represented by the LP They should be related to the common core theme or strand, and coherent with the cognitive complexity of the standards.
- #7 The first of the two types of rubrics we’ll discuss is an analytic (or descriptive) type. Analytic rubrics allow scoring of a task on several different aspects (or criteria) of the task (for example, grammar and creativity in ELA, or graphical skills and algorithmic correctness in math) Components include the general task description, descriptions of the criteria, and specific performance level descriptions for each of the criteria. Such rubrics provide separate judgments for each criteria Good for diagnostic purposes Provides more information from larger tasks However, such rubrics can be time consuming to make, and to use for scoring purposes.
- #11 The second type of rubric is a holistic rubric The holistic type provides a single scale in which all criteria for evaluation are considered together, providing an overall judgment of student work as a whole Such a rubric is quicker to make and use However, it does not provide diagnostic feedback for the student as to which components need improvement. It can also be tricky to use if a student is much better at one subskill than another
- #16 Rubrics are most often used on performance-based tasks, such as extended response items, projects, presentations, or portfolios. They are of course not needed for forced-choice items (like multiple choice or true-false), and are rarely needed for short-answer items.
- #18 There are several key steps to designing a rubric for a particular assessment task First, we need to think about the content, the task, and the students: What did we intend to teach? What is the purpose of the assessment? What we want from our students? Having done this, we need to list the particular details of the learning outcomes and objectives that we would like to see in an ideal response to a completed task. We then organize the results, grouping similar expectations together if we are designing an analytic rubric, and determining which expectations are easier and more difficult Finally, we use a rubric design template to produce the final form of the rubric
- #19 We’ll now be discussing several strategies that can be used to design rubrics for a particular task. The first of these includes using the description of the Learning Progression levels (either within or between grades) as general descriptors to design the specific levels of the rubric. The second is the use of a general rubric strategy such as Bloom’s taxonomy or a modification of it, or the Structure of the Learning Outcome taxonomy (which we’ll show in the next slide), to design the specific levels of the rubric The third is a technique called phenomenographic sorting, which we’ll explore in more detail in a few minutes. This technique basically involves sorting a set of student outcomes according to quality, and using the result to describe the specific rubric levels. Cognitive rigor encompasses the complexity of content, the cognitive engagement with that content, and the scope of the planned learning activities. The Hess Cognitive Rigor Matrix (CRM) is a tool that has significant potential to enhance instructional and assessment planning and practices at the classroom level (Hess, Carlock, Jones, & Walkup, 2009). The CRM superimposes two different cognitive complexity measures – Bloom’s Taxonomy and Webb’s Depth of Knowledge – to produce a means of analyzing the emphasis placed on curricular materials, instructional focus, and classroom assessment
- #20 The “before” example below shows a rubric that measures a lot of things that don’t have anything to do with whether a student understands the lunar phases: things like creativity, attractiveness, and whether the task is completed on time. If the purpose of the assignment is to assess whether students have really learned the lunar phases, the rubric should focus primarily on whether the content in the model is accurate and effective. The “before” example below shows a rubric that measures a lot of things that don’t have anything to do with whether a student understands the lunar phases: things like creativity, attractiveness, and whether the task is completed on time. If the purpose of the assignment is to assess whether students have really learned the lunar phases, the rubric should focus primarily on whether the content in the model is accurate and effective.
- #21 In the revised version below, all of the criteria is focused on the quality and accuracy of the information in the model and measures the desired scientific thinking from building the model in the first place.
- #22 With equal point breakdown WEIGH THE CRITERIA APPROPRIATELY As designers of rubrics, we can signal to students that certain criteria matter more than others. Just because a rubric has four criteria, doesn’t mean that each needs to be worth 25 percent of the score. With the weight of each criterion adjusted, the rubric itself guides students to focus on what is most important. In the example below—which is the same revised rubric from above, but where the teacher wanted to include some accountability for mechanics—all four criteria are weighted exactly the same. This means a student who demonstrates a perfect understanding of the science behind lunar phases, but who struggles with spelling and punctuation, could end up with a C on the project. That would not be a true reflection of mastery.
- #23 With unequal point breakdown
- #24 Using deficit language
- #25 Using can do language
- #26 Phenomenographic sorting involves sorting student responses to a particular item into piles of increasingly good performance (performances within a pile should be similar in level of quality, and piles should be ordered according to quality). Then, the differences between the piles should be described. These can be used to develop descriptions of different levels of the rubric It is often most effective to do the sorting in pairs, and discussing the similarities and differences with a partner
- #27 Rubrics can also be used as a tool with students. This has a number of advantages They can be used to explain what the test will emphasize, and what the expectations will be They can be used to explain how the assessment will be scored, and what the meanings of the criteria are Finally, they can be used to explain how the results will be used
- #28 How to go about using rubrics with students: First, make sure that the language of the rubric can be understood by students (you may have to simplify them for younger children). Holistic rubrics, or a compilation of the criteria for analytic rubrics, are easier for students to follow. They should provide characteristics of a good response, as opposed to information about correct answers It’s a good idea to provide the rubrics as handouts to students in advance of administering the assessment, as they can use them to focus their preparation. For take-home projects, rubrics can be included in the project packet. An alternative use for rubrics is to allow students to participate in a scoring exercise. This works as follows Choose a small number (say, 5-7) of student responses (ideally from another year or class). Remove all identifying information. Have the students work in small groups to score the papers. Require the groups to come to consensus about the scores to be assigned and the reasons why. Allow about half an hour for discussion. If you like, have a whole group discussion about the results. This technique works best with students 10 and older. The idea is to help students understand what teachers are looking for in responses.
- #29 That’s the end of our chapter on rubric development; if you’d like more information, here are some references for you.