Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Creating Descriptive Rubrics for Educational Assessment
Similar to Creating Descriptive Rubrics for Educational Assessment (20)
Recently uploaded
Recently uploaded (20)
Creating Descriptive Rubrics for Educational Assessment
- 1. Creating Descriptive Rubrics for Educational Assessment Ursula Waln, Director of Student Learning Assessment Central New Mexico Community College
- 2. Why Create Rubrics? To communicate learning expectations to students. To facilitate fairness and consistency in evaluation of student learning. To establish a common framework for assessing student learning when using diverse measures.
- 3. Choose a Layout To Fit Your Purpose
- 4. Two Main Types of Descriptive Rubrics Holistic Provides comprehensive descriptions of each level of performance. Useful for quick and general assessment and feedback. Descriptions may be organized in columns or rows. Analytic Breaks out criteria for distinguishing between levels of performance on each criterion. Useful for detailed assessment and feedback. Descriptions are organized in a matrix.
- 5. HOLISTIC RUBRIC FOR RUBRICS (in columns) Highly Effective 3 Useful 2 Rudimentary 1 Focuses on knowledge and skills that confer life-long value through application to novel or complex situations, clearly addressing key criteria that 1) comprise the desired learning outcome, and 2) can be demonstrated through a variety of tasks. Differentiates benchmark levels inherent to development of the competency, thereby facilitating valid assessment of both formative and summative progress. Clearly and concisely communicates the nuances between differing levels of proficiency, and does so in terms of positive evidence. Clearly addresses key criteria that comprise the desired learning outcome. Organizes performance characteristics within progressive categories that accurately reflect observable stages of development. And, clearly describes observable characteristics that demonstrate each level of performance on each criterion. States learning outcomes in terms of broad instructional goals and/or specific learning tasks; presents performance characteristics in terms of correctness, quantity, or other grading criteria; constitutes a Likert scale.
- 6. HOLISTIC RUBRIC FOR RUBRICS (in rows) Highly Effective (3) Focuses on knowledge and skills that confer life-long value through application to novel or complex situations, clearly addressing key criteria that 1) comprise the desired learning outcome, and 2) can be demonstrated through a variety of tasks. Differentiates benchmark levels inherent to development of the competency, thereby facilitating valid assessment of both formative and summative progress. Clearly and concisely communicates the nuances between differing levels of proficiency, and does so in terms of positive evidence. Useful (2) Clearly addresses key criteria that comprise the desired learning outcome. Organizes performance characteristics within progressive categories that accurately reflect observable stages of development. And, clearly describes observable characteristics that demonstrate each level of performance on each criterion. Rudimentary (1) States learning outcomes in terms of broad instructional goals and/or specific learning tasks; presents performance characteristics in terms of correctness, quantity, or other grading criteria; constitutes a Likert scale.
- 7. ANALYTIC RUBRIC FOR RUBRICS Criteria Highly Effective 3 Useful 2 Rudimentary 1 Competency- Based Focuses on knowledge and skills that confer life-long value through application to novel or complex situations, clearly addressing key criteria that 1) comprise the desired learning outcome, and 2) can be demonstrated through a variety of tasks. Clearly addresses key criteria that comprise the desired learning outcome. States learning outcomes in terms of broad instructional goals and/or specific learning tasks. Representative Differentiates benchmark levels inherent to development of the competency, thereby facilitating valid assessment of both formative and summative progress. Organizes performance characteristics within progressive categories that accurately reflect observable stages of development. Presents performance characteristics in terms of correctness, quantity, or other grading criteria. Descriptive Clearly and concisely communicates the nuances between differing levels of proficiency, and does so in terms of positive evidence. Clearly describes observable characteristics that demonstrate each level of performance on each criterion. Constitutes a Likert scale.
- 8. Identify the Performance Criteria What Will Be Assessed
- 9. First, Focus on Competencies Whether designing a rubric for a single assignment or for assessing course or program outcomes, focus on competencies, not specific tasks. Competencies involve knowledge and skills that confer life-long value through application to novel or complex situations. Tasks are used to collect evidence of achievement of the competency, but a competency-based rubric can be applied to a variety of tasks.
- 10. Competencies Apply knowledge of social systems to analyze contemporary global issues. Frame problems within differing ethical perspectives. Evaluate information and sources critically. Employ contextually appropriate etiquette within various forms of electronic communication. Tasks Discuss the role of capitalism in the Chinese construction boom. Critique an opinion article for political bias. Correctly identify websites that are suspect choices as sources for academic research. Write an e-mail to inquire about a job opening. Examples of Competencies vs. Tasks
- 11. Design with Context In Mind Focus on competencies that align to pertinent: Standards (educational, industry, program) Student learning outcome statements (program, course) Instructional goals Learning objectives
- 12. Next, Derive Criteria from Competencies Competencies Broad descriptions of desired learning outcomes. What you want the student to be able to do upon completion of the lesson, course, or program. Criteria Standards on which judgments about level of competency can be made. Characterizing traits (characteristics) of the competencies. What is needed to demonstrate the competencies.
- 13. From Competencies to Criteria Some competencies are straight-forward enough to be used as criteria in rubrics. For example: Competency: The student will be able to… Apply the commutative property in computation. Criterion: To demonstrate application of the commutative property, the student needs to… Apply the commutative property in computation.
- 14. Some competencies are overarching and/or complex and need to be broken down for analytic evaluation. For example: Competency: The teacher will be able to… Create effective rubrics. Criteria: To be effective, the teacher’s rubrics need to be… Competency-based Representative Descriptive
- 15. Break Down Compound Competencies This competency represents a combination of abilities: Produce audience appropriate communication that displays consideration of ethical principles and diverse points of view. Compound competencies are more easily evaluated when broken down into component criteria. Using the above example: Audience appropriateness Consideration of ethical principles Consideration of diverse points of view
- 16. Consider Differences in Manifestation Some competencies call for synthesis of several related manifestations of learning. For example: The competency “Employ contextually appropriate etiquette within various forms of electronic communication” involves: SKILLS/ABILITIES related to language and register-based code-switching. KNOWLEDGE/AWARENESS of etiquette associated with different contexts. USES/APPLICATIONS of electronic communication media. BEHAVIORS/DISPOSITIONS related to courtesy and respect.
- 17. Evaluation can be facilitated by breaking different manifestations of learning out into separate criteria. For example: Competency (from previous slide): The student will be able to… Employ contextually appropriate etiquette within various forms of electronic communication. Criteria: To show proficiency, the student needs to demonstrate… Fluency in code-switching Familiarity with contextual expectations Application of electronic media Courtesy
- 18. Can One Part Be Done Without the Other? “Demonstrate leadership and integrity.” Yes! We can find examples of strong leaders with little integrity and people of high integrity with very poor leadership skills. Therefore, for analytic evaluation purposes, these should probably be separate criteria: Demonstrate leadership. Demonstrate integrity. “Analyze and evaluate college-level texts.” This one is not so clear. Evaluation requires analysis, and effective analysis requires a modicum of evaluative skill. It might be best to leave these together unless you intend to evaluate them as separate skills.
- 19. Is the Proficiency Multi-Faceted? “Evaluate information and sources critically.” Yes, this consists of two separate abilities: Evaluate information critically. Evaluate sources critically. And, each of these could be further broken down: Evaluate information critically → recognize manipulative rhetorical devices, recognize bias, assess contradictory information, identify failures in logic, analyze alternative points of view, etc. Evaluate sources critically → analyze context, infer motive, assess credibility, etc.
- 20. How Far Should You Break Things Down? Consider the purpose of the rubric. Program-level assessment uses broadly defined criteria. Course-level assessment uses more specific criteria. Consider performance evaluation. Break apart criteria if you would use separate measures to evaluate the components. Consider complexity. A component that has nuanced and/or distinguishable performance levels probably needs to be separated out.
- 21. How Far Is Too Far? Rubrics usually have more than two performance levels, so use ‘all or nothing’ as a litmus test for too much specificity. For example: “Differentiate between collective and individualistic societies.” If a student either is or is not able to identify a basic difference (all or nothing), consider using a broader criterion, such as “Describe societal differences that contribute to diverse world views.” On the other hand, if a student might describe a variety of characteristic differences and do so with nuanced degrees of accuracy, then the criterion should probably stand alone.
- 22. Phrasing of Criteria Criteria are usually written as phrases and are formatted according to personal taste. Some examples of different formats: Employs code-switching fluently Employ code-switching fluently Fluent code-switching Code-switching fluency Fluent in code-switching For rubric clarity and cohesion, pick a format and be consistent.
- 23. Create the Headings for a Matrix Analytic Rubrics
- 24. Criteria & Performance Levels In analytic rubrics, the convention is to use the left-hand column for listing the criteria. You can leave the top cell blank or enter a heading, such as “Criteria.” The tops of the remaining columns hold the headings for the various performance levels.
- 25. Level Heading 1 Level Heading 2 Level Heading 3 Criterion 1 Criterion 2 Criterion 3 Criterion 4 Basic Layout
- 26. Determine How Many Columns You Need How many performance levels do you want to describe? The usual is 3 to 5, but some advise using an even number (to avoid a “central” tendency in scoring). Do you want to include an exceptional level? Do you want to include a scoring column? Helpful if the rubric will be used as a scoring instrument. Do you want to include unlabeled columns with numerical values to allow for in-between scoring?
- 27. Level Heading 1 2 Level Heading 3 4 Level Heading 5 Exceptional Level Heading 6 Score Criterion 1 Description Cr. 1 Lvl. 1 Description Cr. 1 Lvl. 3 Description Cr. 1 Lvl. 5 Description Cr. 1 Lvl. 6 Criterion 2 Description Cr. 2 Lvl. 1 Description Cr. 2 Lvl. 3 Description Cr. 2 Lvl. 5 Description Cr. 2 Lvl. 6 Criterion 3 Description Cr. 3 Lvl. 1 Description Cr. 3 Lvl. 3 Description Cr. 3 Lvl. 5 Description Cr. 3 Lvl. 6 Criterion 4 Description Cr. 4 Lvl. 1 Description Cr. 4 Lvl. 3 Description Cr. 4 Lvl. 5 Description Cr. 4 Lvl. 6 Sample Layout with Unlabeled and Scoring Columns
- 28. Choose Ascending or Descending Order The choice of whether to arrange the levels in ascending or descending order is a matter of preference. Does one order make more sense to you than the other? Does your topic suggest a certain organizational logic? Are your students accustomed to a certain layout? Does your school or program have a preferred model?
- 29. Exceptional Level Heading 6 Level Heading 5 4 Level Heading 3 2 Level Heading 1 Score Criterion 1 Description Cr. 1 Lvl. 6 Description Cr. 1 Lvl. 5 Description Cr. 1 Lvl. 3 Description Cr. 1 Lvl. 1 Criterion 2 Description Cr. 2 Lvl. 6 Description Cr. 2 Lvl. 5 Description Cr. 2 Lvl. 3 Description Cr. 2 Lvl. 1 Criterion 3 Description Cr. 3 Lvl. 6 Description Cr. 3 Lvl. 5 Description Cr. 3 Lvl. 3 Description Cr. 3 Lvl. 1 Criterion 4 Description Cr. 4 Lvl. 6 Description Cr. 4 Lvl. 5 Description Cr. 4 Lvl. 3 Description Cr. 4 Lvl. 1 Sample Layout with Levels in Descending Order
- 30. Compose Performance-Level Headings Performance-level headings are usually one or two words each, carefully selected to represent the developing levels of proficiency. If the rubric is to be used as a scoring instrument, include a numeric score value with each heading. You can decide upon headings early on or after writing the descriptions. Sometimes it helps to write the descriptions first. If you want to wait, just enter numbers as place holders.
- 31. Sample Headings Beginner, Amateur, Pre-Professional, Professional Beginning, Emerging, Developing, Proficient, Exemplary Below Expectations, Satisfactory, Exemplary Benchmark, Milestone, Capstone Level 1, Level 2, Level 3, Level 4 Needs Improvement, Acceptable, Exceeds Expectations Needs Work, Acceptable, Excellent Neophyte, Learner, Artist Novice, Apprentice, Expert Novice, Intermediate, Advanced Rudimentary, Skilled, Accomplished Undeveloped, Developing, Developed, Advanced
- 32. Emerging 1 2 Developing 3 4 Proficient 5 Exemplary 6 Score Criterion 1 Criterion 2 Criterion 3 Criterion 4 Example of Headings with Associated Values
- 33. Write Descriptions In the Intersecting Cells
- 34. Describe the Target Outcome Start by describing the highest level of performance expected. Describe any level that exceeds expectations last. Descriptions should focus on what is observable. What does it look like (how can you tell) when the student has achieved the desired learning outcome? Try to capture the essence of the outcome, the important manifestations. Use clear and concise language. It is okay to use sentence fragments, so long as the meaning is clear.
- 35. What Is Observable? Internal processes, such as understanding, appreciating, valuing, respecting, being curious, and being interested are not readily observable. So, we have to identify behaviors/actions that evince those processes, such as explaining, analyzing, classifying, applying, appraising, defending, integrating, transforming, etc. Avoid putting the word demonstrates in front of an internal process to make it observable (e.g., demonstrates understanding or demonstrates appreciation) unless you truly need a vaguely worded rubric for broad application and broad interpretation.
- 36. Describe the Other Levels Seek to describe performance breakthroughs or thresholds that logically distinguish the levels of achievement. If you include an exceptional level (“Exceeds Expectations,” “Outstanding,” etc.), make it clear the proficiency goes well above and beyond the learning expectations. Scoring in this column should be rare; otherwise, you probably need to adjust your learning expectations.
- 37. Use Cumulative or Distinct Descriptions Cumulative Build upon positive indicators of achievement. What the student does do versus does not do. Sophistication of indicators increases with proficiency. Lower-level abilities are assumed to be present within the more advanced levels. Distinct Each level is described in full. Lower levels typically include negative as well as positive indicators. Tend to repeat and add to lower-level positive indicators. Can be tricky to write well because saying a student does not do something is fairly absolute and tends to restrict interpretation.
- 38. Sample Criterion: Discuss Design Concepts Cumulative Descriptors Level 1: Identifies fundamental design concepts. Level 2: Applies design concepts to describe works. Level 3: Applies design concepts to effectively critique works. Distinct Descriptors Level 1: Identifies but does not apply design concepts. Level 2: Identifies design concepts and applies them to describe works but does not effectively apply them in critiques of works. Level 3: Identifies design concepts and applies them to describe and effectively critique works.
- 39. Allow the Creative Process to Unfold Revise Some people can clearly envision the whole and implement it without modification, but most of us adjust a little here and there as we see the totality begin to emerge more clearly. Entertain Ideas The process of creating rubrics often helps teachers become much more clear in their own minds about what they want their students to learn, which can stimulate new ideas about how to promote that learning.
- 40. Rubric for Rubric Component Analysis Good Even Better For Greatest Effect Layout Layout presents or implies a matrix with more than one performance level. Visual organization forms a clear basis for either holistic or analytic evaluation. Choice of layout is appropriate to the intended use and audience. Performance Criteria Criteria clarify key aspects of proficiency. Criteria are distinct, scalable, and measurable. Criteria are clearly worded and consistently formatted. Headings Headings represent performance levels, arranged in either ascending or descending order. Headings accurately reflect developmental processes associated with the desired learning. Word choices are appropriate to the context in which the rubric will be used. Descriptions Descriptions characterize proficiency at successive stages of development. Higher-level descriptions relate breakthroughs in proficiency development. Descriptions emphasize positive indicators of proficiency.