Week 7 Rubrics And Rating Scales

  • 2,531 views
Uploaded on

 

More in: Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,531
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
36
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Assessment in Schools Complex Achievement: Scoring Performance Based Assessments
  • 2. Question (choose the best answer)
    • Which statement provides the best description of an analytic scoring rubric?
    • Rating is based on the overall performance compared to exemplars.
    • Rating is based on a sum of scores for the individual parts of the performance.
    • Rating is based on the student’s analysis of their performance.
    • All the above.
  • 3. Types of Performance
    • Alternative – something other than traditional paper and pencil tests requiring students to demonstrate
    • Authentic – practical application of a task in real world conditions/setting
    • (usually only approximated/simulated)
  • 4. Assessing Performance
    • Why use performances in assessment?
    • Why do we score/measure performance?
    • Communication
    • Comparison
  • 5. Assessing Performance
    • All Claims about the value of performance assessments rest on the assumption that performance can be accurately observed and reliably rated.
  • 6. Conducting Music Assessment
  • 7. Scoring Challenge
    • No one correct or best answer/solution
    • Many different performances or solutions might be judged as excellent (or poor)
    • Requires expert judgment and clearly specified criterion to assess properly.
  • 8. Scoring Limitations
    • Scoring can be inconsistent (unreliable Rating)
    • To compare scores fairly,
      • Task (learning outcome) must be clearly defined and communicated to students
      • Scoring criteria/rubrics must be well defined.
    • Time consuming to complete
      • Must have reasonable amount of time to do
      • Limits the number of tasks that can be done
  • 9. Scoring Issues
    • What are you assessing?
      • Process – approach used, methods & procedures, instrument use, etc.
      • Product – complete performance or resulting artifact
  • 10. Scoring Decisions
    • What assessment instruments will be used?
      • Rubrics, Rating scales, Checklists …
    • How will the results be used/reported?
    • What will you do to make sure the results are accurate (reliable)?
  • 11. Scoring Decisions
    • Who will do the assessment?
      • Teacher, student, peers, others
    • How will they be trained?
  • 12. Guidelines and Suggestions
    • Focus on the learning outcomes that require complex cognitive skills and performances
    • Select tasks that represent important content and skills
    • Minimize the dependence of irrelevant skills not directly related to learning outcome
  • 13. Guidelines and Suggestions
    • Provide scaffolding as needed
    • Construct task directions that clearly explain what students are expected to do
    • Clearly communicate performance expectations (how performance will be judged)
  • 14. Scoring Issues
    • What are your expectations?
      • Criteria – ideas about what is good or desirable when we judge adequacy; also used to defend that judgment.
  • 15. Scoring Criteria Issues
    • Floating Criteria – wait until you see the performance to determine acceptability
      • Ask yourself –
      • Do you know what your are looking for?
      • Can you define and describe the quality of a performance (both good and bad)?
      • Can you provide a defensible basis for rating good and bad performance?
  • 16. Scoring Criteria Issues
    • Criteria –
    • define what is acceptable and unacceptable in ways the student can understand
    • communicate the goal or standards
    • not useful when vague or ambiguous
    • make public what is considered important
    • [9-30] characteristics
  • 17. Instruments
    • Rubrics
    • Rating Scales
    • Checklists
  • 18. Scoring Rubrics
    • Rubrics are a set of guidelines that explain the criteria by which performance will be judged or rated (may include a rating scale) .
    • Rubrics outline performance standards
  • 19. Scoring Rubrics
    • Rubrics can be
      • Analytic – individual aspect of the task are judged and used to determine overall score
      • Holistic – the performance or product is judged as a whole, compared to models or exemplars
      • [see chapter 10]
  • 20. Scoring Rubrics
    • Rubrics typically provide a description of how the rater should determine the quality of various performances at specific levels.
    • Examples [9-29,9-33, pg 272]
    [9-32 rubric development]
  • 21. Group Task
      • Create an rubric you might use to rate or score the Leading Music performance.
  • 22. Rating Scales
    • They provide a convenient recording method, common frame of reference, and focus the raters attention on specific important aspects of the performance
    • Used for (limited to) make quality judgments
    • Requires additional information regarding performance expectations
    • Examples [9-34, pg 274]
  • 23. Rating Scales
    • These take many forms (numerical and descriptive) but are used to provide a uniform way to score performances along a continuum (at least ordinal, preferably interval).
    [9-42 types of rating scales]
  • 24. How often do you (meant to record frequency)
    • Response scale 1
    • Daily
    • 2-3 times per week
    • Once a week
    • 2-3 times a month
    • Once a month
    • Less than once a month
    • Response scale 2
    • Once a day
    • Once a week
    • More than once a day
    • More than once a week
    • As seldom as possible
    Not Ordinal, Not Interval Ordinal but Not Interval
  • 25. Question (choose the best answer)
    • Which of the following is NOT a good principle for constructing a graphic rating scales?
    • Characteristics should be directly observable.
    • Use 3 to 7 points on the scale.
    • Points on the scale must form an ordinal continuum.
    • Each point on the scale must be defined clearly.
  • 26. Rating vs. Ranking
    • Ranking requires a person to place in relative order
    • Raters assigns a specific score
    • Why might you rank instead of rate?
  • 27. Checklists
    • More appropriate for analytic rubrics where you can easily divide the task into a series of specific actions that must be present.
    • Reduces the amount of subjectivity in the judgment (dichotomous decision)
    • Can be problematic when aspects of the performance are valued but not represented in the criteria. (e.g., esthetically pleasing, interesting)
    • Examples [pg 282]
  • 28. Common Rating Errors
    • Personal Bias –
      • Generosity error – too easy, grade inflation
      • Severity error – too hard, no perfect papers
      • Central tendency – rating everyone about average
      • Halo Effect – general impression of individual (positive or negative) influences an individual rating
    • Logical error – rating alike or different based on the belief that factors are related (e.g., studious and able)
      • [see 9-44, pg. 277]
  • 29. Effective Rating Review
    • Focus on educationally significant outcomes
    • Characteristics should be directly observable
    • Clearly define key points on scale
    • Select most appropriate type of instrument
    • Use an appropriate scale (# of points)
  • 30. Effective Rating Review
    • Rate all performances on one task before going on to next.
    • When possible rate performances without knowing the raters name
    • If the assessment has significant impact, several ratings should be used.
    • Example practice [7-22,7-23]