Published on

Published in: Education, Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. This presentation was developed for the exclusive use of students enrolled in: Educational Testing & Grading, Professor Gregory E. Stone. © 2004 Gregory E. Stone. All rights reserved. This presentation may not be reproduced in any form, in part or as a whole, without the express written permission of the author.
  2. 2. Essays & other non-objective items
  3. 3. We’ll include: Essays, Short answers and other items that can fit within the general parameters Mathematical problem solving Other partial credit items
  4. 4. Allow for free expression without artificial restriction and without prompting.
  5. 5. Two Fundamental Types: Restricted Response Extended Response
  6. 6. Restricted Response Limit the ways in which you will permit the students to answer.
  7. 7. Restricted Response Limit the ways in which you will permit the students to answer. � There ARE correct answers. � We allow students to express the answer in their own words.
  8. 8. Restricted Response “Write a brief essay comparing and contrasting the term analysis and synthesis as they relate to constructing (a) objective items and (b) essay items.” am ple ex
  9. 9. Restricted Response “What is the poet’s attitude toward literature as it is apparent in lines 1 to 8? what words in these lines make this apparent. am ple ex
  10. 10. Short Answer Restricted Response “A car traveling 50 mph leaves Chicago at 9am. A train traveling at 70 mph leaves Milwaukee at 10 am. Who will arrive in Toledo (250 miles away) first? Show your work.” am ple ex
  11. 11. Restricted Response Predicated on the notion that students supply the answers rather than selecting the answer from a group options.
  12. 12. Extended Response Allow students to express their own ideas and interrelationships among ideas and use their own strategy for organization. � No “correct” answers but reasonable/un logic.
  13. 13. Extended Response Sometimes people write just for the fun of it. This is a chance for you to have some fun writing. Pretend that you are a pair of tennis shoes. You’ve done all kinds of things with your owner in all kinds of weather. Now you are being picked up again by your owner. Tell what you, as the tennis shoes, think about what’s going to happen to you. Tell how you feel about your owner. ple Space it provided below and on the next page. ex am
  14. 14. Extended Response Devise a plan to determine whether the democrats or republicans are evenly distributed throughout the city, or whether the supporters of each party are concentrated in certain wards. Design an experiment to calculate the height of a redwood tree. am ple ex Create an original story.
  15. 15. Extended Response Because the focus is on logical argument and reasoned answering the teacher must be open and accepting of uncomfortable responses.
  16. 16. Extended Response Based on the creative process of the student - often including reasoning and factual presentations.
  17. 17. Essays have a unique ability to assess multiple taxonomic levels at once - while maintaining a good understanding of strengths and weaknesses. Look at the family photo attached. Describe the mood or feeling in the photo as well as the body language of the people. Use metaphors or similes to make these descriptions. Analysis Knowledge Comprehension
  18. 18. Is your essay a good essay?  Checklist 1. Does the item test an important aspect of the learning objective? Essays should ideally be targeted at no less than Application!
  19. 19. Is your essay a good essay?  Checklist 2. Does the item match the learning objective in terms of performance, emphasis and “number of points” (on the exam)?
  20. 20. Is your essay a good essay?  Checklist 3. Does the item require the students to apply their knowledge and skill to solve a new and novel problem? Essays should ideally be targeted at no less than Application!
  21. 21. Is your essay a good essay?  Checklist 4. When viewed in relation to the other items on the test, does this item measure new information covering the range of content and behavior specified in the learning objectives?
  22. 22. Is your essay a good essay?  Checklist 5. Is the item focused? Does it define a task with specific directions, rather than leave the assignment so broad that almost any answer would be acceptable?
  23. 23. Is your essay a good essay?  Checklist 6. Is the task defined by the item within a level of complexity that is appropriate for the intellectual maturity of the students?
  24. 24. Is your essay a good essay?  Checklist 7. To get a good grade on the item, is the student required to demonstrate more than facts, ideas, lists, definitions, generalizations, etc.
  25. 25. Is your essay a good essay?  Checklist 8. Is the item worded in a way that leads all students to interpret the assignment in the way you intended.
  26. 26. Is your essay a good essay?  Checklist 9. Does the wording of the item make clear to the students: Length, purpose, time needed and the basis for evaluation?
  27. 27. Is your essay a good essay?  Checklist 10. If the essay asks students to state and support their opinions on controversial matters, does the wording of the item make it clear that students’ assessments will be based on the logic and evidence supporting their arguments, rather than on the actual position or opinion?
  28. 28. Creating scoring rubrics Essential for ensuring fair, reliable, valid and meaningful assessments. One of the hardest problems!
  29. 29. Rubrics come in 2 varieties: Holistic (logic of argument) Analytic (correct answers provided in the student’s own words)
  30. 30. Holistic (logic of argument) Best used for Extended Response Essays Grading essays on how well the argument/essay is written in general.
  31. 31. Analytic (correct answers) Best used for Restricted Response Essays Grading essays on whether or not the student has provided the correct answer.
  32. 32. Analytic rubrics require the student to provide specific information in their answer in order to receive credit. No “bluffing” allowed!
  33. 33. To construct a rubric: (3)Ask - what is your objective? (4)Ask - what is the taxonomy? (5)Ask - what pieces of the puzzle do I expect the student to provide?
  34. 34. Example: Comprehend Vertical Leap Concepts: Potential/Kinetic energy Describe the concepts of Potential and Kinetic energy. Give an example of each in your description.
  35. 35. Describe the concepts of Potential and Kinetic energy. Give an example of each in your description. 1. Student correctly defined terms 2 points: Answer was clear and fully correct. 1 point: One or more answers were nearly correct, but student missed key concept. 0 points: Student failed to provide correct answer. 2. Examples were clear and correct 2 points: Both examples were appropriate. 1 point: One example was appropriate. 0 points: Neither example was appropriate.
  36. 36. Creating rubrics is essential. Using the rubrics in a standard, systematic fashion is vital!
  37. 37. The loss of reliability and validity are the leading causes of essay derailment. May be caused by poorly written essays; or May be caused by poor use of the created rubrics.
  38. 38. Inconsistent Standards Raters of essays may vary greatly. Individuals may grade essays differently on one day versus the next. (Reliability loss)
  39. 39. Rater Drift Raters tend to either fail to pay attention to criteria or change them - midstream - over time. (Reliability loss)
  40. 40. Changes in Topic and Prompt Raters often grade essays of different topics differently. (e.g. I like this topic more so I’m easier) (Validity loss)
  41. 41. Reader Bias Halo Effect Grading based on non-graded topics. (e.g If the student uses good grammar, grade the essay higher, even if the student doesn’t fully answer the item)
  42. 42. Reader Bias Carryover Effect If question 1 was answered poorly, the rater already has a bad feeling about the student when they begin grading question 2. Or vice versa!
  43. 43. Scoring Suggestions! You’ve created a rubric, so USE it! Score one item at a time. Score essays “anonymously” Score subject-matter correctness separately. Provide your students with feedback!
  44. 44. Essays & other non-objective items