The document outlines a professional development session aimed at helping teachers understand and use rubrics for scoring constructed response questions in writing and reading comprehension. It discusses the differences between holistic and analytic rubrics, provides examples of scoring student responses, and emphasizes best practices to minimize scoring bias. The session includes hands-on activities for norming student responses and encourages teachers to apply the knowledge gained back in their schools.

1. ConstructedResponse PaloozaWinter 2010

2. Think, pair, shareWhat are some of the challenges you face when grading constructed responsequestions?

3. ObjectivesProvide teachers with a firm understanding of the different types of rubrics and how to use them with different student writing samples.

4. Provide an opportunity to examine and discuss an anchor set for a given writing prompt.

5. Provide teachers with the opportunity to examine student samples from an administered constructed response item and norm along the ANet rubric.OVERARCHING GOAL: To leave with a better understanding of using rubrics to effectively score constructed responses and to take this back to your schools to improve your school’s overall use of constructed response data.

6. agenda I. Rubrics 101 15 min.Different types of rubricsDifferent types of questions/promptsScoring II. Norming around an Anchor Set 45 min.Review of passage, item, and rubricModel of ScoringPaired Scoring and Debrief II. Practice Norming around ANet Responses 1 hourGroups use rubrics to score student responsesDebriefIV. Next Steps 10 min.

7. Rubrics 101Holistic v. AnalyticHolistic Rubricsprovide a single score based on an overall impression of a student’s performance on a task.DC Uses: to score constructed response items aimed at assessing reading comprehensionAdvantages: quick scoring, provides overview of student achievement Disadvantages: does not provide detailed information, may be difficult to provide one overall score Analytic Rubricsprovide specific feedback along several dimensions.DC Uses : to score writing compositions aimed at assessing writing Advantages: more detailed feedback, scoring more consistent across students and graders Disadvantages: time consuming to score

8. ELA Composition vs. Constructed Response QuestionsCompositionAssess writing

9. A starting point to initiate a student’s own thoughts on the topic, supported by their experiences and ideas

10. No “correct” answer (but must address topic specified by the prompt)

11. Style and organization are important to the score

12. Uses Analytic Rubric and scored in two domains: Topic Development and Conventions

13. Scored out of 10 points in grades 4,7 and 10Constructed responseAssess reading comprehension

14. Student must answer the question asked and answer must be supported with details from the text

15. Answers found in reading the passage or can be inferred from information in the passage

16. The explanation and support from the text matter much more than style and organization

17. Uses Holistic Rubric, conventions are not scored

18. Worth 9 total points (3 CRs, each worth 3 pts.)Sample Composition Writing PromptThink about a time you were helpful. For example, maybe you helped a new student feel comfortable in school, helped a teacher with a job in the classroom, helped with a project in your neighborhood or school, or helped to care for an animal.Write a story about a time you were helpful. Give enough details for readers to understand how you were helpful.Example is from the 2009 4th grade ELA MCAS

19. Analytic RubricTopic/Idea Development

20. Analytic RubricStandard English Conventions

21. Sample Constructed Response QuestionBased on the article, describe what peoplehave learned from studying slave quilts.Support your answer with importantinformation from the article.Example is question 35 from the 2008 6th grade ELA MCASPassage “Pattern for Freedom: Women’s Quilt as Art”

22. Holistic Rubric

23. Scoring Bias in Constructed Response items It’s important for validity not to let writing skills impinge on reading comprehension scores.Things to look past (to avoid scorer bias)PenmanshipNeatnessMechanical errorsSpellingLength

24. Process for scoring student responses to Constructed Response PromptsRead the passage and the accompanying prompt.Review the rubric for the prompt and highlight key words.Create a list of scoring notes. Begin to build an anchor set.Score remaining student responses using rubric and anchor set.

25. Read passage and accompanying promptRead the passage “Pattern for Freedom: Women’s Quilt as Art”Read the prompt: Based on the article, describe what people have learned from studying slave quilts. Support your answer with important information from the article.

26. Review the rubric

27. highlight key words in the rubric

28. Create Scoring NotesWhat people have learned from slave quilts:Quilts were forms of self-expression: used to pass on emotion, histories and religious beliefs

29. Quilts were a way to convey messages in African culture:Historical symbols: such as X for a crossroads, snake motif for the West African God of fertility, flower patterns for the Haitian goddess of love, etc.Quilts were a way for slaves to leave a record of their lives (personal histories): The use of color: for example red for a woman’s birth process or a man’s role as hunter and warrior, etc.Quilts were a way to guide runaway slaves (use in the Underground Railroad):Black fabric symbolized a safe house, the “Drunkard’s Path” told slaves to take an indirect route, etc.

30. Create an Anchor setWhat is an Anchor set?An anchor paper is a sample response that illustrates student work at a given score point on a rubric.

31. An anchor set will include examples of student responses at each score point at the rubric.

32. These student samples help illustrate what is provided in the rubric.Student Response #1Score: 0Why a 0?The response is incorrect.Why not a 1?The information contained in the response is incorrect and not supported by the article.

33. Student Response #2Score: 1Why a 1?There is minimal attempt to explain that slave quilts contain meaningful designs. There are no supporting details from the article included.Why not a 0?The information provided was not incorrect. The response, while lacking details from the passage, does show understanding of quilts helping slaves escape from slavery and that designs had different meanings.

34. Student Response #3Score: 2Why a 2?The response is a general description of what people have learned from studying slave quilts. The focus of the discussion is on the symbolism of the colors and patterns used in slave quilts.Why not a 3?The response is partially complete in that it lacks any reference to the use of quilts for passing down history. Details from the story are general.

35. Student Response #4Score: 3Why a 3?The response clearly, completely, and accurately explains that the study of slave quilts has taught people about the personal history of slaves, aspects of African culture, and symbols used in the Underground railroad. Relevant quotes from the article are included to support each idea. The introduction and conclusion do not contribute any additional information to this response.Why not a 2? The essay is clear and complete. Specific details, including quotes from the article, are used in the response. No inaccurate details are included.

36. Think, Pair, ScoreRead student samples #5 and #6.Score each sample using the rubric.Justify your score.Share your thoughts with your neighbors.

37. Student Responses #5 and #6Student Response #5Score: 3Why a 3?The response is a clear, complete and accurate description of what people have learned from studying slave quilts. Details are included to explain the meaning of several historical symbols as well as quilt patterns designed to guide runaway slaves. The symbolic meaning of different colors are also discussed.Student Response #6Score: 2Why a 2?This response provides a partial explanation of what people have learned from studying slave quilts. Information is provided about the symbolism of the color black and the significance of certain patterns, but overall details are limited.

38. Scoring an ANET promptRead the passage and the accompanying prompt.Review the rubric for the prompt and highlight key words.Create a list of scoring notes. Begin to build an anchor set.Score remaining student responses using rubric and anchor set.

39. ANet Writing PromptRead the passage:Take a moment to read the passage from Lost in the Grand Canyonby Deborah Hopkinson.Read the promptAccording to the article, Powell was “a born leader with courage andambition.” Based on the article do you agree or disagree with thisstatement? Why? Use evidence from the article to support youranswer.