505ferdon evalprojectpart


Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

505ferdon evalprojectpart

  1. 1. Springfield Public Schools Springfield, USA Evaluation Report First Street Elementary School2009-2010 School Improvement Plan - Goal 3 May 4, 2010 Report prepared by Susan Ferdon Faculty Member, First Street Elementary SchoolGraduate Student, Master of Educational Technology Program Boise State University Report Submitted to First Street School Improvement Planning Team
  2. 2. 
IntroductionThe program being evaluated is First Street’s School Improvement Plan, Goal 3, for2009-2010 (Appendix A). Required by the Illinois State Board of Education (ISBE) forschools in academic status, the submission of a School Improvement Plan is optionalbut recommended, for schools that are not (http://www.isbe.net/sos/htmls/school.htm).In Springfield schools, School Improvement Plans provide structure for continuousimprovement of instruction. SIP Goal 3 is an outgrowth of a 2008-2009 goal whichfocused on improving teacher feedback and students’ ability to correctly interpretteacher feedback. It was found that “while 88.6% of First Street students, includingkindergarten students, met the goal of correctly interpreting their teachers’ feedback ona specific writing assignment, more instruction and practice is needed in order forstudents to understand the link between feedback and subsequent learning andimprovement” (SIP Goal 3, p. 3).For 2009-2010, the goal was expanded to include student-generated learning goals andsubsequent reflection on those goals. The objective for SIP Goal 3 is that “all studentswill learn how to reflect after receiving specific and timely feedback from their teachers;and all students will learn how to use these personal reflections to generate their ownspecific learning goal(s) for their next assignment. When subsequent assignments areassessed, it will be noted that students tied their improvement to their goal based onprior feedback” (SIP Plan, p. 3).The SIP Team identified Reader Response (RR) writing assignments, administeredthree times during the year, as the data collection tool. Following each of the first twoiterations of the cycle, data were analyzed and recommendations made to the SIPTeam regarding improvements to this process. Though the primary focus of thisevaluation was ultimately narrowed to student goal attainment, a wider view of Goal 3components is also presented.Evaluation Timeline: March 8th Evaluator meets with First Street principal to discuss the three SIP goals and select one to focus on for this evaluation. March 11th SIP sub-committee meets for preliminary planning. March 15th Analysis of February work samples begins. April 7th Presentation of initial findings/recommendations to SIP Team. April SIP Team brings recommendations to grade level teams, new work samples are submitted, and data are analyzed. April 27th Presentation of findings and final recommendations to SIP sub-committee. May 4th Presentation of checklists and updated guidelines to faculty.
  3. 3. 
This evaluation and related reports serve several purposes: 1) To determine howcomprehensively SIP Goal 3 has been addressed thus far, 2) To documentrecommendations made regarding administration of Goal 3 assessments, 3) To shareFebruary and March Reader Response information and data, and 4) To provide detailedprogram information which may be used by First Street’s incoming principal for fall 2010reporting and goal-setting.Program DescriptionThe SIP Team selected Reader Response writing assignments, with work samplessubmitted in February, March, and May, as a means of collecting data. The Teamdeveloped a framework for the process that would be followed and establishedguidelines for the generation of related documents. Grade level teams used thatinformation to create grade-specific forms used by teachers and students in grades K -5 for the Feedback/Goal/Reflection cycle (Appendix E).For each set of writing assignments, students read a story or article and wrote aresponse to a prompt. Teachers provided feedback and used district-approved rubricsto score responses. Student then reflected on feedback and identified a goal. Next,students read a second story or article and, with their goal in mind, wrote a response toanother prompt. The teacher graded the second response and students reflected on theachievement of their goal. While the process followed was the same for all grade levels,procedures varied from one grade level to another due to the developmental nature ofreading and writing skills and the age-range (K-5) of the students who participated inthis program. Whereas kindergarten teachers provided verbal feedback and acted asscribes for goal setting, students were expected to be increasingly independent insuccessive grade levels and over time.At the point that the evaluator entered the process, February work samples had beencollected. In initial meetings with the principal and SIP Team, three evaluation questionswere identified: 1. What modifications can be made to clarify and standardize procedures and processes teachers and students will follow when completing the next two Reader Response writing assignments? 2. What data can be extracted from Reader Response work samples/feedback in order to report on this goal? 3. What additional information/data is needed and how can we collect it?
  4. 4. 
Evaluation MethodsParticipantsIt was initially expected that all students and classroom teachers would participate: 73 -kindergarteners, 81 - 1st graders, 82 - 2nd graders, 104 - 3rd graders, 81 - 4th graders, 98- 5th graders, and 23 teachers. It was found that student participation was notcompatible with the Read 180 program, a replacement program that coincides withclassroom Language Arts instruction. 2009 Data SCHOOL DISTRICT STATE LEP (Limited English 1.50% 0.70% 8% Proficient) Low Income 0.80% 0.30% 42.90% Parental 100% 100% 96.70% Involvement Attendance 96% 95.90% 93.70% Mobility 3.70% 1.90% 13.50%Table 1: First Street Educational Environment Figure 1: First Street DemographicsSource: Interactive Illinois Report Card

Source: Illinois Interactive Report Card - ProfileProceduresPhase I • A discrepancy model was used to identify potential gaps between stated objectives and current practice. SIP Goal 3 (Appendix A) and the fall 2009 Board of Education Presentation (Appendix B) were used to establish objectives and evidence was collected through observation and informal interviews. • February work samples had already been submitted: initial writing assignment, teacher feedback, rubric scores, and information related to goal attainment. o Teacher feedback was examined and trends were noted. o Student goals were examined for quantity, measurability, and to determine what feedback was used in the goal selection process. o Student achievement was examined with individual student scores entered into spreadsheets and mean, median, mode and standard deviation calculated for each classroom of students. o Goal attainment was examined via student and teacher reporting and that information was compared against rubric scores when possible. • Preliminary findings were reported to the SIP Team on April 7th, along with recommendations for the next Feedback/Goal/Reflection cycle (Appendix C).
  5. 5. 
Phase II • Based upon April 7th recommendations, some changes were made in the Feedback/Goal/Reflection process, resulting in greater standardization of processes and practices within each grade level. • Work samples from the second iteration of the Feedback/Goal/Reflection cycle were examined. o Student goals were examined, as above, but only for measurability. Problematic goals were noted. o Student achievement and goal attainment data collected and entered into spreadsheets. • Findings were reported to the SIP subcommittee (Appendix D) with further recommendations for the final Feedback/Goal/Reflection cycle.Data SourcesCopies of documents* may be found in the Appendices. SIP Goal 3* Document submitted to ISBE includes objectives, strategies, activities and monitoring process identified to meet the stated goal. BOE Presentation* PowerPoint presentation to the School Board includes intended processes and tasks related to SIP Goal 3. Reader Responses Student work samples; typically the writing prompt is included. Teacher feedback is frequently written directly on student work. Rubrics* Grade-specific, district-approved. Teacher feedback is frequently written in white space on this form. Feedback/Goal- Grade-specific forms used for students to re-state teacher feedback and Setting Forms* identify a goal. Some forms include subsequent reflection and space to indicate whether or not goal was met.ResultsObjectives and ActivitiesA discrepancy model was used to note differences between objectives stated in SIPGoal 3, the School Board presentation, and related activities that took place. Additionaldetails were presented to the SIP Team on April 7th (Appendix C). The focus of thisevaluation is on part A, Student Strategies and Activities. Parts C, D and E, ProfessionalDevelopment, Parent Involvement, and Monitoring, are not addressed.
  6. 6. 
 Objective: All students will learn how to reflect after receiving specific and timely feedback from their teachers; and all students will learn how to use these personal reflections to generate their own specific learning goal(s) for their next assignment. When subsequent assignments are assessed, it will be noted that students tied their improvement to their goal based on prior feedback. Strategies and Activities: SIP Goal 3 1. All students will take time to reflect on their teachers’ feedback and will briefly summarize the main idea of the feedback. ✔ 2. Using this reflection statement, students will generate learning goals … ✔ 3. Students will generate specific action plans for goal accomplishment … NA 4. Students will generate a plan for self-assessment. NA 5. Students will confer with teachers, demonstrating their understanding of the feedback and need for improvement. ✔ 6. Students will use a reflection and goal setting planner. ✔ Students will: BOE Presentation • continue to work on using feedback effectively by using their reflection planners with greater frequency for subsequent assignments. ND • practice using feedback on writing assignments across the curriculum. ND To accomplish this: • teachers will continue to increase and strengthen the specificity of feedback they give to students. NA • students will need to refer to prior feedback before beginning work on subsequent assignments. ✔ • students and teachers will need to assess whether the need, the feedback, the goal and subsequent improvement are aligned. ✔ Table 2: Discrepancies KEY: ✔ = Evidence supports that this objective/strategy/activity was adequately addressed. NA = Evidence supports that this objective/strategy/activity was not addressed. ND = No data was collected.Within stated program objectives, four components emerged as topics of interest:teacher feedback, student goals, student achievement, and goal attainment. For the firstreport to the SIP Team, information had been gathered for each of these four topics.Following the second report, the focus was on goal attainment.
  7. 7. 
Teacher FeedbackContent of written feedback for the first RR in February was examined. Trends werenoted in relation to patterns, detail, and quantity. It was determined that future focuswould not include feedback, so March information was not collected. Teacher Overview of Feedback Provided to Students in February Same feedback across the board. 1st: "Use more of the question in your answer," and/or 1 "Why did Celina feel ___." 2nd: "Remember to use an example to support your answer." Feedback frequently re-states the rubric and includes a question related to prompt - “How 2 do you know …?” 1st feedback: "You did a good/great job … Next time …" 2nd feedback: "Next time …" 3 Each time comment duplicated rubric info. Positive statement then "remember to …" 2nd time: Positive statement/goal achieved 4 followed by "keep trying to…" or "lets keep working on..." 5 When feedback on rubric page is the same, comments on student work differs. Teacher feedback on student pages differs from rubric - adds text as examples plus editing 6 marks. 7 Feedback frequently limited; typically spelling and punctuation. 8 Starts with positives and goes into detail regarding areas of improvement. 9 Feedback limited and brief, underlined words on the rubric. Spelling, grammar and word choice feedback on student page. Comments (organization, 10 content) on rubric page. Something positive included on each. st Aside from two “answer the question” comments, there is no teacher feedback on 1 11 response. Rubric score for all; different rubrics were used each time. Feedback is evenly split between spelling/punctuation and content. Comments are most 12 often directives or questions. No feedback on student work or score sheet. Rubric score for all, occasional editing marks 13 (paragraph, capital) for some. Detailed written feedback does not duplicate rubric. Mix of positive comments and 14 suggestions for improvement. 15 Feedback typically one sentence plus rubric score, with nothing written on student page. 16 (no work samples submitted in February) 17 Feedback is most often a one-sentence directive and editing marks. 18 Detailed written feedback on student work plus editing marks. 19 Detailed written feedback on student work plus editing marks. st 20 Written feedback on 1 , responses not graded (no rubric). Most have a sentence or two of written feedback on student page. Some are rubric number 21 score only. Table 3: Summary of teacher feedback
  8. 8. 
Student GoalsFeedback/Goal-Setting forms were the source of student goal data. The number ofgoals each student generated (how many students wrote one goal, how many wrotetwo, etc.) was counted as were the number of troublesome goals generated (non-goals,not measurable). Goals deemed “inappropriate” by schools standards also were notedand examples are included below. The recommendation was made to limit students toone goal for May because most students with multiple goals met some goals but notothers, resulting in “not met” categorization.Number of Goals: One Two Three Goal Goals Goals K 49.2% 38.5% 12.3% 1 97.4% 2.6% 0.0% 2 68.4% 30.3% 1.3% 3 64.3% 28.6% 7.1% 4 50.0% 37.5% 12.5% 5 56.3% 36.6% 7.0% Table 4: Number of Goals by Grade Figure 2: Number of Goals, K-5Source of Goals:Teacher feedback and student goals were compared to ascertain sources used togenerate goals. Most often, goals were based on comments. For students in 1st and 2ndgrade, goals were typically copied word-for-word from whatever the teacher wrote onthe page. While this was most common in the primary grades, it was common practiceat all grade levels. “Neither” was listed when it was not apparent where the goal camefrom. Kindergarten feedback was verbal so it is not included. Comment Rubric Both Neither 1 94.8% 1.3% 0.0% 3.9% 2 73.7% 9.2% 14.5% 2.6% 3 34.9% 38.4% 7.0% 19.8% 4 42.6% 8.5% 31.9% 17.0% 5 58.0% 10.1% 18.8% 13.0%Table 5: Source of Goals by Grade Figure 3: Source of Goals, K-5
  9. 9. 
Troublesome Goals:In February work samples, 10% of student goals (34 of 346) were found to betroublesome. In the initial report, that information was shared and SIP Team membersreported back to grade level teams. In March work samples, that number was reducedto 4% (17 of 441). e Quantity Comments K 0 1 1 Most of these “goals” are actually strategies; some cannot be measured (“I will re- 2 13 read my sentences”). Many of these “goals” are also strategies. Some are what is considered an 3 24 inappropriate goal at First Street (e.g., “get at least 8 points” “Use the writing I’m supposed to use”). 4 8 Seven are strategies and one (“make my Reading Response better”) is not. Three are strategies and two (“do what I did on this assignment and keep it up” and 5 5 “do everything my teacher told me to do”) are not. Table 6: Troublesome Student-Generated GoalsStudent AchievementRubric scores were used to track individual student achievement with mean, median,mode and standard deviation calculated for each classroom. Grading practices mayvary between teachers and comparison between groups of students was not desired, soteachers receive student-specific information (Appendix F) and achievement data isreported more generally to others.In most classrooms, the first February Reader Response (RR) had the lowest mean andgreatest standard deviation. From the first RR to the second RR, 14 of 17 classroomsshowed an increase in mean score – higher achievement. 12 of 17 showed a decreasein standard deviation – scores are more “clumped” toward the middle. From the secondRR in February to the first RR in March, nine of 17 classrooms showed a slightregression in mean score (lower achievement) and 11 of 17 classrooms showed anincrease in standard deviation (more highs and lows). In 17 of 21 classrooms, thehighest mean score was on the second RR in March. On the whole, studentachievement improved over time with fewer outliers and greater consistency amongstudent scores.
  10. 10. !Goal AttainmentGoal attainment was examined via student and teacher reporting and information wascompared against rubric scores when possible. FEBRUARY MARCH Goal Not Goal Not Met Met Met Met K 57.7% 42.3% 1 54.9% 45.1% 65.3% 34.7% 2 61.1% 38.9% 69.6% 30.4% 3 84.3% 15.7% 89.3% 10.7% 4 70.0% 30.0% 81.4% 18.6% 5 82.9% 17.1% 85.3% 14.7%Table 7: Goal Attainment by Grade Figure 4: Goal Attainment, K-5DiscussionThe purpose of this study was to provide the First Street SIP Team withrecommendations regarding the refinement of student activities and data collection forSIP Goal 3. Examining processes and content allowed for mid-course corrections that,in turn, provided the SIP Team with increasingly consistent and usable data. Analysis ofFebruary work samples showed that there were some disparities within and betweengrade levels in terms of procedures that were being followed and data that werecollected. Some differences between grade levels were suitable and expected, likeadditional conferencing and teacher involvement needed in kindergarten, but otherswere not: • In K – 2nd grade, there was one week, at most, between the two RRs. In 3rd and 4th grades that span could be as much as several months. Too many other variables would affect the comparison of rubric scores from the first RR to the second, so a guideline was put into place limiting the interval between RR1 and RR2. • In 4th grade, there was no place on the form to indicate if the teacher thought the goal was met and that form was also used by some 5th grade teachers. • 5th grade required more of an overhaul – some work was scored with a rubric and some was not. Some teachers used the 3rd grade form and others used the 4th grade form and different information was collected on each. Sometimes comments were very brief and there wasn’t anything we could measure. For one classroom, RR2 was a revision of RR1 – students used teacher feedback to edit the same piece of writing – and everyone else had two separate writing pieces. A large packet was submitted, of which just a few pages related to the RRs.! 9!