Mesures of Student Learning - Talent Management Pilot - Fall Assessment Implications and Next Steps
Fall 2011 MoSL Analysis and Feedback Bronx Green Middle School’s implementation and implications of MoSL administration Charles Johnson, Principal Victor Rodriguez, Assistant Principal Kristin Crowley, Achievement Coach Jeff Hammer, Math Coach Mercedes Vazquez, ELA CoachPower point developed by Kristin Crowley, Achievement Coach Bronx Green Middle School (11X326)
Rationalization for MoSL at BGMS The variance application which allowed us to create assessments which would drive specific areas of weakness in student achievement at BGMS. ELA – writing has historically been an area where students have struggled. Math – Rations/Proportions (6/7) and Expressions & Equations (8) were determined based on Common Core focus strands. Social Studies – writing in content has been an area that has not had enough attention. Science – Scientific method and lab activities have been an area that students have lacked exposure.
Creations of the Assessments Overview of the process in creating the MoSL We studied historical data at BGMS to determine what would be best to make a focus for the school. We then analyzed assessments that were previously given at BGMS and other schools. We used rubrics and our talent coach for support in the development process. We revised and edited assessments until they only assessed the competencies on the rubric. We provided teachers with lead-in lessons to support areas of instruction that were necessary for assessment completion but not being assessed (i.e. – reading lessons for the ELA assessment). We also considered Lexile scores as well as text readability and interest.
Format of the assessments The assessments were given to the students over the course of 2 – 3 days with lead-in activities that provided scaffolded levels of support. In ELA and Social Studies the lead-in activities provided the students with the time to analyze the reading passages. In Math the lead-in activities provided the students with background knowledge providing them with an overview of the task being addressed. In Science the lead-in activities provided the students with a data and background information in a topic that was recently covered in class.
Focus of the Assessments - ELA ELA Grade 6 Steve Jobs greatest accomplishments Essay Focus – From a list choose and support which Steve Jobs invention had the greatest impact on society. Grade 7 Polar Bears and Tigers and supporting an endangered species Essay Focus – Based on facts provided which animal would you support with class funds – the polar bears or the tigers. Grade 8 Corporal Punishment in schools Essay Focus – Do you believe that schools should have the right to use corporal punishment on students as a form of discipline?
Focus of the Assessments – Social Studies Social Studies Grade 6 Hammurabi’s Code and Ancient law Essay Focus – Using historical documents students had to discuss whether or not they thought Hammurabi’s Code was just. Grade 7 Loyalists vs. the Patriots and the American Revolution Essay Focus – Using historical documents students had to side with either the Loyalists or the Patriots choosing one side and justifying their reasons. Grade 8 Johnson-Reed Act of 1924 and early immigration policy in the U.S. Essay Focus – Using historical documents students had to decide whether or not the Johnson-Reed Act of 1924 was fair or biased and why.
Focus of the Assessments – Science Science Grade 6 Simple Machines Lab Focus – create a lab which tests one hypothesis of a simple machine Grade 7 Classification of Minerals Lab Focus – create a lab which tests the way in which we classify and categorize minerals. Grade 8 Forces and Motion Lab Focus – create a lab which tests on of the theories around forces and motion.
Focus of the Assessments – Math Math Grade 6 Proportional Reasoning Focus – Ratio & Proportion Grade 7 Proportional Reasoning Focus – Ratio & Proportion Grade 8 Expressions & Equations Focus - Slope
Rigor of the AssessmentsDepth of Knowledge Content Area ELA (Level 3) Social Studies (Level 3) Math (Levels 1 – 3) Science (Levels 4)
Scoring of the Assessments Scoring of the assessment took place immediately following the administration process; Assessment were scored using a blind scoring system with the following procedures in place; The only identification on the assessments when they were given to the teachers was a student OSIS number – no class number or names were provided to ensure fairness. Teachers were paired in teams and were assigned a content area and grade. They were then responsible for the completion of the scoring of all the tests in that grade and content area. At least on teacher in the teacher team taught that grade and content area (while we are aware that this is not completely blind scoring teachers did not know the class or name of the students test they were scoring). Norming was done several times before the actual scoring began Teachers were provided no less then 6 hours over the course of 4 weeks to grade assessments (this was still not enough time for some grades and content areas). Back scoring for quality assurance was done throughout the scoring process by members of the leadership team.
Some notes on scoring During the scoring process the following decisions were made; Students who did not test were marked on our data capture sheets as absent (ABS). Students who did not make any attempt to answer a question were coded as A or X for their lack of response. Students who attempted to answer a task but got it completely wrong were given a score point of 0. Students who attempted to answer a task and/or question and indicated some minimal understanding of the concept was given a score point of 1. This was done to ensure that teachers had a way to determine which students had some understanding of the concept and which students had no understanding of it.
ELA Analysis After implementation and scoring of the assessment the following strengths and weaknesses were noted; Overall Strengths – Students wrote longer essays then in previous years Student essays were generally on topic Students were able to introduce a claim Students were able to cite textual evidence Overall Weaknesses – Students did not use transitional phrases from one thought to the next Students often did not include a conclusion Students in grades 7/8 did not consistently acknowledge and/or cite evidence for the counter claim
Social Studies Analysis After implementation and scoring of the assessment the following strengths and weaknesses were noted (and were very similar to ELA); Overall Strengths – Student essays were generally on topic. Students were able to introduce a claim. Students were able to cite textual evidence. Overall Weaknesses – Students did not use transitional phrases from one thought to the next. Students often did not include a conclusion.
Math Analysis After implementation and scoring of the assessment the following strengths and weaknesses were noted; Overall Strengths – Students were able to be exposed to multi-task questions Students were able to understand the relationships between equivalent ratios. Students understand the proportional relationships between decimals, percents, and fractions. Overall Weaknesses – Students in grade 6 were never exposed to proportional reasoning Students in grades 6/7 need to focus on unit rates Students in grades 8 need to work more on linear relationships
Science Analysis After implementation and scoring of the assessment the following strengths and weaknesses were noted; Overall Strengths – Students were able to identify the materials needed for a lab Students were able to ask questions that related to their hypothesis for analysis Overall Weaknesses – Students did not have an understanding of what the variable and independent variable was Students did not write the hypothesis in the “if…then” format Students struggled to create a lab that outlined all elements needed to test a hypothesis
Feedback on the MoSL To effectively capture teacher comments and suggestions everyone at BGMS who administered a MoSL were afforded the opportunity to provide written feedback in the form of the survey; General feedback to the assessment period included; Wanting more time to review the tasks and lead-in activities (especially in science). Changes to sections of the rubric to allow for different elements in certain competencies to be scored separately. More differentiation for ELL and SpEd students. More time in preparing for the assessment.
Driving Instruction at BGMS Each teacher has received the data from the MoSL for a baseline in planning their own individual goals going into mid- year conversations In Science data indicated that student needed much more exposure to lab and lab activities which we will plan on embedding into the curriculum as we reevaluate it for the winter and spring In Social Studies data indicated that students need more exposure to writing in content area and curriculum units of study are being revised to include more writing. In ELA; Teachers used MoSL data to help determine which areas would receive more in class attention during the second of to Argumentative Writing Units (based on data capture that focus will be in transitional words and conclusions. ELA classes also had the first of several on-demand writing assessments (more about this in an upcoming slide). In Math: Data indicated that students need help explaining their work through writing: Students will explain how they find their answers. Math classes need to focus on unit rates and how they are proportionally related.
On Demand Activities In order to ensure that the units of study that are being implemented in the classrooms is truly having an impact on student learning BGMS made the decision to implement a new system to determine students ability to transfer knowledge. Beginning with ELA and moving to all content areas a cross population of students from all grades are picked and given an on-demand assessment based on the current unit of study. As new units of study are developed with Common Core alignment more on-demand activities will be used as a form of formative assessment at BGMS The first on-demand assessment at BGMS was given in ELA to 90 students across three grades (sub-populations included in the testing were CTT, 12:1, Honors, ELLs (B/I/A), Holdover, SETTS, and Gen. Ed Students) The early data from the assessments indicate that students are; Writing with structure in mind (paragraphs and organization) Students are able to make a claim and support it Students are able to to cite text evidence Students are using more transitional words than they did in the MoSL Students are still struggling to develop a logical conclusion
Planning As BGMS moves to align more closely to Common Core Standards and rigor, the following structures have been put in place; ELA teachers have met in extended after school planning sessions to develop units of study that support high cognitive thinking. Math teachers will begin these type of planning sessions in the near future. Social Studies teachers are beginning to work with ELA teachers to find ways in which the curriculums and teaching points cross. Science teachers will be looking to meet in the spring to begin to develop curriculum that has a greater alignment to Common Core. Additionally teacher teams meet weekly to look at student work and unit pacing with members of the leadership team. This provides the leadership team the opportunity to get a pulse on individual classes and overall grade and content progress and where additional in class support is needed
Looking Ahead Spring MoSL administration planning will begin to mid-February to allow for enough time to create the assessments. Mid-Year conversations will be driven by MoSL data and will be used as a basis for all goals being set (expect in ELA where teachers will also be using on-demand assessments). On-demand assessments will become a foundation to determine whether students are retaining and transferring what they are learning in class. 2011 – 2012 formative and summative assessments will be used when curriculum changes are made during the summer.