Assessing Assessments Kristin Crowley Reevaluating a school’s assessment plan and use of data toeffectively meet student needs and teacher effectiveness while overcoming personal obstacles Leadership Competency Data Personal Leadership
Facing Adversity James Lane Allen is quoted as saying –Adversity does not build character, it reveals it.•Accepting my dyslexia is not something that Ieasily embrace.•As an emerging leader I went out of my way toavoid math and numbers.•I also struggle to ask for help because Ispent my entire life hiding my disability andfiguring out ways in which I couldproblem solve myself.
The plan-assess-adjust cycle• According to the state of New York’s School Quality Indicators the plan-assess-adjust cycle shows that there is evidence of continuous improvement of educational practices via data-driven, collaborative instructional planning and continuous monitoring of progress of all students and subgroups, e.g., 4-6-week improvement targets; clearly- defined timeframes to adjust practice during the school year; clear milestones for progress are defined in advance. (Quality indicator 3.7 in School Leadership)• According to the State indications of assessment can include; – Interim assessment reports e.g., Princeton and Acuity Interim Assessments and classroom assessments used in conjunction with pacing calendars – ELA and Mathematics learning activities organized by ELA and Mathematics skill levels – Review, analysis & adjustment conducted periodically – Formal curriculum, instruction and assessment review and revision
The Old Plan (Summer 2011)Planning for the 2011 – 2012 school year,Bronx Green Middle School’s assessment plan included formative and summative assessments
Some of the formative assessments regularly in used • Formative assessments Entrance slips Exit slips Questioning Think, Pair, Share Group discussions (small and whole class) Conferencing (individual and small group) Anticipation Guides Quizzes Acuity iXL Achieve 3000 Learning Logs Self assessments Graphic Organizers
Summative Assessments• Summative assessments; – Beginning, Mid, and Endline assessments in ELA – Baseline and Endline in Math (inconsistently) – Reading Records (September, January, May) – End of unit projects in all content areas – End of unit tests and quizzes – New York State Assessments including ELA, Math Science (grade 8), NYSELAT (including alternate assessment scores) – Promotional Appeal Portfolios – GSIP’s for all IEP students meeting modified criteria
Analysis of the 2011 – 2012 Assessment Plan (based on the 2010 – 2012 school year) • Assessment was commonly used in the classrooms in many forms, however, data was sporadically collected, rarely looked at to inform and drive instruction and did not drive development of teacher and student goals. • Some teachers used various forms of data during team meetings to look at student progress but this was usually limited to state test scores.
The 2011 – 2012 plan• Inclusion in the Talent Management Policy under the New State Law for APPR to determine teacher effectiveness by analyzing the ways in which we assessed our students. This needed to be reassessed and revised.• The school also had to make choices in how we would not only assess students but also measure the growth of a teachers effectiveness in the classroom based on city options for schools who were “opted-in” to the pilot program.
Assessment Options – via DOE • CAT – Computer Adapted Tests • Performance Tasks • SLP – Student Learning Portfolio’s • Group Measure • Variance Option
Variance Plan• After considering the options from the city, I made the decision to apply for the variance application. While it was clearly the most work on the school level because it required schools to create their own assessments, submit them for approval, grade and analyze their own data, and use data to determine teacher growth, it also allowed our school to align assessments specific to the CIE (City-wide Instructional Expectations) for the CCLS (Common Core Learning Standards) and focus on a skill which the historical data for the school has shown that the students have had the least amount of growth – writing.• Working with our talent coach and the leadership team I developed the variance application and submitted it to central and the talent management program. The application is now being used as a model for schools as a reference for future applications.
Implementing the plan • To meet the needs of this new plan the following needed to created; Assessments (that mimicked the state tests in writing) Data Capture Guide for implementation Securement of the assessments Norming Scoring Analysis of data Use of assessments to drive instruction and goal set
Assessment DevelopmentContent Area Grade 6 Grade 7 Grade 8English LanguageArts Essay with a focus on Writing Standard 1 Essay with a focus on Writing Standard 1 Essay with a focus on Writing • The assessments had to; Standard 1 Align to the Common Core Standards (either WritingMathematics Series of extended Series of extended Series of extended response questions response questions response questions modeled after state modeled after state modeled after state assessments assessments assessments Standard 1, or Reading StandardSocial Studies DBQ based on Writing Standard 1 DBQ based on Writing Standard 1 DBQ based on Writing Standard 1 10)Science Series of short and extended responses Series of short and extended responses Series of short and extended responses Be scored using a city-wide developed rubric model after state model after state model after state science assessments. science assessments science assessmentsArt N/A Essay with a focus on Essay with a focus Given at least two different times Students recognize on Students the societal, cultural, and historical recognize the societal, cultural, in the school year significance of art and its connection to and historical significance of art Have a specific means to capture data the real world. and its connection to the real world.Physical Education Fitness Gram Fitness Gram Fitness Gram
Sub-population modifications• Using some of the guiding principles of UDL (Universal Design for Learning) the following was considerations and adjustments were made; Students with Disabilities and English Language Learners were given graphic organizers and vocabulary guides (as needed) Additional time (as per IEP or state testing indicators) Additional scaffolding based on testing modifications
Assessment Administration • The assessments were designed to only measure the students ability to apply their knowledge of Writing Standard 1 • Teachers were given guides to administering the exam which included 1 – 2 lead in sessions which provided the opportunity for students to be taught the reading skills needed to complete the assessment. • A 2nd or 3rd day was then used to administer the actual exam in which the students were able to use their notes from the previous sessions.
Data and Analysis• The Data The data was then collected and captured in a spreadsheet Back scoring was done at random to ensure that teachers were scoring accurately Fall data was then analyzed by the leadership team to help frame goal setting for teachers Teachers then met with the administration to set goals based on fall data At the end of the year the fall and spring data was used to help determine teacher effectiveness and students growth• Data was used to develop the years instructional plan For example in ELA as argumentative units were planned the data was used to help from the unit. Students demonstrated difficulty with transitions and conclusions so overall the team embedded more lessons on these skills.
Data Analysis Up Close• Below is an example of one of the spreadsheets from the 2011 – 2012 school year
Evolving Data Spreadsheets• The spreadsheets were developed even further in the 2012 – 2013 school year to provide an even more focused lens for data analysis;
The good, the bad and the ugly…• After the first argumentative unit of writing we noticed that the students did extremely well on it. It did not match an appropriate level of growth to the assessment. – An on-demand assessment of writing was done with 90 students across three grade levels and sub-groups to assess understanding. • The outcome of the on-demand assessment showed a mix of both to much teacher scaffolding and a more solid delivery of instruction.• We also noticed a more succinct delivery of materials to the students as it was more data driven and grounded in evidence.• Growth was not as high as we had hoped for students at the end of the year but growth was still evident in almost all subject areas.• Many other schools in our network have looked at our assessment plan and begun to implement a version of their own based on our work.
Driving the plan for NYS• Based on the work in 2011 – 2012 on our assessment plan, the NYC DOE Department of Assessment asked us to spearhead a the small scale pilot program to guide the development of; – Implementation on a school level – Assessment driven instructional model – Assessment models
Redirecting my focus• During the 2011 – 2012 school year I spearheaded the assessment development plan.• However my focus needed to be more of delegation rather then development in 2012 – 2013. Therefore; – The ELA coach took over the responsibility to lead the focus team (Grade 6) – The Math coach took over the development of all assessments outside of ELA – Teachers have became trained in assessment development – The Department of Assessment has taken a role in managing the pilot• My role has become more of a management on rather then the lead figure.• The shift has allowed me to focus my time in other aspects of running a school in which I needed including resources and operations. – This includes resources and operations and personal leadership
Additional changes for 2012 - 2013• Three assessments will be administered (September, January, and May) under DYO• Previously unused assessments like Acuity and iXL are being phased out to focus on more specific data.• Teacher teams will meet once a week to specifically look at the data of the students to drive instruction.• Student goal setting based on the outcome of the assessments both formative and summative.
My thoughts on assessment before• Assessment was just something that teachers did in the classroom.• The plan-assess-adjust cycle was only obtainable on a small scale – not school wide. – Assessment development was something that came easily.• Goal setting was not necessarily assessment driven.
My thoughts on assessment after• Prior to this process I did not fully have an understanding of the power of assessment.• Developing and implementing assessment is more difficult of a task then it looks. – A lot of factors can sway the outcome of an assessment. – The way in which you assess also has to be carefully considered.• Goals when grounded in assessment driven data can move instruction.• My work on assessment this year will definitely be fundamental as I move forward in leadership.
Reflecting Back• A school leader uses data to set high learning goals and increase student achievement. This effective use of data is demonstrated in the following elements – Demonstrates ability to understand and analyze data from multiple sources. – Uses data to identify student learning trends, set goals, monitor and modify instruction, and increase student achievement. – Develops school culture and practices that rely on data to inform adult learning, professional development, and decision-making.
Reflecting Back Continued • Triangulation of data is necessary to ensure accuracy. • Without accurate data goal setting, modifications to the curriculum and ultimately student achievement is not possible. • Data and subsequent analysis should be the foundation for all professional and adult development.
Overcoming Adversity • I continue to struggle with embracing my disability. • My fear of numbers is something I have overcome. • I have found ways in which I can begin to delegate more; – I do not do everything myself. – I develop plans to oversee and manage rather then do it alone.
If the only tool you have is a hammer, you tend to see every problem as a nail. Abraham Maslow