An Analysis of Quality Review          Alignment with Selected            Governance Models                          Conne...
Agenda• Overview of Connecticut Quality Review project• Design choices made to align Connecticut Quality Review  with expe...
The team supported the Connecticut State Department of Educationby designing standards, tools, and a strategic process for...
During the QR process, reviewers will assess schools against fouroverarching standards. Overview of Standard              ...
These standards are aligned to 17 indicators of school practice.      Standard               Indicator (To what extent do ...
These standards are aligned to 17 indicators of school practice.       Standard              Indicator (To what extent do ...
The Quality Review process involves pre-review data collection, the    2-day onsite review, and post-review feedback.     ...
The pilot will include two teams of reviewers conducting quality     reviews in 4 East Hartford schools in April/May.     ...
The QR pilot is designed to accomplish three key objectives.                        • Reviewers, districts, and schools wi...
Agenda• Overview of Connecticut Quality Review project• Design choices made to align Connecticut Quality Review  with expe...
CT QR was designed to inform an “experimentalist” problem-solvingprocess at the school level, by providing schools with ta...
CT QR will ensure that expertise flows from local learning to thecenter, and that the state, in turn, will accumulate and ...
Other specific design features of CT QR were also intentionally linkedto experimentalism.      Feature of CT QR          H...
Standard 4: Operational Alignment to Goals  Indicator 4.2: Monitor and Revise Operations              • Minimal process. S...
Other specific design features of CT QR were also intentionally linkedto experimentalism.      Feature of CT QR          H...
Self-Review ensures the QR process is collaborative, and empowersstakeholders by helping them engage in a structured proce...
Other specific design features of CT QR were also intentionally linkedto experimentalism.      Feature of CT QR          H...
Constituent conversations allow reviewers to gain information aboutstakeholders’ perspectives and experiences with the sch...
Agenda• Overview of Connecticut Quality Review project• Design choices made to align Connecticut Quality Review  with expe...
Tension #1: An experimentalist QR relies on autonomy at the schoollevel, but autonomy is noticeably constrained.        EX...
Tension #1 Resolution: Indicator 4.3 Managing Talent was adapted toacknowledge school-level constraints around talent mana...
Tension #2: An experimentalist QR uses broad standards that are notprescriptive, but stakeholders want clear expectations....
Tension #2 Resolution: Revised QR includes evidence document withvignette-like narratives to provide examples of “well-dev...
Agenda• Overview of Connecticut Quality Review project• Design choices made to align Connecticut Quality Review  with expe...
The CSDE faces a decision about whether to use the results of QualityReview as part of its system for school accountabilit...
Performance management suggests using QR for Accountabilitywhereas professionalism would use QR purely developmentally.   ...
The CSDE’s decision about how to use Quality Review is closely related    to its theory of action for school improvement. ...
Discussion teams    Team 1     Dara, Michael, Rohan    Team 2     Seo Yun, Tom, Zahreen, Jason    Team 3     Jill, Sana, B...
Discussion Role Play             •   Take a point of view on whether Quality Review should be                 part of Conn...
As you discuss whether QR should be a factor in school accountability, consider which use of QR is most likely to meet eac...
Report out.                     Factor in Accountability?        Purely Developmental?                 •     To come      ...
Appendix
NCLB exposed poor performance, but is blunt and limited;Connecticut is developing a richer, more nuanced system.          ...
Qualitative Review could be integrated into Connecticut’s newaccountability system for all schools.                       ...
Qualitative Review could be integrated into Connecticut’s newaccountability system as an additional component for schools ...
Qualitative Review could be integrated into Connecticut’s new accountability system as an additional requirement for exiti...
Upcoming SlideShare
Loading in …5
×

Csde final class ppt 1

249 views

Published on

Final Class Presentation - CSDE - QR Team

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
249
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Mention that, among many other things, we discussed QR in the Ct context; what purposes it has served in the past, what purposes it can serve, and how it can help Connecticut both fulfill its obligations under the ESEA waiver and develop its support and accountability system in potentially groundbreaking ways.
  • Csde final class ppt 1

    1. 1. An Analysis of Quality Review Alignment with Selected Governance Models Connecticut Quality Review TeamJamie Alter, Gabriella Barbosa, Ranjana Reddy, Ryan Thorpe, William David Williams April 8, 2013 1
    2. 2. Agenda• Overview of Connecticut Quality Review project• Design choices made to align Connecticut Quality Review with experimentalism• Challenges in designing an experimentalist Quality Review system• Examination of an unresolved question: Should CSDE use Quality Review for school accountability? 2
    3. 3. The team supported the Connecticut State Department of Educationby designing standards, tools, and a strategic process for pilotingQuality Review.Created first iteration ofQR standards, tools, and Conducted focus groups Strategically recruitedprocess based on with CSDE to solicit districts to participate inresearch and effective feedback pilotpractices Supported training of Solicited feedback on Designed Reviewer and reviewers and QR from district leader School Guides orientation for schools and district personnel 3
    4. 4. During the QR process, reviewers will assess schools against fouroverarching standards. Overview of Standard Rationale • High quality schools have an overall vision focused on a 1. Create and monitor a pathway to college and career readiness for all students that vision and theory of has generated a thoughtful theory of action, comprehensiveaction focused on student data gathering, and careful plans for implementing the theory learning of change. • High quality schools evaluate and select appropriate2. Develop and implement strategies, plans, instructional programs, and curriculum for a rigorous instructional their circumstances to support college and career readiness program that meets the for all students. learning needs of all • Similarly, they organize themselves so that school leaders students are helping to facilitate continuous improvement in educator effectiveness at meeting learning goals. • High quality schools promote a positive learning3. Create a school culture environment and engage families and the community inand climate that enables continuous improvement. all students to learn4. Align management and • High quality schools continuously monitor and improve operations to facilitate management practices to ensure the are effectively serving achievement of student student learning as the school’s primary objective learning goals and steadily improve 4
    5. 5. These standards are aligned to 17 indicators of school practice. Standard Indicator (To what extent do schools…) 1.1 Create a Shared Vision for Student Learning Student Learning- Centered Culture 1.2 Develop Theory of Action and Goals 1.3 Target Student Needs 1.4 Assess and Monitor Progress Continuously- 2.1 Develop Rigorous Curriculum Improving Instruction 2.2 Build Instructional Capacity 2.3 Develop Teacher Leadership 2.4 Promote Evidence-based Instruction
    6. 6. These standards are aligned to 17 indicators of school practice. Standard Indicator (To what extent do schools…) 3.1 Establish Welcoming Environment Student and Family Investment 3.2 Engage Teachers and Staff 3.3 Engage Students 3.4 Communicate with and Engage Families 3.5 Build Community and Family Partnerships Operational 4.1 Align Resources and Policies to Goals Alignment to Goals 4.2 Monitor and Revise Operations 4.3 Manage Talent 4.4 Develop and Pursue Innovative Solutions
    7. 7. The Quality Review process involves pre-review data collection, the 2-day onsite review, and post-review feedback. Pre-review Review Post-reviewDescription: Reviewers and schools prepare During a two-day school visit, Reviewers provide verbal and for the formal process of data reviewers collect and synthesize written feedback to schools collection by gathering evidence from a variety of based on evidence collected documents, examining school sources to gain a full picture of during the quality review and data, self-evaluating, and school performance against the facilitate a structured debrief setting logistical parameters. set of standards for effective and reflection on results. schools. Key steps: • Principal Orientation • Classroom Observations • Presentation of Findings • School Self-Evaluation • Constituent Interviews • Goal Setting and Planning • Conversation with District • Observation of Teacher Teams • Rating and Scoring Staff • Reviewer Check-ins • Written Report • Planning Conversation Among • Principal Debriefs • Verification Reviewers • Reconsideration, if necessary • Planning Conversation with School Leader
    8. 8. The pilot will include two teams of reviewers conducting quality reviews in 4 East Hartford schools in April/May. School 1 REVIEWER TEAMS SCHOOLS• Reviewer Include at least 1 • Four elementary state and 1 district Team 1 schools; identified reviewer by CSDE as Focus School 2 Schools• Trained on the QR standards and • Located in Alliance process by CSDE District vendor • District has 61%• Have experience as FRL, 47% proficient educators and/or School 3 reading, 64% educational leaders Reviewer proficient math Team 2 (Grade 3 CMT) School 4
    9. 9. The QR pilot is designed to accomplish three key objectives. • Reviewers, districts, and schools will provide feedback on the pilot tools,To troubleshoot and process, and reports. improve the • This feedback will be used to revise the QR system and to informstandards, tools, and recommendations for roll-out. process • East Hartford will have an opportunity to test-drive quality review. To increase • The CSDE will conduct outreach sessions with other Alliance Districts to encourage more districts to engage in Quality Review in future years.stakeholder support for Quality Review • State reviewers will be trained to conduct reviews, evaluate schools, write reports, provide verbal feedback to schools, and to ensure inter-rater To build capacity reliability through internal training. within the state • The CSDE will develop internal procedures for collecting and analyzing data from quality reviews and managing the pre- and post- review elements of the process. • Personnel from East Hartford will gain experience as co-reviewers. 9
    10. 10. Agenda• Overview of Connecticut Quality Review project• Design choices made to align Connecticut Quality Review with experimentalism• Challenges in designing an experimentalist Quality Review system• Examination of an unresolved question: Should CSDE use Quality Review for school accountability? 10
    11. 11. CT QR was designed to inform an “experimentalist” problem-solvingprocess at the school level, by providing schools with targetedfeedback on areas of improvement Clarify and Translate Visions and Strategy SCHOOL Communicate and Link PROBLEM Enhance Strategic Strategic Objectives SOLVING FOR Feedback and Learning and Measures IMPROVEMENT Plan, Set Targets and Align Strategic Initiatives 11
    12. 12. CT QR will ensure that expertise flows from local learning to thecenter, and that the state, in turn, will accumulate and share locallearning across schools. SCHOOL SCHOOL State Knowledge Base of Effective Practices SCHOOL SCHOOL
    13. 13. Other specific design features of CT QR were also intentionally linkedto experimentalism. Feature of CT QR How it Meets an Experimentalist Objective The system is built around standards that exemplify effective schools, while being broad Strategic, Open-Ended enough to generate fruitful conversation and Standards learning among schools operating in different contexts. Processes such as self-evaluation, principal check-in, and goal setting meetings Involvement of School empower local school leaders to be a critical Throughout Process part of monitoring and adjusting their own performance. Experimentalism allows for a diverse group of Involvement of Other stakeholders to be included as part of an Stakeholders accountability system, including those usually left out of the conversation like parents and the community. 13
    14. 14. Standard 4: Operational Alignment to Goals Indicator 4.2: Monitor and Revise Operations • Minimal process. School does not have in place any coherent system for regularly monitoring the effects of school operations and management systems on student learning Minimal or outcomes.no evidence • Limited process. School has a plan for monitoring effectiveness and efficiency ofProgressing operations and management systems for their effects on student learning outcomes, but toward inconsistently uses the information learned from this monitoring. standard • Strategic process. School effectively and strategically plans and executes systems to regularly monitor and revise effectiveness and efficiency of operations and management Meets systems to improve their impact on student learning outcomes. standard • Strategic process and continuous improvement. School effectively and strategically plans and executes systems to regularly monitor and revise effectiveness and efficiency of Exceeds operations and management systems to improve their impact on student learning standard outcomes. 14
    15. 15. Other specific design features of CT QR were also intentionally linkedto experimentalism. Feature of CT QR How it Meets an Experimentalist Objective The system is built around standards that exemplify effective schools, while being broad Strategic, Open-Ended enough to generate fruitful conversation and Standards learning among schools operating in different contexts. Processes such as self-evaluation, principal check-in, and goal setting meetings Involvement of School empower local school leaders to be a critical Throughout Process part of monitoring and adjusting their own performance. Experimentalism allows for a diverse group of Involvement of Other stakeholders to be included as part of an Stakeholders accountability system, including those usually left out of the conversation like parents and the community. 15
    16. 16. Self-Review ensures the QR process is collaborative, and empowersstakeholders by helping them engage in a structured process ofreflection on strengths and weaknesses. Plan: • What are our goals? • How will we achieve them? • What support do we need? Reflect: Execute: • How do we put our plan into • How are we doing? action? • How do we know? • What more needs to happen • How did we get here? to make the plan a reality? Monitor: Collaborate: • What progress are we • With whom do we need to making toward our goals? partner to support our • How well are we aligning goals? resources to achieving our • Who needs to know about goals? our goals and progress? 16
    17. 17. Other specific design features of CT QR were also intentionally linkedto experimentalism. Feature of CT QR How it Meets an Experimentalist Objective The system is built around standards that exemplify effective schools, while being broad Strategic, Open-Ended enough to generate fruitful conversation and Standards learning among schools operating in different contexts. Processes such as self-evaluation, principal check-in, and goal setting meetings Involvement of School empower local school leaders to be a critical Throughout Process part of monitoring and adjusting their own performance. Experimentalism allows for a diverse group of Involvement of Other stakeholders to be included as part of an Stakeholders accountability system, including those usually left out of the conversation like parents and students. 17
    18. 18. Constituent conversations allow reviewers to gain information aboutstakeholders’ perspectives and experiences with the school acrossmultiple indicators. Focus Group with Teachers Focus Group with Parents Focus Group with Students 18
    19. 19. Agenda• Overview of Connecticut Quality Review project• Design choices made to align Connecticut Quality Review with experimentalism• Challenges in designing an experimentalist Quality Review system• Examination of an unresolved question: Should CSDE use Quality Review for school accountability? 19
    20. 20. Tension #1: An experimentalist QR relies on autonomy at the schoollevel, but autonomy is noticeably constrained. EXPERIMENTALIST BUREAUCRATIC PROPOSAL REALITY Schools should have the Schools have limited discretion to develop flexibility in allocating context-specific strategies financial resources and to close gaps identified by guiding talent the QR system management constrain their ability to make changes advocated by QR RESOLUTION By working with the state and districts to design and implement QR, we ultimately hope to build buy-in for necessary structural changes at the school level. Essentially, states and districts will see that schools have internal capacity to drive change and that they need additional autonomy to facilitate improvement through the QR process.
    21. 21. Tension #1 Resolution: Indicator 4.3 Managing Talent was adapted toacknowledge school-level constraints around talent management. Original Indicator Adapted IndicatorDevelop and implement a strategy for Develop and implement a strategy formanaging human capital, including marshaling resources to manage humanidentifying, recruiting, mentoring, capital, including identifying, recruiting,providing career-enhancing mentoring, providing career-enhancingopportunities for, and retaining the opportunities for, and retaining the mostmost effective teachers and assigning effective teachers and assigning teachersteachers to areas where their skills to areas where their skills best matchbest match student learning needs. student learning needs, given each individual school’s contextual constraints. 21
    22. 22. Tension #2: An experimentalist QR uses broad standards that are notprescriptive, but stakeholders want clear expectations. EXPERIMENTALIST BUREAUCRATIC PROPOSAL REALITY CPRL proposed a rubric with The CSDE and many school intentionally broad standards and district personnel believe to encourage a dialogue that providing specific between reviewers and standards with concrete school personnel and to resist expectations for acceptable the tool’s use as a compliance evidence ensures the process mechanism. is fair and best communicates expectations to stakeholders. RESOLUTION The version of QR being piloted includes a rubric with broad, experimentalist standards and an evidence document with narrative-style vignettes as examples of what "well-developed" looks like for each indicator.
    23. 23. Tension #2 Resolution: Revised QR includes evidence document withvignette-like narratives to provide examples of “well-developed.”Original version included only indicators Adapted version added vignette-style narratives “The school effectively conveys high expectations toIndicator 1.1 students and parents that result in a deep commitment toCreate a widely shared vision of the the school’s vision. Success is celebrated during “Honorsprogress students are expected to Night” where teachers, parents and students are rewarded for exemplary work. The school provides parents withmake while at the school to get and continuous information on how well their children arestay on track to graduate on time learning via report cards, progress reports and Engrade, anand become college and career- on-line reporting system. In an effort to better support students and effectively work with families, the schoolready and foster a deeply embedded administers a student survey that aids the school to betterculture of high expectations understand the circumstances of different families andcommitted to realizing that vision. differentiate support accordingly. Guidance counselors work closely with families and actively assist in the completion of high school applications ensuring that students and families make good choices when selecting high schools. The emphasis on high standards and open exchange of information results in an atmosphere of care and support. Consequently, parents report appreciating the high expectations for attendance, good citizenship and academic effort within a highly inclusive and welcoming environment.”
    24. 24. Agenda• Overview of Connecticut Quality Review project• Design choices made to align Connecticut Quality Review with experimentalism• Challenges in designing an experimentalist Quality Review system• Examination of an unresolved question: Should CSDE use Quality Review for school accountability? 24
    25. 25. The CSDE faces a decision about whether to use the results of QualityReview as part of its system for school accountability. School ratings on the qualitative indicators would be integrated with the quantitative metrics Factor in School currently used (state test scores and graduation Accountability rates) to classify schools and to determine which schools require intervention. Schools would continue to be classified based on Purely only quantitative metrics; Quality Review ratings Developmental will be provided only to school and district leaders, who will be encouraged to use the information to drive their own improvement.
    26. 26. Performance management suggests using QR for Accountabilitywhereas professionalism would use QR purely developmentally. QR as a Factor in Performance Management School Experimentalism Accountability QR as a Purely Professionalism Experimentalism Developmental Tool
    27. 27. The CSDE’s decision about how to use Quality Review is closely related to its theory of action for school improvement. If the CSDE believes… Then…Systems of school Schools are likely toaccountability Teachers and school respond to QR forshould…. leaders…. stakes by… QR should be…• be fair, valid, include • are motivated by fair • taking the standards both leading and and valid more seriously and Factor in lagging indicators, assessments of working harder to student and provide schools performance that are authentically accountability with actionable public and result in improve information sanctions and interventions• help the state • are professionals • “gaming” the quality identify schools that who, when provided review by prepping Purely need the most with information for the review and Developmental support and autonomy, are performing on the motivated to day of the site visit improve without sanctions
    28. 28. Discussion teams Team 1 Dara, Michael, Rohan Team 2 Seo Yun, Tom, Zahreen, Jason Team 3 Jill, Sana, Brian, Jason Team 4 Regina, Alia, Matthew Team 5 New Haven
    29. 29. Discussion Role Play • Take a point of view on whether Quality Review should be part of Connecticut’s school accountability system and why Task • If so, how would this work? – Publically reported – Factor in classifying schools – Factor in determining whether schools are closed or turned around • Split into groups of three or four and take the following roles: Roles – Person who is youngest will represent a Teacher in the School – Person who is second youngest will represent the Leader of the School – Person who is third youngest will represent the District Superintendent – Person who is the oldest will represent the State Commissioner • Each party should advocate for the use quality review that is best from the assigned perspective. • Note that each entity likely has a different viewpoint regarding the purpose for conducting of quality review as well as which use of quality review best serves the stated purpose.
    30. 30. As you discuss whether QR should be a factor in school accountability, consider which use of QR is most likely to meet each of QR’s primary objectives. Part of Purely Acct. Dev. ExplanationMeets primary objectives Improve accountability system Create a common language around school quality Build capacity in districts and state Build support among educators and the public Encourage and enable authentic school improvement Likely UnlikelySource: Interviews with districts and vendors. 30
    31. 31. Report out. Factor in Accountability? Purely Developmental? • To come • To come Teacher • To come • To come Principal • To come • To comeSuperintendent • To come • To come StateCommissioner
    32. 32. Appendix
    33. 33. NCLB exposed poor performance, but is blunt and limited;Connecticut is developing a richer, more nuanced system. NCLB A richer system Use status-based Use rich accountability and accountability and tiered classification to enable identification of “schools in and motivate schools to need of improvement” to improvemotivate schools to improve Support low performing schools in the process of strategic planning and intervention Give targeted low performing schools the flexibility and resources they need to improve Provide all schools with tools and information needed to facilitate improvementImpose sanctions if schools do Impose sanctions if schools do not improve not improve 33
    34. 34. Qualitative Review could be integrated into Connecticut’s newaccountability system for all schools. Quantitative Factors 64 < SPI < 88; 64 < SPI < 88; SPI > 88; SPI < 64 did not meet met annual met state goals annual targets targets Well Transition Progressing Excelling Excelling DevelopedQualitative Factors Proficient Transition Transition Progressing Excelling Developing Review Transition Transition Progressing Under- Review Review Transition Transition developed 34
    35. 35. Qualitative Review could be integrated into Connecticut’s newaccountability system as an additional component for schools on thecusp. Excelling Schools on the cusp of Excelling Schools receive QR forProgressing next three years Schools on the cusp If score Proficient or of Progressing Well-Developed, will be classified in the Transition higher category even if quantitative data Schools on the cusp falls short by slight of Transition margin. Review 35
    36. 36. Qualitative Review could be integrated into Connecticut’s new accountability system as an additional requirement for exiting Focus and Turnaround status. Meet subgroup SPI and grad rateLowest subgroup targetsSPI or grad rates Focus Status Achieve Proficient or Well-Developed QR Score Meet SPI and grad Lowest SPI for rate targets “all students” Turnaround Status Achieve Proficient or Well-Developed QR Score 36

    ×