Recommended
More Related Content
Featured
Featured (20)
Authorizer Accountability Systems: Evaluating Oversight in the Growing Charter School Sector
- 1. #*#ftru*il# ffi.m.ffi nvffiffiilH ["lir.o,t gv'ffiffi*t'lYY r-:,F &i$t grl${iffiS*T'',e Deposit Agreement We represent that we are the creators of the digital material identified herein ("Work"). We represent that the Work is original and that we either own all rights of copyright or have the right to deposit the copy in a digital arshive such as the Conservancy. 'We represent that the regard to any non-original material included in the Work we have sesured written pennission of the copyright owner(s) for this use or believe this use to be allowed by low. We further represent that we have included all appropriate credits and attributions. We hereby grant to the Regents of the University of Minnesota ("University"), through its University Digital conservancy, & non- exclusive right to access, reproduce and distribute the Work, in whole or in pafi, for the purposes of security, preservation and perpetual access. We grant the University a limited non-exclusive right to make derivative works for the purpose of migrating the Work to other media or formats in order to preserve access to the Work. We do not transfer or intend to transfer any right of copyright or other intellectual properfy to the University, Work to bq deposited: Title: krfuu' tutc.fu$'- 6L,}1d Print €u-'u €&fto^"* Print fiali ,4c^htlodq'.- Print Signature Names of Authors: Signature Signatdre Signature
- 3. 1 Authorizer Accountability Systems: Evolving Evaluation and Oversight in the Charter School Sector MPP Professional Paper In Partial Fulfillment of the Master of Public Policy Degree Requirements The Hubert H. Humphrey School of Public Affairs The University of Minnesota Created in partnership with client: The National Association of Charter School Authorizers By: Emily Edstrom Aaliyah Hodge Griffin Merry Caitlin Osborn May 19, 2016 Signature below of Paper Supervisor(s) certifies successful completion of oral presentation and completion of final written version: _________________________ ____________________ ___________________ Laura Bloomberg Date, oral presentation Date, paper completion Associate Dean __________________________ ___________________ ___________________ Melissa Stone Date, oral presentation Date, paper completion Gross Family Professor of Nonprofit Management
- 7. 5 these schools will have a charter school authorizer to which they report, as mandated in state statute. A charter school authorizer, or “sponsor”, is an entity or group that has either been approved by the state or given statutory authority to oversee a portfolio of charter schools. For clarity, we will refer to these entities as authorizers throughout this report. The role of a charter school authorizer is to provide oversight and create accountability tools that help to promote high quality schools. In addition to this, authorizers are responsible for issuing new and renewal contracts to charter school boards, and holding them accountable to their performance as outlined in these contracts. Conversely, charter school authorizers are also responsible for closing a school or not renewing a charter contract when necessary. The authorizing landscape varies greatly throughout the country in terms of both size and number, with some authorizers overseeing a portfolio of one or two schools and others with portfolios of over a hundred. Each state has different laws surrounding who can be a charter school authorizer. As of 2016, there are six types of entities operating as authorizers in the U.S., as defined by NACSA: 1. Higher Education Institutions (HEI): There are currently 12 states where HEIs are actively operating. 2. Independent Charter Boards (ICB): “Also known as “commissions” or “institutes,” these are statewide bodies that have been set up [by the state] for the sole purpose of awarding charters and overseeing charter schools” (NACSA, 2016). Currently, there are 15 states where ICBs are actively operating. 3. Local Education Agencies (LEA): “Usually a local or county-wide district whose school board is the literal “authorizer” since it makes final decisions” (NACSA, 2016). This type of authorizer is most popular and there are 31 states where LEAs are actively operating. 4. Non-Educational Government Entities (NEG): “Mayors and municipalities that serve as the authorizer.” (NACSA, 2016) There are currently only 3 states where NEGs are actively operating. 5. Not-For-Profit Organizations (NFP): There are currently only 2 states where NFPs are actively operating. 6. State Education Agencies (SEA): “Typically housed in a state’s department of education” (NACSA, 2016). There are currently 17 states where SEAs are actively operating.
- 9. 7 Background Since the first charter law was enacted in 1991 there have been numerous studies on charter school outcomes and effectiveness, fueling national debates in education policy (e.g. CREDO, 2009). There have been fewer studies, however, looking at authorizers and their potential impacts on student outcomes. Many new authorizer accountability laws are just starting to take effect and researchers are beginning to turn their focus to authorizers and their impact on charter performance. This increased awareness of authorizers and the role they play in the success of the charter sector has scholars beginning to ask questions about the quality of authorizing and how it relates to student outcomes. Preliminary studies began to appear in the early 2000s, with the first known national-scale study coming from Palmer and Gau in 2003. This study established a baseline account of the state of authorizing in the United States, looking at different state policy environments. The study utilized surveys and collected data from 24 states. Palmer and Gau found six major takeaways from the study, which included (1) poor policy support for charter schools and authorizers and (2) states with fewer authorizers serving more schools generally have better outcomes. Palmer and Gau’s later work focused on the benefits and drawbacks to different kinds of authorizers and how these results might impact state policy (Palmer & Gau, 2005), a characterization of the different types of authorizers in the country (Gau, 2006), and a deeper dive into non-traditional, or alternative types of authorizers (Palmer, 2007). In the 2005 study, they recognized that more information was still needed about the authorizer environment and
- 14. 12 Table 1: Visual Map of State AAS Based on System Attributes Authorizer Practices Student Outcomes Both Sanctions Alabama Missouri Arizona Hawaii Indiana Louisiana Maine Nevada Incentives Both Minnesota Ohio None Washington D.C. Georgia Mississippi Upon review of findings from the literature review, and analysis of Table 1 trends, our team made recommendations for states that would serve as valuable case studies and candidates for further qualitative analysis through the use of interviews. With NACSA’s recommendations, our study narrowed the field to four states: Alabama, Indiana, Ohio, and Washington D.C. These systems were variable enough to give a unique spectrum of accountability system attributes for study (see Appendix B). After deciding on the four case study states, the research team constructed a general interview guide with concern given to both research questions and anomalies found in the literature review. For each of the four chosen states, state-specific questions were constructed and added to the general interview guide. Due to time constraints and availability of interviewees, the number of interviews per state varied. All interviews were conducted by phone, with one for Alabama, three for Indiana, two for Ohio, and one for Washington D.C. All but one of the interviewees were locally operating authorizers, the outlier being a member of a statewide charter policy and advocacy network who was included because they could provide
- 15. 13 perspective on previous authorizing experience. Table 2 is a demographic chart, further illustrating our pool of interviewees. On the Y-axis, interviewees are represented by a letter corresponding to their state, and a number corresponding to the order in which they were interviewed. We asked interviewees if their organization had undergone the evaluation process, which is shown in the second half of the table. Table 2: Demographic Profile of Interviewees Job Role Undergone Evaluation Authorizer State DOE 3rd Party Yes No N/A A-1 X X W-1 X X O-1* X X O-2 X X I-1 X X I-2 X X I-3 X X *O-1 interviewee is currently working for a third party charter advocate, though the information they provided was mainly from their previous experience as an authorizer. The study team analyzed interview data with two goals in mind. First, we sought to determine the extent to which interview data corroborated our previous legal/statute review. This proved helpful, as a number of states possess accountability systems which operate differently in practice then it would appear from a strictly legal reading of statute. Second, the interviews allowed for collection of qualitative data, allowing authorizers to not only offer descriptions of how accountability systems work in their respective states, but also their personal and professional assessments of various aspects of the AAS. The remainder of our primary report uses interview data for corroboration of findings from the legal literature
- 23. 21 Every state with an AAS has an implementation authority established in the government of each state. The level of involvement with the evaluation, however, varies greatly. For instance, statutorily Washington, D.C.’s only authorizer is to be evaluated every 5 years by the Comptroller General of the United States. The last report like this was generated in 2011 and focused mainly on authorizer practice. There are no sanctions or incentives tied to this evaluation, and the interviewee made it clear that this evaluation had little bearing on the authorizer’s work. In contrast with this hands-off approach, the Alabama State Department of Education plans to evaluate and work with authorizers annually as the charter sector starts to establish itself in the state. The interviews did not shed significant light on the AAS implementation body. The main related concern was the expertise of the individuals involved in authorizer evaluation, be they in the government or from a third-party. Specifically, individuals who have not worked as authorizers are making critical decisions that affect the work with unintended consequences. The mismatch between the task of evaluating authorizers and the expertise of those set with creating and implementing said evaluation was brought up in multiple interviews. All of this to say that while we initially looked for commentary on who generally has authority to implement the evaluation, we uncovered widespread concern about lack of expertise within those bodies. Evaluation Focus A crucial difference between systems involves the evaluation’s main focus. In some states, evaluations are aimed only at authorizer practices, such as adherence to nationally recognized principles and standards. While there are some states, such as Missouri, that have
- 24. 22 state mandated practices and principles, this study is focused on nationally recognized standards. Of the thirteen states reviewed, four had systems that focused exclusively on authorizer practices. These include Alabama, Minnesota, Missouri, and Washington, D.C. An alternative method, which is only taking place in Georgia, is to focus an evaluation only on student outcomes, such as proficiency and growth measures, or graduation rates. The most common focus, occurring in eight of thirteen states, is a mix of both authorizer practices and student outcomes. For some states student outcomes are one measure in a larger rubric or report more focused on authorizer practices, such as Hawaii’s annual report. In other states, however, practices of authorizers are one part of an evaluation that hinges on student outcomes, such as Louisiana. This information is illustrated in Table 4 below. Table 4: Evaluation Focus Authorizer Practices Student Outcomes Both Alabama Missouri Minnesota Washington D.C. Georgia Arizona Hawaii Indiana Mississippi Louisiana Maine Nevada Ohio A key component of our analysis was to investigate instances in which two related attributes were found to both be present. For example, we assessed how often those states that include authorizer practices as an evaluation focus also include a requirement for (and definition of) national best practices. Of the 12 states that include authorizer practices in their evaluations (with eight assessing both authorizer practices and student outcomes, and four assessing only authorizer practices), nine of those twelve states require authorizers to adhere to nationally recognized practices. Interestingly, two of the three states that do not require adherence to nationally recognized practices also belong to the category of only focusing
- 25. 23 evaluations on authorizer practices. This is a problematic and counterintuitive finding that begs the question: Should a system that only assesses authorizer practices not include a requirement for authorizers to adhere to nationally recognized standards? This is concerning and may hamper the ability of authorizers to fully understand how they will be evaluated with regard to authorizer practices. Sanctions/Incentives In addition to the focus of the evaluation, a key attribute is how evaluation mandates are enforced. The divisions among states were interesting for this category. Eight of 13 states use only sanctions. We should note here that of those eight, five had only one sanction, which was outright revocation of authorizing authority. Interestingly, none of the states our team reviewed were found to employ only incentives and only two states employed both sanctions and incentives. Perhaps the most interesting finding with regard to incentives and sanctions is the fact that three of 13 states did not clearly outline evaluation consequences at all. None of the statutes for Georgia, Mississippi, or Washington, D.C. explicitly outline possible sanctions or incentives that would result from an evaluation. Of these three states, Mississippi and Washington, D.C. are both states in which there exists only one governmental authorizing body. While both of these states have an accountability system, the lack of defined sanctions and/or incentives may present problems for authorizer accountability, especially within the context of a single governmental authorizing body. For example, one interviewee from D.C. stated, "We are aware of the statute but are not sure if this will be done and have no record of
- 26. 24 communication about it being done this year." This is cause for some concern as, in essence, these bodies are essentially not accountable to any external authority. While it seems clear that revocation is impossible in a single-authorizer state, revocation is not the only form of sanction, which can be seen in other state systems. Both sanctions and incentives exist on a spectrum. Ten of 12 states had some sort of sanction included in the evaluation, but for five of these states the “sanction” did not increase in stages, but was instead an outright revocation of authorizing authority. Similarly, incentives ranged from fast-tracking official affidavits to permission to expand portfolios beyond standard geographical limits. Moreover, it was noted in one interview that while incentives exist, they are sometimes so difficult to attain that in practice the evaluation system operates only as a sanction-driven evaluation. This information is illustrated in Table 5 below. Table 5: Sanctions and Incentives Sanctions Incentives Both None Alabama Missouri Arizona Nevada Hawaii Indiana Louisiana Maine Minnesota Ohio Washington D.C. Georgia Mississippi Another consideration is the political climate or feasibility of various sanctions and incentives. In two separate interviews (both from the same state), it was noted that incentives were politically infeasible, with one interviewee noting that, “It is not the state board’s prerogative to offer incentives. Authorizers should be [performing] because of their mission, not because of incentives.” This illustrates that the presence of sanctions or incentives is often defined by the unique political landscape of each state.
- 27. 25 Implementation Cycle As a part of the investigation into AAS, we found that the timeline on which the evaluations are conducted varies somewhat widely, ranging from annually to every 5 years, with the possibility of intervention in one state being possible at any time (continuous). In the case of the 13 states included in this study, the timelines are as follows (Alabama is not included in this breakdown, as the timeline has not yet been defined in statute): ● Continuous: 1 ● Annual: 6 ● Every 3 years: 3 ● Every 5 years: 2 The timeline of the evaluation was important in thinking about job impact, specifically how burdensome the evaluation was perceived to be for the authorizing entity itself. For smaller authorizers, frequent evaluations were challenging, as the time it takes to gather all materials is very great. One interviewee said that an annual review, especially for high- performing or highly rated authorizers, seemed too much. In their opinion, a “temperature check” for high-performing authorizers every three to five years would be enough. This illustrates that there is a variation in the burden of time, resources, etc. expended on completing evaluations within the AAS. Conclusions This report has provided an introductory glimpse into the similarities and differences present across 13 state AAS. While the analysis has discovered initial trends, we cannot state with certainty which commonalities across systems lead to effective authorizing or improved student outcomes. This results from the fact that a number of systems are either too new or
- 28. 26 have just begun implementation. The frequency with which AAS are updated and changed, as a result of political climate and state’s continuous improvement efforts, make it difficult to establish causality between AAS features and student outcomes. Given the difficulties expressed herein we urge practitioners and policy makers to strongly consider the emerging trends presented in our addendum, titled Further Considerations. Due to the preliminary nature of the analysis in this report, we propose the following recommendations for further research: • The interview sample in this report was regrettably limited. Speaking with or surveying a larger sample of authorizers in addition to a wider range of stakeholders, such as non-authorizers, state legislators, Department of Education personnel, general educators, and school administrators, will widen the scope of the research and provide increased validity to any conclusions. • The most common evaluation focus, occurring in eight of thirteen states, is a mix of both authorizer practices and student outcomes. This indicates a belief in these states that a mix of both improves student outcomes though more specific research must be done to confirm this hypothesis. • There are concerning relationships between national best practice requirements and definition those practices. Only three of the thirteen states require and clearly define national best practices for authorizers. We recommend further research on the specificity, or lack thereof, with regard to requiring and defining national best practices in statute.