The presentation discusses new approaches to framing accountability to communities. It proposes that accountability involves a dialogue between stakeholders and school leaders about goals and performance. The presentation emphasizes transparency about both successes and failures, and communicating strategies for improvement. It observes that some accountability reports focus too much on scoring schools rather than informing stakeholders. The presentation suggests accountability reports could better address equity issues, inclusion of leading indicators, and evidence that poor results lead to changes.

1. Welcome!
http://www.slideshare.net/JFCronin/new-ways-to-
think-about-framing-accountability-to-your-
community-63339505
This presentation can be accessed at:

2. New ways to think about framing
accountability to your community.
John Cronin, Ph.D.
Senior Director of Education Research
NWEA

3. What you will learn in the next
45 minutes
• A proposed meaning for accountability
• The value of transparency.
• A different approach to metrics.
• Observations based on my review of
available community accountability
reports.

4. Accountability is NOT simply about
meeting targets!!

5. What is accountability?
Accountability is a dialogue between the stakeholders
and the leaders of their schools. Part of that dialogue is
understanding and aligning the goals and objectives of
the parents, the schools, and the larger community.
Another part of that dialogue is discussing how your
schools are doing in reaching those goals.

6. What is accountability?
To be “answerable” means that accountability is a
dialogue between the stakeholders and the leaders of
their schools. One part of that dialogue is understanding
the goals and objectives of the community. Another part
of that dialogue is discussing how your schools are doing
in reaching those goals.
The most important part of the dialogue is what you’re
doing to improve performance based on this
information, which is leading.

7. The lesson from the Atlanta cheating scandal

8. Source: Aviv, R (2014, July 21). Wrong Answer. The New Yorker. Retrieved on June 16, 2016
from http://www.newyorker.com/magazine/2014/07/21/wrong-answer
“After more than two thousand interviews, the investigators
concluded that forty-four schools had cheated and that a
“culture of fear, intimidation and retaliation has infested the
district, allowing cheating—at all levels—to go unchecked for
years.” They wrote that data had been “used as an abusive
and cruel weapon to embarrass and punish.” Several
teachers had been told that they had a choice: either make
targets or be placed on a Performance Development Plan,
which was often a precursor to termination. At one
elementary school, during a faculty meeting, a principal
forced a teacher whose students had tested poorly to crawl
under the table.”

9. Differences in fall-spring test durations
An illustration of gaming
15%
25%
60%
Mathematics
Spring < Fall Spring = Fall Spring > Fall
0.0
1.0
2.0
3.0
4.0
5.0
6.0
Spring < Fall Spring = Fall Spring > Fall
GrowthIndex
Mathematics
Differences in growth index score based
on fall-spring test durations

10. The four principles that guide your
communications to your audiences.
• Clearly identify who your audience is.
• Understand what that audience wants to know
about your schools and their performance.
• Be transparent about how you are performing on
these objectives.
• Communicate your strategy to improve
performance, and corrective action when strategy
fails.

11. The community-based
accountability experiment

12. A simple process
• Establish processes to involve the
community in setting accountability goals
as an alternative or complement to ESSA
objectives
• Use these objectives to drive local
performance.
• Report performance and progress on
those goals annually.

13. The annual report contains two
parts, a management letter, and
indicators.

14. The management letter is…
Your narrative. It is the centerpiece of the
accountability report.
The report on indicators is…
Like financials in an annual report. They are a
common set of facts (mastered by all leaders)
that are the basis for discussing student
achievement. They demonstrate transparency
and, if well presented, allow readers to make
their own judgments

15. Part 1 - The management letter…
Is an opportunity to show leadership.
It is a conversation with your stakeholders about the
performance of the school system. In it you explain
your business, your successes and failures, and
discuss your intended actions for improving the
educational experience of students.

16. The tone of the management
letter
• Should be optimistic
• Should be plain yet thoughtful
• Should be conversational
• Should be realistic in addressing successes and
transparent about problems
• Should be effusive in praising of subordinates,
critical of yourself.

17. The management letter…
Is not simply for the community…
It requires the district’s leader(s) to think
seriously about your work and communicate
coherently about the state of the district and
your strategy. It forces you to reflect and clarify
your thinking.

18. • Be transparent about failures
• Have a strategy in place to address it
• Accept responsibility and temper expectations
• Demonstrate progress
How to be transparent without getting killed

19. Our bad news from 2014 comes from (Burlington-Northern Santa Fe) and is
unrelated to earnings. During the year, BNSF disappointed many of its
customers. These shippers depend on us, and service failures can badly hurt
their businesses. (transparency about failure)
BNSF is, by far, Berkshire’s most important non-insurance subsidiary and, to
improve its performance, we will spend $6 billion on plant and equipment in
2015. That sum is nearly 50% more than any other railroad has spent in a
single year and is a truly extraordinary amount, whether compared to
revenues, earnings or depreciation charges. (our strategy to address failure)
An illustration of good tone from the
Berkshire Hathaway Management Letter

20. Though weather, which was particularly severe last year, will always cause
railroads a variety of operating problems, our responsibility is to do whatever it
takes to restore our service to industry-leading levels. That can’t be done
overnight: The extensive work required to increase system capacity sometimes
disrupts operations while it is underway. (recognizing responsibility and
tempering expectations)
Recently, however, our outsized expenditures are beginning to show results.
During the last three months, BNSF’s performance metrics have materially
improved from last year’s figures. (reporting progress)
From the Berkshire Hathaway Management
Letter

21. One of our commitments is to help more students graduate prepared for college.
Advance Placement is a key piece of this strategy and our goals were to increase
participation in the program, increase the number of students sitting for AP tests,
and improving average scores on AP exams.
Our data show that we are doing well on two of the three. In the past three years
we’ve increased the number of students participating in the program by 20%
across our high schools, and the average scores on the exams have generally
improved. In English for example, the average score improved from 3.5 to 3.7 and
In Calculus AB the average score improved from 3.1 to 3.5. That’s impressive
progress and all credit goes to Marsha White, who led this initiative and the
teachers at Wilson and Jefferson high schools who made this happen. (crediting
others for success)
Unfortunately, the storyline isn’t perfect. While more students are taking the
courses, the improvement in the number of students sitting for the tests has not
kept pace. (Admit failure)
How this would read in a community-based
accountability management letter

22. You know from news reports that the costs of poor performance on the state pension
fund have passed to schools. This has stressed our budget and one consequence was
that we froze hiring at the high schools. Because of retirements and transfers at
some of the high schools, we did not replace AP teachers who retired or transferred
and those courses were dropped. I should’ve seen that problem coming (accepting
responsibility and tempering expectations).
This year our principals reviewed teacher assignments and are reinstituting courses
in areas where we have a teacher with appropriate credentials. Because those
teachers will be dealing with new content and larger class sizes, we’re funding
summer training for them in both the AP content and in teaching strategies that
better enable adaption of instruction with larger student groups (demonstrating
progress).
We can’t promise that this will entirely solve the problem, and our pass rate on AP
exams may take a hit while our new AP teachers master their assignment. I’ll keep
you apprised in my monthly newsletter as to how this is going.
How this would read in a community-based
accountability management letter

23. Observations from the review of data in
community-based accountability reports

24. Everyone is entitled to their opinion, but
not to their own facts.
Daniel Patrick Moynihan

25. Your annual report should be your “almanac”
when it comes to student achievement facts.
Any member of your leadership team who
represents educational data should be
thoroughly familiar with it.
In any data discussion, the discussion should
begin with agreement on what the facts actually
are.

26. Learnings from reviews of
annual reports

27. Accomplishments!
• The reports are data rich!
• They are reflect a broader picture of schools than
standardized tests and school report cards.
• They report on aspects of education that the
community values.
• They provide measurements of performance.

28. The volume of data in the reports is
overwhelming
Suggestion – The management letter should bring
focus. Make sure every indicator reflects an
outcome that the community values.
The scorecard section of the report can be an
appendix.

29. The reports focus more on “scoring” schools
than “informing” stakeholders.
Suggestion – Establish goals rather than categories
and report whether schools have “achieved the
goal”, and whether they are “improving”, and
“accelerating”.

30. Exemplary (3
points)
Recognized (2
points)
Acceptable (1
point)
Unacceptable
(0 points)
Percentage of
students
reading at or
above grade
level in third
grade
100-90% 80-89% 70-79%
X
Less than 70%
From a community-based accountability
report

31. Our first grade reading rate continues to be below goal and we were
alarmed by the declines in reading that we were seeing in 2009
through 2013. Two years ago, we brought in a part-time specialist to
help at-risk readers and teachers added more guided reading
practice to the schedule. The last two years show we’ve improved,
and that improvement is accelerating thanks to the hard work
kindergarten and first grade teachers. To get closer to goal this year,
we’re paying for teachers to take additional staff development and
offering release time for teachers to observe and coach each other
as they implement new practices.
How this could be communicated in a
management letter?

32. Four types of metrics:
Achievement – How are students learning
Growth – How much progress do students make
within a year
Improvement – Are we improving achievement
and/or rate of growth of students over time
Acceleration – Is the rate of change
accelerating, decelerating, going negative

33. The reports are heavy on reporting status
and light on reporting trends
Suggestion – Report your key indicators
longitudinally and focus on improvement and
acceleration/deceleration in those indicators.

36. 75% 72%
68%
60%
51% 48% 50%
40%
50%
60%
70%
80%
90%
100%
2015 2014 2013 2012 2011 2010 2009
HS Graduation Rate – Unacceptable, improving,
but improvement is decelerating

37. Grade Students Athletes % Fine Arts %
9 3276 1487 50% 2256 76%
10 3190 1215 41% 1656 56%
11 2967 932 31% 1233 42%
12 2795 703 24% 1028 35%
Percent of students participating in athletics
and fine arts programs

38. The reports generally do not address issues
related to equity.
Suggestion – Identify and include metrics in your
report that show how you are addressing the
particular needs of at-risk and minority populations
and how you are addressing achievement gaps.

39. Calculus AB 2014 2015
Enrolled %
Taking
% 3+ Enrolled % Taking % 3+
All students 347 257 74% 389 300 77%
Enrollment and student performance in AP
Calculus AB

40. Calculus AB 2012 2013
Enroll
ed
% Taking % 3+ Enrolled % Taking % 3+
All students 347 257 74% 389 300 77%
Minority
students
100 40 70% 130 50 70%
Comment –
Our goal has to improve both enrollment and minority
participation in our AP program and our schools have
accomplished that. We’re concerned that while minority
participation increased, the proportion of minority students
actually testing declined this past year. Our principals reported
that cost of the exams has been a barrier for some students, so
we’ve committed to paying the cost of the exam for all students
identified as eligible for free and reduced lunch.
Enrollment and student performance in AP
Calculus AB

41. The reports report satisfaction but show little
interest in the dissatisfied.
Suggestion – Include follow-up questions to
illuminate who and what may contribute to
dissatisfaction and discuss how you are addressing
those issues.

42. Very
Satisfied
Somewhat
Satisfied
Dissatisfied Unsure
2014 30% 55% 9% 3%
Survey Question –
How satisfied are you with the education
that the school system provides your child?

43. Very
Satisfied
Somewhat
Satisfied
Dissatisfied Unsure
2014 30% 55% 9% 3%
Under 30k 12% 60% 22% 6%
30-70k 20% 62% 15% 3%
Above 70k 40% 50% 6% 4%
Survey Question –
How satisfied are you with the education
that the school system provides your child?

44. Top five sources of dissatisfaction – high schools
parents who responded somewhat or dissatisfied
1. Fees for athletic and extracurricular
participation (15%)
2. Lack of communication/responsiveness by some
teachers (12%)
3. Cutbacks in AP course availability (8%)
4. Poor teaching (6%)
5. Elimination of jazz choir program (4%)

45. The reports lack leading indicators that are
predictive of success at the next level.
Example – The report will include dropout rates, but
lacks reporting on elementary and middle school
indicators that are empirically tied to this metric and
would be key to prevention.

46. The reports show little evidence that a poor
result leads to some decision to change it.

47. Florida
District
Highly
Effective
Effective Needs
Improvement
Developing Unsatisfactory VA Score Florida
Ranking
1 44.4% 55.6% 0.0% 0.0% 0.0% 0.39 109
2 25.0% 75.0% 0.0% 0.0% 0.0% 0.37 121
3 90.9% 9.1% 0.0% 0.0% 0.0% -0.14 2802
4 60.7% 39.3% 0.0% 0.0% 0.0% -0.14 2797
5 81.2% 18.8% 0.0% 0.0% 0.0% -0.16 2831
6 37.3% 54.2% 1.7% 0.0% 6.8% 0.12 880
7 81.3% 18.8% 0.0% 0.0% 0.0% 0.22 402
8 41.7% 55.6% 1.4% 1.4% 0.0% -0.34 3274
9 52.2% 47.8% 0.0% 0.0% 0.0% 0.16 664
10 27.0% 66.2% 1.4% 0.0% 5.4% 0 1764
11 7.1% 72.6% 9.5% 10.7% 0.0% -0.08 2445
Teacher Evaluation Ratings in Eleven Florida
Schools 2013

48. Thank you!
E-mail: john.Cronin@nwea.org
Organization Website: www.nwea.org
Contact: