This document discusses the need for performance management systems rather than just data-driven decision making. It argues that decision-makers need a knowledge base that integrates different categories of data, interprets the data using benchmarks, and includes process data to help identify better practices. An effective knowledge base would provide tools to analyze process data in order to improve performance, not just audit it. It provides examples of the types of data that could be included in a knowledge base to help decision-makers enhance student outcomes and return on resources.

1. SchoolMatters.com
A Service of Standard & Poor’s
School Evaluation Services (SES)
Michael Stewart
Director of Research & Analytics
February 3, 2006

2. School improvement efforts have focused on data-driven decision-
making, and the essential need for better databases.
But policy-makers don’t just need data-driven decision-making
systems; they need performance management systems.
What’s the difference between data bases and knowledge bases?
Most databases include one or more broad categories of data:
• Input data – such as financial and human resources
• Contextual data – such such as demographics
• Outcome data – such such as student achievement
These are often stored in fragmented - not integrated - systems.
Decision-Makers Need a Knowledge Base,
Not Just a Better Data Base

3. A Knowledge Base:
• Integrates different categories of data into a unified system
• Interprets the data using performance benchmarks
• Includes, but goes beyond, input, output, and demographic
data, by addressing the need for process data
• Uses a data structure designed to address policy questions
Decision-makers can’t manage their way to better performance if
they know their inputs and outcomes without knowing where to
find better processes.
By providing tools to “mine” process data, a knowledge base can
be used to improve – not merely audit – performance.
This isn’t a call to reinvent the wheel, but to integrate the
components already available in order to build a knowledge base.

4. Student Performance
• State test scores, participation rates, and
trends disaggregated by student group
• College prep test scores & participation
(e.g., ACT, SAT, PSAT, AP)
• Post-secondary patterns (as available)
• Graduation rates
• Dropout rates
• Attendance rates
Return on
Resources
Relationship between student achievement
and spending in demographic context
Spending and Revenue
• Spending per student, by function, and
by program
• How spending increases are
allocated over time
• Local, state and federal revenue
• Operating margins
• Compensation
• Fund balance
• Taxes
• Debt
School
Environment
• Student characteristics
(economically disadvantaged
backgrounds, disabilities, limited
English proficiency)
• Class size and school size
• Teacher characteristics
• School safety
• Facilities, technology, infrastructure
Community
Demographics
• Urban, suburban, rural locale
• Adult education levels
• Household income levels
• Single parent households
• Property values
• Labor force, (un)employment
• Other demographics

5. Performance Benchmarks
States compared to other states.
Districts compared to other districts in same state.
Schools compared other schools in same state.
Successful organizations continually search for new
ideas by comparing themselves to other organizations to
learn from their best practices.
This is known as benchmarking. It’s a data-driven
process by which learning and innovation can trigger
fundamental breakthroughs in thinking and practice

6. www.SchoolMatters.com

7. Standard & Poor's Observations Report is a written analysis on this district's
Return on Resources.™ The report analyzes the district's academic and financial
performance in demographic context. In addition to state comparisons, the report
also compares the district with a composite of demographically similar peer
districts.
Ann Arbor Public Schools

8. Standard & Poor's Observations Report is a written analysis on this district's
Return on Resources.™ The report analyzes the district's academic and financial
performance in demographic context. In addition to state comparisons, the report
also compares the district with a composite of demographically similar peer
districts.
Ann Arbor Public Schools
All Students
Economically
Disadvantaged
Students

9. Reading Proficiency Grade 4 Economically Disadvantaged
Outperformed for 2 Years
Outperformed for 1 Year
Outperformed for 3 Years

11. School improvement
teams can use the
workbook as a guide for
collaborating with “better
performers” to identify
effective processes and
adapt best practices.
Standard & Poor’s also
offers professional
development programs
that provide training to
school improvement
teams in benchmarking
methods.