This document discusses using data to support decision making at the school level. It notes that while school improvement plans often cite data use, the plans rarely impact meaningful change. The document then shares teacher perspectives on data, including concerns about accuracy and usefulness. Teachers want data that focuses on instruction and individual students. The document also discusses skills needed to effectively use data like assessment literacy. It provides tools to assess a district's data capacity and support around data-informed decision making.

1. Organic School Info-Systems:
Decision Making at the School Level
Christopher A. Thorn
University of Wisconsin-Madison
Wisconsin Center for Education Research

2. The School Improvement Planning
Process and Decision Support
 Much my work has been split between district
technology and research staff and schools working on
SIP projects.
 School Improvement Plans have been the justification
for our efforts, but rarely seem to play an important role
in improvement.
 Is it possible to get inside the improvement process and
see why this is failing?

3. What do teachers say about
the utility of more data?
“ We need to have our minds opened to what else could be done with
data.”
“To know what kind of data would be more useful. What’s out there,
what’s meaningless?”
“I think data gives a small part of the picture. Some of it is very accurate,
but I don’t think that it necessarily tells the entire tale, certainly not if
you’re just looking at numbers.”
“It’s not so much getting a hold of the data, but applying it that’s the
problem.”

4. How do teachers feel about using data?
“I tend to be somewhat cynical about data.”
“Information is power, but it's also really scary. It takes our staff out of
their element.”
Teachers have concerns about data in regards to:
 Accuracy
 Quality of data sources and assessments
 Timeliness
 Usefulness in the classroom
 Use for accountability and evaluation

5. What teachers want from data?
“The problem I see is kids not learning, and if data could zero in on
where we can do the most good, in time…”
“We need data that we can manipulate in the school and that can be
accessed.”
 School-level data
 Data with an instructional focus
 Disaggregated data, individual student data
 Longitudinal data, historical records
 Behavior data, attendance data
 Student home and background data
 Quick access to timely data

6. What skills do teachers (and building
administrators) need?
 Assessment literacy and alignment
 Technology (computers, software, databases)
 Data management, analysis, and application
 How to use data to:
 Improve learning in our classroom and building
 Identify strengths and weaknesses in our curriculum and
instruction, and identify strategies for change
 Target student needs
 Ask better questions and get better answers
 To create visual representations
 Communicate with students, parents and other staff

7. Supporting information and
decision making models
 Recognize the role of leaders as sensemaker and that it is at
least as important as the role of decision maker
 Support PD that encourages building of analytical skills and local
data analysis
 Make technology investments that support the instructional
mission
 Make clear degree of alignment between curriculum, PD, and
assessments
 Data at finer levels of temporal resolution are key to
understanding and addressing instructional problems. Local
documentation of practice may help.
 Needs assessment is a skill that seems to be lacking in most
educational training. Rubric-type surveys could serve to
education and provide feedback

8. Information/Flow Inventory:
Where does your district fit?
Role Enter Get Data Provide data for Provide data for Local data
Data Out lower level use upper level use for local use
Students
Parents
Teachers
Instruction Support
Special Services
School
Administrators
District
Administrators
Community Members

9. Decision support problem space
Existing IT Infrastructure
Changing
Professional
Classroom
Development
Practices
Accountability
Framework

10. Capacity Assessment Tools
Capacity Type
Inquiry Process Technical Organizational
Decision Making /
Data Selection
Data Acquisition
Data Management
Analysis and Reporting
Application
Evaluation

11. Using rubrics to guide the process
1)How well supported are the organizational and personnel issues
(meeting time/space provided, work release time provided, leadership
guidance, etc.) associated with managing and maintaining data?
Check the statement that best describes Education Assessing School
your school. Plan Classroom Wide
Scenario Learning Problem
Little or no current support exists.
Current support exists, but is inadequate
for current data needs.
Current support is adequate for current
data needs, but not for future needs.
Current support is adequate for current
and future data needs.

12. Tracking local assessments:
Traditional and Standards-based

13. Linking to Standards to classroom
work and assessments

14. Dimensions of Organizational Complexity
Sponsor One None/many
Objectives SMART* Vague
Unresolved Issues Few Multiple
District Policies Established Non-existent
Govt. Regulations Few/Simple Many/complex
Leadership Team Experienced Inexperienced
Time to completion Loose/flexible Tight
Geography One location System-wide
Business rules Established Non-existent
*Specific, Measurable, Achievable, Relevant to strategy, Time specific

15. Dimensions of Technical Complexity
Architecture In place Not in place
Technical Novelty Proven New
Comm. system Proven/stable New
Decision making team Experienced Inexperienced
Team location One Many
System platforms One Many
Level of integration Stand-alone Fully integrated
Transaction volume Low High
Fault tolerance High Low

16. School-level areas of
intervention & innovation
 School Level
 Ongoing dialogue about what constitutes relevant evidence
about student and teacher performance towards larger
goals
 Strategic collection of classroom-level data that supports
core goals, not data for data’s sake
 Flexibility to combine locally relevant data about practice
with external data (district, cross-school, etc.)

17. Improvements to data access
and manipulation
 District Level
 Program identifiers for tracking program impact across
schools
 Embedding longitudinal models into centrally-held
information
 Providing ongoing training for teachers and administrators
on data collection and analysis
 Measuring what we care about rather than the stuff that is
easy to measure