Evidence-based practice for schools: One school's story of developing a data dashboard 14.45-15.50 CASE Europe, Brighton Monday 9 March, 2009 Brian Bedrick and David Willows

Evidence-based practice for schools What are some of the biggest challenges you face when managing and using data?

Facing up to where we were… We collected data but did nothing with it We weren’t sure what data was the most important We weren’t sure how to report data No one listened anyway It (not surprisingly) did not lead to changes in behaviour

Facing up to where we were…

We collected data but did nothing with it

We weren’t sure what data was the most important

We weren’t sure how to report data

No one listened anyway

It (not surprisingly) did not lead to changes in behaviour

Solution: The K-CAR approach to developing a data strategy K ey Questions What is it that we really want to know? C ollection What kind of information do we need? A nalysis What conclusions can we draw? R eporting How and with whom do we share?

K ey Questions

What is it that we really want to know?

C ollection

What kind of information do we need?

A nalysis

What conclusions can we draw?

R eporting

How and with whom do we share?

Key question no. 1 How do we know if we are being successful in our mission of inclusion, challenge and success? Step 1 Key Questions

… which quickly led to a whole bunch of other questions: How do we gauge the health of the organisation? How do we learn from the past ? How do we make best use of management meetings? How do we know if we have met our goals ? Do we even have goals for all our strategic objectives ? Are they the right goals ?

… which quickly led to a whole bunch of other questions:

How do we gauge the health of the organisation?

How do we learn from the past ?

How do we make best use of management meetings?

How do we know if we have met our goals ?

Do we even have goals for all our strategic objectives ?

Are they the right goals ?

Determine data that would answer key questions Identify which data that you already have or can realistically get Prioritize and organize until you have something manageable Finding simplicity in the complexity Step 2 Collection

Determine data that would answer key questions

Identify which data that you already have or can realistically get

Prioritize and organize until you have something manageable

Finding simplicity in the complexity

Step 3 Analysis and Reporting The School on One Page

Interactive Demo – Must Be Viewed in “Slideshow” Mode

Interactive Demo – Must Be Viewed in “Slideshow” Mode

Data So what? Are we having better conversations ? Are we more disciplined ? Are we agreed on the meaning and significance of this data ? Has anything changed because of the way we use data?

Data So what?

Are we having better conversations ?

Are we more disciplined ?

Are we agreed on the meaning and significance of this data ?

Has anything changed because of the way we use data?

Data So what? Are we having better conversations ? From ‘things seem slower’ to ‘below target for this month, but still on track year to date’ Are we more disciplined ? Board now expects to see data each meeting Team meetings now focused on actions taken / results / additional actions required Are we agreed on the meaning and significance of this data ? Dashboard provides common vocabulary and ‘one version of the truth’ Has anything changed because of the way we use data? Employee behaviour, focus, morale Becoming part of culture and continuing to gain momentum Management and decision making processes

Data So what?

Are we having better conversations ?

From ‘things seem slower’ to ‘below target for this month, but still on track year to date’

Are we more disciplined ?

Board now expects to see data each meeting

Team meetings now focused on actions taken / results / additional actions required

Are we agreed on the meaning and significance of this data ?

Dashboard provides common vocabulary and ‘one version of the truth’

Has anything changed because of the way we use data?

Employee behaviour, focus, morale

Becoming part of culture and continuing to gain momentum

Management and decision making processes

Data is not just a product , but also influences process

ISB operates on 26% turnover each year. The Board of Trustees changes at the same rate. Our ability to ‘forget’ in times of crisis is perhaps our biggest threat of all. An example of its use in practice

WHAT DOES HISTORY TEACH US?

WHAT DOES HISTORY TEACH US? 1st Oil Crisis Gulf War 9/11 2nd Oil Crisis

WHAT DOES HISTORY TEACH US? Despite a trend upwards, enrolment at ISB is ALWAYS cyclical Periods of economic downturn have ALWAYS impacted enrolment at ISB Typically the negative impact is felt over a 2-3 year period The largest cumulative fall in enrolment was 1974-82: 22% But remember that at this time the market was very different: >50% US prior to 1984, 24% today

Despite a trend upwards, enrolment at ISB is ALWAYS cyclical

Periods of economic downturn have ALWAYS impacted enrolment at ISB

Typically the negative impact is felt over a 2-3 year period

The largest cumulative fall in enrolment was 1974-82: 22%

But remember that at this time the market was very different: >50% US prior to 1984, 24% today

WHAT DOES HISTORY TEACH US? A lot of learning from just annual enrolment figures In the future, we will will have additional data such as visit/application patterns leading up to and following the current financial crisis Moving from ‘guess and react’ to ‘forecast and prepare’

A lot of learning from just annual enrolment figures

In the future, we will will have additional data such as visit/application patterns leading up to and following the current financial crisis

Moving from ‘guess and react’ to ‘forecast and prepare’

How is data like coffee ?

Glass pot Individual Pads Nespresso Home Roasted How is data like coffee ?

Summary Our Story Knew data was important, but was overwhelmed and not using it effectively Began conversations to gain an external, independent perspective Implemented a process to help focus thinking and activities (KCAR) Launched programme even though data and processes were still imperfect Beginning to make steady progress along the continuum Better conversations More discipline Real and beneficial change Understanding that this is a journey. There is no ‘silver bullet’ solution.

Summary Our Story

Knew data was important, but was overwhelmed and not using it effectively

Began conversations to gain an external, independent perspective

Implemented a process to help focus thinking and activities (KCAR)

Launched programme even though data and processes were still imperfect

Beginning to make steady progress along the continuum

Better conversations

More discipline

Real and beneficial change

Understanding that this is a journey. There is no ‘silver bullet’ solution.

Discussion comments suggestions questions ideas

Backup Slides