This presentation was delivered at People Helping People - The future of public services - 3rd September 2014. For more information on the event visit http://www.nesta.org.uk/event/people-helping-people-future-public-services

1. Introduction Katy Owen Senior Policy Advisor Cabinet Office
City Year

2. Projects we’re funding

3. Evidence
•Educational Endowment Fund (EEF) Toolkit
•Strong theories of change / logic models
•Promising evidence
•Independent evaluations commissioned

4. Evidence
“Meta-analyses indicate that pupils [receiving 1-to-1 tuition] might make about 4 or 5 months progress during an intensive programme.”

5. Evidence
“City Year provides enthusiastic and appropriately trained young volunteers to act as mentors and role models for the pupils. Together with teachers, they help establish and maintain very good relationships between pupils and staff.”

6. Evidence
Percentage of School Leavers Attaining a University Place in 2012

7. Building the evidence base
•Impact evaluations
•Process evaluations

8. The Marketplace Michael Stevenson Education Consultant and Nesta Associate
Code Club

9. •Besieged by suppliers unwanted emails and calls
•Suppliers failing to target their offers
•Suppliers not able to articulate educational outcomes
•Most evidence-based research is difficult to use and not context specific
Schools struggle to make informed and objective decisions when purchasing interventions
Key findings
9
•Highly fragmented market and difficult to know exactly what is out there to choose from
•Unclear credentials
Uncertainty about what services exist
Shot-gun approach to selling
Lack of evidence on quality outcomes
•Buying frameworks are in place
•Purchaser rarely consults program specific research
•Purchasing decision usually based on recommendations from other teachers
Purchasing is recommendation-led
1
3
4
5
•Schools liked the idea of an online directory and teacher recommendation portal
•Schools loved the idea that this could include independent third-party impact evaluation
A ‘TripAdvisor’, digital-based solution is attractive
6
•Significant variation in the service quality experienced
•Misdistribution of high quality services, limited outside of London and especially in rural areas
Variable quality and poor distribution of services
2

10. Other market participants advocate a broader approach to securing an effectively functioning market
Key findings
10
Digital platform
•What should students know and be able to do?
•How should we measure success?
•Clear set of maintained standards
•Industry-wide training
•Support for a healthy and efficient market of high quality interventions

11. Key solution components already exist and can be built upon
Proposed solution
11
Provider (product name)
Simple rating system
Enables user feedback
Program specific impact data
Provides evidence based reviews
Cost to use
Usability
Note
Education Endowment Foundation (Toolkit)
Social Research Unit, Dartington (Investing in Children)
Institute for Effective Education, University of York (Evidence4Impact)
Notes: (*) Online version in pilot. All other products shown here are accessible online. Phase 2 (earliest 2015) will incorporate predictive capability to assess needs and use third- party to review sourced impact data
FREE
FREE
FREE
Increasing credibility
Click to buy
•Analyses categories of service
Easy
Capability
Partial capability
No capability
Easy
Easy
•Interventions are rated only if evidence is available
Key component
Database / framework
Database / framework
Database / framework
Teachers Development Trust (Good CPD Guide)
Edukit*
£70 per child
FREE
Easy
•Supports student diagnosis and provides a matching service
Proto- platform
Proto- platform

12. The digital platform should identify student needs, match needs to programs and provide comprehensive assessment of quality
Proposed solution
12
Functionality
Minimum viable product
Ideal product
Identify student needs
•Identify and prioritize student’s needs, using teacher and student data
•Identify and prioritize student’s needs, using teacher, student and school data
•Introduce online tests to help students identify their needs
•Student MOT at 11
Match student’s needs to supplier’s service
•Search database
•Identify and prioritize relevant programs
•Show location and price of program
•Deploy behavioral economic techniques
•Search results to show independent impact analysis, where available
•Search results to show characteristics of schools or students who’ve used program
Assessment of service impact
•Show ratings and awards
•Show supplier’s analysis of impact on learning outcomes
•Show user references
•Show user complaints
•Track service uptake, via online purchasing data
•Show tracking analysis of teacher prediction at 11 versus actual outcome (post program)

13. Assuming the pilot is successful, the sector (schools, suppliers, organisations) should work closely with government to steer the development of the market as proposed
Proposed actions
13
1.Clarify and introduce assessment of what students should know and be able to do at 16
2.Design a long-term approach to measuring the impact of individual services and programs
3.Design and establish effective regulation: platform and services
4. Continue to strengthen the profession’s competence in the use of data to commission services and programs
5.Support suppliers to strengthen the demonstrable impact of their programs and services on education outcomes