Research dissertation on the impact of school inspection on teaching, shared at ResearchED London September 9th 2017

1. I used to think that the policy intent of school
inspection in England was monitoring, and that
this was how Ofsted saw its role; but at an
Ofsted inspector training session a few years
ago, an inspector explained that Ofsted’s role is
primarily school improvement. Their motto,
“raising standards improving lives” shows how
Ofsted brands itself as an engine for school
improvement.
School Inspection – why?

2. Researchers have traced this back to 2000, when
Chris Woodhead as Chief Inspector propounded
an “almost messianic belief” in the power of
inspection to transform schools, despite what
they describe as a lack of evidence at the time
showing that inspection achieved this
goal.(Clarke and Ozga 2011)

3. What are the numbers?
££ spent on Ofsted 2013-14 £150.6 million
“I will use the inspection recommendations
to move the school/my teaching forward”.
c 23K staff responses to post inspection
surveys 2009-14
98%
“Inspections have impacted positively on my
performance at work.” (804 teachers -
(Education Support Partnership 2015)
10%
“Inspections have negatively impacted
on my motivation to stay in a teaching
career.” (Ibid)
74%

4. What are the numbers?
“Inspectors focus sharply on
teaching.” 829 School
leaders, Ofsted survey 2015
89%
Leaders who ranked feedback to senior
leaders on teaching as one of their top
two most helpful aspects of inspection
(Ofsted, 2015)
23%
How many of 1,362 secondary
schools with teaching graded as
Good since 2012 had a specific
action for improvement including
“teaching” ?
1,169

5. Where is the evidence on school
inspection and school improvement?
That inspection identifies areas for development, and is a factor
that leads to improvement in, failing schools was well supported
by research (Matthew and Sammons 2004)(Luginbuhl et al. 2009;
Penninckx et al. 2014) but there was less clear evidence to link
inspection to improvement in effective schools, and there was
some evidence of unintended consequences where the inspection
regime is high stakes, for example narrowing the curriculum or
‘ossification’ of teaching methods (Altrichter and Kemethofer
2015). Some studies described leaders getting distracted from
improving teaching by activities in preparation for the next
inspection (Matthew and Sammons 2004).

6. How might school inspection improve
teaching in effective schools?
UCL/IoE MSc dissertation 2016
Systematic approach to develop a
rough theory of change which was
reviewed by leaders of teaching &
learning.

7. What is quality teaching?
OECD definition: Teachers who have ‘a high level
of general intelligence, a solid mastery of the
subjects to be taught, and a demonstrated
aptitude for engaging students and helping
them understand what is being taught’.(Sellar
and Lingard 2013)

8. Ball: “Schools may pay some attention to a
policy and ‘fabricate’ a response that is
incorporated into school documentation for
purposes of accountability and audit, rather
than to effect pedagogic or organisational
change” (2001, cited in (Ball et al. 2012),

9. What research is there on inspection
and quality of teaching?
Standing International Conference of Inspectorates
(SICI) has supported 32 member inspectorates to
research and evaluate inspection, so there is now a
wealth of published research on school inspection
in Europe.
Lessons could be drawn from this for school leaders
in England, who are usually provided with
evaluation reports written by their own
Inspectorate – Ofsted, which may be subject to bias
and are limited to the experiences of English
schools.

11. (Ehren 2014) has used this survey and reviewed
findings of others who have also published
studies on the same data, to write a detailed
analysis of the impact of inspection on teaching
and learning. This study offered useful insights
and an overview of further investigation needed
to answer my research question. For example,
views of the impact of inspection on classroom
practice differed between headteachers and
teachers.
SICI longitudinal study over 3 years (2010-2013) of >2,000
principals across 6 European countries with inspection
regimes, investigating differences in attitudes over time.

12. My findings were that it is not so much
inspection that changes teachers’ behaviour, but
more the anticipation of inspection and the use
of school-self evaluation by school leaders in
preparation for inspection. The extent to which
this process might improve teaching relies on
the quality and clarity of expectations for
teaching set out in the inspection framework,
and how well these are understood and
interpreted by inspectors and school leaders.

13. Significant evidence in the literature
base that high stakes inspection led to
unintended consequences:
- ossification(teaching methods that do not improve over time,
but stay the same);
- myopia (pursuit of short term goals);
- risk avoidance;
- measure fixation;
- tunnel vision(focus on what the inspectorate wants rather
than broader sense of what’s best for the school);
- narrowing the curriculum;
- teachers ‘teaching for inspection’ when they are observed,
putting on a show for inspectors rather than demonstrating
their usual practice;
- leaders distracted from improving teaching by activities in
preparation for the next inspection.

14. Further research needed to check the
practitioners’ view that the 2015 Ofsted
Framework and the removal of grades for
individual lessons have minimised such
unintended consequences.
And to establish causal links between Ofsted
and improvements in teaching quality.

15. Two main weaknesses in the research
• difficult to judge the quality of teaching
accurately and consistently, so it is difficult to
gather evidence that establishes whether or
not teaching has improved
• previous reviews of the literature have been
unable to make causal links to prove whether
it is inspection that leads to improvements
made by schools, and studies highlight a lack
of evidence on this

17. Rough Theory of Change
Context… Mechanism… Outcome
• The Inspection Framework (CMO1)
• School Self-Evaluation (CMO2)
• Inspectors gather evidence on teaching quality
(CMO3)
• Inspectors give feedback on how to improve
teaching (CMO4)
• Leadership and teachers make use of
inspection (& SSE) process to improve
teaching (CMO5)

18. • Carrots and sticks (CMO6)
• Teachers held accountable – high stakes
accountability (CMO7)
• Gaming – activities whose sole purpose was to
get positive inspection feedback (CMO8)
• Accountability to stakeholders – report
published (CMO9)
Context… Mechanism… Outcome

19. What did practitioners say?
Where teachers had been graded outstanding
under old frameworks, they were described as
being resistant to feedback and feeling that they
did not need to progress further. Meanwhile
when teachers were graded as below good it
damaged their self-worth and it took them a
long time to recover. This applied for both
internal and external graded observations.

20. What did practitioners say?
A practitioner who had also been a HOD since
2010 described writing and updating the
departmental self-evaluation as one of the jobs
they found the hardest, yet it had very little
impact and “disappeared on the computer”.
One practitioner described sparse self-
evaluation by leadership, and departments not
using/ updating their self-evaluations, or if they
were this was not linked to work to improve
teaching in the department.

21. Joint observation with inspectors has given leaders an
insight into how inspectors work, leaving one
practitioner asking (re a 2016 inspection in a school
she supports), “Why are they not seeing this? Why
are they not highlighting that?” and concluding that
when observing classroom practice inspectors still
focus very much on their own interest.
Inspectors may be mis-interpreting the freedom as an
opportunity to judge teaching and feedback based on
their own personal preferences and expectations.
What did practitioners say?

22. All practitioners reported inconsistency of judgement by
inspectors, particularly noticeable under the pre 2010
framework when more lessons were observed in longer
inspections; for example a practitioner described that
during an inspection in which seven of her lessons were
observed, one inspector limited the lesson judgement
because pupils “moved around too much”, whilst another
praised the movement in a different lesson. Things have
not necessarily been resolved by recent changes. As one
practitioner put it regarding the September 2015
framework, “They are out to judge something, but nobody
really knows what they have to judge.”
What did practitioners say?

23. One practitioner described an inspection under
the 2010 – 2011 framework as a “horrific
experience for teachers, you’re on stage as you
would be at the theatre, without all the
controlled variables. I know it’s now about
embedded routines, but do they have the time
to judge that, in one or two days..?” She
questioned the extent to which an inspection
captured valid and reliable data to assess the
SSE.
What did practitioners say?

24. Trust was also an issue – the tension between
speaking honestly with an inspector during a
conversation about leadership of teaching -
which could provide useful verbal feedback from
the inspector on how to improve - yet the
practitioner was left worrying, “Do they try and
catch you off the cuff?”
What did practitioners say?

25. Practitioner feedback indicated that for effective
schools, inspection is less of a driver for
improvement than:
• other accountability measures;
• appraisal;
• leadership, particularly the headteacher.

26. Implications for school staff
• School self-evaluation based on the inspection
framework is a key mechanism for school
improvement. Training is needed for school
leaders to achieve this.
• When observed by inspectors, teachers will be
more likely to act on their feedback to
improve if they go to see the inspectors to
hear it directly from them.

27. Not obsessing over the “Outstanding”
criteria
(maybe because I have internalised
them…)
Just focusing on the
next steps
And blessed to be fully staffed