V1 Hampshire Collegiate School: 29 June, 2018

+
Dr Gary Jones - @DrGaryJones – jones.gary@gmail.com
https://evidencebasededucationalleadership.blogspot.com
Hampshire Collegiate School
Friday, 29 June, 2018
Evidence-based
practice: What it is and
what it isn’t

+ Expert opinion likely to be accepted when
The author of the opinion must have the
relevant expertise.
The author ideally should not have a conflict
of interest that could influence, consciously
or unconsciously, the formulated opinion.
The opinion should not conflict with the
opinion of other qualified experts. If experts
disagree, further probing is required.
5
Ennis, R. H. (1962). A Concept of Critical Thinking. Harvard Educational Review.
32. 1. 81-111

+ Expert opinion likely to be accepted when
The author of the opinion must have the
relevant expertise.
The author ideally should not have a conflict
of interest that could influence, consciously
or unconsciously, the formulated opinion.
The opinion should not conflict with the
opinion of other qualified experts. If experts
disagree, further probing is required.
7
Ennis, R. H. (1962). A Concept of Critical Thinking. Harvard Educational Review.
32. 1. 81-111

+
In this session I intend to
 Provide a working definition of evidence-based
practice.
 Explain a number of tools and techniques which you
can use to improve your evidence-based practice.
 Signpost post some criticisms of evidence-based
practice
9

+
Why do we need evidence-
based practice?
10

+
EBP is defined as the
production or creation of
desirable change and the
prevention of undesirable
change, somehow guided
by evidence of what
works.
11

+
Ethics
 When evidence is not used during practice,
important failures in decision making occur:
 ineffective interventions are introduced;
 interventions that do more harm than good are
introduced;
 interventions that do more good than harm are not
introduced;
 and interventions that are ineffective or do more
harm than good are not discontinued.
13
Gray, J. (2001). Evidence-based healthcare: how to make health policy and
management decisions (Second ed.). London: Churchill Livingstone.

15
BARENDS, E., ROUSSEAU, D. & BRINER, R. 2014. Evidence-Based Management
: The Basic Principles. Amsterdam: Center for Evidence-Based Management.

+
Common misconceptions about
EBP (1)
 Evidence-based practice ignores the expertise and knowledge
of teachers and head-teachers.
 Evidence-based practice is the same as research-based
practice
 Evidence-based practice involves teachers undertaking
research
 Evidence-based practice is all about numbers and statisticians

+
Common misconceptions about
EBP (2)
 School leaders need to make decisions quickly and don’t have
time for evidence-based practice?
 Each school is unique, so the usefulness of scientific evidences
is limited
 If you do not have high-quality evidence, you cannot do
anything
 Good quality evidence gives you the answer to the problem
18

19
BARENDS, E., ROUSSEAU, D. & BRINER, R. 2014. Evidence-Based Management
: The Basic Principles. Amsterdam: Center for Evidence-Based Management.

+
Asking well-formulated and
answerable questions?
20

+
The PICO format
 P — Pupil or Problem. How would you describe
the group of pupils or problem?
 I — Intervention. What are you planning to do
with your pupils?
 C — Comparison. What is the alternative to the
intervention/action/innovations
 O — Outcomes. What are the effects of the
intervention/action/intervention?

+
Using the PICO format
 For pupils requiring additional learning support (P)
how does the provision of 1 to 1 support (I)
compared with group support (C) affect achievement
rates.

+
Now try
 P — Pupil or Problem. How
would you describe the group of
pupils or problem?
 I — Intervention. What are you
planning to do with your pupils?
 C — Comparison. What is the
alternative to the
intervention/action/innovations
 O — Outcomes. What are the
effects of the
intervention/action/intervention?
24

+
Using the SPICE framework
 Setting – where?
 Perspective – for whom?
 Intervention – what?
 Comparison – compared with what?
 .Evaluation – with what result
25

+
SPICE examples
 From the perspective of a newly qualified teacher (NQT)
secondary school, is formal coaching more effective than
informal mentoring in ensuring NQTs successfully complete his
or her probationary year.
 From the perspective of a newly appointed primary school
headteacher in a school that ‘requires improvement’ is seeking
to replace the existing leadership team with a new leadership
more likely to lead to the school becoming good or outstanding.
 From the perspective of Y6 pupils are friendship groups more
important than parental views in influencing the preferred
choice of secondary education
26

+
Now try - SPICE
 Setting – where?
 Perspective – for whom?
 Intervention – what?
 Comparison – compared with
what?
 .Evaluation – with what result
27

+
Helen Kara and 10 ways to access
journal articles
 Openly accessible literature
 Conventionally published
literature
 Use Google Scholar
 ResearchGate
 Use Twitter
 Google books
 Amazon -
 Affiliate to a University
 Contacts – get a pdf
 Contact the author
31
https://www.slideshare.net/MichelleHaywood5

+ 34
Strip it
Trace it
Analyse it
Should I do it?

+
Appraising research evidence
 Why am I reading this?
 What are the authors trying to
achieve in writing this?
 What are the authors claiming
that is relevant to my work?
 How convincing are these
claims, and why?
 In conclusion, what use can I
make of this?
35

37Author/sources Description Year Setting Who Commentary
Research
literature
Sims, Moss
and Marshall*
2017 Two mixed 11-
18 schools
Ofsted -
outstanding
10 teachers
in school A
and 7
teachers in
school
The research finds that
journal clubs are a viable,
scalable model of teacher-
led professional
development, capable of
creating sustained increases
in evidence-informed
practice.
School data
(quantitative)
Professional
Learning
Programme
2017 The school All staff Some space in professional
learning calendar for half-
termly journal clubs
Stakeholder views School staff
meeting
2017 The school All teaching
staff including
teaching
assistants
General acceptance of idea
in principle, though
suggested it should be
trialled with a group of 7 -10
volunteers
Practitioner
Expertise
Senior
Leadership
Team
2017 The school HT, 2 DHs
and School
Research
Lead
No direct experience though
school research lead has
attended sessions on journal
clubs at researchED and is
aware of available resources

+
 1. Inclusive culture, leadership
and management
 2. High quality teaching
 3. Use of expertise
 4. Personalisation
 5. Flexible use of evidence-
based strategies
 6. Progress tracking
 7. Communication and
collaboration
38

+
The components of an argument: a
Toulminian analysis
 The claim (C) or conclusion
 Facts of grounds we appeal as a foundation for C, called
grounds or data (D)
 How do we go from D to C – proposition that provides
justification – called the warrant (W) – provide a licence to the
inference for doing from C to D.
 Standing behind our warrant will be backing (B)
42
Kvernbekk, T. (2013). Evidence-Based Practice: On the Function of
Evidence in Practical Reasoning. Studier i Pædagogisk Filosofi. 2. 2. 19-
33.

+
The components of an argument
Data/gounds Some of students
read poorly
Some of my
students read poorly
Some of students read
poorly
Claim or
conclusion
I shall ask a
colleague, what she
has done in the past
and do that
I shall give them the
same exercises as I
used last year
I shall use XYZ method to
improve reading
Warrant Experienced teacher
who has been
successful with this
type of students
These students
appear to have the
same problems and
these exercises
seemed to work
Since this approach helps
with phonological measures
43

+
Where does research fit in
 Suppose some pupils are reading delayed (data D)
 Infer children should receive extra word and text training (claim C)
 Well-tried remedy to help children with reading (warrant D)
 What backing B do you have for your W?
 RCT – two groups intervention group and control
 Intervention group received intervention for two, 10-week periods
 On average pupils made significant progress, letter knowledge,
single word reading and phoneme awareness (B)
44

+
The components of an argument
 Some pupils have difficulties paying attention (D)
 I shall encourage regular physical activity throughout the day
(C )
 Carrying out moderate levels of physical activity improves
attention and behaviour throughout the day (W)
 Reeves et al 2016 (B)
46

+ Applying: incorporating the
evidence into the decision-
making process
49

Framework for analysis of context
Participants’ will
Sizing up a context
No
Commitment
Some
Commitment
Strong
Commitment
Extant know-
how limited
Limited
capacity
Very small-
scale test
Very small-
scale test
Very small-
scale test
Good
capacity
Very small-
scale test
Very small-
scale test
Small-scale
test
Substantial
know-how
exists
Limited
capacity
Very small-
scale test
Small-scale
test
Large-scale
test
Good
capacity
Small-scale
test
Large-scale
test
Implement
BRYK, A. S., GOMEZ, L. M., GRUNOW, A. & LEMAHIEU, P. G. 2015.
Learning to improve: How America's schools can get better at getting better.

+
Assessing the impact of any
decision taken?
51

+
After-Action-Review
 What did we set out to do?
 What actually happened?
 Why did it happen?
 What are we going to do next time?
52

+
Critics of evidence-based practice
 Biesta, G. (2007). Why “What Works” Won’t Work:
Evidence‐Based Practice and the Democratic Deficit in
Educational Research. Educational theory. 57. 1. 1-22.
 Elliott, J. (2001). Making Evidence‐Based Practice Educational.
British Educational Research Journal. 27. 5. 555-574.
 Hammersley, M. (2004). Some Questions About Evidence-Based
Practice in Educaiton In Thomas, G. and R. Pring. Evidence-
Based Practice in Educations Maidenhead. Open University
Press.
 MacLure, M. (2005). ‘Clarity Bordering on Stupidity’: Where’s the
Quality in Systematic Review? Journal of Education Policy. 20. 4.
393-416.
 Sanderson, I. (2003). Is It ‘What Works’ That Matters? Evaluation
and Evidence‐Based Policy‐Making. Research papers in
education. 18. 4. 331-345.
54

+
Issues and controversies
 the epistemological differences between academic researchers and
policy-makers in terms of what counts as evidence, the quality of evidence
and what evidence can or can't tell us;
 whether the evidence-informed movement serves to work against the
practitioners' professional judgement;
 issues in relations to how formal academic knowledge and professional or
tacit knowledge might be effectively combined;
 differentials in power that can affect or limit interactions between teachers
or policy-makers and research/ers;
 controversies in relation to to some of the methods commonly associated
with enhancing evidence uses;
 how the capacity to engage with academic research might be enhanced;
 issues such as the inaccessibility of research to teachers and policy-
makers, both in terms of where it is published and the language that is
typically used in such publications. (adapted from p1)
55
Brown, C. (2015). Evidence-Informed Policy and Practice in Education: A
Sociological Grounding. London. Bloomsbury Publishing.

+
In summary
 Evidence-based practice is an ethical endeavour
 Some reasonably straightforward techniques which anyone can
use to help make decisions evidence-based
 However, EBP is not uncontested
56

+
@DrGaryJones
jones.gary@gmail.com

+
@DrGaryJones
jones.gary@gmail.com
http://evidencebasededucationalleadership.blogspot.com

+
Evidence to decision
Priority
Logic model
Benefits
Costs
Certainty of the
evidence
Balance
Resource use
Acceptability
Equity
Feasible
Recommendation
Further comments
59

+
How well did I do
 Asking
 Accessing
 Appraising
 Aggregating
 Acting
 Appraising
63

+
An agenda for real evidence-based
education
 Makes the ethical leadership and management of school its top
priority.
 Demands multiple sources of evidence in a format that governors,
school leaders, teachers, support staff, parents and pupils can
understand.
 Is characterised by expert judgement rather than mechanical rule
following.
 Involves shared decision-making between governors, leaders,
teachers, pupils, parents and other stakeholders.
 Builds on the fundamentally human aspect of education to foster
mutual respect between everyone within a school community.
 Applies these principles to all aspects of the work of the school.
64

V1 Hampshire Collegiate School: 29 June, 2018