James Nottingham, 16th Aug 2012



      Engaging all students
       with visible learning

ChallengingLearning.com
james@p4c.com
@JamesNottinghm
Facebook.com/ChallengingLearning
900+ Meta-analyses (covering 50,000+ studies)




       2009                      2011
Maths              Percentage gains


                     level


                An Effect Size
A common scale for measuring progress in student achievement
All studies used in
 the visible learning
 research
Measured growth or
difference

eg impact of feedback

eg boys vs girls
Graph representing the spread of effect sizes
25000




20000
        No. of Effects




15000




10000




5000




    0




                             Hinge point of 0.4
Rank these 10 effects

        a) Pupils frequently moving schools
        b) Computer assisted instruction
        c) Formative evaluation of teachers
        d) How pupils rate their teachers
        e) Homework
        f)   Fewer pupils in each class
        g) Teachers’ subject knowledge
        h) Classroom discussion
        i)   Matching learning styles
        j)   Teacher-student relationships
c) Formative evaluation of teachers (0.90)
d) How pupils rate their teachers (0.90)
h) Classroom discussion (0.82)
j) Teacher-student relationships (0.72)

b) Computer assisted instruction (0.37)
e) Homework (0.29)
f) Fewer pupils in each class (0.21)
i) Matching learning styles (0.17)
g) Teachers’ subject knowledge (0.09)

a) Pupils moving schools (-0.34)
Teachers’ subject knowledge (0.09)

Professor David Starkey, CBE
Matching Learning Styles (0.17)
Rank              Influence            Studies Effects    ES
 91    Inquiry based teaching           205      420     .31
 94    Homework                         161      295     .29
 98    Teaching test taking/coaching    275      372     .27
109    Individualized instruction       638      1185    .22

                                         Primary Schools 0.01

                                         Secondary Schools
                                         0.56
Top 75
Bottom 75
John Hattie   Jamie Oliver
Visible Learning is intended to stimulate dialogue


What questions do we have
about the relative effects?              What are self
                                        report grades?

                                      Why doesn’t class
                                      size matter much?

                                       Are we making
                                      effective progress
                                           with our
                                         innovations?
Don’t waste time on things that probably
         don’t matter much
Not everything counts

Not everything that counts can be
 counted, and not everything that
 can be counted counts

Sign hanging in
Einstein's office at Princeton
Top 75
Rank                Influence           Studies   Effects    ES
  1    Assessment capable students        209       305     1.44
  5    Providing formative evaluation     30        78      .90
  6    Micro teaching                     402       439     .88
 10    Feedback                          1310      2086     .75
Visible Learners (and leaders) accurately assess
themselves
Where am I going?


 How am I doing?


    What are my next steps?


Three key questions for assessment capable students
Learning intentions and success criteria help identify where & how


Learning Intentions
o To find out what links the Vikings with
North East England


Success Criteria        Co-constructed
o Know when and where the Vikings came from
o Identify names and places associated with the Vikings
o Ask relevant questions
Why did they         Gate
      AD 700 - 1100         attack Lindisfarne?    Bairns
                                                    Lad
                                                    Tarn
                  Vikings                         Thriding
  Norse                              Rape &
language                             pillage

                                         Did they believe in
    Longships                                  God?
  Dragon                          Horned
   ships                          helmets
Marzano – groups of 3 work best

                             Informal
                             Formal
                             Long-term
Why did they         Gate
                    AD 700 - 1100          attack Lindisfarne?    Bairns
    Captured
                                                                   Lad
  Yorvik in 866
                                                                   Tarn
                                 Vikings                         Thriding
               Norse                                Rape &
             language                               pillage

   King Cnut                                            Did they believe in
ruled England      Longships                                  God?
  from 1016     Dragon                           Horned
                                                 helmets      Gods included
                  ships                                       Odin, Thor, Fri
                            Eric Bloodaxe
                                                                gg & Loki
    Dead warriors went       died in 954
        to Valhalla
Sharp pencil                         ✔
Title                                ✔
Date                                 ✔
Capital Letters                      ✔
Full stops                           ✗
Describe the character               ✔
Describe the place                   ✗
First, next, then, finally           ✔
And, but, so, while, because         ✗
Fun action words (bounded, sprang)   ✔
Rhyming words (loud, proud, crowd)   ✔
Marking sheet for history essays (Frank Egan)

Introduction                Conclusion
 4+ sentences               3+ sentences
 Proposition stated         Summation
 Outline of narrative       Proof of proposition
 Context of topic           Specific reference to
                            assess/evaluate as last sentence


Body of essay               Literacy
 3+ paragraphs              Spelling accuracy
 6+ facts per paragraph     Grammar structures
 Inter-relationships
                                  “I can actually see how to
 Argument is relevant
                                    improve, it’s obvious.”
 Quote with source given
Learning Detectives
Download slides from …

www.challenginglearning.com




        @JamesNottinghm
Which factors influence student-teacher relationships?


 A.    A focus on learning 0.47
 B.    Warmth 0.68
 C.    Encouragement of higher order thinking skills 0.61
 D.    Adapting to differences 0.41
 E.    Being genuine 0.29
 F.    Non-directivity 0.74
 G.    Beliefs about the students 0.10
 H.    Empathy 0.68

  Cornelius-White notes that students who do not wish to come to
school or who dislike school do so primarily because they dislike their
                    teacher.” (Hattie, 2009, p119)

Holtet Videregående Skole

  • 1.
    James Nottingham, 16thAug 2012 Engaging all students with visible learning ChallengingLearning.com james@p4c.com @JamesNottinghm Facebook.com/ChallengingLearning
  • 4.
    900+ Meta-analyses (covering50,000+ studies) 2009 2011
  • 5.
    Maths Percentage gains level An Effect Size A common scale for measuring progress in student achievement
  • 6.
    All studies usedin the visible learning research Measured growth or difference eg impact of feedback eg boys vs girls
  • 7.
    Graph representing thespread of effect sizes 25000 20000 No. of Effects 15000 10000 5000 0 Hinge point of 0.4
  • 8.
    Rank these 10effects a) Pupils frequently moving schools b) Computer assisted instruction c) Formative evaluation of teachers d) How pupils rate their teachers e) Homework f) Fewer pupils in each class g) Teachers’ subject knowledge h) Classroom discussion i) Matching learning styles j) Teacher-student relationships
  • 9.
    c) Formative evaluationof teachers (0.90) d) How pupils rate their teachers (0.90) h) Classroom discussion (0.82) j) Teacher-student relationships (0.72) b) Computer assisted instruction (0.37) e) Homework (0.29) f) Fewer pupils in each class (0.21) i) Matching learning styles (0.17) g) Teachers’ subject knowledge (0.09) a) Pupils moving schools (-0.34)
  • 10.
    Teachers’ subject knowledge(0.09) Professor David Starkey, CBE
  • 11.
  • 12.
    Rank Influence Studies Effects ES 91 Inquiry based teaching 205 420 .31 94 Homework 161 295 .29 98 Teaching test taking/coaching 275 372 .27 109 Individualized instruction 638 1185 .22 Primary Schools 0.01 Secondary Schools 0.56
  • 13.
  • 14.
  • 15.
    John Hattie Jamie Oliver
  • 16.
    Visible Learning isintended to stimulate dialogue What questions do we have about the relative effects? What are self report grades? Why doesn’t class size matter much? Are we making effective progress with our innovations?
  • 17.
    Don’t waste timeon things that probably don’t matter much
  • 18.
    Not everything counts Noteverything that counts can be counted, and not everything that can be counted counts Sign hanging in Einstein's office at Princeton
  • 19.
  • 20.
    Rank Influence Studies Effects ES 1 Assessment capable students 209 305 1.44 5 Providing formative evaluation 30 78 .90 6 Micro teaching 402 439 .88 10 Feedback 1310 2086 .75
  • 21.
    Visible Learners (andleaders) accurately assess themselves
  • 22.
    Where am Igoing? How am I doing? What are my next steps? Three key questions for assessment capable students
  • 23.
    Learning intentions andsuccess criteria help identify where & how Learning Intentions o To find out what links the Vikings with North East England Success Criteria Co-constructed o Know when and where the Vikings came from o Identify names and places associated with the Vikings o Ask relevant questions
  • 24.
    Why did they Gate AD 700 - 1100 attack Lindisfarne? Bairns Lad Tarn Vikings Thriding Norse Rape & language pillage Did they believe in Longships God? Dragon Horned ships helmets
  • 25.
    Marzano – groupsof 3 work best Informal Formal Long-term
  • 26.
    Why did they Gate AD 700 - 1100 attack Lindisfarne? Bairns Captured Lad Yorvik in 866 Tarn Vikings Thriding Norse Rape & language pillage King Cnut Did they believe in ruled England Longships God? from 1016 Dragon Horned helmets Gods included ships Odin, Thor, Fri Eric Bloodaxe gg & Loki Dead warriors went died in 954 to Valhalla
  • 29.
    Sharp pencil ✔ Title ✔ Date ✔ Capital Letters ✔ Full stops ✗ Describe the character ✔ Describe the place ✗ First, next, then, finally ✔ And, but, so, while, because ✗ Fun action words (bounded, sprang) ✔ Rhyming words (loud, proud, crowd) ✔
  • 30.
    Marking sheet forhistory essays (Frank Egan) Introduction Conclusion  4+ sentences  3+ sentences  Proposition stated  Summation  Outline of narrative  Proof of proposition  Context of topic  Specific reference to assess/evaluate as last sentence Body of essay Literacy  3+ paragraphs  Spelling accuracy  6+ facts per paragraph  Grammar structures  Inter-relationships “I can actually see how to  Argument is relevant improve, it’s obvious.”  Quote with source given
  • 32.
  • 33.
    Download slides from… www.challenginglearning.com @JamesNottinghm
  • 34.
    Which factors influencestudent-teacher relationships? A. A focus on learning 0.47 B. Warmth 0.68 C. Encouragement of higher order thinking skills 0.61 D. Adapting to differences 0.41 E. Being genuine 0.29 F. Non-directivity 0.74 G. Beliefs about the students 0.10 H. Empathy 0.68 Cornelius-White notes that students who do not wish to come to school or who dislike school do so primarily because they dislike their teacher.” (Hattie, 2009, p119)

Editor's Notes

  • #5 The evidence was collected from existing meta-analyses – the actual research that is the basis of the meta-analyses included published material and quality assured research papers and student projects (eg unpublished PhDs theses). John Hattie is constantly updating the meta-analyses so you may find slight variations in the effects across publications. The material in this workshop will be kept up to date and the effect size tables in the workbook will be accurate.
  • #6 This slide represents the way that the multitude of assessment results can be compared once they are put into an effect size – and put onto a common scale. It is a way of taking different types of assessment results and making a common comparison.
  • #7 So the point was to measure progress over time
  • #9 This follows the previous activity where we now take some school systems/innovations/general practices and consider which ones have the biggest effects on student achievement. Clarify any queries around the meaning of some of these effects and what they mean. (p XXX)What effect do you think the following processes/systems/strategies/events have on student achievement – list them from the ones that have the greatest effect to the ones that have the least effect
  • #10 This follows the previous activity where we now take some school systems/innovations/general practices and consider which ones have the biggest effects on student achievement. Clarify any queries around the meaning of some of these effects and what they mean. (p XXX)What effect do you think the following processes/systems/strategies/events have on student achievement – list them from the ones that have the greatest effect to the ones that have the least effect
  • #13 Individualised InstructionBased on the idea that students are all unique and have individual needs, individualised instruction provides a programme of learning for each student.  The 0.22 effect size does not support the notion of individualised instruction as other research shows that students learn better when interacting with peers and when programs are adapted by teachers to suit individual needs within the classroom situation. Teachers are adept at managing classes of 30 or more students and have developed strategies to manage these large groups. Teaching test taking/coachingThere are a wide range of activities related to test preparation and there has been much research to correlate the effect of the activities on test scores. See VL pp 179 - 181 A related aspect of teaching test taking is where it may also have the effect of lowering student anxiety. Reducing anxiety has an effect size of 0.40.HomeworkThe overall effects of homework are positive but there are some important moderators. The key ones are that there is a greater positive effect of homework for high school students (0.59) than for elementary school students (0.04). Systems accountabilityInquiry based teachingInquiry based teaching methods have been mainly studied in the context of science. They includeopen- ended learning situations where students are not expected to come up with a ‘right’ answer but rather be involved in a process of observing, experimenting, exploration etc. Inquiry based learning is a current popular practice that is widely used in schools. There is a concern that novice learners may suffer unless the learning situation is scaffolded for them. There is a risk that in an unstructured learning environment that a learner who has little basic knowledge or skills in the areas of inquiry may have no effective way of managing the inquiry and that their learning may be harmed. In an inquiry based lesson we must provide effective support for all learners to have success.
  • #16 The evidence was collected from existing meta-analyses – the actual research that is the basis of the meta-analyses included published material and quality assured research papers and student projects (eg unpublished PhDs theses). John Hattie is constantly updating the meta-analyses so you may find slight variations in the effects across publications. The material in this workshop will be kept up to date and the effect size tables in the workbook will be accurate.
  • #17 The evidence was collected from existing meta-analyses – the actual research that is the basis of the meta-analyses included published material and quality assured research papers and student projects (eg unpublished PhDs theses). John Hattie is constantly updating the meta-analyses so you may find slight variations in the effects across publications. The material in this workshop will be kept up to date and the effect size tables in the workbook will be accurate.
  • #18 The evidence was collected from existing meta-analyses – the actual research that is the basis of the meta-analyses included published material and quality assured research papers and student projects (eg unpublished PhDs theses). John Hattie is constantly updating the meta-analyses so you may find slight variations in the effects across publications. The material in this workshop will be kept up to date and the effect size tables in the workbook will be accurate.
  • #21 Student expectations/self report gradesStudents who are able to articulate what their learning outcomes/test results might be (when they can self-report their grades) are more likely to be successful than other learners. The key premise is that when students know their learning so well they will be able to do this. These students will be engaged with their learning and be active participants in their learning. This links to the idea of building students who are assessment capable. They can answer the questions: Where am I going? How am I going? and where to next?Providing formative evaluationTeachers regularly receive formative evaluation about their teaching programmes and classroom practice through feedback and/or by looking at their student results. Teachers need to receive ongoing feedback about their teaching to improve student results. Micro teachingMicro teaching is where teachers examine aspects of a lesson, or group of lessons, by working with a colleague/s to look at specific aspects of their teaching. It may be questioning skills, teaching strategies etc. It may also involve videotaping a lesson and then examining an aspect of practice. Teachers need to receive ongoing feedback about their teaching to improve student results.