Embedding formativeassessment with teacherlearning communitiesDylan Wiliam               www.dylanwiliam.net
Overview: Science and Design2       We need to improve student achievement       This requires improving teacher quality...
Raising achievement matters3       For individuals:         Increased lifetime   salary         Improved   health      ...
What is the purpose of education?4       Four main philosophies of education           Personal empowerment           C...
Recession (2008-2010) and recovery (2010-2012)5       Those with a high school diploma or less         lost 5.6 million ...
…with profound impacts on some workers6    Education level                    Change in salary 1978 to 2005    Postgraduat...
The coming war for jobs (Clifton, 2011)   Right now    7  billion people on earth     5 billion adults     3 billion p...
A daunting target8       Programme for International Student Assessment        (PISA)         United States   496       ...
The world of work is changing…9    Skill category                    Percentage change 1969-                              ...
Off-shoring and automation10                    Off-shoreable         Not off-shoreable               Radiographer        ...
Meet Maddie Parlier…11     Davidson (2012)
There is only one 21st century skill12     So the model that says learn while you’re at school, while     you’re young, th...
Successful education?13     The test of successful education is not the amount of knowledge     that a pupil takes away fr...
Where’s the solution?14        Structure:          Smaller/larger high schools          K–8 schools/“All-through” schoo...
Are private schools the answer?15        In PISA, U.S. students in private schools out-         perform public school stu...
Pause for reflection   What’s the most interesting, surprising, or    challenging thing you have heard so far?   See if ...
School effectiveness17     Three generations of school effectiveness research:        Raw results approaches:          D...
We need to focus on classrooms, not schools18        In the USA, variability at the classroom level is at         least f...
And most of all, on teachers19        Take a group of 50 teachers:          Students taught by the most effective teache...
Improving teacher quality takes time20        A classic labor force issue with two (non-exclusive)         solutions:    ...
Replace existing teachers with better ones?21        De-select (i.e., fire) ineffective teachers?          Replace   lea...
How do we speed up teacher improvement?22        Merit pay for effective teachers?          Can’t   be done fairly, and ...
The evidence base for formative assessment23        Fuchs & Fuchs (1986)               Nyquist (2003)        Natriello ...
Which of these are formative?24     A. A district science supervisor uses test results to plan        professional develop...
The formative assessment hijack25        Long-cycle:          Span: across units, terms          Length: four weeks to ...
Main approaches to formative assessment26        Professional Learning Communities          “…an inclusive group of peopl...
Complementary processes27     Data-driven PLCs                   Classroom FA TLCs     •   Quality control                ...
Unpacking formative assessment28                 Where the               learner is going   Where the learner is How to ge...
Formative assessment and other priorities29        Formative assessment is an integral part of many         current polic...
Framework for teaching (Danielson 1996)30        Four domains of professional practice         1.   Planning and preparat...
The framework in detail31        Domain 2: The classroom environment          2a: Creating an environment of respect and...
Formative assessment and domain 332     Framework for teaching        Classroom formative assessment        Communicating...
Differentiated instruction: not a new idea33        Differentiation in action (Stradling & Saunders, 1993)        Differ...
Most definitions of DI are vague34     “While the concept of ‘differentiated instruction’ can be defined in     many ways,...
Differentiated instruction and formative     assessment35                                Aspects of differentiated instruc...
Response to (instruction and) intervention36     “Response to intervention integrates assessment and intervention     with...
Response to (instruction and) intervention37        Key points          Tier 1 must be high-quality, evidence-based     ...
And one big idea38                 Where the               learner is going   Where the learner is How to get there     Te...
An educational positioning system39        A good teacher:          Establishes  where the students are in their learnin...
Strategies and practicaltechniques for classroomformative assessment
Clarifying, sharing andunderstanding learning intentions
Sharing learning intentions42        3 teachers each teaching 4 7th grade science         classes in two US schools     ...
Sharing learning intentions43                              Comprehensive Test of Basic Skills     Group                   ...
Share learning intentions44         Explain learning intentions at start of lesson/unit:           Learning intentions  ...
Engineering effective discussions,activities, and classroom tasks that elicitevidence of learning
Kinds of questions: Israel46                                           1    2    1    1     Which fraction is the smallest...
Draw an upside-down triangle…47
Inverted red triangle…48
Eliciting evidence49        Key idea: questioning should           cause thinking           provide data that informs t...
Questioning in science: Discussion50     Ice-cubes are added to a glass of water. What     happens to the level of the wat...
Questioning in science: Diagnosis51     The ball sitting on the table is not moving. It is not moving     because:     A. ...
Questioning in math: Discussion52     Look at the following sequence:     3, 7, 11, 15, 19, ….     Which is the best rule ...
Questioning in math: Diagnosis53     In which of these right-angled triangles is a2 + b2 = c2 ?           A            b  ...
Questioning in English: Discussion54     Macbeth: mad or bad?
Questioning in English: Diagnosis55     Where is the verb in this sentence?     The dog ran across the road          A B  ...
Questioning in English: Diagnosis (2)56     Which of these is correct?     A. Its on its way.     B. It’s on its way.     ...
Questioning in English: Diagnosis (3)57     Identify the adverbs in these sentences:      1. The boy ran across the street...
Questioning in English: Diagnosis (4)58     Which of these is the best thesis statement?     A. The typical TV show has 9 ...
Questioning in history: Discussion59     In which year did World War II begin?     A.   1919     B.   1938     C.   1939  ...
Questioning in history: Diagnosis60     Why are historians concerned with bias when analyzing     sources?     A. People c...
Questioning in MFL: Discussion61     Is the verb “être” regular in French?
Questioning in MFL: Diagnosis62     Which of the following is the correct translation for “I     give the book to him”?   ...
Hinge questions63        A hinge question is based on the important concept in a         lesson that is critical for stud...
Real-time test: Figurative language64                            1.   He was like a bull in a china shop.                 ...
Providing feedback that moveslearners forward
Kinds of feedback: Israel66        264 low and high ability grade 6 students in 12 classes in 4         schools; analysis...
Responses67                           Achievement            Attitude     Scores                   no gain      High score...
Kinds of feedback: Israel (2)68        200 grade 5 and 6 Israeli students        Divergent thinking tasks        4 matc...
Effects of feedback69        Kluger & DeNisi (1996) review of 3000 research reports        Excluding those:            ...
Getting feedback right is hardResponse type                  Feedback indicates performance…                  falls short ...
Provide feedback that moves learning on71        Key idea: feedback should:          Cause thinking          Provide gu...
Activating students as learningresources for one another
Help students be learning resources73        Students assessing their peers’ work:          “Pre-flight checklist”      ...
Benefits of structured interaction74        15-yr-olds studying World History were tested on their         understanding ...
Impact on achievement75                 100                   90                                                Structured...
Activating students as owners of theirown learning
Self-assessment: Portugal77       45 teachers studying for a Masters degree in Education,       matched in age, qualificat...
Details of the intervention78     Weeks      Intervention     1 to 2     Individual choice from a range of work provided b...
Impact on student achievement79                    Pre-test   Post-test   Gain   Effect size     Control         65.1     ...
Help students own their own learning80        Students assessing their own work:          With rubrics          With ex...
81
82
83
+/–/interesting: responses for “+”84        I got that ball-park estimates are supposed to be simple        I know that ...
+/–/interesting: responses for “–”85        I am still a tiny bit confused about subtraction regrouping        I am a li...
+/–/interesting: responses for “interesting”86        Carrying the number over to the next number        It’s interestin...
Self-assessment in pre-K87
All ready for action in third grade…89
Tell me about you…
IKEA mats…
So much for the easy bit
The happiness hypothesis (Haidt, 2005)                           +                            – The rider      Rational   ...
Strategies for change (Heath & Heath, 2010)    Direct the rider      Follow  the bright spots      Script the critical ...
A model for teacher learning95        Content, then process        Content (what we want teachers to change):          ...
Choice
A strengths-based approach to change97        Belbin inventory (Management teams: Why they         succeed or fail):     ...
Flexibility
Strategies and techniques99        Distinguish between strategies and techniques:          Strategiesdefine the territor...
Small steps
Expertise101         According to Berliner (1994), experts:             Excel mainly in their own domain             Of...
Knowing more than we can say102         Six video extracts of a person delivering          cardiopulmonary resuscitation ...
Looking at the wrong knowledge103         The most powerful teacher knowledge is not explicit:             That’s why te...
Sensory capacity104                                               Conscious                        Total bandwidth       S...
Hand hygiene in hospitals                  Study                     Focus     Compliance ratePreston, Larson, & Stamm (19...
Accountability
Making a commitment107         Action planning:             Forces teachers to make their ideas concrete and creates a r...
And being held to it108       “I think specifically what was helpful was the ridiculous NCR [No Carbon       Required] for...
Support
Supportive accountability110         What is needed from teachers:          A   commitment to:             The continua...
111Teacher learning communities
112         We need to create time and space for teachers to          reflect on their practice in a structured way, and ...
Teacher learning communities113         Plan that the TLC will run for two years.         Identify 10 to 12 interested c...
A “signature pedagogy” for teacher learning114         Every monthly TLC meeting should follow the same          structur...
Every TLC needs a leader115         The job of the TLC leader(s):           To ensure that all necessary resources (incl...
Peer observation116         Run to the agenda of the observed, not the          observer:           Observed      teache...
Summary117         Raising achievement is important.         Raising achievement requires improving teacher          qua...
Comments? Questions?www.dylanwiliam.net
Force-field analysis (Lewin, 1954)119         What are the forces that       What are the forces that          will supp...
Dylan Wiliam seminar for district leaders  accelerate learning with formative assessment 2013
Upcoming SlideShare
Loading in...5
×

Dylan Wiliam seminar for district leaders accelerate learning with formative assessment 2013

803

Published on

Dylan Wiliam, internationally recognized researcher, formative assessment expert and founder of Keeping Learning on Track® believes districts that want to improve academic performance should make embedded formative assessment a priority.

Published in: Education, Career
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
803
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
40
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Dylan Wiliam seminar for district leaders accelerate learning with formative assessment 2013

  1. 1. Embedding formativeassessment with teacherlearning communitiesDylan Wiliam www.dylanwiliam.net
  2. 2. Overview: Science and Design2  We need to improve student achievement  This requires improving teacher quality  Improving the quality of entrants takes too long  So we have to make the teachers we have better Science  We can change teachers in a range of ways  Some will benefit students, and some will not  Those that do involve changes in teacher practice  Changing practice requires new kinds of teacher learning Design  And new models of professional development
  3. 3. Raising achievement matters3  For individuals:  Increased lifetime salary  Improved health  Longer life  For society:  Lower criminal justice costs  Lower healthcare costs  Increased economic growth:  Net present value to the U.S. of a 25-point increase on PISA: $40 trillion (three times the National Debt)  Net present value to the U.S. of getting all students to 400 on PISA: $70 trillion
  4. 4. What is the purpose of education?4  Four main philosophies of education  Personal empowerment  Cultural transmission  Preparation for citizenship  Preparation for work  All are important  Any curriculum is a (sometimes uneasy) compromise between these four forces
  5. 5. Recession (2008-2010) and recovery (2010-2012)5  Those with a high school diploma or less  lost 5.6 million jobs in the recession, and  lost a further 230,000 jobs in the recovery  Those with an Associate’s degree  lost1.75 million jobs in the recession, but  gained 1.6 million jobs in the recovery  Those with at least a Bachelor’s degree  gained 187,000 jobs in the recession, and  gained a further 2 million jobs in the recovery Carnevale, Jayasundera, and Cheah (2012)
  6. 6. …with profound impacts on some workers6 Education level Change in salary 1978 to 2005 Postgraduate qualification +28% BA/BSc +19% Some college 0% High school diploma 0% High school dropout -16% Economic Policy Institute (2010)
  7. 7. The coming war for jobs (Clifton, 2011) Right now 7 billion people on earth  5 billion adults  3 billion people who want to work  90% of these want to work full time As a consequence  2.7billion full-time formal jobs are wanted  with only 1.2 billion full-time formal jobs available A shortfall of 1.5 billion jobs So, for every US worker, there are 10 people who would like their job…
  8. 8. A daunting target8  Programme for International Student Assessment (PISA)  United States 496  Canada 527  Finland 544  Shanghai 579
  9. 9. The world of work is changing…9 Skill category Percentage change 1969- 1999 Complex communication +14% Expert thinking/problem solving +8% Routine manual –3% Non-routine manual –5% Routine cognitive –8% Autor, Levy & Murnane (2003)
  10. 10. Off-shoring and automation10 Off-shoreable Not off-shoreable Radiographer Surgeon (?) Skilled Security analyst Bricklayer Tax accountant Hairdresser Food packager Grocery store clerk Unskilled Data entry clerk Receptionist Call centre operator Retail salesperson
  11. 11. Meet Maddie Parlier…11 Davidson (2012)
  12. 12. There is only one 21st century skill12 So the model that says learn while you’re at school, while you’re young, the skills that you will apply during your lifetime is no longer tenable. The skills that you can learn when you’re at school will not be applicable. They will be obsolete by the time you get into the workplace and need them, except for one skill. The one really competitive skill is the skill of being able to learn. It is the skill of being able not to give the right answer to questions about what you were taught in school, but to make the right response to situations that are outside the scope of what you were taught in school. We need to produce people who know how to act when they’re faced with situations for which they were not specifically prepared. (Papert, 1998)
  13. 13. Successful education?13 The test of successful education is not the amount of knowledge that a pupil takes away from school, but his appetite to know and his capacity to learn. If the school sends out children with the desire for knowledge and some idea how to acquire and use it, it will have done its work. Too many leave school with the appetite killed and the mind loaded with undigested lumps of information. The good schoolmaster is known by the number of valuable subjects that he declines to teach. The Future in Education (Livingstone, 1941 p. 28)
  14. 14. Where’s the solution?14  Structure:  Smaller/larger high schools  K–8 schools/“All-through” schools  Alignment:  Curriculum reform  Textbook replacement  Governance:  Charter schools  Vouchers  Technology:  Computers  Interactive whiteboards  Workforce reforms
  15. 15. Are private schools the answer?15  In PISA, U.S. students in private schools out- perform public school students by 25 points  But, after controlling for social class, public school students in the U.S. out-perform private school students by 10 points.
  16. 16. Pause for reflection What’s the most interesting, surprising, or challenging thing you have heard so far? See if you can get consensus with your neighbors
  17. 17. School effectiveness17 Three generations of school effectiveness research:  Raw results approaches:  Different schools get different results.  Conclusion: Schools make a difference.  Demographic-based approaches:  Demographic factors account for most of the variation.  Conclusion: Schools don’t make a difference.  Value-added approaches:  School-level differences in value-added are relatively small.  Classroom-level differences in value-added are large.  Conclusion: An effective school is a school full of effective classrooms.
  18. 18. We need to focus on classrooms, not schools18  In the USA, variability at the classroom level is at least four times that at school level.  Aslong as you go to school, it doesn’t matter very much which school you go to.  But it matters very much which classrooms you are in.  It’s not class size.  It’s not the between-class grouping strategy.  It’s not the within-class grouping strategy.
  19. 19. And most of all, on teachers19  Take a group of 50 teachers:  Students taught by the most effective teacher in that group of 50 teachers learn in six months what those taught by the average teacher learn in a year.  Students taught by the least effective teacher in that group of 50 teachers will take two years to achieve the same learning (Hanushek & Rivkin, 2006)  And furthermore:  In the classrooms of the most effective teachers, students from disadvantaged backgrounds learn at the same rate as those from advantaged backgrounds (Hamre & Pianta, 2005).
  20. 20. Improving teacher quality takes time20  A classic labor force issue with two (non-exclusive) solutions:  Replace existing teachers with better ones.  Help existing teachers become even more effective.
  21. 21. Replace existing teachers with better ones?21  De-select (i.e., fire) ineffective teachers?  Replace least effective 10% with average teachers 2 points on PISA (right away, if it can be done)  Raising the bar for entry into the profession?  Require teachers to have masters degrees 0 points on PISA (ever)  Exclude the lowest performing 30% from getting in 5 points on PISA (in 30 years time)  So we have to help the teachers we have improve  The “love the one you’re with” strategy
  22. 22. How do we speed up teacher improvement?22  Merit pay for effective teachers?  Can’t be done fairly, and doesn’t work  Improve the effectiveness of existing teachers:  It can be done:  Providedwe focus rigorously on the things that matter  Even when they’re hard to do  Create a culture of continuous improvement  But what should we help teachers improve?
  23. 23. The evidence base for formative assessment23  Fuchs & Fuchs (1986)  Nyquist (2003)  Natriello (1987)  Brookhart (2004)  Crooks (1988)  Allal & Lopez (2005)  Bangert-Drowns, et al. (1991)  Köller (2005)  Dempster (1991, 1992)  Brookhart (2007)  Elshout-Mohr (1994)  Wiliam (2007)  Kluger & DeNisi (1996)  Hattie & Timperley (2007)  Black & Wiliam (1998)  Shute (2008)
  24. 24. Which of these are formative?24 A. A district science supervisor uses test results to plan professional development workshops for teachers B. Teachers doing item-by-item analysis of 5th grade math tests to review their 5th grade curriculum C. A school tests students every 10 weeks to predict which students are “on course” to pass the state test in March D. “Three-fourths of the way through a unit” test E. Students who fail a test on Friday have to come back on Saturday F. Exit pass question: “What is the difference between mass and weight?” G. “Sketch the graph of y equals one over one plus x squared on your mini-white boards.”
  25. 25. The formative assessment hijack25  Long-cycle:  Span: across units, terms  Length: four weeks to one year  Impact: Student monitoring; curriculum alignment  Medium-cycle:  Span: within and between teaching units  Length: one to four weeks  Impact: Improved, student-involved assessment; teacher cognition about learning  Short-cycle:  Span: within and between lessons  Length:  day-by-day: 24 to 48 hours  minute-by-minute: five seconds to two hours  Impact: classroom practice; student engagement
  26. 26. Main approaches to formative assessment26  Professional Learning Communities “…an inclusive group of people, motivated by a shared learning vision, who support and work with each other, finding ways, inside and outside their immediate community, to enquire on their practice and together learn new and better approaches that will enhance all pupils’ learning.” (Stoll et al., 2006)  Two main approaches  Focus on outcomes for students (DuFour)  Focus on increased teacher capacity (Wiliam)
  27. 27. Complementary processes27 Data-driven PLCs Classroom FA TLCs • Quality control • Quality assurance • Common assessments • Highly structured meetings • Improvement through better • Improvement through team work and systems increased teacher capacity • Focus on individual outcomes • Focus on teachers’ individual for students accountability for change • Regular meetings focused on • Regular meetings focused on data teacher change • 16 points on PISA (in two to • 30 points on PISA (in two to three years) three years)
  28. 28. Unpacking formative assessment28 Where the learner is going Where the learner is How to get there Engineering effective Providing discussions, tasks, and feedback that Teacher activities that elicit moves learners Clarifying, evidence of learning forward sharing and understanding Peer Activating students as learning learning intentions resources for one another Learner Activating students as owners of their own learning
  29. 29. Formative assessment and other priorities29  Formative assessment is an integral part of many current policy priorities:  Framework for teaching (Danielson)  Common formative assessments (DuFour)  Differentiated instruction (Tomlinson)  Response to (instruction and) intervention
  30. 30. Framework for teaching (Danielson 1996)30  Four domains of professional practice 1. Planning and preparation 2. Classroom environment 3. Instruction 4. Professional responsibilities  Links with student achievement (Sartain, et al. 2011)  Domains 1 and 4: no impact on student achievement  Domains 2 and 3: some impact on student achievement
  31. 31. The framework in detail31  Domain 2: The classroom environment  2a: Creating an environment of respect and rapport  2b: Establishing a culture for learning  2c: Managing classroom procedures  2d: Managing student behavior  2e: Organizing physical space  Domain 3: Instruction  3a: Communicating with students  3b: Using questioning and discussion techniques  3c: Engaging students in learning  3d: Using assessment in instruction  3e: Demonstrating flexibility and responsiveness
  32. 32. Formative assessment and domain 332 Framework for teaching Classroom formative assessment  Communicating with  Sharing learning intentions students with students  Using questioning and  Eliciting evidence discussion techniques  Engaging students in  Feedback learning  Using assessment in  Students as learning instruction resources  Demonstrating flexibil-  Students as owners of ity and responsiveness their learning
  33. 33. Differentiated instruction: not a new idea33  Differentiation in action (Stradling & Saunders, 1993)  Differences in  educational goals  curriculum structure  course content  learning tasks  teaching approach  pace of learning  assessment  review
  34. 34. Most definitions of DI are vague34 “While the concept of ‘differentiated instruction’ can be defined in many ways, as good a definition as any is ensuring that what a student learns, how he/she learns it, and how the student demonstrates what he/she has learned is a match for that students readiness level, interests, and preferred mode of learning.” (Tomlinson, 2004 p. 188) “To differentiate instruction is to recognize students varying background knowledge, readiness, language, preferences in learning and interests; and to react responsively. Differentiated instruction is a process to teaching and learning for students of differing abilities in the same class.” (Hall, Strangman, & Meyer, 2011)
  35. 35. Differentiated instruction and formative assessment35 Aspects of differentiated instruction (Hall, Strangman & Meyer, 2008) FA? Several elements and materials are used Products Process Content Align tasks and objectives to learning goals  Instruction is concept-focused and principle-driven Flexible grouping is consistently used Classroom management benefits students and teachers Initial and on-going assessment of student readiness and growth  Students are active and responsible explorers  Vary expectations and requirements for student responses  Clarify key concepts and generalizations Miscellaneous Use assessment as a teaching tool  Emphasize critical and creative thinking as a goal in lesson design Engaging all learners is essential  Balance between teacher-assigned and student-selected tasks
  36. 36. Response to (instruction and) intervention36 “Response to intervention integrates assessment and intervention within a multi-level prevention system to maximize student achievement and reduce behavior problems. With RTI, schools identify students at risk for poor learning outcomes, monitor student progress, provide evidence-based interventions and adjust the intensity and nature of those interventions depending on a student’s responsiveness, and identify students with learning disabilities.” (National Center on Response to Intervention, 2010)  Two “creation myths” for RT(I)I  A protocol for preventing academic failure (progress monitoring, early—research-based—intervention)  An alternative to IQ testing in the identification of learning disabilities
  37. 37. Response to (instruction and) intervention37  Key points  Tier 1 must be high-quality, evidence-based instruction  Student progress must be monitored  Failure to progress triggers additional support  Formative assessment  Makes tier 1 instruction as effective as it can be  Allows assessment of progress (for tier 2 assessment)
  38. 38. And one big idea38 Where the learner is going Where the learner is How to get there Teacher Using evidence of achievement to adapt what Peer happens in classrooms to meet learner needs Learner
  39. 39. An educational positioning system39  A good teacher:  Establishes where the students are in their learning  Identifies the learning destination  Carefully plans a route  Begins the learning journey  Makes regular checks on progress on the way  Makes adjustments to the course as conditions dictate
  40. 40. Strategies and practicaltechniques for classroomformative assessment
  41. 41. Clarifying, sharing andunderstanding learning intentions
  42. 42. Sharing learning intentions42  3 teachers each teaching 4 7th grade science classes in two US schools  14 week experiment  7 two-week projects, each scored 2-10  All teaching the same, except:  For a part of each week  Two of each teacher’s classes discuss their likes and dislikes about the teaching (control)  The other two classes discuss how their work will be assessed White & Frederiksen (1998) Cognition & Instruction, 16(1)
  43. 43. Sharing learning intentions43 Comprehensive Test of Basic Skills Group Low Middle High Likes and dislikes 4.6 5.9 6.6 Reflective assessment 6.7 7.2 7.4 Who benefits most from reflective assessment? 1. Low achievers 2. Average students 3. High achievers 4. All students benefit equally
  44. 44. Share learning intentions44  Explain learning intentions at start of lesson/unit:  Learning intentions  Success criteria  Consider providing learning intentions and success criteria in students’language.  Use posters of key words to talk about learning:  E.g., describe, explain, evaluate  Use planning and writing frames.  Use annotated examples of different standards to “flesh out” assessment rubrics (e.g., lab reports).  Provide opportunities for students to design their own tests.
  45. 45. Engineering effective discussions,activities, and classroom tasks that elicitevidence of learning
  46. 46. Kinds of questions: Israel46 1 2 1 1 Which fraction is the smallest? a) , b) , c) , d) . 6 3 3 2 Success rate 88% 4 3 5 7 Which fraction is the largest? a) , b) , c) , d) . 5 4 8 10 Success rate 46%; 39% chose (b) Vinner (1997)
  47. 47. Draw an upside-down triangle…47
  48. 48. Inverted red triangle…48
  49. 49. Eliciting evidence49  Key idea: questioning should  cause thinking  provide data that informs teaching  Improving teacher questioning  generating questions with colleagues  low-order vs. high-order not closed vs. open  appropriate wait-time  Getting away from I-R-E  basketball rather than serial table-tennis  ‘No hands up’ (except to ask a question)  ‘Hot Seat’ questioning  All-student response systems  class poll, ABCD cards, ‘show-me’ boards, exit passes
  50. 50. Questioning in science: Discussion50 Ice-cubes are added to a glass of water. What happens to the level of the water as the ice-cubes melt? A. The level of the water drops B. The level of the water stays the same C. The level of the water increases D. You need more information to be sure
  51. 51. Questioning in science: Diagnosis51 The ball sitting on the table is not moving. It is not moving because: A. no forces are pushing or pulling on the ball. B. gravity is pulling down, but the table is in the way. C. the table pushes up with the same force that gravity pulls down D. gravity is holding it onto the table. E. there is a force inside the ball keeping it from rolling off the table Wilson & Draney (2004)
  52. 52. Questioning in math: Discussion52 Look at the following sequence: 3, 7, 11, 15, 19, …. Which is the best rule to describe the sequence? A. n + 4 B. 3 + n C. 4n - 1 D. 4n + 3
  53. 53. Questioning in math: Diagnosis53 In which of these right-angled triangles is a2 + b2 = c2 ? A b B c a a c b C a D c b b c a E a F b c c b a
  54. 54. Questioning in English: Discussion54 Macbeth: mad or bad?
  55. 55. Questioning in English: Diagnosis55 Where is the verb in this sentence? The dog ran across the road A B C D
  56. 56. Questioning in English: Diagnosis (2)56 Which of these is correct? A. Its on its way. B. It’s on its way. C. Its on it’s way. D. It’s on it’s way.
  57. 57. Questioning in English: Diagnosis (3)57 Identify the adverbs in these sentences: 1. The boy ran across the street quickly. (A) (B) (C) (D) (E) 2. Jayne usually crossed the street in a leisurely fashion. (A) (B) (C) (D) (E) 3. Fred ran the race well but unsuccessfully. (A) (B) (C) (D) (E)
  58. 58. Questioning in English: Diagnosis (4)58 Which of these is the best thesis statement? A. The typical TV show has 9 violent incidents B. The essay I am going to write is about violence on TV C. There is a lot of violence on TV D. The amount of violence on TV should be reduced E. Some programs are more violent than others F. Violence is included in programs to boost ratings G. Violence on TV is interesting H. I don’t like the violence on TV
  59. 59. Questioning in history: Discussion59 In which year did World War II begin? A. 1919 B. 1938 C. 1939 D. 1940 E. 1941
  60. 60. Questioning in history: Diagnosis60 Why are historians concerned with bias when analyzing sources? A. People can never be trusted to tell the truth B. People deliberately leave out important details C. People are only able to provide meaningful information if they experienced an event firsthand D. People interpret the same event in different ways, according to their experience E. People are unaware of the motivations for their actions F. People get confused about sequences of events
  61. 61. Questioning in MFL: Discussion61 Is the verb “être” regular in French?
  62. 62. Questioning in MFL: Diagnosis62 Which of the following is the correct translation for “I give the book to him”? A. Yo lo doy el libro. B. Yo doy le el libro. C. Yo le doy el libro. D. Yo doy lo el libro. E. Yo doy el libro le. F. Yo doy el libro lo.
  63. 63. Hinge questions63  A hinge question is based on the important concept in a lesson that is critical for students to understand before you move on in the lesson.  The question should fall about midway during the lesson.  Every student must respond to the question within two minutes.  You must be able to collect and interpret the responses from all students in 30 seconds
  64. 64. Real-time test: Figurative language64 1. He was like a bull in a china shop. 2. This backpack weighs a ton. A. Alliteration 3. The sweetly smiling sunshine… B. Hyperbole 4. He honked his horn at the cyclist. C. Onomatopoeia 5. He was as tall as a house. D. Personification E. Simile
  65. 65. Providing feedback that moveslearners forward
  66. 66. Kinds of feedback: Israel66  264 low and high ability grade 6 students in 12 classes in 4 schools; analysis of 132 students at top and bottom of each class  Same teaching, same aims, same teachers, same classwork  Three kinds of feedback: scores, comments, scores+comments Achievement Attitude Scores no gain High scorers : positive Low scorers: negative Comments 30% gain High scorers : positive Low scorers : positive Butler(1988) Br. J. Educ. Psychol., 58 1-14
  67. 67. Responses67 Achievement Attitude Scores no gain High scorers : positive Low scorers: negative Comments 30% gain High scorers : positive Low scorers : positive What happened for students given both scores and comments? A. Gain: 30%; Attitude: all positive B. Gain: 30%; Attitude: high scorers positive, low scorers negative C. Gain: 0%; Attitude: all positive D. Gain: 0%; Attitude: high scorers positive, low scorers negative E. Something else
  68. 68. Kinds of feedback: Israel (2)68  200 grade 5 and 6 Israeli students  Divergent thinking tasks  4 matched groups  experimental group 1 (EG1); comments  experimental group 2 (EG2); grades  experimental group 3 (EG3); praise  control group (CG); no feedback  Achievement  EG1>(EG2≈EG3≈CG)  Ego-involvement  (EG2≈EG3)>(EG1≈CG) Butler (1987)
  69. 69. Effects of feedback69  Kluger & DeNisi (1996) review of 3000 research reports  Excluding those:  without adequate controls  with poor design  with fewer than 10 participants  where performance was not measured  without details of effect sizes  left 131 reports, 607 effect sizes, involving 12652 individuals  On average, feedback increases achievement  Effect sizes highly variable  38% (50 out of 131) of effect sizes were negative
  70. 70. Getting feedback right is hardResponse type Feedback indicates performance… falls short of goal exceeds goalChange behavior Increase effort Exert less effortChange goal Reduce aspiration Increase aspirationAbandon goal Decide goal is too hard Decide goal is too easyReject feedback Feedback is ignored Feedback is ignored
  71. 71. Provide feedback that moves learning on71  Key idea: feedback should:  Cause thinking  Provide guidance on how to improve  Comment-only grading  Focused grading  Explicit reference to rubrics  Suggestions on how to improve:  Not giving complete solutions  Re-timing assessment:  E.g., three-fourths-of-the-way-through-a-unit test
  72. 72. Activating students as learningresources for one another
  73. 73. Help students be learning resources73  Students assessing their peers’ work:  “Pre-flight checklist”  “Two stars and a wish”  Choose-swap-choose  Daily sign-in  Training students to pose questions/identifying group weaknesses  End-of-lesson students’ review
  74. 74. Benefits of structured interaction74  15-yr-olds studying World History were tested on their understanding of material delivered in lectures  Half the students were trained to pose questions as they listened to the lectures  At the end of the lectures, students were given time to review their understanding of the material Individual Group Unstructured Independent review Group discussion Structured Structured self- Structured peer- questioning questioning
  75. 75. Impact on achievement75 100 90 Structured peer questioning 80 Structured self- questioning Score 70 Group discussion 60 Independent 50 review 40 Pre Post 10-day King, A. (1991). Applied Cognitive Psychology, 5(4), 331-346.
  76. 76. Activating students as owners of theirown learning
  77. 77. Self-assessment: Portugal77 45 teachers studying for a Masters degree in Education, matched in age, qualifications and experience using the same curriculum scheme for the same amount of time Control group (N=20) follow Experimental group (N=25) regular MA program develop self-assessment with their students 117 students aged 8 years 125 students aged 8 years 119 students aged 9 years 121 students aged 9 years 77 students aged 10 - 14 years 108 students aged 10 - 14 years Fontana & Fernandes, Br. J. Educ. Psychol. 64: 407-417
  78. 78. Details of the intervention78 Weeks Intervention 1 to 2 Individual choice from a range of work provided by the teacher. Student self-assessment using materials provided 3 to 6 Children construct own problems like those in weeks 1 and 2 and select structured math apparatus to aid solutions 7 to 10 Children presented with a new learning objectives, and make up their own problems, without exemplars by the teacher 11 to 14 Children set their own learning objectives, construct appropriate problems, and use appropriate self-assessment 15 to 20 As weeks 1 to 14, but with less monitoring from the teacher and increased freedom of choice and personal responsibility
  79. 79. Impact on student achievement79 Pre-test Post-test Gain Effect size Control 65.1 72.9 7.8 0.34 Experimental 58.7 73.7 15.0 0.66
  80. 80. Help students own their own learning80  Students assessing their own work:  With rubrics  With exemplars  Self-assessment of understanding:  Learning portfolio  Traffic lights  Red/green discs  Colored cups  Plus/minus/interesting
  81. 81. 81
  82. 82. 82
  83. 83. 83
  84. 84. +/–/interesting: responses for “+”84  I got that ball-park estimates are supposed to be simple  I know that you have to look at it and say “OK”  I know that when I am adding the number I end up with must be bigger than the one I started at  I get most of the problems  It was easy for me because on the first one it says 328 so I took the 2 and made it a 12  I know that we would have to regroup  I know how to do plus and minus because we have been doing it for a long time  I get it when you cross out a number and make it a new one  I know that when you can’t – from both colomes you go to the third colome and take that from it  I know that when my answer is right the ball park estimate is close to it
  85. 85. +/–/interesting: responses for “–”85  I am still a tiny bit confused about subtraction regrouping  I am a little bit confused about ball park estimates  I get confused because sometimes I don’t get the problem  I am confused when you subtract really big numbers like 1,000 something  I’m still a little bit confused about regrouping  Minus is confusing when you have to regroup twice  Minus is a little bit hard when you have to regroup  I don’t understand when you borrow which colome you borrow from when both are 0  I am still confused about showing what I did to solve the problem  I am a little confused about when you need to subtract
  86. 86. +/–/interesting: responses for “interesting”86  Carrying the number over to the next number  It’s interesting how some people go to the nearest hundred while some go to the nearest ten  It’s interesting how some have to regroup twice  It’s pretty interesting about how you have to work really hard  I am interested in borrowing because I didn’t just get it yet. I want to really get to know it  I find it weird that you could just keep going from colome to colome when you need to borrow  On the ball park estimate it is easy but sometimes hard  I really think that regrouping is pretty amazing  It is cool how addition and subtraction regrouping is just moving numbers and you could get it right easily
  87. 87. Self-assessment in pre-K87
  88. 88. All ready for action in third grade…89
  89. 89. Tell me about you…
  90. 90. IKEA mats…
  91. 91. So much for the easy bit
  92. 92. The happiness hypothesis (Haidt, 2005) + – The rider Rational Weak Good at complex analysis Easily distracted Focused on the long-term Gets bogged down in detail Thinks about the future Tires quickly The elephant Instinctive Emotional Compassionate Skittish Sympathetic Focused on the short-term Loyal Thinks about the present Protective Powerful
  93. 93. Strategies for change (Heath & Heath, 2010)  Direct the rider  Follow the bright spots  Script the critical moves  Point to the destination  Motivate the elephant  Find the feeling  Shrink the change  Grow your people  Shape the path  Tweak the environment  Build habits  Rally the herd
  94. 94. A model for teacher learning95  Content, then process  Content (what we want teachers to change):  Evidence  Ideas (strategies and techniques)  Process (how to go about change):  Choice  Flexibility  Smallsteps  Accountability  Support
  95. 95. Choice
  96. 96. A strengths-based approach to change97  Belbin inventory (Management teams: Why they succeed or fail):  Eight team roles (defined as “a tendency to behave, contribute and interrelate with others in a particular way”):  Company worker; innovator; shaper; chairperson; resource investigator; monitor/evaluator; completer/finisher; team worker  Key ideas:  Each role has strengths and allowable weaknesses.  People rarely sustain “out-of-role” behavior, especially under stress.  Each teacher’s personal approach to teaching is similar:  Some teachers’ weaknesses require immediate attention.  For most, however, students benefit more from the development of teachers’ strengths.
  97. 97. Flexibility
  98. 98. Strategies and techniques99  Distinguish between strategies and techniques:  Strategiesdefine the territory of formative assessment (no-brainers).  Teachers are responsible for choice of techniques:  Allows for customization; caters for local context  Creates ownership; shares responsibility  Key requirements of techniques:  They embody the deep cognitive and affective principles that research shows are important.  They are seen as relevant, feasible, and acceptable.
  99. 99. Small steps
  100. 100. Expertise101  According to Berliner (1994), experts:  Excel mainly in their own domain  Often develop automaticity for the repetitive operations that are needed to accomplish their goals  Are more sensitive to the task demands and social situation when solving problems  Are more opportunistic and flexible in their teaching than novices  Represent problems in qualitatively different ways than novices  Have faster and more accurate pattern recognition capabilities  Perceive meaningful patterns in the domain in which they are experienced  Begin to solve problems slower but bring richer and more personal sources of information to bear
  101. 101. Knowing more than we can say102  Six video extracts of a person delivering cardiopulmonary resuscitation (CPR):  Fiveof the video extracts feature students.  One of the video extracts feature an expert.  Videos shown to three groups  students, experts, instructors  Success rate in identifying the expert:  Experts: 90%  Students: 50%  Instructors: 30% (Klein & Klein, 1981)
  102. 102. Looking at the wrong knowledge103  The most powerful teacher knowledge is not explicit:  That’s why telling teachers what to do doesn’t work.  What we know is more than we can say.  And that is why most professional development has been relatively ineffective.  Improving practice involves changing habits, not adding knowledge:  That’s why it’s hard:  And the hardest bit is not getting new ideas into people’s heads.  It’s getting the old ones out.  That’s why it takes time.  But it doesn’t happen naturally:  If it did, the most experienced teachers would be the most productive, and that’s not true (Hanushek & Rivkin, 2006).
  103. 103. Sensory capacity104 Conscious Total bandwidth Sensory system bandwidth (in bits/second) (in bits/second) Eyes 10,000,000 40 Ears 100,000 30 Skin 1,000,000 5 Taste 1,000 1 Smell 100,000 1 (Nørretranders, 1998)
  104. 104. Hand hygiene in hospitals Study Focus Compliance ratePreston, Larson, & Stamm (1981) Open ward 16% ICU 30%Albert & Condie (1981) ICU 28% to 41%Larson (1983) All wards 45%Donowitz (1987) Pediatric ICU 30%Graham (1990) ICU 32%Dubbert (1990) ICU 81%Pettinger & Nettleman (1991) Surgical ICU 51%Larson, et al. (1992) Neonatal ICU 29%Doebbeling, et al. (1992) ICU 40%Zimakoff, et al. (1992) ICU 40%Meengs, et al. (1994) ER (Casualty) 32%Pittet, Mourouga, & Perneger (1999) All wards 48% ICU 36% (Pittet, 2001)
  105. 105. Accountability
  106. 106. Making a commitment107  Action planning:  Forces teachers to make their ideas concrete and creates a record  Makes the teachers accountable for doing what they promised  Requires each teacher to focus on a small number of changes  Requires the teachers to identify what they will give up or reduce  A good action plan:  Does not try to change everything at once  Spells out specific changes in teaching practice  Relates to the five “key strategies” of AFL  Is achievable within a reasonable period of time  Identifies something that the teacher will no longer do or will do less of
  107. 107. And being held to it108 “I think specifically what was helpful was the ridiculous NCR [No Carbon Required] forms. I thought that was the dumbest thing, but I’m sitting with my friends and on the NCR form I write down what I am going to do next month. “Well, it turns out to be a sort of ‘I’m telling my friends I’m going to do this’ and I really actually did it and it was because of that. It was because I wrote it down. “I was surprised at how strong an incentive that was to do actually do something different…that idea of writing down what you are going to do and then because when they come by the next month you better take out that piece of paper and say ‘Did I do that?’…just the idea of sitting in a group, working out something, and making a commitment…I was impressed about how that actually made me do stuff.” —Tim, Spruce Central High School
  108. 108. Support
  109. 109. Supportive accountability110  What is needed from teachers: A commitment to:  The continual improvement of practice  Focus on those things that make a difference to students  What is needed from leaders: A commitment to engineer effective learning environments for teachers by:  Creating expectations for continually improving practice  Keeping the focus on the things that make a difference to students  Providing the time, space, dispensation, and support for innovation  Supporting risk-taking
  110. 110. 111Teacher learning communities
  111. 111. 112  We need to create time and space for teachers to reflect on their practice in a structured way, and to learn from mistakes. (Bransford, Brown & Cocking, 1999)  “Always make new mistakes.” —Esther Dyson  “Ever tried. Ever failed. No matter. Try again. Fail again. Fail better.” (Beckett, 1984)
  112. 112. Teacher learning communities113  Plan that the TLC will run for two years.  Identify 10 to 12 interested colleagues:  Composition:  Similar assignments (e.g., early years, math/science)  Mixed subject/mixed phase  Hybrid  Secure institutional support for:  Monthly meetings (75–120 minutes each, inside or outside school time)  Time between meetings (two hours per month in school time):  Collaborative planning  Peer observation  Any necessary waivers from school policies
  113. 113. A “signature pedagogy” for teacher learning114  Every monthly TLC meeting should follow the same structure and sequence of activities:  Activity 1: Introduction and starter (10 minutes)  Activity 2: How’s it going? (25–50 minutes)  Activity 3: New learning about formative assessment (20–40 minutes)  Activity 4: Personal action planning (15 minutes)  Activity 5: Review of learning (5 minutes)
  114. 114. Every TLC needs a leader115  The job of the TLC leader(s):  To ensure that all necessary resources (including refreshments!) are available at meetings  To ensure that the agenda is followed  To maintain a collegial and supportive environment  But most important of all:  It is not to be the formative assessment “expert.”
  115. 115. Peer observation116  Run to the agenda of the observed, not the observer:  Observed teacher specifies focus of observation:  E.g., teacher wants to increase wait time.  Observed teacher specifies what counts as evidence:  Provides observer with a stopwatch to log wait times.  Observed teacher owns any notes made during the observation.
  116. 116. Summary117  Raising achievement is important.  Raising achievement requires improving teacher quality.  Improving teacher quality requires teacher professional development.  To be effective, teacher professional development must address:  What teachers do in the classroom  How teachers change what they do in the classroom  Formative assessment + teacher learning communities:  A point of (uniquely?) high leverage  A “Trojan horse” into wider issues of pedagogy, psychology, and curriculum
  117. 117. Comments? Questions?www.dylanwiliam.net
  118. 118. Force-field analysis (Lewin, 1954)119  What are the forces that  What are the forces that will support or drive the will constrain or prevent adoption of formative the adoption of formative assessment practices in assessment practices in your school/district? your school/district? + —
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×