Download to read offline
EpA_Implemetnación08.ppt
- 1. Classroom Assessment: Minute-by-minute and day-by-day DylanWiliam www.dylanwiliam.net
- 2. Overview of presentation Whyraising achievement is important Why investing in teachers is the answer Why formative assessment should be the focus Why teacher learning communities should be the mechanism How we can put this into practice
- 3. Raising achievement matters Forindividuals Increased lifetime salary Improved health Longer life For society Lower criminal justice costs Lower health-care costs Increased economic growth
- 4. Where’s the solution? Structure Smallerhigh schools K-8 schools Alignment Curriculum reform Textbook replacement Governance Charter schools Vouchers Technology Computers Interactive white-boards
- 5. School effectiveness Three generationsof school effectiveness research Raw results approaches Different schools get different results Conclusion: Schools make a difference Demographic-based approaches Demographic factors account for most of the variation Conclusion: Schools don’t make a difference Value-added approaches School-level differences in value-added are relatively small Classroom-level differences in value-added are large Conclusion: An effective school is a school full of effective classrooms
- 6. How important isteacher quality? How much progress will an average student make when taught by a great teacher (i.e., the best teacher in a group of 50)? A. An extra month per year B. An extra two months per year C. An extra three months per year D. An extra four months per year E. An extra six months per year
- 7. Teacher quality A laborforce issue with 2 solutions Replace existing teachers with better ones? No evidence that more pay brings in better teachers No evidence that there are better teachers out there deterred by burdensome certification requirements Improve the effectiveness of existing teachers The “love the one you’re with” strategy It can be done We know how to do it, but at scale? Quickly? Sustainably?
- 8. 20-25% Total “explained” difference <5% Furtherprofessional qualifications (NBPTS) 10-15% Pedagogical content knowledge <5% Advanced content matter knowledge The ‘dark matter’ of teacher quality Teachers make a difference But what makes the difference in teachers?
- 9. Cost/effect comparisons Intervention Extramonths of learning per year Cost/yr Class-size reduction (by 30%) 3 $30k Increase teacher content knowledge from weak to strong 1.5 ? Formative assessment/ Assessment for learning 6 to 9 $3k
- 10. The research evidence Severalmajor reviews of the research Natriello (1987) Crooks (1988) Kluger & DeNisi (1996) Black & Wiliam (1998) Nyquist (2003) All find consistent, substantial effects
- 11. Types of formativeassessment Long-cycle Span: across units, terms Length: four weeks to one year Impact: Student monitoring; curriculum alignment Medium-cycle Span: within and between teaching units Length: one to four weeks Impact: Improved, student-involved, assessment; teacher cognition about learning Short-cycle Span: within and between lessons Length: day-by-day: 24 to 48 hours minute-by-minute: 5 seconds to 2 hours Impact: classroom practice; student engagement
- 12. Unpacking formative assessment Keyprocesses Establishing where the learners are in their learning Establishing where they are going Working out how to get there Participants Teachers Peers Learners
- 13. Aspects of formativeassessment Where the learner is going Where the learner is How to get there Teacher Clarify and share learning intentions Engineering effective discussions, tasks and activities that elicit evidence of learning Providing feedback that moves learners forward Peer Understand and share learning intentions Activating students as learning resources for one another Learner Understand learning intentions Activating students as owners of their own learning
- 14. Sharing learning intentions Explaininglearning intentions at start of lesson/unit Learning intentions Success criteria Intentions/criteria in students’ language Posters of key words to talk about learning eg describe, explain, evaluate Planning/writing frames Annotated examples of different standards to ‘flesh out’ assessment rubrics (e.g. lab reports) Opportunities for students to design their own tests
- 15. Eliciting evidence ofachievement Key idea: questioning should cause thinking provide data that informs teaching Improving teacher questioning generating questions with colleagues closed vs. open or low-order vs. high-order appropriate wait-time Getting away from I-R-E basketball rather than serial table-tennis ‘No hands up’ (except to ask a question) ‘Hot Seat’ questioning All-student response systems ABCD cards, Mini white-boards, Exit passes
- 16. Feedback that moveslearning on Key idea: feedback should cause thinking provide guidance on how to improve Comment-only grading Focused grading Explicit reference to mark-schemes and scoring guides Suggestions on how to improve ‘Strategy cards’ ideas for improvement Not giving complete solutions Re-timing assessment (eg two-thirds-of-the-way-through-a-unit test)
- 17. Students as ownersof their learning Students assessing their own work with rubrics with exemplars Self-assessment of understanding Traffic lights Red/green discs Colored cups
- 18. Students as instructionalresources Students assessing their peers’ work “pre-flight check-list” “two stars and a wish” Training students to pose questions/identifying group weaknesses End-of-lesson students’ review
- 19. …and one bigidea Use evidence about learning to adapt teaching and learning to meet student needs
- 20. Keeping Learning onTrack (KLT) A pilot guides a plane or boat toward its destination by taking constant readings and making careful adjustments in response to wind, currents, weather, etc. A KLT teacher does the same: Plans a carefully chosen route ahead of time (in essence building the track) Takes readings along the way Changes course as conditions dictate
- 21. Putting it intopractice
- 22. Implementing FA/AfL requires changingteacher habits Teachers “know” most of this already So the problem is not a lack of knowledge It’s a lack of understanding what it means to do FA/AfL That’s why telling teachers what to do doesn’t work Experience alone is not enough—if it were, then the most experienced teachers would be the best teachers—we know that’s not true (Hanushek, 2005; Day, 2006) People need to reflect on their experiences in systematic ways that build their accessible knowledge base, learn from mistakes, etc. (Bransford, Brown & Cocking, 1999)
- 23. A model forteacher learning Content, then process Content (what we want teachers to change) Evidence Ideas (strategies and techniques) Process (how to go about change) Choice Flexibility Small steps Accountability Support
- 24. Strategies and techniques Distinctionbetween strategies and techniques Strategies define the territory of AfL (no brainers) Teachers are responsible for choice of techniques Allows for customization/ caters for local context Creates ownership Shares responsibility Key requirements of techniques embodiment of deep cognitive/affective principles relevance feasibility acceptability
- 25. Teacher learning takestime To put new knowledge to work, to make it meaningful and accessible when you need it, requires practice. A teacher doesn’t come at this as a blank slate. Not only do teachers have their current habits and ways of teaching— they’ve lived inside the old culture of classrooms all their lives: every teacher started out as a student! New knowledge doesn’t just have to get learned and practiced, it has to go up against long-established, familiar, comfortable ways of doing things that may not be as effective, but fit within everyone’s expectations of how a classroom should work. It takes time and practice to undo old habits and become graceful at new ones. Thus… Professional development must be sustained over time
- 26. That’s what teacherlearning communities (TLCs) are for: TLCs contradict teacher isolation TLCs reprofessionalize teaching by valuing teacher expertise TLCs deprivatize teaching so that teachers’ strengths and struggles become known TLCs offer a steady source of support for struggling teachers They grow expertise by providing a regular space, time, and structure for that kind of systematic reflecting on practice They facilitate sharing of untapped expertise residing in individual teachers They build the collective knowledge base in a school
- 27. How to setup a TLC Plan that the TLC will run for two years Identify 8 to 10 interested colleagues Should have similar assignments (e.g. early years, math/sci) Secure institutional support for: Monthly meetings (75 to 120 minutes each, inside or outside school time) Time between meetings (2 hrs per month in school time) Collaborative planning Peer observation Any necessary waivers from school policies
- 28. A ‘signature pedagogy’for teacher learning? Every monthly TLC meeting should follows the same structure and sequence of activities Activity 1: Introduction & Housekeeping (5-10 minutes) Activity 2: How’s It Going (35-50 minutes) Activity 3: New Learning about AfL (20-45 minutes) Activity 4: Personal Action Planning (10 minutes) Activity 5: Summary of Learning (5 minutes)
- 29. The TLC leader’srole To ensure the TLC meets regularly To ensure all needed materials are at meetings To ensure that each meeting is focused on AfL To create and maintain a productive and non-judgmental tone during meetings To ensure that every participant shares with regard to their implementation of AfL To encourage teachers to provide their colleagues with constructive and thoughtful feedback To encourage teachers to think about and discuss the implementation of new AfL learning and skills To ensure that every teacher has an action plan to guide their next steps But not to be the AfL “expert”
- 30. Peer observation Run tothe agenda of the observed, not the observer Observed teacher specifies focus of observation Observe teacher specifies what counts as evidence e.g., teacher wants to increase wait-time provides observer with a stop-watch to log wait-times
- 31. Implementations Current pilots in: ClevelandMunicipal School District, OH Austin Independent School District, TX Chico Unified School District, CA Mathematics and Science Partnership of Greater Philadelphia, PA/NJ St. Mary’s County Public Schools, MD State-wide pilot in 10 schools in Vermont Further details: www.ets.org/klt
- 32. Summary Raising achievement isimportant Raising achievement requires improving teacher quality Improving teacher quality requires teacher professional development To be effective, teacher professional development must address What teachers do in the classroom How teachers change what they do in the classroom AfL/FA + TLCs A point of (uniquely?) high leverage A “Trojan Horse” into wider issues of pedagogy, psychology, and curriculum
Editor's Notes
- #15 Most of these strategies are self-explanatory. Planning and writing frames provide a structure to help students develop a response. While some teachers see such frames as constricting, for most students they provide valuable ‘scaffolding’ for their answers.
- #16 In many classrooms teachers use questions as a way of directing the attention of the class, and keeping students ‘on task’. This may keep students ‘on their toes’ but rarely helps learning. It may be better in a whole-class lesson, to have an extended exchange with a single student. This ‘hot seat’ questioning also has benefits for other students who appear to learn vicariously from the exchange. Other students may see extended exchanges between the teacher and another student as a chance to relax and go ‘off task’, but if theteacher asks them what they have learned from a particular exchange between another student and the teacher, their concentration is likely to be quite high! How much time a teacher allows a student to respond before evaluating the response is also important. It is well known that teachers do not allow students much time to answer questions, and, if they don’t receive a response quickly, they will ‘help’ the student by providing a clue or weakening the question in some way, or even moving on to another student. However, what is not widely appreciated is that the amount of time between the student providing an answer and the teacher’s evaluation of that answer is much more important. Of course, where the question is a simple matter of factual recall, then allowing a student time to reflect and expand upon the answer is unlikely to help much. But where the question requires thought, then increasing the time between the end of the student’s answer and the teacher’s evaluation from the average ‘wait-time’ of less than a second to three seconds, produces measurable increases in learning (although increases beyond three seconds have little effect, and may cause lessons to lose pace). In fact, questions need not always come from the teacher. There is substantial evidence that students’ learning is enhanced by getting them to generate their own questions. If instead of writing an end-of-topic test herself, the teacher asks the students to write a test that tests the work the class has been doing, the teacher can gather useful evidence about what the students think they have been learning, which is often very different from what the teacher thinks the class has been learning. This can be a particularly effective strategy with disaffected older students, who often feel threatened by tests. Asking them to write a test for the topic they have completed, and making clear that the teacher is going to mark the question rather than the answers, can be a hugely liberating experience for many students.
- #17 The research evidence suggests that feedback in terms of scores, grades and levels is unlikely to improve achievement, but that feeding back in terms of comments (whether written or verbal) is. This immediately raises the question “what kind of comments”, and although there is no specific research evidence on this point, it seems that, to be useful, a comment should cause thinking to take place. This feature of ‘mindfulness’ is one of the crucial features of effective formative assessment—effective learning involves having most of the students thinking most of the time. This notion of ‘mindfulness’ also gives some clues about what sort of marking is most helpful. Many teachers say that formative feedback is less useful in mathematics, because an answer is either wrong or right. But even where answers are wrong or right, we can still encourage students to think. For example, rather than marking answers right and wrong and telling the students to do corrections, teachers could, instead, feed back saying simply “Three of these ten questions are wrong. Find out which ones and correct them”. After all, we are often telling our students to check their work, but rarely help them develop the skills to do so. Other strategies that are useful are focused gradin—ie grading a particular piece of work for one aspect (such as sentence structure, expression or spelling) rather than trying to correct everything. This is particularly useful if the comments can be related directly to the assessment criteria for the work. Of course, it is very difficult for feedback to function formatively at the end of a unit so rather than an ‘end-of-unit test’ it may be more useful to have a ‘two-thirds-of-the-way-through-a-unit test’. Those students who have understood something can then help those who haven’t. Many teachers sometimes worry that such strategies may hold back abler students, but the research evidence suggests that it is the students who give help who benefit most from such peer-tutoring. While this may not accelerate more able students through the curriculum, it does lead to better long-term retention.
- #18 Again, most of these strategies are self-explanatory. With ‘traffic-lights’ the teacher identifies a small number of objectives for the lesson (perhaps only one), which are made as clear as possible to the students at the beginning of the lesson. At the end of the lesson, students are asked to indicate their understanding of each objective by a green, yellow or red circle, according to whether they feel they have achieved the objective fully, partially or not at all. This provides useful feedback to the teacher at two levels—to see if there are parts of the lesson that it would be worth re-doing with the whole class, but also to get feedback about which students would particularly benefit from individual support (one technique here has been to ask ambers and greens to work together, while the teacher works with the reds). However, the real benefit of such a system is that it forces the student to reflect on what she or he has been learning. Other teachers have used ‘smiley faces’ which have the advantage, if drawn in pencil, of being modifiable when the student is more confident about their understanding. The other strategy that may require explanation is that of ‘end-of-lesson’ reviews. The idea here is that at the beginning of the lesson, one student is appointed as a ‘rapporteur’ for the lesson. The teacher then teaches a whole-class lesson on some topic, and finishes the lesson ten or fifteen minutes before the end of the lesson. The student rapporteur then gives a summary of the main points of the lesson, and tries to answer any remaining questions that students in the class may have. If he or she can’t answer the questions, then the rapporteur asks members of the class to help out. What is surprising is that teachers who have tried this out have found that students are queuing up to play the role of rapporteur, provided this is started at the beginning of the school year, or even better, when students are new to the school.
- #19 Again, most of these strategies are self-explanatory. With ‘traffic-lights’ the teacher identifies a small number of objectives for the lesson (perhaps only one), which are made as clear as possible to the students at the beginning of the lesson. At the end of the lesson, students are asked to indicate their understanding of each objective by a green, yellow or red circle, according to whether they feel they have achieved the objective fully, partially or not at all. This provides useful feedback to the teacher at two levels—to see if there are parts of the lesson that it would be worth re-doing with the whole class, but also to get feedback about which students would particularly benefit from individual support (one technique here has been to ask ambers and greens to work together, while the teacher works with the reds). However, the real benefit of such a system is that it forces the student to reflect on what she or he has been learning. Other teachers have used ‘smiley faces’ which have the advantage, if drawn in pencil, of being modifiable when the student is more confident about their understanding. The other strategy that may require explanation is that of ‘end-of-lesson’ reviews. The idea here is that at the beginning of the lesson, one student is appointed as a ‘rapporteur’ for the lesson. The teacher then teaches a whole-class lesson on some topic, and finishes the lesson ten or fifteen minutes before the end of the lesson. The student rapporteur then gives a summary of the main points of the lesson, and tries to answer any remaining questions that students in the class may have. If he or she can’t answer the questions, then the rapporteur asks members of the class to help out. What is surprising is that teachers who have tried this out have found that students are queuing up to play the role of rapporteur, provided this is started at the beginning of the school year, or even better, when students are new to the school.