Hélène Galdin-O'Shea, profile picture
Uploaded byHélène Galdin-O'Shea
PPTX, PDF2,450 views

Dylan Wiliam's slides - 2013 11-21 bcssa

This document discusses principles of curriculum and assessment design for schools. It outlines seven principles for curriculum design: balanced, rigorous, coherent, vertically integrated, appropriate, focused, and relevant. It argues the most important principle is appropriate, as curriculum should match students' developmental levels. The least important is focused/parsimonious, as not overloading students is less critical than other principles. The document also discusses different functions of assessment, including evaluating institutions, describing individuals, and supporting learning.

Embed presentation

Downloaded 18 times
Principled curriculum and assessment design: Tools for schools Dylan Wiliam British Columbia School Superintendents Association Fall Conference November 2013, Vancouver, BC. 21 Nov 2013 www.dylanwiliam.org @dylanwiliam
Outline  Curriculum  What is curriculum?  Seven principles for curriculum design  Assessment  Assessment for accountability  Quality in assessment
Why do we educate young people?  Broad views on the philosophy of education (Williams, 1961)  Transmission of culture (e.g., Arnold)  Preparation for work (e.g., OECD)  Preparation for effective citizenship (e.g., Freire)  Preparation for life  Any curriculum is a sometimes messy compromise between all of these
Curriculum: an evolving concept   The courses taken (Scottish HE, late 17th century) Four questions (Tyler, 1949)       What educational purposes should the school seek to attain? What educational experiences … are likely to attain these purposes? How can these educational experiences be effectively organized? How can we determine whether these purposes are being attained? “All the learning which is planned or guided by the school, whether it is carried on in groups or individually inside or outside the school.” (Kerr, 1968 p. 16) “the school curriculum (in the wider sense) is essentially a selection from the culture of a society.” (Lawton 1975 p. 7)
What is curriculum really?  Three levels of curriculum  The intended curriculum  The curriculum  The mandated by government agencies implemented curriculum  The curriculum realized in textbooks, schemes of work, lesson plans, etc.  The achieved curriculum  The lived  daily experience of learners in schools Each of these has explicit and tacit (hidden) aspects
And what is not there is also important…  The null curriculum: “the options students are not afforded; the perspectives they may never know about, much less be able to use; the concepts and skills that are not part of their intellectual repertoire” (Eisner 1985, p.107).
The role of teachers (Stenhouse, 1975)   “A curriculum is an attempt to communicate the essential principles and features of an educational proposal in such a form that it is open to critical scrutiny and capable of effective translation into practice.” (p. 5) The proposal should have three parts: A. In planning B. In empirical study C. In relation to justification
Principles of curriculum design  A good curriculum is:  Balanced  Rigorous  Coherent  Vertically integrated  Appropriate  Focused/parsimonious  Relevant
Balanced: which subjects?            English Mathematics Science Technology Modern foreign languages Geography History Music Art Physical education Religious education            Drama Dance Chess Engineering Geology Astronomy Media studies Law Psychology Sociology Politics
Rigorous: subjects, disciplines, or skills?  Disciplinary habits of mind are important, specific, powerful ways of thinking that are developed through sustained engagement with the discipline.  Mathematics: transformation and invariance  History: provenance and context  Statistics: dispersion as well as central tendency  Sociology: structure and agency
21st Century skills  Cognitive competencies     Intra-personal competencies     Cognitive processes and strategies Knowledge Creativity Intellectual openness Work ethic/conscientiousness Positive core self-evaluation Inter-personal competencies   Team-work Leadership Pellegrino and Hilton (2012)
Coherent: subjects or themes?   Subject-based curricula support disciplines but tend to undermine coherence across different aspects of learning Theme-based curricula support coherence, but tend to undermine disciplinary development
Reading skills: what are they really? A manifold, contained in an intuition which I call mine, is represented, by means of the synthesis of the understanding, as belonging to the necessary unity of self-consciousness; and this is effected by means of the category. What is the main idea of this passage? 

1. Without a manifold, one cannot call an intuition ‘mine.’ 

2. Intuition must precede understanding. 

3. Intuition must occur through a category. 

4. Self-consciousness is necessary to understanding Hirsch (2006)
John walked to first, stole second, got bunted over to third, and reached home on a sacrifice fly. How many outs were there when John got to the plate? A. 0 B. 1 C. 2
Lost in translation? Comprehension depends on constructing a mental model that makes the elements fall into place and, equally important, enables the listener or reader to supply essential information that is not explicitly stated. In language use, there is always a great deal that is left unsaid and must be inferred. This means that communication depends on both sides, writer and reader, sharing a basis of unspoken knowledge. This large dimension of tacit knowledge is precisely what is not being taught adequately in our schools. Hirsch (2009 loc. 176)
Reading is complex… (Scarborough, 2001)
Skill is content, content is skill Five propositions about academic skills (Hirsch, 2009) 1 The character of an academic skill is constrained by the limitations of short-term working memory. 2 Academic skills have two components: procedures and contents. 3 Procedural skills such as turning letters into sounds must initially be learned as content, along with other content necessary to higher-order skills. 4 An advance in skill, whether in procedure or content, entails an advance in speed of processing. 5 A higher-order academic skill such as reading comprehension requires prior knowledge of domain-specific content; the higher-order skills for that domain does not readily transfer to other content domains.
 SOLO taxonomy (Biggs & Collis, 1982)  Structure of observed learning outcomes  Levels of structure  Unistructural  Multi-structural  Relational  Cause and effect in history  Single cause  Multiple causes  Multiple interacting causes
Vertically integrated: emphasis on progression In which order would you teach the areas of the following shapes (currently arranged alphabetically)?  Parallelogram  Rectangle  Square  Trapezium  Triangle
Learning hierarchies  Universal  Addition  before multiplication Natural  Multiplication before division  Differentiation before integration  Arbitrary  Areas  of triangles before areas of parallelograms Optional  The Romans before the Vikings
The spiral curriculum The “spiral curriculum.” If one respects the ways of thought of the growing child, if one is courteous enough to translate material into his logical forms and challenging enough to tempt him in advance, then it is possible to introduce him at an early age to the ideas and styles that in later life make an educated man. We might ask, as a criterion for any subject taught in primary school, whether, when fully developed, it is worth an adult’s knowing, and whether having known it as a child makes a person a better adult. If the answer to both questions is negative or ambiguous, then the matter is cluttering the curriculum. Bruner, J. (1960). The Process of Education, Cambridge, MA: Harvard University Press, pp. 52-54 (my emphasis).
Kinds of spiral  Kinds of spiral  Trivial: anything can usefully be revisited  Deep: spirals are an important part of a curriculum  Inclusion criteria  You might need this later  You will need this later  This is useful now, even if you do not go further  You will need this later, and you will be significantly disadvantaged if you do not learn it now
Backward design    The tragedy of life is that one can only understand life backwards, but one must live it forwards (Søren Kierkegaard) In the same way, curricula need to be designed backwards, but delivered forwards Should a curriculum be specified in terms of  Experiences?  Outcomes?  Both?
Curriculum for excellence: Dance Through dance, learners have rich opportunities to be creative and to experience inspiration and enjoyment. Creating and performing will be the core activities for all learners, and taking part in dance contributes to their physical education and physical activity. Learners develop their technical skills and the quality of their movement, and use their imagination and skills to create and choreograph dance sequences. They further develop their knowledge and understanding and their capacity to enjoy dance through evaluating performances and commenting on their work and the work of others. Scottish Government. (2007). “Curriculum for Excellence: expressive arts experiences and outcomes” p. 5.
Appropriate: 860+570=? 25 1.00 Over 5 years, the increase in facility is 75%—an average of 15% per year. 0.90 0.80 Facility 0.70 0.60 0.50 0.40 0.30 In other words, in a class of 30, only four or five children learn this each year. 0.20 0.10 0.00 6 7 8 9 10 11 12 Age (years) Source: Leverhulme Numeracy Research Programme
Consequences (1) 26
Consequences (2) 27 SD = chronological age/4 0 . 5 05 . 4 0 . 4 05 . 3 0 . 3 05 . 2 0 . 2 05 . 1 0 . 5 05 . 4 0 . 4 05 . 3 0 . 3 05 . 2 0 . 2 0 . 1 05 . 0 0 05 . 1 0 . 1 05 . 0 0 4 5 6 7 8 9 e 1 0 a lg c ia 1 1 lg o o o 1 rn 2 h 4 5 67 8 9 0 a 11 2 ti tn 1 a 3 m 11 4 e n tg 1 5 a e 11 6 1 3 1 4 78 11 9 1 0 2 1 5 1 6 c
Focused: Successful education “The test of successful education is not the amount of knowledge that a pupil takes away from school, but his appetite to know and his capacity to learn. If the school sends out children with the desire for knowledge and some idea how to acquire and use it, it will have done its work. Too many leave school with the appetite killed and the mind loaded with undigested lumps of information. The good schoolmaster is known by the number of valuable subjects that he declines to teach. (Livingstone, 1941 p. 28)”
Big ideas of science (Harlen et al., 2011) 29 1 2 3 4 All material in the Universe is made of very small particles. Objects can affect other objects at a distance. Changing the movement of an object requires a net force acting on it. The total amount of energy in the Universe is always the same but energy can be transformed when things change or are made to happen. 5 The composition of the Earth and its atmosphere and the processes occurring within them 6 The solar system is a very small part of one of millions of galaxies in the Universe. 7 Organisms are organised on a cellular basis. 8 Organisms require a supply of energy and materials for which they are often dependent on or in competition with other organisms. 9 Genetic information is passed from one generation of organisms to another. 10 The diversity of organisms, living and extinct, is the result of evolution.
Big ideas about science (Harlen et al., 2011) 1 Science assumes that for every effect there is one or more causes. 2 Scientific explanations, theories and models are those that best fit the facts known at a particular time. 3 The knowledge produced by science is used in some technologies to create products to serve human ends. 4 Applications of science often have ethical, social, economic and political implications.
Relevant: informed choice    About how to learn (Pedagogy) About what to learn (Curriculum) Degree of choice should be influenced by    Consequences (for the individual and for society) Maturity Consequences of choices (and especially poor choices) about what is to be learned are generally greater than choices about how learning should be achieved, so   For younger learners, many if not most learning outcomes need to be non-negotiable. As they get older their wishes should become predominate their interests (progressive lowering of the “safety net”) From the earliest age, however, learners should be involved in decisions about how they learn best.
Informed choice about curriculum • Intrinsic factors – – – – • Extrinsic factors – – • What is the subject really like? Authenticity of experience Habits of mind Developing identity (e.g., mathematics, plumbing) “Critical filters” for particular careers Financial rewards Consequences – – Closing down of options (“leaky pipes”) Sensitive periods
Informed choice in mathematics Torricelli’s trumpet Euler’s relation F+V=E+2 ip e +1= 0 Goldbach’s conjecture The alternating harmonic series
Principles of curriculum design        Balanced Rigorous Coherent Vertically integrated Appropriate Focused Relevant   Which of the seven principles of curriculum design do you think is most important? Which one of the seven principles do you think is least important?
Functions of assessment
Functions of assessment  Three functions of assessment:  For evaluating institutions (evaluative)  For describing individuals (summative)  For supporting learning  Monitoring learning: Whether learning is taking place  Diagnosing (informing) learning: What is not being learnt  Forming learning: What to do about it
Assessment: good servant, bad master
Written examinations 38 “They have perverted the best efforts of teachers, and narrowed and grooved their instruction; they have occasioned and made well nigh imperative the use of mechanical and rote methods of teaching; they have occasioned cramming and the most vicious habits of study; they have caused much of the overpressure charged upon schools, some of which is real; they have tempted both teachers and pupils to dishonesty; and last but not least, they have permitted a mechanical method of school supervision.” White (1888, pp. 517-518)
The Macnamara Fallacy (Handy, 1994 p. 219) 39  The first step is to measure whatever can be easily measured.   The second step is to disregard that which can’t easily be measured or to give it an arbitrary quantitative value.   This is artificial and misleading. The third step is to presume that what can’t be measured easily really isn’t important.   This is OK as far as it goes. This is blindness. The fourth step is to say that what can’t be easily measured really doesn’t exist.  This is suicide.
Goodhart’s law (Campbell’s law) 40  All performance indicators lose their meaning when adopted as policy targets:  Inflation and money supply  Airline schedules  School achievement targets  The clearer you are about what you want, the more likely you are to get it, but the less likely it is to mean anything
The “Lake Wobegon” effect Grade equivalents Test C Test B Test C 4.4 4.3 4.2 4.1 4.0 3.9 3.8 3.7 3.6 3.5 3.4 1986 1987 Koretz, Linn, Dunbar and Shepard (1991) 1988 1989 1990
Effects of narrow assessment 42  Incentives to teach to the test Focus on some subjects at the expense of others  Focus on some aspects of a subject at the expense of others  Focus on some students at the expense of others (“bubble” students)   Consequences  Learning that is Narrow  Shallow  Transient 
And yet…    High-stakes assessment systems do improve outcomes for students on a range of measures The effects can be substantial (an extra two months of learning per year) The challenge:  Realize the benefits of high stakes assessments  Avoid the unintended adverse consequences
The challenge comes down to… 44  To design an assessment system that is:  Distributed   Synoptic   So that all important aspects are covered (breadth and depth) Manageable   So that learning has to accumulate Extensive   So that evidence collection is not undertaken entirely at the end So that costs are proportionate to benefits Trusted  So that stakeholders have faith in the outcomes
Quality in assessment
Validity 46  Evolution of the idea A property of a test  A property of students’ results on a test  A property of the inferences drawn on the basis of test results   For any test: some inferences are warranted  some are not    “One validates not a test but an interpretation of data arising from a specified procedure” (Cronbach, 1971; emphasis in original) No such thing as a valid assessment!
Threats to validity Construct of interest Assessment that is “too small” Assessment that is “too big”
Threats to validity 48  Construct-irrelevant variance  Systematic: some variation in performance on the assessment is attributable to abilities not related to the construct of interest  Random: some variation in performance is related to chance factors, such as luck (effectively poor reliability)  Construct under-representation  Good performance on the assessment can be achieved without demonstrating all aspects of the construct of interest
Teacher assessment is essential 49  The only way to improve the validity of assessments is to make them longer:  Increase testing time  Use information from teachers   Teachers’ involvement is not optional but essential However, teacher assessment brings problems of its own  Standardization  Random  Bias variation
Assessment design
Four-process architecture     Task selection Task presentation Evidence identification Evidence accumulation Almond, Steinberg and Mislevy (2002)
Task selection
Kinther Layticks 53 Skondo has often been described as one of the fantem growing plaidos in the UK during the last 10 years, but the lure of chemicks about in tabsel has continued to attract the attention of moorick numbers of Britons. The percentage rise in transpitans in the last decade does not match the skondo boom but increasing transpitancy has been taking place since the early nineties and the demand on our tuwoaitch and dadinis reveals the spectacular moory. Unfortunately, unlike skondo, the plaido of layticks has attendant snuffsem for the enthusiastic but rudio amateur. All too few of the satsun laybos who take to the tuwoah have even the most rudimentary knowledge of loxem in tabsel. 1. Name two popular plaidos. 2. Have there been many deaths from Skondo? 3. Which country has a lot of kinther layticks? 4. Write down two precautions to take for layticks 5. What is snuffsem about skondo? 6. What would you find in dadinis?
Discussion Discussion question  How can we ensure that the questions that we use to assess our students assess deep, rather than surface, features of the material to be learned?
Task presentation
Item formats  “No assessment technique has been rubbished quite like multiple choice, unless it be graphology” Wood, 1991, p. 32)  Myths about multiple-choice items  They are biased against females  They assess only candidates’ ability to spot or guess  They test only lower-order skills
Questionaing in English (4)  In a piece of persuasive writing, which of these would be the best thesis statement? A. B. C. D. E. F. G. H. The typical TV show has 9 violent incidents There is a lot of violence on TV The amount of violence on TV should be reduced Some programs are more violent than others Violence is included in programs to boost ratings Violence on TV is interesting I don’t like the violence on TV The essay I am going to write is about violence on TV
Evidence identification
Referents in assessment  Norm-referenced   Cohort-referenced   explicit and precise performance criteria Ipsative   the group assessed at the same time Criterion-referenced   a group who were assessed previously defined only within an individual Construct-referenced  a shared construct in a community of practice
Quality “Maxims cannot be understood, still less applied by anyone not already possessing a good practical knowledge of the art. They derive their interest from our appreciation of the art and cannot themselves either replace or establish that appreciation”. (Polanyi, 1958 p. 50). “Quality doesn’t have to be defined. You understand it without definition. Quality is a direct experience independent of and prior to intellectual abstractions”. (Pirsig, 1991 p. 64).
Moderation and standardisation 61 Moderation Standardisation Backward looking Forward looking Quality control Quality assurance Inspects quality in Builds quality in Static Dynamic Flat cost profile Reducing cost profile Ephemeral evidence ignored Ephemeral evidence used
Discussion Discussion question  How can we ensure that different teachers reach similar judgments about the quality of student work without assessing only surface features of the learning?
Evidence accumulation
Memory on land and underwater   18 (5f, 13m) student members of a university diving club were tested on their recall of two- and three-syllable words from four 36-word lists taken from the Toronto Word Bank spoken to them twice. Students learned, and were tested on, the words while underwater, and while on the shore, resulting in four conditions:     DD (learn dry, recall dry) DW (learn dry, recall wet) WD (learn wet, recall dry) WW (learn wet, recall wet)
Memory is context-dependent 65 Recall environment Dry Learning environment Wet Dry 13.5 8.6 Wet 8.4 11.4 No significant main effects; interaction effect: F=22.0; df = 1, 12; p= <0.001 Godden and Baddeley (1975)
Discussion question Discussion 66  How will you decide how much evidence is needed to decide whether a student has reached a particular outcome?
Recording
Identify milestones (and inch pebbles)  Development of science skills in eighth grade Use of laboratory equipment  Metric unit conversion  Density calculations  Density applications  Density as a characteristic property  Phases of matter  Gas laws  Communication (graphing)  Communication (lab reports)  Inquiry skills 
Homework 2 ✓ Communication (report) Communication (graph) ✓ ✓ ✓ ✓ Module test ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Homework 4 Final exam Gas laws ✓ Homework 3 Laboratory 2 Phases of matter Density properties Equipment ✓ ✓ Homework 1 Laboratory 1 Density calculations Metric units Assessment matrix ✓ ✓ ✓ ✓ ✓ ✓
Reporting
Effects of feedback    Kluger & DeNisi (1996) Review of 3000 research reports Excluding those:        without adequate controls with poor design with fewer than 10 participants where performance was not measured without details of effect sizes left 131 reports, 607 effect sizes, involving 12652 individuals On average feedback does improve performance, but   Effect sizes very different in different studies 40% of effect sizes were negative
Getting feedback right is hard Response type Feedback indicates performance… exceeds goal falls short of goal Change behavior Exert less effort Increase effort Change goal Increase aspiration Reduce aspiration Abandon goal Decide goal is too easy Decide goal is too hard Reject feedback Feedback is ignored Feedback is ignored
Discussion question Discussion 74   How can feedback be designed so as to cue appropriate action by learners? What information should be reported to stakeholders, and how often?
Building an assessment system
Mapping out the terrain 76 Timescale Annual Interim Weekly Daily Hourly Academic promotion Benchmark Common formative assessments Before the endof-unit tests Exit pass End-of-course exams High-stakes accountability Growth End-ofunit tests Hinge-point questions Instructional Guidance (“formative”) Describing Individuals (“summative”) Function Institutional Accountability (“evaluative”)
Perspectives on assessment Role Learners Summative assessment Formative assessment Understanding the assessment intentions, so they produce relevant evidence become Learners members of the same community of practice of which their teachers are already members Teachers A community of practice in which teachers share a construct of quality Teachers possess an anatomy of quality
Self-assessment 78 My red folder in the fourth year wants me to be clear and positive about what I achieve in school “in my own words” which are foreign to me. In my own words in my own language (which has no place here) how can I feel clear and positive? Raychaudhuri (1998) My red folder in the fourth year wants me to be positive about my grade E in English History: the heritage and glory of the British Empire “in my own words”. My red folder in the fourth year suddenly out of nowhere wants me to assert what I achieve in school “in my own words”. How can I blow the trumpet they’ve taken from me?
Thank you

More Related Content

PPTX
Instrumentation in Mathematics TNESC
bynick ratunil
 
PPTX
Session 1 -- global challenges in education
byMadan Pant
 
PDF
Literacy Based Intervention: From Theory to Practice
byBilinguistics
 
PPTX
PYP Language Workshop for Parents (January 2013)
bybisedu
 
PPT
Pyp activities to share
byforeignlanguagescommittee
 
PDF
5, 4, 3, 2, 1... Launching the PYP! Tips, Strategies, and Resources
byStephanie Barrus
 
PPTX
Pfs ss 131004160005-phpapp01
byColette Selepe
 
PPTX
Hendricks workshop on decolonising the curriculum may 24 2016
byBrenda Leibowitz
 
Instrumentation in Mathematics TNESC
bynick ratunil
 
Session 1 -- global challenges in education
byMadan Pant
 
Literacy Based Intervention: From Theory to Practice
byBilinguistics
 
PYP Language Workshop for Parents (January 2013)
bybisedu
 
Pyp activities to share
byforeignlanguagescommittee
 
5, 4, 3, 2, 1... Launching the PYP! Tips, Strategies, and Resources
byStephanie Barrus
 
Pfs ss 131004160005-phpapp01
byColette Selepe
 
Hendricks workshop on decolonising the curriculum may 24 2016
byBrenda Leibowitz
 

What's hot

PPTX
Can Social Realism Do Social Justice? Debating the Warrants for Curriculum Kn...
byedsrpp
 
PPT
Nyu Pte Methods For Math & Science Teachers Oct 8, 2009 1
byProject for Developing Chinese Language Teachers
 
PPT
Loves math 5
byLovemath Scream
 
PPTX
[BIS] Intro to PYP For Parents Sept 2012
byirfansatria
 
PPT
Lecture: Teacher identity and impact on literacy
byAnnie Muir
 
PPTX
Pre writing and learning styles
byCasa Thomas Jefferson
 
PDF
Designing Blended Learning Experiences - Handout
byBrent Jones
 
PPTX
Threshold Concepts: A Discipline-based approach to learning and design
bydisciplinarythinking
 
PDF
Parent workshop pyp intro 2015
byISM-Arusha
 
PPT
The Primary Years: Basis for Practice
byPatrick O'Sullivan
 
PPT
Theory Paper Presentation for Ed. D. Program
byLarry Weas
 
PPTX
Ib parent presentation
bymhdeller
 
PPTX
Facilitating the Learning of Diverse Students
byMara Mastro
 
PDF
La Scuola's Learning Journey to IB-PYP
bylascuola
 
DOC
Curricula
byFrancisco Perez
 
PPT
21st century Edn. and strategies( School Based Inservice program)
byChumey MSS
 
PPT
Introductory Pp Models
byguest7d9132
 
PPTX
2012 pyp for parents
byatomikey
 
PPT
Teaching For Rigor And Relevance
byJennifer Dorman
 
PPT
Portppt 564
byCarla Piper
 
Can Social Realism Do Social Justice? Debating the Warrants for Curriculum Kn...
byedsrpp
 
Nyu Pte Methods For Math & Science Teachers Oct 8, 2009 1
byProject for Developing Chinese Language Teachers
 
Loves math 5
byLovemath Scream
 
[BIS] Intro to PYP For Parents Sept 2012
byirfansatria
 
Lecture: Teacher identity and impact on literacy
byAnnie Muir
 
Pre writing and learning styles
byCasa Thomas Jefferson
 
Designing Blended Learning Experiences - Handout
byBrent Jones
 
Threshold Concepts: A Discipline-based approach to learning and design
bydisciplinarythinking
 
Parent workshop pyp intro 2015
byISM-Arusha
 
The Primary Years: Basis for Practice
byPatrick O'Sullivan
 
Theory Paper Presentation for Ed. D. Program
byLarry Weas
 
Ib parent presentation
bymhdeller
 
Facilitating the Learning of Diverse Students
byMara Mastro
 
La Scuola's Learning Journey to IB-PYP
bylascuola
 
Curricula
byFrancisco Perez
 
21st century Edn. and strategies( School Based Inservice program)
byChumey MSS
 
Introductory Pp Models
byguest7d9132
 
2012 pyp for parents
byatomikey
 
Teaching For Rigor And Relevance
byJennifer Dorman
 
Portppt 564
byCarla Piper
 

Viewers also liked

PPTX
Module 4: Lesson 1 Assessing the Curriculum
byJUJIE ATILANO
 
PPTX
Tools To Assess The Quality Of The Curriculum
bydbrady3702
 
PPTX
Assessing the curriculum
byJessica Ampater
 
PPTX
Linking curriculum and assessment
bycinth26
 
PPTX
intended vs implemented vs achieved curriculum
byobemrosalia
 
PPT
Curriculum development
byRona De la Rama
 
PPTX
Implemented curriculum vs achieved curriculum
byJERCAS Castillo
 
PPTX
Linking curriculum,instruction and assessment(cia)making fit
byCarl Richard Dagalea
 
PPT
Re Thinking Curriculum And Assessment With Digital Tools
bykampology
 
PPT
Curriculum development
byRona De la Rama
 
Module 4: Lesson 1 Assessing the Curriculum
byJUJIE ATILANO
 
Tools To Assess The Quality Of The Curriculum
bydbrady3702
 
Assessing the curriculum
byJessica Ampater
 
Linking curriculum and assessment
bycinth26
 
intended vs implemented vs achieved curriculum
byobemrosalia
 
Curriculum development
byRona De la Rama
 
Implemented curriculum vs achieved curriculum
byJERCAS Castillo
 
Linking curriculum,instruction and assessment(cia)making fit
byCarl Richard Dagalea
 
Re Thinking Curriculum And Assessment With Digital Tools
bykampology
 
Curriculum development
byRona De la Rama
 

Similar to Dylan Wiliam's slides - 2013 11-21 bcssa

PPTX
SECOND-THIRD-TOPIC_THE-TEACHER-THE-SCHOOL-CURRICULUM.pptx
bymekaylaatendido1
 
PPTX
The-Teacher-and-the-School-Curriculum-FINAL.pptx
byChrisiaMaeGonzales1
 
PPTX
Curriculum de vt esg11
bybulacan state university
 
PPTX
EDU-227A_DILUVIO.pptx
byGuillermoJrDiluvio
 
PPTX
The Components of Curriculum Design.pptx
byOtenciano
 
PPT
Curriculum development new
byMohammed Shareef
 
PPT
Curriculum development new
byMohammed Shareef
 
PPTX
Presentation curriculum
byFariba Ataie
 
PPTX
Presentation curriculum
byFariba Ataie
 
PPTX
MODUEL 2 THE TEACHER AS A KNOWER OF THE CURRICULUM.pptx
byJirehAlbay
 
PPTX
Curr. 1
byAkmal Akmaludin
 
PDF
curriculum development in special education
byAlinaNoreen1
 
PPTX
Pedagogy, curriculum and assessment
bythomastallis
 
PPTX
Group 5 phases_of_curriculum_development
byJohn Ervin
 
PPT
Chapter 6 curriculum design
bytraciemdixon
 
PPTX
Curriculum meaning definitions curriculum vs syllabus
byDr. Deivam M
 
PPTX
Curriculum Development Module 2 lesson 1-3
byalkhaizar
 
DOCX
Curriculum
byxlyz
 
PPT
Curriculum
byCarla Piper
 
PPT
Curriculum II
byLily Pad
 
SECOND-THIRD-TOPIC_THE-TEACHER-THE-SCHOOL-CURRICULUM.pptx
bymekaylaatendido1
 
The-Teacher-and-the-School-Curriculum-FINAL.pptx
byChrisiaMaeGonzales1
 
Curriculum de vt esg11
bybulacan state university
 
EDU-227A_DILUVIO.pptx
byGuillermoJrDiluvio
 
The Components of Curriculum Design.pptx
byOtenciano
 
Curriculum development new
byMohammed Shareef
 
Curriculum development new
byMohammed Shareef
 
Presentation curriculum
byFariba Ataie
 
Presentation curriculum
byFariba Ataie
 
MODUEL 2 THE TEACHER AS A KNOWER OF THE CURRICULUM.pptx
byJirehAlbay
 
Curr. 1
byAkmal Akmaludin
 
curriculum development in special education
byAlinaNoreen1
 
Pedagogy, curriculum and assessment
bythomastallis
 
Group 5 phases_of_curriculum_development
byJohn Ervin
 
Chapter 6 curriculum design
bytraciemdixon
 
Curriculum meaning definitions curriculum vs syllabus
byDr. Deivam M
 
Curriculum Development Module 2 lesson 1-3
byalkhaizar
 
Curriculum
byxlyz
 
Curriculum
byCarla Piper
 
Curriculum II
byLily Pad
 

More from Hélène Galdin-O'Shea

PPTX
The internet debate
byHélène Galdin-O'Shea
 
PDF
EBT
byHélène Galdin-O'Shea
 
PPTX
film opening conventions and discussion of genre
byHélène Galdin-O'Shea
 
PPTX
Berger slides
byHélène Galdin-O'Shea
 
PPTX
Beats by Dr Dre - Class ppt
byHélène Galdin-O'Shea
 
PPTX
Dylan Wiliam 2013 03-22 future of fe conference
byHélène Galdin-O'Shea
 
PPT
Marlynne Grant, researchED conference 6th september 2014
byHélène Galdin-O'Shea
 
PDF
SLEs: the foot soldiers of system leadership?
byHélène Galdin-O'Shea
 
PPTX
#TLT13 Presentation on Questioning
byHélène Galdin-O'Shea
 
PPTX
21 october Presentation
byHélène Galdin-O'Shea
 
PPTX
Presentation Mea bfi Conference 14
byHélène Galdin-O'Shea
 
PPT
Pans labyrinth blog
byHélène Galdin-O'Shea
 
PPTX
Representations of the law (blog version)
byHélène Galdin-O'Shea
 
PPTX
G322 INSTITUTIONS AND AUDIENCES past questions
byHélène Galdin-O'Shea
 
PPTX
Ppt blog version
byHélène Galdin-O'Shea
 
PPTX
Analysing adverts womenrep
byHélène Galdin-O'Shea
 
PDF
genre-crime-tv-drama Article from Media mag
byHélène Galdin-O'Shea
 
PPT
Technical Codes in Music Videos
byHélène Galdin-O'Shea
 
PPTX
The bill shots
byHélène Galdin-O'Shea
 
PPTX
G322 TV DRAMA past questions
byHélène Galdin-O'Shea
 
The internet debate
byHélène Galdin-O'Shea
 
EBT
byHélène Galdin-O'Shea
 
film opening conventions and discussion of genre
byHélène Galdin-O'Shea
 
Berger slides
byHélène Galdin-O'Shea
 
Beats by Dr Dre - Class ppt
byHélène Galdin-O'Shea
 
Dylan Wiliam 2013 03-22 future of fe conference
byHélène Galdin-O'Shea
 
Marlynne Grant, researchED conference 6th september 2014
byHélène Galdin-O'Shea
 
SLEs: the foot soldiers of system leadership?
byHélène Galdin-O'Shea
 
#TLT13 Presentation on Questioning
byHélène Galdin-O'Shea
 
21 october Presentation
byHélène Galdin-O'Shea
 
Presentation Mea bfi Conference 14
byHélène Galdin-O'Shea
 
Pans labyrinth blog
byHélène Galdin-O'Shea
 
Representations of the law (blog version)
byHélène Galdin-O'Shea
 
G322 INSTITUTIONS AND AUDIENCES past questions
byHélène Galdin-O'Shea
 
Ppt blog version
byHélène Galdin-O'Shea
 
Analysing adverts womenrep
byHélène Galdin-O'Shea
 
genre-crime-tv-drama Article from Media mag
byHélène Galdin-O'Shea
 
Technical Codes in Music Videos
byHélène Galdin-O'Shea
 
The bill shots
byHélène Galdin-O'Shea
 
G322 TV DRAMA past questions
byHélène Galdin-O'Shea
 

Recently uploaded

PDF
GIÁO ÁN KẾ HOẠCH BÀI DẠY NĂNG LỰC SỐ MÔN TIẾNG ANH LỚP 10 CẢ NĂM - GLOBAL SUC...
byNguyen Thanh Tu Collection
 
PDF
BÀI GIẢNG POWERPOINT CHÍNH KHÓA PHIÊN BẢN AI TIẾNG ANH 6 CẢ NĂM, THEO TỪNG BÀ...
byNguyen Thanh Tu Collection
 
PPTX
special senses-eye, ear, nose, tongue.pptx
byAshish Umale
 
PPTX
Accounting Theory Group Presentation - Why Study Accounting Theory
byAbrahamAtuanor1
 
PDF
Agents in Artificial Intelligence: Types, Architecture and Real-World Examples
byARTHIDEVARANIP
 
PPTX
Insertion of Suppositories..........pptx
byAneetaSharma15
 
PPTX
The Creation Pattern Physical Health.pptx
byTammy Hulse
 
PDF
The Unique Wildlife of Ethiopia: From the Simien to the Bale Mountains
byfirtunagebremicheal
 
PPTX
STERILITY INDICATOR Pharmaceutical microbiology
byChetana Wagh
 
PPTX
VAGINAL IRRIGATION..................pptx
byAneetaSharma15
 
PPTX
Activity on Job position in Odoo 19 Recruitment
byCeline George
 
PPTX
VITAMINS CHAPTER NO.05 PHARMACOGNOSY D. PHARMACY
byGanu Gavade
 
PPTX
How to Manage Scheduling Activities in Odoo 18 CRM
byCeline George
 
PPTX
Ultrasound .pptx by Nagmani ojha (paramedical department)(Radiation technolog...
bynagmaniojha8
 
PPTX
YSPH VMOC Special Report - Measles - The Americas 1-18-2026
byYale School of Public Health - The Virtual Medical Operations Center (VMOC)
 
PPTX
How to Create Opening Balance in Odoo 18
byCeline George
 
PPTX
Nanomaterials and its types - Dr.M.Jothimuniyandi
byJothimuniyandi
 
PPTX
Exploring car engine oil and lubricants purpose and functions
bymjb77ny
 
PPTX
How to Manage Empty Location in Odoo 18 Inventory
byCeline George
 
PPTX
How to Manage Product Types in Odoo 18 Sales
byCeline George
 
GIÁO ÁN KẾ HOẠCH BÀI DẠY NĂNG LỰC SỐ MÔN TIẾNG ANH LỚP 10 CẢ NĂM - GLOBAL SUC...
byNguyen Thanh Tu Collection
 
BÀI GIẢNG POWERPOINT CHÍNH KHÓA PHIÊN BẢN AI TIẾNG ANH 6 CẢ NĂM, THEO TỪNG BÀ...
byNguyen Thanh Tu Collection
 
special senses-eye, ear, nose, tongue.pptx
byAshish Umale
 
Accounting Theory Group Presentation - Why Study Accounting Theory
byAbrahamAtuanor1
 
Agents in Artificial Intelligence: Types, Architecture and Real-World Examples
byARTHIDEVARANIP
 
Insertion of Suppositories..........pptx
byAneetaSharma15
 
The Creation Pattern Physical Health.pptx
byTammy Hulse
 
The Unique Wildlife of Ethiopia: From the Simien to the Bale Mountains
byfirtunagebremicheal
 
STERILITY INDICATOR Pharmaceutical microbiology
byChetana Wagh
 
VAGINAL IRRIGATION..................pptx
byAneetaSharma15
 
Activity on Job position in Odoo 19 Recruitment
byCeline George
 
VITAMINS CHAPTER NO.05 PHARMACOGNOSY D. PHARMACY
byGanu Gavade
 
How to Manage Scheduling Activities in Odoo 18 CRM
byCeline George
 
Ultrasound .pptx by Nagmani ojha (paramedical department)(Radiation technolog...
bynagmaniojha8
 
YSPH VMOC Special Report - Measles - The Americas 1-18-2026
byYale School of Public Health - The Virtual Medical Operations Center (VMOC)
 
How to Create Opening Balance in Odoo 18
byCeline George
 
Nanomaterials and its types - Dr.M.Jothimuniyandi
byJothimuniyandi
 
Exploring car engine oil and lubricants purpose and functions
bymjb77ny
 
How to Manage Empty Location in Odoo 18 Inventory
byCeline George
 
How to Manage Product Types in Odoo 18 Sales
byCeline George
 

Dylan Wiliam's slides - 2013 11-21 bcssa

  • 1.
    Principled curriculum and assessmentdesign: Tools for schools Dylan Wiliam British Columbia School Superintendents Association Fall Conference November 2013, Vancouver, BC. 21 Nov 2013 www.dylanwiliam.org @dylanwiliam
  • 2.
    Outline  Curriculum  What is curriculum? Seven principles for curriculum design  Assessment  Assessment for accountability  Quality in assessment
  • 3.
    Why do weeducate young people?  Broad views on the philosophy of education (Williams, 1961)  Transmission of culture (e.g., Arnold)  Preparation for work (e.g., OECD)  Preparation for effective citizenship (e.g., Freire)  Preparation for life  Any curriculum is a sometimes messy compromise between all of these
  • 4.
    Curriculum: an evolvingconcept   The courses taken (Scottish HE, late 17th century) Four questions (Tyler, 1949)       What educational purposes should the school seek to attain? What educational experiences … are likely to attain these purposes? How can these educational experiences be effectively organized? How can we determine whether these purposes are being attained? “All the learning which is planned or guided by the school, whether it is carried on in groups or individually inside or outside the school.” (Kerr, 1968 p. 16) “the school curriculum (in the wider sense) is essentially a selection from the culture of a society.” (Lawton 1975 p. 7)
  • 5.
    What is curriculumreally?  Three levels of curriculum  The intended curriculum  The curriculum  The mandated by government agencies implemented curriculum  The curriculum realized in textbooks, schemes of work, lesson plans, etc.  The achieved curriculum  The lived  daily experience of learners in schools Each of these has explicit and tacit (hidden) aspects
  • 6.
    And what isnot there is also important…  The null curriculum: “the options students are not afforded; the perspectives they may never know about, much less be able to use; the concepts and skills that are not part of their intellectual repertoire” (Eisner 1985, p.107).
  • 7.
    The role ofteachers (Stenhouse, 1975)   “A curriculum is an attempt to communicate the essential principles and features of an educational proposal in such a form that it is open to critical scrutiny and capable of effective translation into practice.” (p. 5) The proposal should have three parts: A. In planning B. In empirical study C. In relation to justification
  • 8.
    Principles of curriculumdesign  A good curriculum is:  Balanced  Rigorous  Coherent  Vertically integrated  Appropriate  Focused/parsimonious  Relevant
  • 9.
    Balanced: which subjects?            English Mathematics Science Technology Modernforeign languages Geography History Music Art Physical education Religious education            Drama Dance Chess Engineering Geology Astronomy Media studies Law Psychology Sociology Politics
  • 10.
    Rigorous: subjects, disciplines,or skills?  Disciplinary habits of mind are important, specific, powerful ways of thinking that are developed through sustained engagement with the discipline.  Mathematics: transformation and invariance  History: provenance and context  Statistics: dispersion as well as central tendency  Sociology: structure and agency
  • 11.
    21st Century skills  Cognitivecompetencies     Intra-personal competencies     Cognitive processes and strategies Knowledge Creativity Intellectual openness Work ethic/conscientiousness Positive core self-evaluation Inter-personal competencies   Team-work Leadership Pellegrino and Hilton (2012)
  • 12.
    Coherent: subjects orthemes?   Subject-based curricula support disciplines but tend to undermine coherence across different aspects of learning Theme-based curricula support coherence, but tend to undermine disciplinary development
  • 13.
    Reading skills: whatare they really? A manifold, contained in an intuition which I call mine, is represented, by means of the synthesis of the understanding, as belonging to the necessary unity of self-consciousness; and this is effected by means of the category. What is the main idea of this passage? 

1. Without a manifold, one cannot call an intuition ‘mine.’ 

2. Intuition must precede understanding. 

3. Intuition must occur through a category. 

4. Self-consciousness is necessary to understanding Hirsch (2006)
  • 14.
    John walked tofirst, stole second, got bunted over to third, and reached home on a sacrifice fly. How many outs were there when John got to the plate? A. 0 B. 1 C. 2
  • 15.
    Lost in translation? Comprehensiondepends on constructing a mental model that makes the elements fall into place and, equally important, enables the listener or reader to supply essential information that is not explicitly stated. In language use, there is always a great deal that is left unsaid and must be inferred. This means that communication depends on both sides, writer and reader, sharing a basis of unspoken knowledge. This large dimension of tacit knowledge is precisely what is not being taught adequately in our schools. Hirsch (2009 loc. 176)
  • 16.
  • 17.
    Skill is content,content is skill Five propositions about academic skills (Hirsch, 2009) 1 The character of an academic skill is constrained by the limitations of short-term working memory. 2 Academic skills have two components: procedures and contents. 3 Procedural skills such as turning letters into sounds must initially be learned as content, along with other content necessary to higher-order skills. 4 An advance in skill, whether in procedure or content, entails an advance in speed of processing. 5 A higher-order academic skill such as reading comprehension requires prior knowledge of domain-specific content; the higher-order skills for that domain does not readily transfer to other content domains.
  • 18.
     SOLO taxonomy (Biggs& Collis, 1982)  Structure of observed learning outcomes  Levels of structure  Unistructural  Multi-structural  Relational  Cause and effect in history  Single cause  Multiple causes  Multiple interacting causes
  • 19.
    Vertically integrated: emphasison progression In which order would you teach the areas of the following shapes (currently arranged alphabetically)?  Parallelogram  Rectangle  Square  Trapezium  Triangle
  • 20.
    Learning hierarchies  Universal  Addition  beforemultiplication Natural  Multiplication before division  Differentiation before integration  Arbitrary  Areas  of triangles before areas of parallelograms Optional  The Romans before the Vikings
  • 21.
    The spiral curriculum The“spiral curriculum.” If one respects the ways of thought of the growing child, if one is courteous enough to translate material into his logical forms and challenging enough to tempt him in advance, then it is possible to introduce him at an early age to the ideas and styles that in later life make an educated man. We might ask, as a criterion for any subject taught in primary school, whether, when fully developed, it is worth an adult’s knowing, and whether having known it as a child makes a person a better adult. If the answer to both questions is negative or ambiguous, then the matter is cluttering the curriculum. Bruner, J. (1960). The Process of Education, Cambridge, MA: Harvard University Press, pp. 52-54 (my emphasis).
  • 22.
    Kinds of spiral  Kindsof spiral  Trivial: anything can usefully be revisited  Deep: spirals are an important part of a curriculum  Inclusion criteria  You might need this later  You will need this later  This is useful now, even if you do not go further  You will need this later, and you will be significantly disadvantaged if you do not learn it now
  • 23.
    Backward design    The tragedyof life is that one can only understand life backwards, but one must live it forwards (Søren Kierkegaard) In the same way, curricula need to be designed backwards, but delivered forwards Should a curriculum be specified in terms of  Experiences?  Outcomes?  Both?
  • 24.
    Curriculum for excellence:Dance Through dance, learners have rich opportunities to be creative and to experience inspiration and enjoyment. Creating and performing will be the core activities for all learners, and taking part in dance contributes to their physical education and physical activity. Learners develop their technical skills and the quality of their movement, and use their imagination and skills to create and choreograph dance sequences. They further develop their knowledge and understanding and their capacity to enjoy dance through evaluating performances and commenting on their work and the work of others. Scottish Government. (2007). “Curriculum for Excellence: expressive arts experiences and outcomes” p. 5.
  • 25.
    Appropriate: 860+570=? 25 1.00 Over 5years, the increase in facility is 75%—an average of 15% per year. 0.90 0.80 Facility 0.70 0.60 0.50 0.40 0.30 In other words, in a class of 30, only four or five children learn this each year. 0.20 0.10 0.00 6 7 8 9 10 11 12 Age (years) Source: Leverhulme Numeracy Research Programme
  • 26.
  • 27.
    Consequences (2) 27 SD =chronological age/4 0 . 5 05 . 4 0 . 4 05 . 3 0 . 3 05 . 2 0 . 2 05 . 1 0 . 5 05 . 4 0 . 4 05 . 3 0 . 3 05 . 2 0 . 2 0 . 1 05 . 0 0 05 . 1 0 . 1 05 . 0 0 4 5 6 7 8 9 e 1 0 a lg c ia 1 1 lg o o o 1 rn 2 h 4 5 67 8 9 0 a 11 2 ti tn 1 a 3 m 11 4 e n tg 1 5 a e 11 6 1 3 1 4 78 11 9 1 0 2 1 5 1 6 c
  • 28.
    Focused: Successful education “Thetest of successful education is not the amount of knowledge that a pupil takes away from school, but his appetite to know and his capacity to learn. If the school sends out children with the desire for knowledge and some idea how to acquire and use it, it will have done its work. Too many leave school with the appetite killed and the mind loaded with undigested lumps of information. The good schoolmaster is known by the number of valuable subjects that he declines to teach. (Livingstone, 1941 p. 28)”
  • 29.
    Big ideas ofscience (Harlen et al., 2011) 29 1 2 3 4 All material in the Universe is made of very small particles. Objects can affect other objects at a distance. Changing the movement of an object requires a net force acting on it. The total amount of energy in the Universe is always the same but energy can be transformed when things change or are made to happen. 5 The composition of the Earth and its atmosphere and the processes occurring within them 6 The solar system is a very small part of one of millions of galaxies in the Universe. 7 Organisms are organised on a cellular basis. 8 Organisms require a supply of energy and materials for which they are often dependent on or in competition with other organisms. 9 Genetic information is passed from one generation of organisms to another. 10 The diversity of organisms, living and extinct, is the result of evolution.
  • 30.
    Big ideas aboutscience (Harlen et al., 2011) 1 Science assumes that for every effect there is one or more causes. 2 Scientific explanations, theories and models are those that best fit the facts known at a particular time. 3 The knowledge produced by science is used in some technologies to create products to serve human ends. 4 Applications of science often have ethical, social, economic and political implications.
  • 31.
    Relevant: informed choice    Abouthow to learn (Pedagogy) About what to learn (Curriculum) Degree of choice should be influenced by    Consequences (for the individual and for society) Maturity Consequences of choices (and especially poor choices) about what is to be learned are generally greater than choices about how learning should be achieved, so   For younger learners, many if not most learning outcomes need to be non-negotiable. As they get older their wishes should become predominate their interests (progressive lowering of the “safety net”) From the earliest age, however, learners should be involved in decisions about how they learn best.
  • 32.
    Informed choice aboutcurriculum • Intrinsic factors – – – – • Extrinsic factors – – • What is the subject really like? Authenticity of experience Habits of mind Developing identity (e.g., mathematics, plumbing) “Critical filters” for particular careers Financial rewards Consequences – – Closing down of options (“leaky pipes”) Sensitive periods
  • 33.
    Informed choice inmathematics Torricelli’s trumpet Euler’s relation F+V=E+2 ip e +1= 0 Goldbach’s conjecture The alternating harmonic series
  • 34.
    Principles of curriculumdesign        Balanced Rigorous Coherent Vertically integrated Appropriate Focused Relevant   Which of the seven principles of curriculum design do you think is most important? Which one of the seven principles do you think is least important?
  • 35.
  • 36.
    Functions of assessment  Threefunctions of assessment:  For evaluating institutions (evaluative)  For describing individuals (summative)  For supporting learning  Monitoring learning: Whether learning is taking place  Diagnosing (informing) learning: What is not being learnt  Forming learning: What to do about it
  • 37.
  • 38.
    Written examinations 38 “They haveperverted the best efforts of teachers, and narrowed and grooved their instruction; they have occasioned and made well nigh imperative the use of mechanical and rote methods of teaching; they have occasioned cramming and the most vicious habits of study; they have caused much of the overpressure charged upon schools, some of which is real; they have tempted both teachers and pupils to dishonesty; and last but not least, they have permitted a mechanical method of school supervision.” White (1888, pp. 517-518)
  • 39.
    The Macnamara Fallacy(Handy, 1994 p. 219) 39  The first step is to measure whatever can be easily measured.   The second step is to disregard that which can’t easily be measured or to give it an arbitrary quantitative value.   This is artificial and misleading. The third step is to presume that what can’t be measured easily really isn’t important.   This is OK as far as it goes. This is blindness. The fourth step is to say that what can’t be easily measured really doesn’t exist.  This is suicide.
  • 40.
    Goodhart’s law (Campbell’slaw) 40  All performance indicators lose their meaning when adopted as policy targets:  Inflation and money supply  Airline schedules  School achievement targets  The clearer you are about what you want, the more likely you are to get it, but the less likely it is to mean anything
  • 41.
    The “Lake Wobegon”effect Grade equivalents Test C Test B Test C 4.4 4.3 4.2 4.1 4.0 3.9 3.8 3.7 3.6 3.5 3.4 1986 1987 Koretz, Linn, Dunbar and Shepard (1991) 1988 1989 1990
  • 42.
    Effects of narrowassessment 42  Incentives to teach to the test Focus on some subjects at the expense of others  Focus on some aspects of a subject at the expense of others  Focus on some students at the expense of others (“bubble” students)   Consequences  Learning that is Narrow  Shallow  Transient 
  • 43.
    And yet…    High-stakes assessmentsystems do improve outcomes for students on a range of measures The effects can be substantial (an extra two months of learning per year) The challenge:  Realize the benefits of high stakes assessments  Avoid the unintended adverse consequences
  • 44.
    The challenge comesdown to… 44  To design an assessment system that is:  Distributed   Synoptic   So that all important aspects are covered (breadth and depth) Manageable   So that learning has to accumulate Extensive   So that evidence collection is not undertaken entirely at the end So that costs are proportionate to benefits Trusted  So that stakeholders have faith in the outcomes
  • 45.
  • 46.
    Validity 46  Evolution of theidea A property of a test  A property of students’ results on a test  A property of the inferences drawn on the basis of test results   For any test: some inferences are warranted  some are not    “One validates not a test but an interpretation of data arising from a specified procedure” (Cronbach, 1971; emphasis in original) No such thing as a valid assessment!
  • 47.
    Threats to validity Constructof interest Assessment that is “too small” Assessment that is “too big”
  • 48.
    Threats to validity 48  Construct-irrelevantvariance  Systematic: some variation in performance on the assessment is attributable to abilities not related to the construct of interest  Random: some variation in performance is related to chance factors, such as luck (effectively poor reliability)  Construct under-representation  Good performance on the assessment can be achieved without demonstrating all aspects of the construct of interest
  • 49.
    Teacher assessment isessential 49  The only way to improve the validity of assessments is to make them longer:  Increase testing time  Use information from teachers   Teachers’ involvement is not optional but essential However, teacher assessment brings problems of its own  Standardization  Random  Bias variation
  • 50.
  • 51.
    Four-process architecture     Task selection Taskpresentation Evidence identification Evidence accumulation Almond, Steinberg and Mislevy (2002)
  • 52.
  • 53.
    Kinther Layticks 53 Skondo hasoften been described as one of the fantem growing plaidos in the UK during the last 10 years, but the lure of chemicks about in tabsel has continued to attract the attention of moorick numbers of Britons. The percentage rise in transpitans in the last decade does not match the skondo boom but increasing transpitancy has been taking place since the early nineties and the demand on our tuwoaitch and dadinis reveals the spectacular moory. Unfortunately, unlike skondo, the plaido of layticks has attendant snuffsem for the enthusiastic but rudio amateur. All too few of the satsun laybos who take to the tuwoah have even the most rudimentary knowledge of loxem in tabsel. 1. Name two popular plaidos. 2. Have there been many deaths from Skondo? 3. Which country has a lot of kinther layticks? 4. Write down two precautions to take for layticks 5. What is snuffsem about skondo? 6. What would you find in dadinis?
  • 54.
    Discussion Discussion question  How canwe ensure that the questions that we use to assess our students assess deep, rather than surface, features of the material to be learned?
  • 55.
  • 56.
    Item formats  “No assessmenttechnique has been rubbished quite like multiple choice, unless it be graphology” Wood, 1991, p. 32)  Myths about multiple-choice items  They are biased against females  They assess only candidates’ ability to spot or guess  They test only lower-order skills
  • 57.
    Questionaing in English(4)  In a piece of persuasive writing, which of these would be the best thesis statement? A. B. C. D. E. F. G. H. The typical TV show has 9 violent incidents There is a lot of violence on TV The amount of violence on TV should be reduced Some programs are more violent than others Violence is included in programs to boost ratings Violence on TV is interesting I don’t like the violence on TV The essay I am going to write is about violence on TV
  • 58.
  • 59.
    Referents in assessment  Norm-referenced   Cohort-referenced   explicitand precise performance criteria Ipsative   the group assessed at the same time Criterion-referenced   a group who were assessed previously defined only within an individual Construct-referenced  a shared construct in a community of practice
  • 60.
    Quality “Maxims cannot beunderstood, still less applied by anyone not already possessing a good practical knowledge of the art. They derive their interest from our appreciation of the art and cannot themselves either replace or establish that appreciation”. (Polanyi, 1958 p. 50). “Quality doesn’t have to be defined. You understand it without definition. Quality is a direct experience independent of and prior to intellectual abstractions”. (Pirsig, 1991 p. 64).
  • 61.
    Moderation and standardisation 61 Moderation Standardisation Backwardlooking Forward looking Quality control Quality assurance Inspects quality in Builds quality in Static Dynamic Flat cost profile Reducing cost profile Ephemeral evidence ignored Ephemeral evidence used
  • 62.
    Discussion Discussion question  How canwe ensure that different teachers reach similar judgments about the quality of student work without assessing only surface features of the learning?
  • 63.
  • 64.
    Memory on landand underwater   18 (5f, 13m) student members of a university diving club were tested on their recall of two- and three-syllable words from four 36-word lists taken from the Toronto Word Bank spoken to them twice. Students learned, and were tested on, the words while underwater, and while on the shore, resulting in four conditions:     DD (learn dry, recall dry) DW (learn dry, recall wet) WD (learn wet, recall dry) WW (learn wet, recall wet)
  • 65.
    Memory is context-dependent 65 Recallenvironment Dry Learning environment Wet Dry 13.5 8.6 Wet 8.4 11.4 No significant main effects; interaction effect: F=22.0; df = 1, 12; p= <0.001 Godden and Baddeley (1975)
  • 66.
    Discussion question Discussion 66  How willyou decide how much evidence is needed to decide whether a student has reached a particular outcome?
  • 67.
  • 68.
    Identify milestones (andinch pebbles)  Development of science skills in eighth grade Use of laboratory equipment  Metric unit conversion  Density calculations  Density applications  Density as a characteristic property  Phases of matter  Gas laws  Communication (graphing)  Communication (lab reports)  Inquiry skills 
  • 69.
    Homework 2 ✓ Communication (report) Communication (graph) ✓ ✓ ✓ ✓ Module test ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Homework4 Final exam Gas laws ✓ Homework 3 Laboratory 2 Phases of matter Density properties Equipment ✓ ✓ Homework 1 Laboratory 1 Density calculations Metric units Assessment matrix ✓ ✓ ✓ ✓ ✓ ✓
  • 71.
  • 72.
    Effects of feedback    Kluger& DeNisi (1996) Review of 3000 research reports Excluding those:        without adequate controls with poor design with fewer than 10 participants where performance was not measured without details of effect sizes left 131 reports, 607 effect sizes, involving 12652 individuals On average feedback does improve performance, but   Effect sizes very different in different studies 40% of effect sizes were negative
  • 73.
    Getting feedback rightis hard Response type Feedback indicates performance… exceeds goal falls short of goal Change behavior Exert less effort Increase effort Change goal Increase aspiration Reduce aspiration Abandon goal Decide goal is too easy Decide goal is too hard Reject feedback Feedback is ignored Feedback is ignored
  • 74.
    Discussion question Discussion 74   How canfeedback be designed so as to cue appropriate action by learners? What information should be reported to stakeholders, and how often?
  • 75.
  • 76.
    Mapping out theterrain 76 Timescale Annual Interim Weekly Daily Hourly Academic promotion Benchmark Common formative assessments Before the endof-unit tests Exit pass End-of-course exams High-stakes accountability Growth End-ofunit tests Hinge-point questions Instructional Guidance (“formative”) Describing Individuals (“summative”) Function Institutional Accountability (“evaluative”)
  • 77.
    Perspectives on assessment Role Learners Summative assessment Formative assessment Understandingthe assessment intentions, so they produce relevant evidence become Learners members of the same community of practice of which their teachers are already members Teachers A community of practice in which teachers share a construct of quality Teachers possess an anatomy of quality
  • 78.
    Self-assessment 78 My red folderin the fourth year wants me to be clear and positive about what I achieve in school “in my own words” which are foreign to me. In my own words in my own language (which has no place here) how can I feel clear and positive? Raychaudhuri (1998) My red folder in the fourth year wants me to be positive about my grade E in English History: the heritage and glory of the British Empire “in my own words”. My red folder in the fourth year suddenly out of nowhere wants me to assert what I achieve in school “in my own words”. How can I blow the trumpet they’ve taken from me?
  • 79.

Editor's Notes

  • #3 In the philosophy of Kant the manifold is the unorganized flux presented to the senses, but not experienced, since experience results from the mind structuring the manifold by means of concepts. The nature of the unstructured manifold is unknowable (transcendental).