‘… assessments which assist learning by giving feedback which indicates how the student is progressing in terms of knowledge, skills and understanding of a subject. In CAA this often takes the form of objective questions with feedback given to the student either during or immediately after the assessment. Formative assessment may be monitored by the tutor, used purely for self-assessment, or used to contribute marks to a module grade’. (p. xiv)
feedback is complex (Shute 2008), or a two-way or multi-way process?
- we contrast Bull with Nicol & Macfarlane-Dick’s 2006 criticisms of transmission-focused feedback:
‘ feedback messages are invariably complex and difficult to decipher’ in transmission contexts - students find it hard to take appropriate actions
feedback tends to emphasis only cognition, not motivation and beliefs which are vital to internalization of knowledge and understanding
transmission-based feedback is used in highly inventive ways, e.g. to direct learners within a carefully structured test environment towards appropriate pathways which enhance motivation and self-regulation (Winkley)
What types of knowledge, skills & understanding are to be included?
How do feedback patterns fit into broader conceptual frameworks for learning and teaching?
The contribution of more/faster/more frequent/ automated feedback to formative assessment ( Conole & Warburton, 2005 )??
The use of digital means to support formative assessment??
Formative features of assessment, which are afforded by specific features of digital media??
“ An assessment functions formatively when evidence about student achievement elicited by the assessment is interpreted and used to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions that would have been made in the absence of that evidence” (Dylan Wiliam)
Formative = feedback + moments of contingency "... These create "moments of contingency," in which the direction of the instruction will depend on student responses. Teachers provide feedback that engages students, make time in class for students to work on improvement, and activate students as instructional resources for one another." (Leahy, Lyon, Thompson, and Wiliam 2005)
the limit on growth is not the capacity to produce, but the knowledge to do it right. Problem: The Design Divide the gap between those who have the expertise to develop high-quality tools and resources and those who don’t (Mor & Winters, 2008*)
Design patterns Formative e-Assessment PED 5, Dec 2008 [describe] a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice (Alexander et al., 1977) C o n t e x t Problem Solution
Formative e-Assessment PED 5, Dec 2008 Problem Keep the rain out Context Cold, wet, poor. Method of solution Thatched roof Related Timber frame, Slanted roof, Chimney
example: activity nodes Formative e-Assessment PED 5, Dec 2008 Design problem Community facilities scattered individually through the city do nothing for the life of the city. Design solution Create nodes of activity throughout the community, spread about 300 yards apart. http://www.uni-weimar.de/architektur/InfAR/lehre/Entwurf/Patterns/030/ca_030.html
‘ E-assessment… there is still much work to be done. One of the most important messages…is simply the variety and scope for imagining new forms of assessment – and of using technology to support it in imaginative ways ’ (Whitelock and Watt 2008, p. 153)
‘ Modernising assessment’ involves blurring the boundaries between formative and summative processes (Elliott 2008)
Rather than thinking in terms of ‘formative assessment’, it might be more appropriate to think in terms of how assessment can be used ‘formatively’ (Wiliam July 08)
Cases Assessment focus Technology used Technology role Socio-ped setting Institutional setting HE under-graduate + post-16 self instant feedback to individual st bespoke string comparator accuracy of language items String comparison HE distance tutor self-assess-ment graphical feedback to tutors web-based tool tutor socio-emotive feedback Open mentor HE vet training WBL self peer-peer t-st, t-group reflection & multi-player feedback social networking/ mob device recording/ reflecting clinical exp. Como: mobiles + flikr HE under-graduate t-st deliver tutor feedback Audacity & dicta-phones concepts in sociology Audiofiles M-level ITE peer-peer t-st t-group represent & share thinking wiki academic writing in teacher ed Academic writing
Solution A bespoke string (sequence) comparator was designed; uses fine-granularity sequence comparison to compare correct language strings to a user’s answer. Students answer questions and the comparator marks up errors in their input using colour coding (and font style) to highlight the different types of error. If an answer contains errors the student is given a second attempt in which to correct the submission based on the feedback received.
Open mentor (Denise Whitelock) http://purl.org/planet/Cases/OpenMentor
Try Once, Refine Once (Aliy Fowler) http://pul.org/planet/Patterns/TryOnceRefineOnce
Problem Lack of immediate feedback for students leads to fossilisation of errors and misconceptions providing immediate feedback in an iterative fashion can also hinder effective learning since students are able to "grope their way" step-by-step to a correct solution without necessarily having to think about each answer as a whole.
Feedback is mediated by technology that allows it to be captured and processed in real time
Topic of study is subject to both grading and formative feedback.
Solution Embed a mechanism in the learning and teaching system that regularly captures tutor feedback, analyses it, and presents them with graphical representation of the types of feedback they have given. Ideally, this should also include constructive advice as to how to shift from less to more effective forms. In computer supported environments (e.g. VLEs), this mechanism could be integrated into the system, providing tutors with immediate analysis of their feedback, as well as long-term aggregates.
Often the advantage of the ‘e’ is that it enables rapid communication of ideas across a range of audiences, and the technology allows this range to be controlled it can be just one person, a group, a class or more
This communication aspect means that aspects of communication can be captured and given a degree of semi-permanence
Semi- permanence supports the sharing of intellectual objects.
V. Construction and representation
Representation – the ability to represent ideas in a variety of ways and to move and translate between these representations
Technology can support learners in the construction of representations of their own ideas.
Shared objects are not fixed. They can be changed easily and quickly.
Practitioners (educational / software) acknowledge the value of patterns, when served with side dishes of cases + scenarios
Collaborative elicitation of patterns from cases could be a potent form of professional development.
Thank you The Formative e-Assessment project: http://projects.lkl.ac.uk/feasst Final report http://telearn.noe-kaleidoscope.org/open-archive/browse?resource=1875 This event http://projects.lkl.ac.uk/feasst/april-28th/ This presentation http://www.slideshare.net/yish/formative-eassessment-a-scoping-study