formative e-assessment: cases, patterns and scenariosPresentation Transcript
formative e-assessment: case stories, design patterns, and future scenarios Caroline Daly, Harvey Mellar, Yishay Mor, Norbert Pachler, Institute of Education, University of London
Scoping study commissioned by JISC
Short term, small budget, intended to inform future funding frameworks
Established a commited user group of higher-education teachers & researchers
Adopted and adapted the Planet Project's Participatory Methodology for Practical Design Patterns, and used the Planet platform
5 Practical Enquiry Days
Combination of collaborative reflection, report back from team, and guest plenaries
Launch day, 3 Planet workshops, developers' day
What is formative e-assessment?
No consistent view in the literature
From “anything test before the final” to “synonymous with learning”
The use of digital means to support formative assessment
Formative features of assessment, which are afforded by specific features of digital media
OK, so what is Formative Assessment?
“ An assessment functions formatively when evidence about student achievement elicited by the assessment is interpreted and used to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions that would have been made in the absence of that evidence”
Formative = feedback + moments of contingency
"... These create "moments of contingency," in which the direction of the instruction will depend on student responses. Teachers provide feedback that engages students, make time in class for students to work on improvement, and activate students as instructional resources for one another."
A bespoke string (sequence) comparator was designed; uses fine-granularity sequence comparison to compare correct language strings to a user’s answer. Students answer questions and the comparator marks up errors in their input using colour coding (and font style) to highlight the different types of error. If an answer contains errors the student is given a second attempt in which to correct the submission based on the feedback received.
Open mentor (Denise Whitelock) http://purl.org/planet/Cases/OpenMentor
A few patterns..
Try Once, Refine Once
Feedback on Feedback
Try Once, Refine Once (Aliy Fowler) http://pul.org/planet/Patterns/TryOnceRefineOnce
Problem Lack of immediate feedback for students leads to fossilisation of errors and misconceptions providing immediate feedback in an iterative fashion can also hinder effective learning since students are able to "grope their way" step-by-step to a correct solution without necessarily having to think about each answer as a whole.
Mode of instruction
Blended / on-line. Computer tested.
Feedback on Feedback (Linda McGuigan) http://purl.org/planet/Patterns/FeedbackonFeedback
Good feedback should -
Alert learners to their weaknesses.
Diagnose the causes and dynamics of these.
Include operational suggestions to improve the learning experience.
Address socio-emotive factors.
Tutors know this, but are pressed for time. Or not aware of their feedback strategies Large teaching organisations are not equipped to provide tutors with personal feedback on their teaching
Large scale, technology supported, graded courses
many tutors instructing many students.
Feedback is mediated by technology that allows it to be captured and processed in real time
Topic of study is subject to both grading and formative feedback.
Embed a mechanism in the learning and teaching system that regularly captures tutor feedback, analyses it, and presents them with graphical representation of the types of feedback they have given. Ideally, this should also include constructive advice as to how to shift from less to more effective forms.
In computer supported environments (e.g. VLEs), this mechanism could be integrated into the system, providing tutors with immediate analysis of their feedback, as well as long-term aggregates.
Needs to collate works in a single easy to access location.
Learners uncomfortable about presenting their work in public
Legal or other restrictions on sharing work.
Small / medium (6-60)
Mode of instruction:
Continuous, over a period
Involves construction / media production
Augmented domain map
When using Try Once Refine Once , there is a risk that high-achievers do not receive feedback.
Use Showcase Learning to celebrate students’ work and provoke feedback from peers and tutors.
Use Feedback on Feedback to alert tutors to the problem.
Tip of the iceberg
Practitioners (educational / software) acknowledge the value of patterns, when served with side dishes of cases + scenarios
Collaborative elicitation of patterns from cases could be a potent form of professional development.
Thank you The Formative e-Assessment project: http://projects.lkl.ac.uk/feasst Final report http://telearn.noe-kaleidoscope.org/open-archive/browse?resource=1875 Yishay Mor http://people.lkl.ac.uk/yishay email@example.com This presentation http://www.slideshare.net/yish/formative-eassessment-cases-patterns-and-scenarios