Feedback
Marking
Standards
Pedagogy
Assessment
literacy
Change
management
Technical skills
Evaluation
Dissemination
Sustainability
Community of
practice
Train the trainer
Cascading
training
Continuous
professional
development
Peer support
Resources and
materials
Blended
approaches
Just in time
training
Recognition
How to design (develop & evaluate) sociotechnical-educational learning
successfully? What is "successful" for whom? What elements can be designed? What R&D methods are appropriate?
How to design (develop & evaluate) sociotechnical-educational learning
successfully? What is "successful" for whom? What elements can be designed? What R&D methods are appropriate?
Synthesis Report onAssessment and Feedback with Technology Enhancement (SRAFTE)Denise Whitelock
Presentation of key findings from Synthesis Report on Assessment and Feedback with Technology Enhancement (SRAFTE) project conducted for the HEA by University of Southampton and The Open University.
Good pedagogical practice driving learning analytics: OpenMentor, Open Commen...Denise Whitelock
OpenMentor, Open Comment are two systems that provide immediate feedback to tutors and students about their learning and support of learning. All these systems were built on the formalisation of sound pedagogical practice. SAFeSEA is a new EPSRC funded project with Oxford University investigating summarisation techniques to give feedback on draft essays.
Technology Enhanced Activities for Learning Science for Children in Hospital ...Denise Whitelock
This research set out to construct a Roadmap for the role that technology can play in the teaching of science to chronically ill children. The modus operandi employed for this study was adapted from Gordon & Glenn's (2003) range of roadmap methodologies.
Supporting Science Studies for children with long term health problems using ...Denise Whitelock
Children with long term health problems cannot always maintain their schooling and keep up with the curriculum (Prevatt et al., 2000; McDougall et al., 2004). They also spend prolonged periods in hospital where access to science teaching is very limited (Sobrino, Lizasoain & Ochoa, 2001). In order to address this problem, the Nefreduca project was designed to develop a short science curriculum for children in Spain with chronic kidney disease. The Nefreduca project was designed to develop a series of open source science inquiry based web learning materials for children with chronic kidney disease. In this paper the learning design strategies employed to build the Nefreduca platform are described , together with how the students’ conceptions of the Kidney’s role in the nutrition process were extended whilst trialling the Nefreduca materials. The students’ notions of the kidney also changed after using the Nefreduca programme. Their answers illustrated a deeper understanding of the urine production process, it’s constituency; and its connection to the blood filtration that occurs in the kidney. These findings suggest that the Nefreduca activities could serve as appropriate teaching material for scaffolding the students’ mental models of the kidney.
Technology-Enhanced Assessment and Feedback: How is evidence-based literature...Denise Whitelock
This desktop research commissioned by the Higher Education Academy set out to consult with the academic community about which references on assessment and feedback with technology enhancement were most useful to practitioners. While all the recommended publications may be characterised as reputable and the majority were peer-reviewed (67.7%), only a minority provided quantitative data (28.2%), of which relatively few provided appropriate experimental designs or statistical analysis (18.5%). The majority of publications were practitioner-led case studies. The references that were recommended to us are clearly having an impact on current practice and are found valuable by practitioners. The key messages from these sources are consistent and often give detailed and practical guidance for other academics. We found that most of the recommended literature focused on the goals that technology enhancement can enable assessment and feedback to meet and how assessment and feedback can be designed to make best use of the technology.
Workshop 1 introduction, case studies and context for wikigemkimble
UNAWE EU Astronomy Education
Evaluation Workshop Session 1:
House of Astronomy, Heidelberg, October 8th 2013
Facilitator: Grace Kimble
Content can be freely used.
The challenges of Assessment and Feedback: findings from an HEA projectDenise Whitelock
The challenges of Assessment and Feedback: findings from an HEA project – Denise Whitelock (IET)
This project was undertaken by IET and colleagues from the University of Southampton and is just producing its final report. The project's aim was to produce a synthesis of evidence based research which throws light on the progress made in the practice of Assessment and Feedback in H.E. This presentation will highlight findings with respect to authentic assessment, e-portfolios, peer assessment, feedback for language learning and Advice for Action.
CHALLENGES OF ENGAGING STUDENTS THROUGH VIRTUAL LEARNING ENVIRONMENTS. A NEW ...Andy Lima
Presentation given for the 13th Conference on Social Science at University of Vienna, October 6th/7th.
Paper published on the EUROPEAN JOURNAL OF SOCIAL SCIENCES EDUCATION AND RESEARCH September-December 2017, Volume 11, Nr. 2, pg 39-51
“Many Universities manage billions in research funding, but there is usually no R&D budget for their own product, namely delivering education to willing buyers” (Michael Stanton, 2014) Education are missing on an explosive growth sector: Their Own. (Harvard Business Review)
Education has undoubtedly been commoditised and as a good. As suggested by American entrepreneur Peter Theil (Cited in Hellweg, 2013), it may function as both learning and insurance access platform.
Another aspect of HE is that, in contrast to most other industries that have faced disruption, the top 10 providers are still the same for the last 40 decades or so. The question is how much longer will they be there? Not even Coursera’s founder Daphne Koller could anticipate the scale and impact of the MOOCs.
Udacity, Coursera and edX, aka ‘The Big Three’ have crossed the barrier of 40 million active students. What does it mean for traditional universities? MIT’s president Raphael Reif struggles to see how his institution can carry on justifying charging $50,0000 for tuition much longer and that is a big problem not only for them for many other universities that are starting to see their numbers dwindling.
This paper is the first in a series of five papers looking at the future of higher education and learning
A collaborative presentation on different type of assessments in education as a group project for Masters in education program. Co-presented with Ms. Anari, Ms. Divya, Ms. Jie , and Mr. Simon
Synthesis Report onAssessment and Feedback with Technology Enhancement (SRAFTE)Denise Whitelock
Presentation of key findings from Synthesis Report on Assessment and Feedback with Technology Enhancement (SRAFTE) project conducted for the HEA by University of Southampton and The Open University.
Good pedagogical practice driving learning analytics: OpenMentor, Open Commen...Denise Whitelock
OpenMentor, Open Comment are two systems that provide immediate feedback to tutors and students about their learning and support of learning. All these systems were built on the formalisation of sound pedagogical practice. SAFeSEA is a new EPSRC funded project with Oxford University investigating summarisation techniques to give feedback on draft essays.
Technology Enhanced Activities for Learning Science for Children in Hospital ...Denise Whitelock
This research set out to construct a Roadmap for the role that technology can play in the teaching of science to chronically ill children. The modus operandi employed for this study was adapted from Gordon & Glenn's (2003) range of roadmap methodologies.
Supporting Science Studies for children with long term health problems using ...Denise Whitelock
Children with long term health problems cannot always maintain their schooling and keep up with the curriculum (Prevatt et al., 2000; McDougall et al., 2004). They also spend prolonged periods in hospital where access to science teaching is very limited (Sobrino, Lizasoain & Ochoa, 2001). In order to address this problem, the Nefreduca project was designed to develop a short science curriculum for children in Spain with chronic kidney disease. The Nefreduca project was designed to develop a series of open source science inquiry based web learning materials for children with chronic kidney disease. In this paper the learning design strategies employed to build the Nefreduca platform are described , together with how the students’ conceptions of the Kidney’s role in the nutrition process were extended whilst trialling the Nefreduca materials. The students’ notions of the kidney also changed after using the Nefreduca programme. Their answers illustrated a deeper understanding of the urine production process, it’s constituency; and its connection to the blood filtration that occurs in the kidney. These findings suggest that the Nefreduca activities could serve as appropriate teaching material for scaffolding the students’ mental models of the kidney.
Technology-Enhanced Assessment and Feedback: How is evidence-based literature...Denise Whitelock
This desktop research commissioned by the Higher Education Academy set out to consult with the academic community about which references on assessment and feedback with technology enhancement were most useful to practitioners. While all the recommended publications may be characterised as reputable and the majority were peer-reviewed (67.7%), only a minority provided quantitative data (28.2%), of which relatively few provided appropriate experimental designs or statistical analysis (18.5%). The majority of publications were practitioner-led case studies. The references that were recommended to us are clearly having an impact on current practice and are found valuable by practitioners. The key messages from these sources are consistent and often give detailed and practical guidance for other academics. We found that most of the recommended literature focused on the goals that technology enhancement can enable assessment and feedback to meet and how assessment and feedback can be designed to make best use of the technology.
Workshop 1 introduction, case studies and context for wikigemkimble
UNAWE EU Astronomy Education
Evaluation Workshop Session 1:
House of Astronomy, Heidelberg, October 8th 2013
Facilitator: Grace Kimble
Content can be freely used.
The challenges of Assessment and Feedback: findings from an HEA projectDenise Whitelock
The challenges of Assessment and Feedback: findings from an HEA project – Denise Whitelock (IET)
This project was undertaken by IET and colleagues from the University of Southampton and is just producing its final report. The project's aim was to produce a synthesis of evidence based research which throws light on the progress made in the practice of Assessment and Feedback in H.E. This presentation will highlight findings with respect to authentic assessment, e-portfolios, peer assessment, feedback for language learning and Advice for Action.
CHALLENGES OF ENGAGING STUDENTS THROUGH VIRTUAL LEARNING ENVIRONMENTS. A NEW ...Andy Lima
Presentation given for the 13th Conference on Social Science at University of Vienna, October 6th/7th.
Paper published on the EUROPEAN JOURNAL OF SOCIAL SCIENCES EDUCATION AND RESEARCH September-December 2017, Volume 11, Nr. 2, pg 39-51
“Many Universities manage billions in research funding, but there is usually no R&D budget for their own product, namely delivering education to willing buyers” (Michael Stanton, 2014) Education are missing on an explosive growth sector: Their Own. (Harvard Business Review)
Education has undoubtedly been commoditised and as a good. As suggested by American entrepreneur Peter Theil (Cited in Hellweg, 2013), it may function as both learning and insurance access platform.
Another aspect of HE is that, in contrast to most other industries that have faced disruption, the top 10 providers are still the same for the last 40 decades or so. The question is how much longer will they be there? Not even Coursera’s founder Daphne Koller could anticipate the scale and impact of the MOOCs.
Udacity, Coursera and edX, aka ‘The Big Three’ have crossed the barrier of 40 million active students. What does it mean for traditional universities? MIT’s president Raphael Reif struggles to see how his institution can carry on justifying charging $50,0000 for tuition much longer and that is a big problem not only for them for many other universities that are starting to see their numbers dwindling.
This paper is the first in a series of five papers looking at the future of higher education and learning
A collaborative presentation on different type of assessments in education as a group project for Masters in education program. Co-presented with Ms. Anari, Ms. Divya, Ms. Jie , and Mr. Simon
4 March 2010 (Thursday) | 15:30 - 17:40 | http://citers2010.cite.hku.hk/abstract/20 | Dr. Barbara MEANS | Center for Technology in Learning, SRI International
Student Generated Content and Guidelines to StudentsEunbae Lee
This presentation is created to share with my EDIT 2000 class at the University of Georgia.
This presentation aims to provide a framework for designing and implementing student-generated content (SGC) activities using free Web 2.0 tools. I discuss the characteristics of SGC and guidelines to students when they engage with SGC activities.
Similar to Dmw. E Assessmentlive July2009 Videos (20)
Should feedback be at the centre of Personalised Learning?Denise Whitelock
Should feedback be at the centre of Personalised Learning?
The advent of e-Learning has prompted the development of web-based learning systems, recognising there is no fixed learning pathway that will be appropriate for all learners. However, most learning platforms with personalised learning sequencing rely on a learner’s preferences.
However if we want students to be able to learn to make reliable judgements about their learning and to identify any further support they require to meet their learning goals, then personalised automatic feedback should play an important role. This presentation explores the role that technology enhanced feedback can play in the pursuit of a personalised learning agenda.
References
Whitelock, D., Twiner, A., Richardson, J.T.E., Field, D. & Pulman, S. (2015). Feedback on academic essay writing through pre-emptive hints: Moving towards ‘advice for action’. Winner of Best Research Paper Award. Special Issue of European Journal of Open, Distance and E-Learning, Best of EDEN RW8, 8th EDEN Research Workshop (eds. U. Bernath and A. Szucs). Published by European Distance and E-Learning Network, 1-15. ISSN 1027 5207
Whitelock, D., Twiner, A., Richardson, J.T.E., Field, D. & Pulman, S. (2015). OpenEssayist: A supply and demand learning analytics tool for drafting academic essays. The 5th International Learning Analytics and Knowledge (LAK) Conference, Poughkeepsie, New York, USA. 16-20 March 2015. ISBN 978-1-4503-3417-4
Technology Enhanced Assessment: Do we have a wolf in sheep's clothing?Denise Whitelock
Technology Enhanced Assessment: Do we have a wolf in sheep’s clothing?
A sea change in assessment, precipitated by both researchers and practitioners alike, was crystallised by a statement issued by the Assessment Reform Group, who have rejected the notion of Assessment that foregrounds cognitive ability tests that are valued for their predicted validity (Broadfoot, Daugherty, Gardner, Harlen, James & Stobart, 2002). The ARG set out to promote better alignment between teaching, learning & assessment and endorsed the term ‘Assessment for Learning’. This presentation explores the role that technology enhanced assessment can play in encouraging the assessment for learning agenda. It presents a number of cases of peer, self and computer assessments that display a range of characteristics for the next generation of assessment tasks.
The discussion of the cases reveals a missing characteristic, which is a form of feedback to the students that will take their learning forward which I refer to as “Advice for Action” (Whitelock, 2011). Recent developments in automatic feedback systems for essay writing (Whitelock, Twiner, Richardson, Field & Pulman, 2015a and 2015b) will be presented and the role of visualisations and socio-emotive feedback in conveying meaningful feedback will also be discussed.
Since any feedback that is not understood or cannot be acted upon is likely to be ignored it will not facilitate learner improvement or confidence. This will always be a challenge – but how can technology enhanced assessment pursue this agenda?
References
Broadfoot, P., Daugherty, R., Gardner, J., Harlen, W., James, M. & Stobart, G. (2002). Assessment for learning: 10 principles, Research-based principles to guide classroom practice. London: Assessment Reform Group. Retrieved 4 April 2017, from http://sunnyspelles.co.uk/Pedagogy%20Resources/A4L/10principles.pdf
Whitelock, D. (2011). Activating Assessment for Learning: are we on the way with Web 2.0? In M.J.W. Lee & C. McLoughlin (Eds.) Web 2.0-Based-E-Learning: Applying Social Informatics for Tertiary Teaching. IGI Global. 319-342.
Whitelock, D., Twiner, A., Richardson, J.T.E., Field, D. & Pulman, S. (2015a). Feedback on academic essay writing through pre-emptive hints: Moving towards ‘advice for action’. Winner of Best Research Paper Award. Special Issue of European Journal of Open, Distance and E-Learning, Best of EDEN RW8, 8th EDEN Research Workshop (eds. U. Bernath and A. Szucs). Published by European Distance and E-Learning Network, 1-15. ISSN 1027 5207
Whitelock, D., Twiner, A., Richardson, J.T.E., Field, D. & Pulman, S. (2015b). OpenEssayist: A supply and demand learning analytics tool for drafting academic essays. The 5th International Learning Analytics and Knowledge (LAK) Conference, Poughkeepsie, New York, USA. 16-20 March 2015. ISBN 978-1-4503-3417-4
Who has the crystal ball for moving forward with Digital Assessment?Denise Whitelock
Who has the crystal ball for moving forward with Digital Assessment?
Digital assessment is an evolving construct used in education to enrich, inform and complement the teaching process. Using automatic feedback however, has been under-utilised and under-valued throughout this process and further highlighted with the introduction of electronic teaching and assessments.
This presentation will discuss the issues raised by teachers and students in this arena. It will provide exemplars of how their concerns are currently being addressed by both researchers and software developers in order to support educator feedback to students. Finally, the issue of potential disrupters will be raised which moves us into the realm of crystal ball gazing.
Keynote address presented at WISEflow Conference, Brunel University
Designing and testing visual representations of draft essays for Higher Educa...Denise Whitelock
This presentation reports the findings of an empirical investigation which set out to test a set of rainbow exercises. The rainbow diagrams are pictorial representations of formal graphs that are derived automatically from student essays. They were designed to allow students to discover how key concepts in a well written essay are connected together. The students would then be able to compare a rainbow diagram of their own essay with a good essay and make changes to it before submission to their tutor. A trial was undertaken with academics, teaching and learning staff, doctoral students at The Open University of Catalonia and the Open University UK, before implementation into the web application known as OpenEssayist.
OpenEssayist: Feedback and moving forward with draft essaysDenise Whitelock
The presentation assists users when submitting draft essays for analysis by OpenEssayist to make full use of the feedback in order to draft another version of their essay.
A reflection on where we are with learning analytics as a new multi-discipline research area. Reflections from the Learning Analytics Conference 2013 with respect to Assessment.
Academics' Understanding of Authentic AssessmentDenise Whitelock
This paper reports on a project undertaken at The Open University which set out to explore academics’ notion and practice of authentic assessment through the exploration of the following research objectives:
1. To understand what is meant by authentic assessment in the literature by examining a set of examples of authentic assessments.
2. To construct a questionnaire which could be used by Open University academics to explore their understanding of authentic assessment.
3. To investigate through means of a questionnaire the types of assessment academics were currently undertaking and whether they fitted into a broad definition of authentic assessment.
The findings from the electronic survey suggest that Open University academics are on the way to designing meaningful assessments for their students. Although many of the courses were employing assessment tasks that could be considered as ‘authentic’, only 25% of the academics had heard of the terms ‘authentic learning’ and ‘authentic assessment’, which is a low response compared with ‘learning design’. However, there has been a well publicised Learning Design initiative taking place across the University.
This presentation models the feedback teachers give to students who are using an interactive computer tutorial (Nefreduca) to help them understand key biological concepts related to the kidney. Nefreduca consists of a series of open source science inquiry based web learning materials targeted at children with chronic kidney disease. It utilises a scenario based approach drawing on the work of Lijnse (1995) and Buty, Tiberghien & Le Marechal (2004). The hospital teachers who assisted the children with their use of Nefreduca reported learning gains with the system but there is no automatic feedback for autonomous learning with Nefreduca when children are unsupported by a teacher. An ethnographic approach was adopted to then understand how teachers provided feedback to the students in order to devise a model of supportive feedback to the chronically ill students, which can be implemented in the next version of Nefreduca.
Framing Feedback for Formative Assessment, Denise WhitelockDenise Whitelock
Presentation given to formative e-assessment project group at the Institute of Education, 3rd July 2008. Synopsis of projects at The Open University, UK.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
Honest Reviews of Tim Han LMA Course Program.pptxtimhan337
Personal development courses are widely available today, with each one promising life-changing outcomes. Tim Han’s Life Mastery Achievers (LMA) Course has drawn a lot of interest. In addition to offering my frank assessment of Success Insider’s LMA Course, this piece examines the course’s effects via a variety of Tim Han LMA course reviews and Success Insider comments.
June 3, 2024 Anti-Semitism Letter Sent to MIT President Kornbluth and MIT Cor...Levi Shapiro
Letter from the Congress of the United States regarding Anti-Semitism sent June 3rd to MIT President Sally Kornbluth, MIT Corp Chair, Mark Gorenberg
Dear Dr. Kornbluth and Mr. Gorenberg,
The US House of Representatives is deeply concerned by ongoing and pervasive acts of antisemitic
harassment and intimidation at the Massachusetts Institute of Technology (MIT). Failing to act decisively to ensure a safe learning environment for all students would be a grave dereliction of your responsibilities as President of MIT and Chair of the MIT Corporation.
This Congress will not stand idly by and allow an environment hostile to Jewish students to persist. The House believes that your institution is in violation of Title VI of the Civil Rights Act, and the inability or
unwillingness to rectify this violation through action requires accountability.
Postsecondary education is a unique opportunity for students to learn and have their ideas and beliefs challenged. However, universities receiving hundreds of millions of federal funds annually have denied
students that opportunity and have been hijacked to become venues for the promotion of terrorism, antisemitic harassment and intimidation, unlawful encampments, and in some cases, assaults and riots.
The House of Representatives will not countenance the use of federal funds to indoctrinate students into hateful, antisemitic, anti-American supporters of terrorism. Investigations into campus antisemitism by the Committee on Education and the Workforce and the Committee on Ways and Means have been expanded into a Congress-wide probe across all relevant jurisdictions to address this national crisis. The undersigned Committees will conduct oversight into the use of federal funds at MIT and its learning environment under authorities granted to each Committee.
• The Committee on Education and the Workforce has been investigating your institution since December 7, 2023. The Committee has broad jurisdiction over postsecondary education, including its compliance with Title VI of the Civil Rights Act, campus safety concerns over disruptions to the learning environment, and the awarding of federal student aid under the Higher Education Act.
• The Committee on Oversight and Accountability is investigating the sources of funding and other support flowing to groups espousing pro-Hamas propaganda and engaged in antisemitic harassment and intimidation of students. The Committee on Oversight and Accountability is the principal oversight committee of the US House of Representatives and has broad authority to investigate “any matter” at “any time” under House Rule X.
• The Committee on Ways and Means has been investigating several universities since November 15, 2023, when the Committee held a hearing entitled From Ivory Towers to Dark Corners: Investigating the Nexus Between Antisemitism, Tax-Exempt Universities, and Terror Financing. The Committee followed the hearing with letters to those institutions on January 10, 202
Welcome to TechSoup New Member Orientation and Q&A (May 2024).pdfTechSoup
In this webinar you will learn how your organization can access TechSoup's wide variety of product discount and donation programs. From hardware to software, we'll give you a tour of the tools available to help your nonprofit with productivity, collaboration, financial management, donor tracking, security, and more.
Biological screening of herbal drugs: Introduction and Need for
Phyto-Pharmacological Screening, New Strategies for evaluating
Natural Products, In vitro evaluation techniques for Antioxidants, Antimicrobial and Anticancer drugs. In vivo evaluation techniques
for Anti-inflammatory, Antiulcer, Anticancer, Wound healing, Antidiabetic, Hepatoprotective, Cardio protective, Diuretics and
Antifertility, Toxicity studies as per OECD guidelines
Embracing GenAI - A Strategic ImperativePeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
Francesca Gottschalk - How can education support child empowerment.pptxEduSkills OECD
Francesca Gottschalk from the OECD’s Centre for Educational Research and Innovation presents at the Ask an Expert Webinar: How can education support child empowerment?
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
Palestine last event orientationfvgnh .pptxRaedMohamed3
An EFL lesson about the current events in Palestine. It is intended to be for intermediate students who wish to increase their listening skills through a short lesson in power point.
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
Adversarial Attention Modeling for Multi-dimensional Emotion Regression.pdf
Dmw. E Assessmentlive July2009 Videos
1. Electronic Assessment: a reflection on old
practice and an appraisal of future
directions
Denise Whitelock
The Open University, Walton Hall,
Milton Keynes MK7 6AA
d.m.whitelock@open.ac.uk
DMW, Loughborough, July 2009
8. MCQs: Variation on a theme (1)
The question is an example of a COLA The question was developed using
assessment used at the Reid Kerr Questionmark Perception at the
College, Paisley. It is a Multiple University of Dundee. It is part a set of
response Question used in one of their formative assessment for medical
modules. students.
DMW, Loughborough, July 2009
9. Example of use of TRIADS, Molecular Cell Biology module –
1st year BSc Biological Sciences,
Birkbeck, Richard Rayne
DMW, Loughborough, July 2009
10. MCQs: Variation on a theme (2)
Example of LAPT Certainty-Based
Marking, UK cabinet ministers demo Drug Chart Errors and Omissions,
exercise showing feedback, Medicines Administration Assessment,
University College, Tony Gardner- Chesterfield Royal Hospital
Medwin
DMW, Loughborough, July 2009
11. MCQs: High Stakes Assessment
Example of practice Thinking Skills
Assessment" (TSA) question, Example of practice Thinking Skills
Admissions Interview, Assessment" (TSA) feedback,
Cambridge Assessment, Steve Lay Admissions Interview,
Cambridge Assessment, Steve Lay
DMW, Loughborough, July 2009
12. Scaffolding and High Stakes assessment
• Math for Science
• Tutor less course
• Competency led
• No point to cheat
• Web home exam
• Invigilation technologies
DMW, Loughborough, July 2009
13. Theoretical drivers for Asesessment
• Piaget (1930) Individual manipulating surroundings
--mental representations can be tested Induction of
rules and their testing ...Nuffield science
• Cognitive pyschology( 70s 80s) How are these mental
representations stored? Classification development and
how this can go wrong. .... Misconceptions and mal
rules........ AI
• Bruner (1982) assessment tasks will match
competency levels depending on level of
help...SCAFFOLDING
DMW, Loughborough, July 2009
14. Scaffolding and Interactivity; OU Science
Foundation Course
• Interaction, Feedback loops
• Tell, Explore, Check
• Predict, Look and Explain
• Entering the discourse of a subject via audio
feedback
• Scaffolded text feedback (Bruner & Woods)
• SHOW ME button
DMW, Loughborough, July 2009
16. Interactive Tasks
• Games
• Simulations
• Making the abstract
concrete
• Directing the sequence
of an animation
DMW, Loughborough, July 2009
17. Theoretical drivers:Social Constructivism
• Vygotsky (1978) individuals shape cultural settings
which shapes minds no longer individual
• Activity theory (Engstrom 1987) Tool mediation
• Situated Cognition (Lave and Wenger 1991) Authentic
assessment
• Peer interaction and assessment
• Learning conversations Laurillard (2002)
DMW, Loughborough, July 2009
18. Characteristics Descriptor
Authentic Involving real-world knowledge and skills
Personalised Tailored to the knowledge, skills and interests of each
student
Negotiated Agreed between the learner and the teacher
Engaging Involving the personal interests of the students
Recognise existing skills Willing to accredit the student’s existing work
Deep Assessing deep knowledge – not memorization
Problem oriented Original tasks requiring genuine problem solving skills
Collaboratively produced Produced in partnership with fellow students
Peer and self assessed Involving self reflection and peer review
Tool supported Encouraging the use of ICT
Elliott’s characteristics of Assessment 2.0 activities
DMW, Loughborough, July 2009
19. Authentic assessments :e-portfolios
Electronic NVQ portfolio cover contents page, OCR IT Practitioner, EAIHFE, Robert Wilsdon
DMW, Loughborough, July 2009
21. Building e-portfolios on a chef’s course
Evidence of food preparation skill for e-
portfolio, Modern Apprenticeship in
Hospitality and Catering, West Suffolk
food preparation for e-portfolio, Modern College, Mike Mulvihill
Apprenticeship in Hospitality and Catering,
West Suffolk College, Mike Mulvihill
DMW, Loughborough, July 2009
22. Mobile Technologies and Assessment
• MCQs ,PDAs
Valdiva &
Nussbaum(2009)
• Polls,instant surveys
• Simpson & Oliver
(2007)
• Draper (2009) EVS
DMW, Loughborough, July 2009
23. Gains from Formative Assessment
Mean effect size on standardised tests between 0.4 to
0.7 (Black & Williams, 1998)
Particularly effective for students who have not done
well at school
http://kn.open.ac.uk/document.cfm?docid=10817
Can keep students to timescale and motivate them
How can we support our students to become more
reflective learners and engage in formative assessment
tasks?
DMW, Loughborough, July 2009
24. Collaborative formative assessment with
Global Warming
DMW, Institute of Educati onal Technology, September 1997
DMW, Loughborough, July 2009
28. Research issues from CAA: Marking Free
text
• e-Rater Attali & Burnstein (2006)
• LSA (Laudaurer,Laham& Foltz 2003)
• Christie SEAR (1999)
• Knowledge engineering Pulman &Sukkarieh (2005)
• IAT (Jordan and Mitchell 2009)
• Genre Analysis Moreale& Vargas –Vera (2005)
DMW, Loughborough, July 2009
29. Open Comment addresses the problem of
free text entry
• Automated formative assessment tool
• Free text entry for students
• Automated feedback and guidance
• Open questions, divergent assessment
• No marks awarded
• For use by Arts Faculty
DMW, Loughborough, July 2009
30. Three Causal Question Types proved to be
suitable for Open Comment
3. Analysis of statistics, usually presented to the student
as a table
4. Comprehension of a set text
5. Identifying similarities and differences for a given event
DMW, Loughborough, July 2009
31. Open Comment components
• A Java-based feedback system
• A web service shell
• A graphical interface for testing
• A Moodle-based question type
• A forms-based editing tool
DMW, Loughborough, July 2009
36. Stages of analysis by computer of students’ free text
entry for Open Comment: advice with respect to
content (socio-emotional support stylised example)
• STAGE 1a: DETECT ERRORS E.g. Incorrect dates,
facts. (Incorrect inferences and causality is dealt with
below)
• Instead of concentrating on X, think about Y in order to
answer this question Recognise effort (Dweck) and
encourage to have another go
• You have done well to start answering this question but
perhaps you misunderstood it. Instead of thinking about
X which did not…….. Consider Y
DMW, Loughborough, July 2009
37. Computer analysis continued
• STAGE 2a: REVEAL FIRST OMISSION
• Consider the role of Z in your answer Praise what is
correct and point out what is missing Good but now
consider the role X plays in your answer
• STAGE 2b: REVEAL SECOND OMISSION
• Consider the role of P in your answer Praise what is
correct and point out what is missing Yes but also
consider P. Would it have produced the same result if P
is neglected?
DMW, Loughborough, July 2009
38. Final stages of analysis
• STAGE 3:REQUEST CLARIFICATION OF KEY POINT 1
• STAGE 4:REQUEST FURTHER ANALYSIS OF KEY POINT
1(Stages 3 and 4 repeated with all the key points)
• STAGE 5:REQUEST THE INFERENCE FROM THE ANALYSIS
OF KEY POINT 1 IF IT IS MISSING
• STAGE 6:REQUEST THE INFERENCE FROM THE ANALYSIS
OF KEY POINT 1 IF IT IS NOT COMPLETE
• STAGE 7:CHECK THE CAUSALITY
• STAGE 8:REQUEST ALL THE CAUSAL FACTORS ARE
WEIGHTED
DMW, Loughborough, July 2009
39. Where are we now?
• Opening up with Open Source
• Moving towards vision and not losing sight of it through
tool adaptation
• More work to do for Arts
• Open Comment - pedagogical model open to test
• Feedback
• Changing pedagogy
• Another handle on misconceptions
DMW, Loughborough, July 2009
40. Open Comment drivers for reflection
• Students are able to find facts similar to X
• Know how X might be disputed
• Are able to make predictions about X
• Know how to use X in an argument
• Know how far X can be pushed
• Supported with tools and strategies
DMW, Loughborough, July 2009
41. LISC: Aily Fowler
• Kent University ab-initio Spanish module
• Large student numbers
• Skills-based course
• Provision of sufficient formative assessment
meant unmanageable marking loads
• Impossible to provide immediate feedback
• leading to fossilisation of errors
DMW, Loughborough, July 2009
42. The LISC solution: developed by Ali Fowler
• A CALL system designed to enable students
to:
• Independently
practise sentence
translation
• Receive immediate
(and robust)
feedback on all
errors
• Attend immediately
to the feedback
(before
fossilisation can
occur)
DMW, Loughborough, July 2009
43. How is the final mark arrived at in the
LISC System?
• The two submissions are unequally weighted
• Best to give more weight to the first attempt
• since this ensures that students give careful
consideration to the construction of their first
answer
• but can improve their mark by refining the answer
• The marks ratio can vary (depending on
assessment/feedback type)
• the more information given in the feedback, the
lower the weight the second mark should carry
DMW, Loughborough, July 2009
44. Heuristics for the final mark
• If the ratio is skewed too far in favour of the
first attempt…
• students are less inclined to try hard to correct
non-perfect answers
• If the ratio is skewed too far in favour of the
second attempt…
• students exhibit less care over the
construction of their initial answer
DMW, Loughborough, July 2009
45. Feedback: Advice for Action
• Students must decode
feedback and then act on it
Boud (2000)
• Students must have the
opportunity to act on
feedback Sadler (1989)
• Gauging efficacy through
student action
DMW, Loughborough, July 2009
47. Characteristics Descriptor
Authentic Involving real-world knowledge and skills
Personalised Tailored to the knowledge, skills and interests of each
student
Negotiated Agreed between the learner and the teacher
Engaging Involving the personal interests of the students
Recognise existing skills Willing to accredit the student’s existing work
Deep Assessing deep knowledge – not memorization
Problem oriented Original tasks requiring genuine problem solving skills
Collaboratively produced Produced in partnership with fellow students
Peer and self assessed Involving self reflection and peer review
Tool supported Encouraging the use of ICT
Advice for Action
Elliott’s characteristics of Assessment 2.0 activities
DMW, Loughborough, July 2009
48. Embedding Assessment into Pedagogical
Frameworks
• Luca & Oliver (2002)
• Boud & Falchikov
(2007)
• Bartlett-Bragg
(2008)
DMW, Loughborough, July 2009
50. The 4Ts Pyramid
Transfer
Learning
Transformation
Tasks
Training of staff
Tool Development
DMW, Loughborough, July 2009
51. e-Assessment: effectiveness vs.
innovation
35
Dundee
30
Derby
OU
East Antrium Ulster
Plymouth
Birkbeck
25 Glamorgan Loughborough
Southampton
West Suffolk
Effectiveness
20 Heriot Watt
Cambridge Assessment
Belfast
Coleg Sir Gar
15 Warwickshire College
10 UCL Surrey
Calibrand
5
0 COLA
4 6 8 10 12 14 16
Innovation
DMW, Loughborough, July 2009
52. e-Assessment: Maturity vs. Availability
Scatter diagram to illustrate relationship between Maturity vs Availability
32
Heriot Watt
OU
Surrey
30 Derby Dundee
Heriot Watt
Ulster
Dundee
28 East Antrium Plymouth
Surrey Birkbeck Southampton
Loughborough
26 OU
Ulster
Southampton
UCL
Loughborough
24
Maturity
Plymouth Glamorgan
Cambridge Assessment Derby
22 Warwickshire College Coleg Sir Gar Birkbeck
Warwickshire College
UCL
Coleg Sir Gar
20 East Antrium
West Suffolk Calibrand Belfast
18 West Suffolk
COLA
Cambridge Assessment
16 Belfast Calibrand
14
12 14 16 18 20 22 24
Availability
DMW, Loughborough, July 2009
53. Training
Question
development
Quality REAQ
Plagiarism software
Advice for Action
Socio e-emotive
content
Maintain empathy
with the Learner
DMW, Loughborough, July 2009
54. e-Assessment Futures
• Research issues
• Adaptive testing
• Automatic marking
• Data mining
• Web 2.0, 3.0 ...
DMW, Loughborough, July 2009
55. Rearranging
• Promoting
champions
• Multidisciplinary
teams
• Open source
• Pedagogical
frameworks
DMW, Loughborough, July 2009
56. National Union of Students’ Principles of Effective
Assessment Times Higher Education, 29th January
2009
• Should be for learning, not simply of learning
• Should be reliable, valid, fair and consistent
• Should consist of effective and constructive feedback
• Should be innovative and have the capacity to inspire
and motivate.
• Should be conducted throughout the course, rather
than being positioned as a final event
• Should develop key skills such as peer and reflective
assessment
DMW, Loughborough, July 2009