Ajjawi online eassessment conference final


Published on

Slides from eAssessment Conference 05Dec 2012

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • £125,021 for 3 years
  • Effective feedback can lead to change in learners through affective processes of increased effort, motivation and engagement, or through cognitive processes including restructuring of knowledge and alternative strategies to understanding (Hattie and Timperley, 2007).Bullet point 2: no feedback can be demotivating and humiliating.
  • Dialogical process NOT a product to be delivered
  • The rationale for self-review1. The purpose of feedback is to develop the students’ capacity to make evaluative judgements of their own work, without help of external agent.2. Developing this ability would arguably make students better at using teacher feedback3. Some research suggests that the more feedback teachers give, the more dependent some students become. .
  • Watling feedback credibility based on perceptions of trust and relationship between the people – so where does that leave us when we don’t ever meet many of our students in person? How to take into account fear confidence and receptivity when online?Written feedback – losing audio/visual/kinestheticsTechnological advances but perhaps not quite there? Audio feedback (not less time but richer).WorkloadsMotivation from tutors
  • Ajjawi online eassessment conference final

    1. 1. interACT: InteractiveAssessment andCollaboration viaTechnology
    2. 2. Outline• Background to project• What we did• Where we are up to• Preliminary evaluation data• Challenges and future development
    3. 3. The Project Team Rola Ajjawi Karen Barton Grant Murray Susie SchofieldProject Director Project Officer Learning Technologist Project Manager Natalie Lafferty Sean McAleer David Walker e-learning Advisor Assessment Advisor Learning Technology Advisor
    4. 4. Recognition of a need for change….• External examiner reports (2006-2011)• End of course evaluations (2006-2011)• Additional evaluation surveys conducted in 2010 and 2011 as part of curriculum review• HEA postgraduate student experience survey• Staff discussions• Feedback audit (5% random sample of all assignments submitted in 2011 audited)Outlined that…
    5. 5. Specific issues raised:1. Inconsistency in the quality and quantity of feedback provided2. Assessment design (e.g. over-assessment, over-reliance on essays, lack of formative assessment)3. Timeliness of the feedback4. Lack of assessment and feedback dialogue
    6. 6. A critque of monologic feedback1-6• Lack of learner engagement with feedback• Lack of understanding of feedback• Transmitted feedback creates dependency on teacher• Not utilising self/peer feedback• Lack of a shared context for assessment for teacher and learner• High teacher effort— low efficiency• Reduced staff satisfaction as evidence of feedforward not seen
    7. 7. Shift conceptions of feedback• Feedback should develop the students’ capacity to make evaluative judgements about their own and others work1,2• Feedback should serve the function of progressively enabling students to better monitor, evaluate and regulate their own learning, independently of the teacher3
    8. 8. So what did we do…http://youtu.be/S5bBFEbXDD0
    9. 9. Re-engineered our assessment • Modular approach to sequencing of assessment • Explicating feedback times in a student-tutor charter • Development of assessment rubrics for all summative • Increased use of formative assessments
    10. 10. PG Certificate in Medical Education Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 chapters chapters chapters chapters chapters chapters chapters chapters chapters chapters Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 chapters chapters chapters chapters chapters chapters chapters chapters chapters chapters Revised PG Certificate in Medical Education Formative assessmentChapters 2 4 6 8 10 Medium stakes assessment High stakes assessmentChapters 2 4 6 8 10Chapters 2 4 6 8 10 ©Mark Russell 2010Chapters 2 4 6 8 10 University of Hertfordshire ESCAPE project
    11. 11. Faculty development • Workshops about feedback and importantly agreement on how to give feedback (process)
    12. 12. Include self-review • Self-review is a scenario where students make evaluative judgements about the own work and produce a written feedback commentary4
    13. 13. Content (understanding of theory / principles and Self-evaluation:application to own context)- Understanding of learning theory- Evidence of critical reflection on learning theory andkey learning and teaching principles- Evidence of application of learning theory to ownpractice Tutor feedback:Style, format and language (e.g. structure, coherence, Self-evaluation:flow, formatting, use of language) Tutor feedback:Sources and references (e.g. range of references cited, Self-evaluation:relevance, consistency, accuracy and completeness ofreferencing) Tutor feedback:Which aspect(s) of your assignment would you Student comment:specifically like feedback on? Tutor feedback:How did previous feedback inform this assignment? Student comment:Name of Tutor: Date:
    14. 14. Engaging students in processing of feedback through the wiki When you receive your assignment with feedback please upload a copy to your wiki and include a reflection on the following four questions: 1. How well does the tutor feedback match with your self-evaluation? 2. What did you learn from the feedback process? 3. What actions, if any, will you take in response to the feedback process? 4. What if anything is unclear about the tutor feedback?
    15. 15. Student engagement• interACT launched 30th April 2012• 100% completion of the cover page• Wiki engagement varies from 65-20%
    16. 16. Evaluation interviews to date• Conducted interviews with 5 students• They find the instructions to be clear (questions to technical team have dropped)• They value the opportunity to ask for specific feedback on aspects of their work and ability to dialogue with tutors• The cover page self-evaluation is repetitive for references and style sections• Building on this with a survey to be launched in the next week
    17. 17. Aspects I would like feedback on:• Ideas of not preparing too rigidly in order to be flexible within sessions – practical advice would be welcomed!• As the first essay I have written in nearly 20 years, I would like to know whether the standard overall was acceptable• Please advise me how I can enter a specific page number in a reference when using Endnote (I wanted to add “p. 58” to the first reference used in the text since this is an exact quote, but failed to find out how I can do it, despite using the help option of the software).• Feedback on whether my peers have had similar thoughts for their own teaching, or other ideas that have been commonly developed would be beneficial in case I have not thought or considered them.• any part of it
    18. 18. How did previous feedback inform this assignment• It made me realise that instead of focusing on a single or a few key teaching principles, I focused on many of them without going into much detail. Also I had used bullet points in the text.• Feedback that my writing style was agreeable was reassuring. I appreciated knowing my use of literature was valid and supportive in the previous assignment, so have tried to continue applying the literature to my work.• I tried to be careful to define and reference jargon• It was really helpful in writing present assignment
    19. 19. I really like the cover sheet and request for selfassessment and what you would like feedbackabout ... consistent with principles of feedback!!(M&L end of module feedback)I like this better, as there is an element ofexpectation that there will be some dialogue …What I really like about it, is that it gives me achance to request, to maybe direct a bit thespecificity of what kind of information I am going toget (student interview #3)
    20. 20. Added challenges of: For sharing feedback • Relationship and trust • Asynchronous dialogue and shared understanding • Reliance on written feedback • Technology
    21. 21. Benefits of doing the audit• Raises awareness of quantity and quality of feedback provided individually and across the centre• Opportunity for faculty development• Objective measure of changes in feedback culture – The process will be repeated on 2012 assignments at the beginning of 2013
    22. 22. Future developments• Develop a toolkit of e-activities designed to improve self, peer and tutor dialogic feedback through the use of available technologies such as blogs, Friendfeed, Skype/webinars, or synchronous chat• Introduce a patchwork assignment at the end of the Certificate and Diploma• Work to refine process
    23. 23. Project websitehttp://blog.dundee.ac.uk/interact
    24. 24. Contact DetailsRola AjjawiCentre for Medical EducationUniversity of DundeeEmail: r.ajjawi@dundee.ac.ukTwitter: @r_ajjawihttp://blog.dundee.ac.uk/interact/Karen BartonCentre for Medical EducationUniversity of DundeeEmail: interact@dundee.ac.uk
    25. 25. References1. Sadler, D. R. (2010) Beyond feedback: developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35:5, 535-550.2. Boud, D., & Associates. (2010). Assessment 2010: Seven propositions for assessment reform in higher education. Sydney: Australian Learning and Teaching Council3. Nicol, D. J., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218.4. Nicol, D. (2012). Assessment and feedback - in the hands of the student [Online]. JISC. Available: http://jiscdesignstudio.pbworks.com/w/page/50118521/Assessment%2 0and%20feedback%20- %20in%20the%20hands%20of%20the%20student [Accessed 01/02/12].5. Brown, E. & Glover, C. (2006) Evaluating written feedback. in: B. C. & K. Klegg (Eds) Innovative assessment in higher education. London, Routledge), 81-91.6. Hattie, J. & Timperley, H. (2007) The Power of Feedback. Review of Educational Research, 77, 81-112.