Getting Vocal About Feedback


Published on

Presentation delivered by Malcolm Mactavish, University of Abertay Dundee, at the 2011 e-Assessment Scotland conference.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Getting Vocal About Feedback

  1. 1. Getting Vocal About Feedback Malcolm Mactavish School of Computing & Engineering Systems Abertay University [email_address]
  2. 2. <ul><li>The Problem: </li></ul><ul><ul><ul><li>Large classes submitting complex assessments (mainly practical coursework) </li></ul></ul></ul><ul><ul><ul><li>Desire to optimise student performance </li></ul></ul></ul>
  3. 3. <ul><li>Ideally: </li></ul><ul><ul><ul><li>Set milestones through the project … </li></ul></ul></ul><ul><ul><ul><li>… review student progress at each … </li></ul></ul></ul><ul><ul><ul><li>… and provide formative feedback </li></ul></ul></ul><ul><ul><ul><li>Students take account of feedback to reinforce early work … </li></ul></ul></ul><ul><ul><ul><li>… resulting in improved final submissions </li></ul></ul></ul>
  4. 4. <ul><li>The Problem: </li></ul><ul><ul><ul><li>Feedback is necessary </li></ul></ul></ul><ul><ul><ul><li>Don't have enough time to give all the feedback I want </li></ul></ul></ul>
  5. 5. <ul><li>The Result: </li></ul><ul><ul><ul><li>Weaker students submit poor work and get poor grades … </li></ul></ul></ul><ul><ul><ul><li>… if they submit anything at all ! </li></ul></ul></ul>
  6. 6. <ul><li>A Solution? </li></ul><ul><ul><ul><li>Reduce time spent on feedback by recording comments rather than writing them </li></ul></ul></ul>
  7. 7. <ul><li>Trial run over three sessions, and using three different systems: </li></ul><ul><ul><ul><li>Email attachment </li></ul></ul></ul><ul><ul><ul><li>Email with link to file </li></ul></ul></ul><ul><ul><ul><li>Email with link to file + PIN number </li></ul></ul></ul>
  8. 8. <ul><li>Email Attachment: </li></ul><ul><ul><ul><li>Great for small files … </li></ul></ul></ul><ul><ul><ul><li>… but larger files might exceed any maximum size for attachments </li></ul></ul></ul>
  9. 9. <ul><li>Email With Link (URL) To File: </li></ul><ul><ul><ul><li>audio file is saved on a web server </li></ul></ul></ul><ul><ul><ul><li>an email is created </li></ul></ul></ul><ul><ul><ul><li>email contains a link to the student's audio file </li></ul></ul></ul><ul><ul><ul><li>other students can 'hack' their own link to retrieve other students' files </li></ul></ul></ul>
  10. 10. <ul><li>Email With Link To File + PIN number: </li></ul><ul><ul><ul><li>audio files are stored in a 'secret' location </li></ul></ul></ul><ul><ul><ul><li>system creates the email as before </li></ul></ul></ul><ul><ul><ul><li>BUT also generates a random four-digit number </li></ul></ul></ul><ul><ul><ul><li>student follows the link and enters their PIN </li></ul></ul></ul>
  11. 11. <ul><li>Email With Link To File + PIN number: </li></ul><ul><ul><ul><li>the student's download can be detected and recorded … </li></ul></ul></ul><ul><ul><ul><li>… so that take-up can be monitored and gauged </li></ul></ul></ul>
  12. 12. <ul><li>Results? </li></ul><ul><ul><ul><li>Feedback for good students takes very little time to record … </li></ul></ul></ul><ul><ul><ul><li>… leaving more time to devote to weaker students … </li></ul></ul></ul>
  13. 13. <ul><li>Results? </li></ul><ul><ul><ul><li>… we can give more feedback overall than we could be writing it out … </li></ul></ul></ul><ul><ul><ul><li>… and, for example, vocal inflection helps us place more emphasis on important areas </li></ul></ul></ul>
  14. 14. <ul><li>Results? </li></ul><ul><ul><ul><li>Students almost universally in favour </li></ul></ul></ul><ul><ul><ul><li>Perhaps predictably, the better students more likely to retrieve feedback … </li></ul></ul></ul><ul><ul><ul><li>… much of the extended feedback to weaker students went unheard. </li></ul></ul></ul><ul><ul><ul><li>However, tracking of downloads allowed follow-up with these students </li></ul></ul></ul>
  15. 15. <ul><li>Results? </li></ul><ul><ul><ul><li>UAD grading scale is 0-20 </li></ul></ul></ul><ul><ul><ul><ul><li>grades 0-3 are irretrievable fails </li></ul></ul></ul></ul><ul><ul><ul><ul><li>6-8 are Marginal Fails </li></ul></ul></ul></ul><ul><ul><ul><ul><li>9-11 are 'pass by skin of teeth' </li></ul></ul></ul></ul>
  16. 16. <ul><li>Results? </li></ul><ul><ul><ul><li>With audio feedback, </li></ul></ul></ul><ul><ul><ul><ul><li>no improvement on irretrievable fails </li></ul></ul></ul></ul><ul><ul><ul><ul><li>fewer marginal fails </li></ul></ul></ul></ul><ul><ul><ul><ul><li>average improvement of 2 points in the bands 6-14 </li></ul></ul></ul></ul>
  17. 17. <ul><li>Potential Elephant Traps: </li></ul><ul><ul><ul><li>Once you've said something, you've said it </li></ul></ul></ul><ul><ul><ul><li>To revise your feedback could mean re-recording from scratch </li></ul></ul></ul><ul><ul><ul><li>You may regret some or all of the following … </li></ul></ul></ul>
  18. 18. <ul><li>Potential Elephant Traps: </li></ul><ul><ul><ul><li>You may regret what you thought were … </li></ul></ul></ul><ul><ul><ul><ul><li>innocent, ad hoc remarks </li></ul></ul></ul></ul><ul><ul><ul><ul><li>comments that you think are constructive but which the students views otherwise </li></ul></ul></ul></ul>
  19. 19. <ul><li>Potential Elephant Traps: </li></ul><ul><ul><ul><li>You may regret what you thought were … </li></ul></ul></ul><ul><ul><ul><ul><li>comments intended within the context of a particular assignment … </li></ul></ul></ul></ul><ul><ul><ul><ul><li>… but the student interprets within a wider context </li></ul></ul></ul></ul>
  20. 20. <ul><li>Potential Elephant Traps: </li></ul><ul><ul><ul><li>You may regret what you thought were … </li></ul></ul></ul><ul><ul><ul><ul><li>comments intended to show the student an effective approach … </li></ul></ul></ul></ul><ul><ul><ul><ul><li>… while the student thinks you're clearly wrong because they know a 'better' way </li></ul></ul></ul></ul>
  21. 21. <ul><li>However … </li></ul><ul><ul><ul><li>Don't we get these very same problems with written feedback? </li></ul></ul></ul>
  22. 22. <ul><li>Important End Note: </li></ul><ul><li>Following a subsequent policy decision, all work had to be given written feedback </li></ul><ul><li>No scope to provide both written and audio feedback </li></ul><ul><li>Grades fell back – roughly to pre-audio levels </li></ul>