RethinkingStudent Feedback<br />13th July 2011 3:00 pm Venetian Congress Center - Titian 2206<br />Dr Malcolm MurrayDurham...
Acknowledgement<br />Much of this presentation draws on conversations with and research by my colleagues Janet Lavery and ...
Slides freely available<br />http://www.slideshare.net/malcolmmurray<br />
Recorded Delivery<br />http://www.audioh.com/projects/recorded_delivery.html <br />
Session Plan<br />Perceptions of Students<br />Why give feedback?<br />What we’ve tried<br />What staff want<br />What stu...
http://www.flickr.com/photos/rightreading/1243239706/sizes/m/in/photostream/<br />Perceptions of Students<br />
Student as Consumer<br />Ham KhanTimes Higher Educational Supplement14th December 2007<br />
Education by Numbers<br />http://www.bbc.co.uk/news/education-13874483<br />
Student as Participant<br />in anAcademicCommunityof Practice<br />http://knowledgemanagement-review.blogspot.com/2011/02/...
Implications<br />Consumers<br />Focus on the end result<br />Never look back<br />Participants<br />Learning the rules<br...
http://www.flickr.com/photos/fin5bjh/4442706561/sizes/l/in/photostream/<br />Why Feedback?<br />
Five Functions of Feedback<br />Learners can use feedback to:<br />confirm<br />add to<br />overwrite<br />tune<br />restr...
Feedback is a Dialogue<br />Feedback on assessment whether formative or summative is a personal dialogue between a lecture...
Feedback is about improvement<br />“Feedback that does not tell you how to improve is pointless” said one of the students ...
Laurillard’s Conversational Framework<br />http://www2.smumn.edu/deptpages/~instructTech/lol/laurillard/index.htm<br />
Effect of Modularization<br />59%<br />of students responded that feedback was given too late to be helpful, as they got i...
Speaking Different Languages<br />“students who do not yet share a similar understanding of academic discourse as the tuto...
Feedback by Larson<br />
What we have tried<br />
Tablet Annotation<br />
Tablets: Good Points<br /><ul><li>Novelty makes the process more fun
Don't have to fiddle with envelopes
Admin processing time is significantlyreduced
You can type comments on the work as well so you don’t have to write the same thing out again and again
Rubbing out your writing is easy
The output looks quite professional</li></ul>Dr Grant Ingram<br />School of Engineering & Computer Sciences<br />
Tablets: Down Side<br /><ul><li>Size of tablet - small movements are required and writing in the margins is tricky
Zooming needed to get good quality handwriting - makes getting an overview of the work harder
The quality of your handwriting is much poorer than with an ordinary pen
A great deal of electronic fiddling: concatenating the marking matrix, naming the PDF files correctly, converting from Wor...
The marking time is comparable to doing it with an ink pen</li></li></ul><li>Inline Annotation: Grademark<br />
Rubric Deployed<br />
Dr Steve LyonAnthropology<br />Rationale<br />“…when we started doing this  [on paper] we had students coming and saying, ...
Staff Feedback<br />Fatigue is less of an issue…  The twentieth bad essay no longer invokes lots of exclamation marks, ‘Wh...
Student Feedback<br /><ul><li>The student gets at least six comments even if the postgraduate marker [is] less confident…
Students really like it especially as I give them the rubric before the assignment so they know what they are working to.
Students like being able to get the feedback online.
They get more feedback … the essays will be riddled with these little comments that are specific to a paragraph</li></li><...
Student Voice<br />“it’s not face to face but it’s certainly one to one”<br />I think this is a much better system than pa...
Katherine Griffiths<br />Alia Moser<br />German Department<br />School of Modern Languages<br />1:1 Feedback using Blogs<b...
Staff edit the posts, marking mistakes in red and any corrections in green.
Students are invited to correct their mistakes in blue(crossing out the original mistakes, but not deleting them) and to l...
Thanks to the colour code, staff can then quickly check whether the students have accurately corrected their mistakes</li>...
Staff Rationale<br />“the previous year I was disappointed because students were doing their translations uncritically, wi...
Peer Feedback: Blogs<br />It ensures corrections are completed and<br />are easily re-marked – cuts out time, and<br />if ...
Student Feedback: Blogs<br />I prefer posting my written work on the blog to handing in handwritten work because of its po...
Understanding how Students work<br />
Summary of findings<br />
Feedback – staff<br />Find simpler means to development and deliver feedback online<br />The current mechanisms for develo...
Feedback - students<br />Praise for feedback limited to that ‘like school’ or ‘what we are used to’.<br />More unhappiness...
http://www.flickr.com/photos/techsavi/3946238357/sizes/o/in/photostream/<br />Next Steps<br />
Potential Feedback Tool<br />
Complications – Naming Policy<br />
Choice: Feedback Format<br />
Upcoming SlideShare
Loading in...5
×

Rethinking student feedback

3,167

Published on

Student feedback is a hot topic in higher education, with students demanding more of it, quicker. This session discusses a project that attempted to define the concept of feedback from both a student and faculty perspective and then develop workflows and possible extensions to Blackboard to improve the creation, delivery and learning from feedback.

Published in: Education, Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,167
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
99
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • Student feedback is a hot topic in higher education, with students demanding more of it, quicker. This session discusses a project that attempted to define the concept of feedback from both a student and faculty perspective and then develop workflows and possible extensions to Blackboard to improve the creation, delivery and learning from feedback.
  • Essentially students focused on end-point. Surface learners, note grade and move on. Tend to ignore feedback.
  • Note – the student satisfaction figures are fake
  • I admit, stereotyping a bit here
  • Butler D.l. &amp; Winne P.H. (1995) “Feedback and Self-Regulated Learning: A Theoretical Synthesis” Review of Educational Research 65 (3) pp 245-281
  • Hartley, J. &amp; Chesworth, K. (2000) Qualitative and quantitative methods in research on essaywriting: no one way, Journal of Further and Higher Education, 24(1), 15–24. – study of 102 Psychology students at KeeleCoursework is largely summatively assessed, formative squeezed out…
  • Weaver, M.R. (2006). ‘Do Students value feedback? Student perceptions of tutors’ written response’ in Assessment and Evaluation in Higher Education, 31 (3), 379-394.
  • Grant Ingram used a Wacom Bamboo tablet (other tablets are available) and Xournal note taking software – see http://xournal.sourceforge.net/Trust – detailed feedback, students believed that their work really had been scrutinised- less challenges?
  • Also easy to make/take multiple copies, e.g. for audit
  • Grant concludes: Overall I am quite enthusiastic about the approach largely because I think the students willappreciate the quality of the output and the lack of envelopes and paper to cart around.
  • Beginning the conversation early…
  • Not everyone leaves things to the last minute. No corelation between submission time and plagiarism matches. Still working on final marks!
  • Walk the user through the steps…
  • All good and well, but this encourages offline marking.
  • Not yet in production either. Available for licensing by Blackboard Mobile – contact my agent 
  • Genuine list from a third year history student. Could we develop a feedback portal? Students want to be able to interact and filter the data just like instructors do with the Grade Center…
  • Note this is just a conceptual mock-up. Predicted Grade Calculator? Set aims? Peer Comparison, Tasks – self and tutor, etc. etc.
  • Excuse the punHiggins, R., Hartley, P. &amp; Skelton, A. (2001). Getting the message across: the problem of communicating assessment feedback. Teaching in Higher Education, 6(2), 269–74.
  • The last point is the most important, and possibly the hardest to answer. Staff may be constrained by the ways they have learnt to use feedback, probably stemming from a paper-based system they themselves experienced.
  • Transcript of "Rethinking student feedback"

    1. 1. RethinkingStudent Feedback<br />13th July 2011 3:00 pm Venetian Congress Center - Titian 2206<br />Dr Malcolm MurrayDurham University, UK<br />
    2. 2. Acknowledgement<br />Much of this presentation draws on conversations with and research by my colleagues Janet Lavery and Judith Jurowska<br />janet.lavery@durham.ac.ukj.e.jurowska@durham.ac.uk<br />
    3. 3. Slides freely available<br />http://www.slideshare.net/malcolmmurray<br />
    4. 4. Recorded Delivery<br />http://www.audioh.com/projects/recorded_delivery.html <br />
    5. 5. Session Plan<br />Perceptions of Students<br />Why give feedback?<br />What we’ve tried<br />What staff want<br />What students want<br />What next?<br />
    6. 6. http://www.flickr.com/photos/rightreading/1243239706/sizes/m/in/photostream/<br />Perceptions of Students<br />
    7. 7. Student as Consumer<br />Ham KhanTimes Higher Educational Supplement14th December 2007<br />
    8. 8. Education by Numbers<br />http://www.bbc.co.uk/news/education-13874483<br />
    9. 9. Student as Participant<br />in anAcademicCommunityof Practice<br />http://knowledgemanagement-review.blogspot.com/2011/02/knowledge-management-and-community-of.html<br />
    10. 10. Implications<br />Consumers<br />Focus on the end result<br />Never look back<br />Participants<br />Learning the rules<br />Want to gain acceptance<br />
    11. 11. http://www.flickr.com/photos/fin5bjh/4442706561/sizes/l/in/photostream/<br />Why Feedback?<br />
    12. 12. Five Functions of Feedback<br />Learners can use feedback to:<br />confirm<br />add to<br />overwrite<br />tune<br />restructure<br />information in their memory<br />Butler & Winne (1995)<br />
    13. 13. Feedback is a Dialogue<br />Feedback on assessment whether formative or summative is a personal dialogue between a lecturer and a student about the student’s assessment and aspirations. <br />It is not necessarily a face-to-face discussion, but it is a dialogue. <br />Janet Lavery<br />
    14. 14. Feedback is about improvement<br />“Feedback that does not tell you how to improve is pointless” said one of the students in a focus group to the roar of approval from the other students. <br />For improvement to occur feedback has to be personal, i.e. in context of the specific assessment, identify strengths and weakness in the assessment, and provide insights into how the student can improve in time for the next assessment. <br />Janet Lavery<br />
    15. 15. Laurillard’s Conversational Framework<br />http://www2.smumn.edu/deptpages/~instructTech/lol/laurillard/index.htm<br />
    16. 16. Effect of Modularization<br />59%<br />of students responded that feedback was given too late to be helpful, as they got it after the end of the module<br />Hartley & Chesworth (2000)<br />
    17. 17. Speaking Different Languages<br />“students who do not yet share a similar understanding of academic discourse as the tutor would… …have difficulty in understanding and using the feedback”<br />Melanie Weaver (2006)<br />
    18. 18. Feedback by Larson<br />
    19. 19. What we have tried<br />
    20. 20. Tablet Annotation<br />
    21. 21. Tablets: Good Points<br /><ul><li>Novelty makes the process more fun
    22. 22. Don't have to fiddle with envelopes
    23. 23. Admin processing time is significantlyreduced
    24. 24. You can type comments on the work as well so you don’t have to write the same thing out again and again
    25. 25. Rubbing out your writing is easy
    26. 26. The output looks quite professional</li></ul>Dr Grant Ingram<br />School of Engineering & Computer Sciences<br />
    27. 27. Tablets: Down Side<br /><ul><li>Size of tablet - small movements are required and writing in the margins is tricky
    28. 28. Zooming needed to get good quality handwriting - makes getting an overview of the work harder
    29. 29. The quality of your handwriting is much poorer than with an ordinary pen
    30. 30. A great deal of electronic fiddling: concatenating the marking matrix, naming the PDF files correctly, converting from Word format for students who don't follow instructions and so on.
    31. 31. The marking time is comparable to doing it with an ink pen</li></li></ul><li>Inline Annotation: Grademark<br />
    32. 32. Rubric Deployed<br />
    33. 33. Dr Steve LyonAnthropology<br />Rationale<br />“…when we started doing this [on paper] we had students coming and saying, “I got lots of comments and so and so only got two little lines.” <br />I wanted somehow to, not impose, but encourage a more consistent amount of feedback for everyone and ensure similar things were being flagged up.”<br />
    34. 34. Staff Feedback<br />Fatigue is less of an issue… The twentieth bad essay no longer invokes lots of exclamation marks, ‘What is this!’<br />The ‘marking rubric’… helps automate the process of allocating marks against set criteria. This was particularly helpful to the teaching assistants.<br />Lyon, Steve. “Making the grade: Helping postgraduate teaching assistants with their marking and feedback..” QED (Durham University)2008.<br />
    35. 35. Student Feedback<br /><ul><li>The student gets at least six comments even if the postgraduate marker [is] less confident…
    36. 36. Students really like it especially as I give them the rubric before the assignment so they know what they are working to.
    37. 37. Students like being able to get the feedback online.
    38. 38. They get more feedback … the essays will be riddled with these little comments that are specific to a paragraph</li></li></ul><li>Audio Feedback<br />Prof Ian GreenerSchool of Applied Social Sciences<br />Quicker than typing<br />More effective<br />Students pay attention<br />Tone of voice / inflection<br />Personal - Allows you to talk direct to student work<br />Upload process is complicated<br />
    39. 39. Student Voice<br />“it’s not face to face but it’s certainly one to one”<br />I think this is a much better system than paper feedback.  I know I got a lot more from it than if you had just had to tick boxes and given me a comment in that little box on the piece of paper. <br />http://www.flickr.com/photos/jiscimages/436585751/sizes/m/in/photostream/<br />
    40. 40. Katherine Griffiths<br />Alia Moser<br />German Department<br />School of Modern Languages<br />1:1 Feedback using Blogs<br /><ul><li>Language students given weekly homework tasks using a blog
    41. 41. Staff edit the posts, marking mistakes in red and any corrections in green.
    42. 42. Students are invited to correct their mistakes in blue(crossing out the original mistakes, but not deleting them) and to look at the rest of the corrections
    43. 43. Thanks to the colour code, staff can then quickly check whether the students have accurately corrected their mistakes</li></li></ul><li>Sample Feedback: Blogs<br />
    44. 44. Staff Rationale<br />“the previous year I was disappointed because students were doing their translations uncritically, without thinking or looking at the feedback.”<br />Alia Moser<br />“With the blog you have the work and feedback online and we can look back and say, “Look, we have mentioned adjective endings before, so why is it still going wrong?” Students can look back at areas where they did well or were weak. This helps build confidence. It was important to make them feel we had a record they could be proud of.”<br />Katherine Griffiths<br />
    45. 45. Peer Feedback: Blogs<br />It ensures corrections are completed and<br />are easily re-marked – cuts out time, and<br />if this had been hand written +<br />marked, I would probably put it straight in my<br />file without bothering with corrections!<br />
    46. 46. Student Feedback: Blogs<br />I prefer posting my written work on the blog to handing in handwritten work because of its positive effect on my learning.<br />The opportunity to be able to go back for amendments and revision is important for me.<br />I can learn from reading my peers’ work and the tutor’s feedback on their work.<br />I am happy for my peers to comment on my work.<br />80%of students agreed<br />
    47. 47. Understanding how Students work<br />
    48. 48. Summary of findings<br />
    49. 49. Feedback – staff<br />Find simpler means to development and deliver feedback online<br />The current mechanisms for developing and delivering digital feedback (text, audio, etc…) are according to many staff: “inefficient”, “require too many steps”, and generally too difficult.<br />Staff want access to sophisticated feedback generation tools that are easy to use and provide simple delivery mechanisms. <br />
    50. 50. Feedback - students<br />Praise for feedback limited to that ‘like school’ or ‘what we are used to’.<br />More unhappiness then happiness with the actual content of feedback.<br />Enthusiastic about receiving feedback developed using new technologies such as audio recordings or digital mark-up systems. <br />Students very happy with their current feedback were noticeably less enthusiastic about the possibility of new technologies – prefer handwritten comments on a copy of the essays and face-to-face discussions.<br />
    51. 51. http://www.flickr.com/photos/techsavi/3946238357/sizes/o/in/photostream/<br />Next Steps<br />
    52. 52. Potential Feedback Tool<br />
    53. 53. Complications – Naming Policy<br />
    54. 54. Choice: Feedback Format<br />
    55. 55. Simplifying Download<br />
    56. 56. Simplifying Mark Entry<br />
    57. 57. Simplifying Upload<br />
    58. 58. Mobile Marking<br />
    59. 59. My Grades<br />2/10 Blackboard<br />-Must try harder…<br />
    60. 60. Feedback Dashboard<br />
    61. 61. Simple, but Consistent<br />Look at the processes for submission & feedback<br />Push for standardization<br />in the lecturer’s experience<br />in the student’s experience<br />
    62. 62. Conclusion<br />
    63. 63. Rubric cubism (sorry)<br />feedback [is] an essentially problematic form of communication involving particular social relationships…<br />…external conditions interplay, mediate (and are mediated by) patterns of power, authority, emotion and identity<br />Higgins, Hartley & Skelton (2001)<br />http://www.forevergeek.com/2009/09/rubiks_cube_fancy_dress_costume/<br />
    64. 64. Findings so far…<br />Creative ways exist for staff & peers to provide feedback<br />Most have ugly workflows, discouraging wider adoption<br />No one-size fits all – (5 purposes of feedback)<br />Need to make things better without breaking those that already work<br />Electronic delivery can make feedback more visible, thus more likely to result in learning<br />Must provide/promote timely feedback<br />We need to learn how our students could use feedback to feed forward…<br />
    65. 65. Please provide feedback for this session by emailingBbWorldFeedback@blackboard.com. <br />The title of this session is:<br />Rethinking Student Feedback<br />
    66. 66. Get in touch:<br />malcolm.murray@durham.ac.uk<br />@learntechdurham<br />@malcolmmurray<br />http://www.dur.ac.uk/lt.team/blog/<br />Questions?<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×