Student feedback is a hot topic in higher education, with students demanding more of it, quicker. This session discusses a project that attempted to define the concept of feedback from both a student and faculty perspective and then develop workflows and possible extensions to Blackboard to improve the creation, delivery and learning from feedback.
1. RethinkingStudent Feedback 13th July 2011 3:00 pm Venetian Congress Center - Titian 2206 Dr Malcolm MurrayDurham University, UK
2. Acknowledgement Much of this presentation draws on conversations with and research by my colleagues Janet Lavery and Judith Jurowska janet.lavery@durham.ac.ukj.e.jurowska@durham.ac.uk
9. Student as Participant in anAcademicCommunityof Practice http://knowledgemanagement-review.blogspot.com/2011/02/knowledge-management-and-community-of.html
10. Implications Consumers Focus on the end result Never look back Participants Learning the rules Want to gain acceptance
12. Five Functions of Feedback Learners can use feedback to: confirm add to overwrite tune restructure information in their memory Butler & Winne (1995)
13. Feedback is a Dialogue Feedback on assessment whether formative or summative is a personal dialogue between a lecturer and a student about the student’s assessment and aspirations. It is not necessarily a face-to-face discussion, but it is a dialogue. Janet Lavery
14. Feedback is about improvement “Feedback that does not tell you how to improve is pointless” said one of the students in a focus group to the roar of approval from the other students. For improvement to occur feedback has to be personal, i.e. in context of the specific assessment, identify strengths and weakness in the assessment, and provide insights into how the student can improve in time for the next assessment. Janet Lavery
16. Effect of Modularization 59% of students responded that feedback was given too late to be helpful, as they got it after the end of the module Hartley & Chesworth (2000)
17. Speaking Different Languages “students who do not yet share a similar understanding of academic discourse as the tutor would… …have difficulty in understanding and using the feedback” Melanie Weaver (2006)
26. The output looks quite professionalDr Grant Ingram School of Engineering & Computer Sciences
27.
28. Zooming needed to get good quality handwriting - makes getting an overview of the work harder
29. The quality of your handwriting is much poorer than with an ordinary pen
30. A great deal of electronic fiddling: concatenating the marking matrix, naming the PDF files correctly, converting from Word format for students who don't follow instructions and so on.
33. Dr Steve LyonAnthropology Rationale “…when we started doing this [on paper] we had students coming and saying, “I got lots of comments and so and so only got two little lines.” I wanted somehow to, not impose, but encourage a more consistent amount of feedback for everyone and ensure similar things were being flagged up.”
34. Staff Feedback Fatigue is less of an issue… The twentieth bad essay no longer invokes lots of exclamation marks, ‘What is this!’ The ‘marking rubric’… helps automate the process of allocating marks against set criteria. This was particularly helpful to the teaching assistants. Lyon, Steve. “Making the grade: Helping postgraduate teaching assistants with their marking and feedback..” QED (Durham University)2008.
35.
36. Students really like it especially as I give them the rubric before the assignment so they know what they are working to.
39. Student Voice “it’s not face to face but it’s certainly one to one” I think this is a much better system than paper feedback. I know I got a lot more from it than if you had just had to tick boxes and given me a comment in that little box on the piece of paper. http://www.flickr.com/photos/jiscimages/436585751/sizes/m/in/photostream/
40.
41. Staff edit the posts, marking mistakes in red and any corrections in green.
42. Students are invited to correct their mistakes in blue(crossing out the original mistakes, but not deleting them) and to look at the rest of the corrections
43.
44. Staff Rationale “the previous year I was disappointed because students were doing their translations uncritically, without thinking or looking at the feedback.” Alia Moser “With the blog you have the work and feedback online and we can look back and say, “Look, we have mentioned adjective endings before, so why is it still going wrong?” Students can look back at areas where they did well or were weak. This helps build confidence. It was important to make them feel we had a record they could be proud of.” Katherine Griffiths
45. Peer Feedback: Blogs It ensures corrections are completed and are easily re-marked – cuts out time, and if this had been hand written + marked, I would probably put it straight in my file without bothering with corrections!
46. Student Feedback: Blogs I prefer posting my written work on the blog to handing in handwritten work because of its positive effect on my learning. The opportunity to be able to go back for amendments and revision is important for me. I can learn from reading my peers’ work and the tutor’s feedback on their work. I am happy for my peers to comment on my work. 80%of students agreed
49. Feedback – staff Find simpler means to development and deliver feedback online The current mechanisms for developing and delivering digital feedback (text, audio, etc…) are according to many staff: “inefficient”, “require too many steps”, and generally too difficult. Staff want access to sophisticated feedback generation tools that are easy to use and provide simple delivery mechanisms.
50. Feedback - students Praise for feedback limited to that ‘like school’ or ‘what we are used to’. More unhappiness then happiness with the actual content of feedback. Enthusiastic about receiving feedback developed using new technologies such as audio recordings or digital mark-up systems. Students very happy with their current feedback were noticeably less enthusiastic about the possibility of new technologies – prefer handwritten comments on a copy of the essays and face-to-face discussions.
61. Simple, but Consistent Look at the processes for submission & feedback Push for standardization in the lecturer’s experience in the student’s experience
63. Rubric cubism (sorry) feedback [is] an essentially problematic form of communication involving particular social relationships… …external conditions interplay, mediate (and are mediated by) patterns of power, authority, emotion and identity Higgins, Hartley & Skelton (2001) http://www.forevergeek.com/2009/09/rubiks_cube_fancy_dress_costume/
64. Findings so far… Creative ways exist for staff & peers to provide feedback Most have ugly workflows, discouraging wider adoption No one-size fits all – (5 purposes of feedback) Need to make things better without breaking those that already work Electronic delivery can make feedback more visible, thus more likely to result in learning Must provide/promote timely feedback We need to learn how our students could use feedback to feed forward…
65. Please provide feedback for this session by emailingBbWorldFeedback@blackboard.com. The title of this session is: Rethinking Student Feedback
66. Get in touch: malcolm.murray@durham.ac.uk @learntechdurham @malcolmmurray http://www.dur.ac.uk/lt.team/blog/ Questions?
Editor's Notes
Student feedback is a hot topic in higher education, with students demanding more of it, quicker. This session discusses a project that attempted to define the concept of feedback from both a student and faculty perspective and then develop workflows and possible extensions to Blackboard to improve the creation, delivery and learning from feedback.
Essentially students focused on end-point. Surface learners, note grade and move on. Tend to ignore feedback.
Note – the student satisfaction figures are fake
I admit, stereotyping a bit here
Butler D.l. & Winne P.H. (1995) “Feedback and Self-Regulated Learning: A Theoretical Synthesis” Review of Educational Research 65 (3) pp 245-281
Hartley, J. & Chesworth, K. (2000) Qualitative and quantitative methods in research on essaywriting: no one way, Journal of Further and Higher Education, 24(1), 15–24. – study of 102 Psychology students at KeeleCoursework is largely summatively assessed, formative squeezed out…
Weaver, M.R. (2006). ‘Do Students value feedback? Student perceptions of tutors’ written response’ in Assessment and Evaluation in Higher Education, 31 (3), 379-394.
Grant Ingram used a Wacom Bamboo tablet (other tablets are available) and Xournal note taking software – see http://xournal.sourceforge.net/Trust – detailed feedback, students believed that their work really had been scrutinised- less challenges?
Also easy to make/take multiple copies, e.g. for audit
Grant concludes: Overall I am quite enthusiastic about the approach largely because I think the students willappreciate the quality of the output and the lack of envelopes and paper to cart around.
Beginning the conversation early…
Not everyone leaves things to the last minute. No corelation between submission time and plagiarism matches. Still working on final marks!
Walk the user through the steps…
All good and well, but this encourages offline marking.
Not yet in production either. Available for licensing by Blackboard Mobile – contact my agent
Genuine list from a third year history student. Could we develop a feedback portal? Students want to be able to interact and filter the data just like instructors do with the Grade Center…
Note this is just a conceptual mock-up. Predicted Grade Calculator? Set aims? Peer Comparison, Tasks – self and tutor, etc. etc.
Excuse the punHiggins, R., Hartley, P. & Skelton, A. (2001). Getting the message across: the problem of communicating assessment feedback. Teaching in Higher Education, 6(2), 269–74.
The last point is the most important, and possibly the hardest to answer. Staff may be constrained by the ways they have learnt to use feedback, probably stemming from a paper-based system they themselves experienced.