This document discusses technology-enhanced feedback approaches and what works. It begins by outlining the importance of feedback and challenges to providing effective feedback, including disconnects between staff and student perspectives, resource constraints, large class sizes, and modularization. Contemporary perspectives view feedback as an ongoing dialogue rather than a delivered message, and emphasize feedforward and self-regulated learning. Recommended approaches include online written feedback using tools like annotated PDFs and rubrics, in-class feedback using clickers or backchannels, audio-visual feedback like screencasts, and peer feedback technologies. Programmatic approaches aim to coordinate feedback across a program. Technology can help increase feedback quantity and quality, support dialogue, and provide flexibility.

1. Technology Enhanced Feedback
Approaches: What Works and Why?
Lisa O’Regan and Morag Munro
March 2016

2. Workshop Outline
• Why feedback matters
• Challenges to feedback provision
• Contemporary perspectives on feedback
• Technology-enabled approaches that work!
• Y1Feedback Project

3. Why is Feedback Important?
“Feedback is one of the most powerful influences on
learning and achievement” (Hattie and Timperley 2007:
81).
Timely and effective feedback can:
• correct errors, and close the gap between current and
desired performance;
• identify strengths and weaknesses;
• build student confidence and motivation; and
• foster self-regulated learning.
(Hounsell 2003, Hattie and Timperley 2007, Sadler 2010,
Carless et al. 2011, Merry et al. 2013)

4. Challenges to Effective Feedback
http://www.tricider.com/brainstorming/3OYMW
FoR2m7
What are the key challenges
that you, and/or your students
experience in relation to
feedback?

5. Challenges: Disconnect Between Staff and
Student Perspectives
UK, Australia and Asia: Carless 2006; James et al. 2010; Price et al. 2010; Radloff & Coates 2010;
HEFCE 2014, 2015; Jessop et al. 2014; Mulliner & Tucker 2015.
Ireland: Irish Survey of Student Engagement (ISSE) 2013, 2014; Y1Feedback 2016a.
Students say:
• We don’t get enough feedback
• We don’t receive feedback on
time
• We don’t understand our
feedback
• Feedback practices are
inconsistent
Staff say:
• Students are being provided
with timely feedback
• Students often don’t engage
with, or act on, feedback
• Students are only interested
in their grades
• Students don’t recognise
when feedback is being
provided
• Students don’t even collect
their feedback

6. Challenges: Resource Constraints
• Between 2008 and 2014:
– core expenditure per student reduced by 15%
– staff numbers reduced by 2000
– The HE system accommodated an extra 25,000 students
• In comparison with the OECD norm of 1:16, the staff-student
ratio in Irish HEIs increased from 1:15 in 2007 to 1:19.5 in
2014, and is projected to rise to 1:20 in 2016
Source: HEA 2014. Higher Education system performance. First report 2014 -2016. Report of
the Higher Education authority to the Minister for Education and Skills. Dublin: Higher
Education Authority (HEA).

7. Challenges: Large Classes
• Additional capacity accommodated via larger teaching workloads and
growing class sizes
• Can detrimentally impact on:
– Frequency;
– Timeliness; and
– Quality of feedback
(Carless, Joughin and Liu 2006, Ackerman and Gross 2010, Nicol 2010).

8. Challenges: Modularisation
• Can lead to ‘bunching’ of assessment deadlines towards
end of semesters (Carless, Joughin and Liu 2006, Irons
2007, Deepwell and Benfield 2013).
Image source: Map my programme

9. Challenges: Modularisation
Impacts of assessment bunching:
• Discourages students from spreading their
learning efforts throughout the semester: can
lead to student overload
• Students do not receive feedback until after the
module is over.
• Students may be un motivated to, or may find it
difficult to apply feedback received in one
module to subsequent pieces of work
(Carless, Joughin and Liu 2006, Irons 2007,
Deepwell and Benfield 2013)

10. Challenges: Multiple Purposes of
Assessment

11. ADDRESSING THE CHALLENGES:
CONTEMPORARY PERSPECTIVES ON
FEEDBACK

12. Contemporary perspectives
Feedback should:
1. Take place in formal and informal learning
settings beyond assessment;
2. Feed forward to future work; and
3. Be a dialogic process that ultimately supports
learners to become self-regulating.
Y1Feedback 2016b. Technology Enabled Feedback in First
Year: A Synthesis of the Literature. Available from:
http://y1feedback.ie/

13. Beyond assessment feedback
• Feedback tends to be associated with assessment
(Boud and Molloy 2013, Hounsell 2015, O’Donovan,
Rust and Price 2015)
• Assessment often the main locus for much of the
feedback in HE
• Feedback potentially the key element in enabling
learning from an assessment task (Sadler 2013,
O’Donovan, Rust and Price 2015)
• Contemporary thinking underpinned by a broader
conception of feedback as: “All dialogue to support
learning in both formal and informal situations”
(Askew and Lodge 2000: 1)

14. Feedback and Feedforward
• Feedback: focused on current performance and is
typically employed in order to justify the grade
awarded.
• Feedforward: looks beyond the context of the
current piece of work towards future
assessments, modules, and courses
• Good feedback acts as both feedback and
feedforward

15. From feedback as delivered message
towards dialogue and engagement
• Transmission model of feedback:
– “One-way flow of information from a knowledgeable
person to a less knowledgeable person” (Boud and
Molloy 2013: 7).
– Written commentary on end of module, and in most
cases graded, assignments (Nicol 2010, Carless 2013).
FEEDBACK

16. From feedback as delivered message
towards dialogue and engagement
• Contemporary thinking reframes feedback
as dialog (Nicol 2010, Beaumont,
O’Doherty and Shannon 2011, Carless et
al. 2011, Boud and Molloy 2013, Merry et
al. 2013, Carless 2015).
• Why?
– Students do not learn by passively absorbing
information conveyed to them.
– One way transmission of feedback does not
automatically lead to learning or to changes in
behaviour.
– “Information only becomes feedback when it is
used productively” (Carless 2015: 192)
Image source: http://www.cliparthut.com/people-talking-clip-art-clipart-npcgGT.html

17. Towards sustainable feedback and self
regulated learning
• Teacher no longer the only source of feedback
• Additional sources are a students’ peers, and his or her self
• Internal dialog with the self is fundamental to sustainable
feedback: students should ultimately become capable of
critically evaluating and monitoring their work
independently of the teacher, that is, they should become
self-regulating (Hounsell 2007, Sadler 2010, Carless 2013,
Carless 2015, Ajjawi and Boud 2015).
• “Our work as educators is sustainable when students have
learnt with us, and are able to continue learning without
us” Carless (2013: 113)

18. In class feedback quiz!
• Go to quiz.al
• Select “I am a student”
• Enter class code: xkp454
• Add your name to the list
• Play the quiz!

19. TECHNOLOGY ENABLED FEEDBACK
APPROACHES

20. Recommended Feedback Approaches
• In-class dialog and informal feedback
• Feed forward strategies
– Flipping feedback: feedback provided in-task, rather than at the end.
– Multi-stage assignments: an assignment comprises two or more related stages
interweaved with feedforward comments.
– Linked assignments: two or more assignments are designed so that each piece of
work builds on the next.
• Rubrics and marking guides
• Exemplars
• Peer feedback
• Generic feedback
• Distancing feedback and grades
• Programmatic

21. Potential of Technology for Feedback
• Can help to support provision of a greater volume
of timely feedback;
• Can lead to improved student understanding of,
and engagement with, feedback;
• Can support a greater variety in feedback formats
and approaches;
• Can help to generate opportunities for dialogic
feedback; and
• Can offer greater flexibility and accessibility in
relation to feedback access and use.

22. Technology Enabled Feedback Approaches
• Online Written
• In-class
• Audio Visual
• Peer feedback technologies
• Automated

23. Why Online Written Feedback Works?
Increased quality and quantity of feedback
• Develop comments in greater detail and depth with editing software
• Typed comments may perceived to be more considered and
thoughtful than handwritten feedback (Parkin et al. 2012).
Time Saving
• In-text annotations and comments may allow for the provision of more
feedback in less time (Buckley and Cowap 2013, van der Hulst, van
Boxel and Meeder 2014)
• Rubrics structure can provide time-savings (Atkinson and Lim 2013)
Accessibility and flexibility
• More frequently accessed (Ferrill 2014)
• Can increase timeliness (Parkin et al. 2015, Carless 2015)
• Permanence – More likely to revisit (Ferrill 2014, Parkin et al. 2012)
• Privacy (Ferrill 2014, Parkin et al. 2012, Sopina and McNeill 2014)

24. Online Written Feedback Approaches
• Digital comments
• Annotated PDF
• Comment banks
• Rubrics and marking guides
• Separating grades and feedback
• E-portfolios
• Others?

25. Online Written Feedback in Moodle
• Pic of Annotated Doc

26. Moodle Feedback Files

27. Online Written Feedback Example
Sopina & McNeill 2015
• First Year
• Population Health
• 335 students
• Examined the relationship
between the way feedback
is delivered and quality of
feedback
• Showed that electronic
marking can result in more
timely feedback for
students without impacting
on quality
• Student preference for the
electronic submission and
return
• Students appeared to be
more satisfied with the
annotated format of the
feedback, which “solved the
issue of illegible
handwriting” (p. 675).
• Marking electronically took
on average 3 minutes less
than hard copy

28. Moodle Marking Guides

29. Y1Feedback Example:
Moodle Marking Guide
• Conor McKevitt DKIT
• Using Rubrics and the Moodle Gradebook to
promote engagement with formative feedback
in Social Care within a Multi-Stage Assignment
• Moodle Assignment
• Moodle Marking Guide

30. Turnitin Rubrics

31. In-class Feedback Approaches
• In-class dialog
• In-class back channel
• ‘Muddiest Point’
• ‘One minute paper’
• Multiple choice questions
– Individual & Group
• Think-Pair-Share/Peer-
Instruction
• Suggestions – What have
you used?
Why?
• Identify and address
problems that could
adversely impact on
learning if left unresolved
(Goldstein 2007,
McArthur et al. 2011,
Tang 2013, Wang et al.
2013).

32. In-Class Feedback Technologies
• Clickers (eg Turning Technologies)
• Twitter: In-class back channel
• Tricider (www.tricider.com)
• Padlet (padlet.com)
• Zeetings (www.zeetings.com)
• Socrative (socrative.com) (50 students max)
• Google Docs: Real-time feedback
• Other suggestions?

33. Y1Feedback Example: In-Class Feedback Using Padlet
Dr Pauline Cullen, Dept of Sociology Maynooth University

34. Y1Feedback Example:
Real-time In-Class Feedback Response App
• McLoone et al 2015
• Electronic Engineering
• Student sketch App
• A lecturer ‘view-and-edit’ App
• A cloud-based service for co-
ordinating information
between these applications.
• Allows students responses to
include “higher quality and
more relevant information and
thus, improves their active
learning” (p. 2071).

35. Why Audio-Visual Works?
Audio
• Personal nature of audio
feedback, particularly where
names used (Ice et al. 2007;
Gould & Day 2013; Knauf 2015)
• More content in less time The
average number of words in the
audio feedback was 12 times
higher than in the written
feedback (Voelkel and Mello
2014)
• Richer Feedback than with
written feedback Chalmers &
MacCallum (2014)
Screencast
• More personal, Easy to Follow,
Richer Feedback
• “The students liked the fact that
they could see the instructor
actually making corrections and
hear the instructor’s thought
process through the audio
commentary”. (Ghosn-Chelala &
Al-Chibani 2013, p.271)
• Screencast feedback equal to or
better than the written feedback
in quality and quantity. (Haxton
&McGarvey 2011)

36. Audio Visual Technologies
Audio
• Smartphone
• Audacity
– www.audacityteam.org
• Garageband (Mac)
• VoiceThread
– https://voicethread.com/
• Digital Voice Recorder
Screencast
• Screencast-O-matic
– https://screencast-o-matic.com
• Camtasia
– www.techsmith.com
• Adobe Voice iOS app
• iMovie (Mac)
• Screenflow (Mac)
– https://www.telestream.net/screenflow/

37. Y1Feedback Examples: Screencasts
Multi-Stage Assignment Feedback
David Cranny, Dundalk Institute of
Technology
Summative Feedback on Exams
Mary Corcoran Department of Sociology,
Maynooth University.

38. Why Online Peer Feedback?
• Students exposed to greater quantity and variety of timely
feedback (Carless, Joughin and Liu 2006, Nicol, Thomson
and Breslin 2014).
• Peer feednack may be more accessible and understandable
than that provided by teachers (Falchikov 2004).
• Closes the between receipt of feedback and application
(Nicol, Thomson and Breslin 2014).
• Constructing and receiving peer feedback can support and
improve learning (Falchikov 2001, Falchikov 2004, Cho and
MacArthur 2011, Nicol, Thomson and Breslin 2014).
• Online tools make peer feedback feasible with large
numbers

39. Some Peer Feedback Approaches
Feedback as constructive dialogue
• In-class peer feedback on in-class activity
• Peer feedback on drafts using rubrics or marking
guides
• Peer feedback as part of a multi-stage assignment

40. Peer Feedback Technologies
• Moodle Workshop
• Turnitin PeerMark
• WebPA (webpa.ac.uk)
• Teammates (teammatesv4.appspot.com)
• Peerwise (https://peerwise.cs.auckland.ac.nz)
• Sparkplus (Peer Feedback in Groups)
• Wikis eg PbWorks
• Blogs eg Wordpress, Moodle OU Blog

41. Example: Peer Feedback Via Turnitin
PeerMark
• Engineering design class (Nicol
et al.2014)
• Two reviews and one self-
review
• Allocation and student review
process facilitated through
PeerMark
• Not asked to mark their peers
(unusual)
• Teacher provided 4 questions
to scaffold student peer-
feedback review
• Positive experience of
students compared with peer
assessment

42. Automated Feedback – Why?
• Provide students with immediate, targeted feedback
– can support self-regulatory learning
• Flexibility
• Direct students to sources of action
• Provide you with feedback on student’s
understandings and misunderstandings

43. Automated Feedback Technologies
Quizzes
• Moodle Quiz
• External Quiz (McGrawHill Connect, Quizalize …)
Writing
• OpenEssayIT: http://www.open.ac.uk/researchprojects/safesea
• WriteLab: https://www.writelab.com
Computer Programming
• Virtual Programming Lab: https://moodle.org/plugins/mod_vpl
• Web-Cat: http://web-cat.org
Others?

44. Virtual Programming Lab

45. Programmatic Approaches to Feedback
• Complement programmatic assessment strategies
• Promote feedforward between assessment tasks
• Potentially mitigate effects of modularisation (Boud & Molloy
2013, Carless 2015)
• Need for programme teams to work together to develop a
shared culture in relation to issues such as consistency and
timing of feedback. (Gibbs 2015, Jessop et al. 2014)
– programme team discussions around feedback criteria; marking
workshops; mentoring structures for new staff; and by promoting
greater visibility of feedback (Jessop, El Hakim and Gibbs 2014).
• Note … Scant evidence of implementation

46. FASTECH – JISC Project
FASTECH’s first principle is to work with
programme team members, using baseline
evidence, to develop technology strategies to
address assessment and feedback challenges.

47. May My Programme Tool - University of Greenwich
Project Website: https://sites.google.com/site/mapmyprogramme/home
• Assignment hand-in date
analysis;
• Assignment diet analysis.
• Real-time visualisation of
programme assessment;
• Estimate of marking loads
and effort;
• Links between formative
and summative
assignments & feedback.

48. Break-Out Padlet Brainstorm
1. In practice, what would the first steps to
implementing programmatic feedback be?
2. List at least one way technology could help?
– Groups of 4
– Brainstorm in Padlet – 10 Minutes
• Group 1: http://padlet.com/lisa_oregan/wwawdcu1
• Group 2: http://padlet.com/lisa_oregan/wwawdcu2
• Group 3: http://padlet.com/lisa_oregan/wwawdcu3
• Group 4: http://padlet.com/lisa_oregan/wwawdcu4
– Sharing approaches with groups

49. Supporting Transition:
Enhancing Feedback in First Year Using Digital Technologies

50. Our
Project
Partners

51. Y1Feedback
• Two year project
• Lead by Maynooth University - Lisa O’Regan
• Supporting Transition: Enhancing Feedback in First
Year Using Digital Technologies
• Funded by the National Forum for the Enhancement
for Teaching & Learning
• Focus on First Year Feedback to Support Transition
• Case Study Phase 2016 (WIP)
• National Symposium on Feedback in First Year – Jan
2017

52. Two Publications in Press
Will be available at www.y1feedback.ie

53. Y1Feedback DCU Contacts
• Mark Brown NIDL
(Institutional Lead)
• Orna Farrell NIDL

54. Y1Feedback DKIT Contacts
• Moira Maguire
(Institutional Lead)
• Gerry Gallagher
• David Cranny
• Conor McKevitt

55. Y1Feedback AIT Contacts
• Nuala Harding
(Institutional Lead)
• Seamus Ryan
• Geraldine McDermott

56. Thank you
@Y1Feedback