Lecturer perspectives of using technology for student feedback - A presentation shared at the Harper Adams Research Forum on 15/10/15

1. Lecturer
perspectives of
using technology
for student
feedback
LYDIA ARNOLD

2. Rationale
 The feedback challenge (Achilles heal)
 Use of technology - uncritical acceptance?
 Staff voices in the discourse
 Harper Adams' own feedback dilemmas
 Contribution to the Ed Dev community

3. Feedback: Definitions and features
 Formal vs informal (Kahu, 2008)
 Formative vs summative
 Self, peer, automated
 Dialogic, cyclical vs transmission (See Nicol, 2010; Barker & Pinard, 2014)
 Emotional process (e.g. Pekrun, Cusack, Murayama, Elliot, & Thomas, 2014)
 Market forces: Furedi (2009) feedback as flattery
 Culturally routed (Chen, May, Klenowski, & Kettle, 2014)
 Feedback as an extension of pedagogic values (Ellery, 2008)
 "Information given on a performance which the aim of creating improvement"

4. Literature
 The problems with feedback (not clear, legibility, mark attribution, code and
annotation and more!)
 Varied responses (more of the same) (Robinson, Pope, & Holyoak, 2013).
 Emphasis on student views (Evans, 2013) – 7% on lecturer perspectives
 Lecturer views as an appendage
 Technology as novel
 The hope of technology
 The risk of romantisizing (Clapham, 2012)
 Research on technology by the enthusiastic few

5. More literature
 Research as benefit conflation (Shelton, 2014)
 Technology and the Halo effect (Lunt & Curran,2013)
 A hunch about the impact of technology (shared by Watkins et al,
2014)
 Existence if feedback orientations (Tang & Harrison, 2011) – so what?
 Ideas that technology choices for feedback were shaped by
professional destinations of students

6. Models of technology dissemination
 Practice landscape (Geoghegan, 1994)
 Support and policy assumes homogeneity
 Conditions favour the early adopter
 Alienation of the mainstream
 Absence of a compelling reason for adoption
Rogers, 2003 (original 1963)
Does little to explain or
understand; infers
technology is ‘good’;
derogatory language

7. Questions
 Why do tutors turn to technology for feedback?
 What are the influences on choice and use of technology?
 How does practice develop?
 What is the effect of using technology in feedback?

8. Methodology
 Critical realist approach
 Neither just objective, nor just subjective
 Stratification – three layers of reality
 Structure and agency
 Individuals with identity
 Aim to explain and devise theoretical models
 Does not seek to generalise

9. Methods
 Narrative interviews – empathy as key
 Harper Adams, Staffordshire, Keele,
Sheffield Hallam, Chester, Manchester
Met.
 Connecting analysis case by case
 Portraits (Seidman, 2013) – available
online
 Categorising (themes) – Maxqda11
 Combined – collocated themes,
strong themes, common themes
 Modelling
See lydiaarnold.wordpress.com

10. Findings

11. Findings: Why use technology?
 Triggers
 External examiner feedback
 Team choices
 Change of delivery mode
 Realisations about writing
 Change in student group (international)
 Workload increase and a search for efficiency
 Convergence
 Industry experience
 Family members
 Career stage
 Own experience
 Passion
 Market and competitor position
 Recognition of learning preferences

12. Two underlying factors to trigger
reflection on practice
 Change
 External interactions
Social practice / praxis

13. Feedback beliefs – academic
identity
1. Universal acceptance of feedback importance
2. Varied ‘power’ to influence the impact feedback (affects volume and type and tech choices)
3. Ideas vs structure (degree of negotiation varies according to power)
4. Limited confidence in power of officialised feedback influences choices
5. Professionalism (how much feedback is part if the role)
Challenges the idea that poor feedback is apathy (linked to academic identity and beliefs about
feedback impact)

14. What beliefs relate to technology
 Where it can make a difference to student use (how so? ….. Media,
access)
 Where it can make a difference to upholding professional
responsibility

15. Feedback as important
 “if we don't give feedback, we are failing as educators”
 “I think it’s probably the most important thing. Because I think that,
something I found lacking, and it’s not a criticism of having gone
through the course here, it’s a criticism of being at four unis doing
various things and feedback generally is pretty poor” (Anna).
 “In my teaching it's, I've always considered it an important part of
the teaching process, of the learning process of the students…To
some extent I certainly think most of the students learn something
from it, either what you write, or when they come and try and speak
to you about an assignment, and you can try and explain what your
feedback actually means”

16. Officialised feedback
 “I think the reason why there is so much focus on feedback is due to
the NSS [National Student Survey] and quality assurance so I think
there is this push from above and I think perhaps the student and
what the student wants is being forgotten and I sometimes think the
students don’t want the feedback”

17. Variable use
[How the students use feedback is] the million-dollar question. It's
variable. I know from speaking to students about this, some do really
value the feedback but it depends on how it's presented to them. And
it depends on lots of other things like the mark they get, whether they
like the module, whether it's an area that they want to extend in future,
and they want to do really well in that module because, that's taking
them in a direction. So I think it's incredibly variable. There are some
students who, even if they get fantastic feedback, still wouldn't take
any notice of it. They're still only interested in the mark”

18. Influences on choice of tool
 Efficiency and enhancement
 Personal media preferences
 Beliefs about student learning preferences of students
 Emotional concern
 Fit with existing practice (rubrics)
 Fit with structure or ideas emphasis

20. Fractured practice: A double fault
line
Tensions occurred in relation to:
 Use of technology and the alienation of those not involved
 Perceived poor feedback irrespective of technology being used
 Perceived poor feedback and a lack of willingness to use
technology to address specific issues
 Perceived reticence to try new technologies for the benefit of
students
 Workload imbalances associated with trying to develop and sustain
good practice

21. Academic identity – technology
 Beliefs about being a professional academic
 Prioritise other activities over investing time in learning to use
technology
 A very positive feedback orientation, he just avoided using
technology production for fear of distraction
 Pace of change in technology was seen as all consuming

23. Spreading practice
 Acculturation
 Office location, campus location
 Matching (knowing the options)
 Technology first – externality
 Course team

24. Impact on practice
 increase in feedback volume and legibility, student engagement
and consistency
 consider their assumptions and practices related to feedback
 Attribution of marks
 Information overload
 Rubrics and the top end
 Frustration

25. Key Case differences
Unit of the course team for managing feedback
Feedback in design
Quality assurance requirements
Technology as normal - “I don’t do anything else on paper. I’m not
used to actually the physical hold of the pen anymore and I never
have ever since computers existed, or even typewriters. I never have
been able to compose with a pen. I have always written on a
machine”

26. Course team
1. Programme level operation:
a) have their practice recognised as a core part of pedagogy
b) exploit the particular benefits of tools in a way that is coordinated
from both a pedagogic perspective and a workflow perspective
c) strengthen meaningful networks for the development, discussion,
sharing and refinement of practice

27. 1. Systems thinking about change (But systems which complement academic identity)
2. Forming ways to work with (accept or change) different academic identities
3. Consider the fault lines in practice (tech and feedback)
4. QA and clarity
5. External contact increases
6. STOP considering efficiencies without consideration of quality issues
7. Change as a trigger for reframing practice
8. Exposure to CPD in a way that fits with identity
9. Feedback induction and space management

28. Next
 Spaces and feedback practice
 Structure and ideas – orientations
 Critical realism as a methodology
 Academic identity and the role of feedback

29. Want to know more?
 Full report available at http://repository.liv.ac.uk/2014121/
 Portraits available at
https://lydiaarnold.wordpress.com/2014/10/10/using-technology-
for-student-feedback-lecturer-perspectives-in-their-words/
 Internal related case study (with Carl Kennard) coming soon in The
International Journal of Assessment and Evaluation