Presentation at #3riv18 2018 Three Rivers Conference delivered by Candace Nolan-Grant and Malcolm Murray. Discusses findings from four projects carried out looking at the staff and student experience of assessment.

1. Learning from student
perspectives on digital
assessment
Candace Nolan-Grant
Dr Malcolm Murray
Photo by Gaelle Marcel on Unsplash
@crnolangrant
@malcolmmurray

2. Introduction
Photo by Christian Joudrey on Unsplash

3. Tools for digital assessment at Durham
Photo by Annie Spratt on Unsplash
blogs
wikis
journals
discussion boards
assignments

4. Almost every departmentVery limited use Heavy use in a few departmentsA few departments
Use of tools for digital assessment at Durham
Photo by Annie Spratt on Unsplash

5. Enquiry into Assessment: 2017-18
Photo by Tim Gouw on Unsplash

6. 1. Process Mapping
Photo by Štefan Štefančík on Unsplash

7. 2. Focus Groups

8. 3. JISC Student Tracker

9. 4. Design thinking

10. Findings: submission methods
Photo by Jerry Kiesewetter on Unsplash

11. Digital or analogue?
Survey of Departmental submission methods: staff
33%
11%
66%
Online only
Physical only
Both
Photo by Florian Klauer on Unsplash

12. Variety of Assessment
Different online summative assessment types: students
Photo by Sharon McCutcheon on Unsplash
0 1 2 3 4
15%
39%
28%
8% 2%
Number of different assessment types used on their courses
118,574
Turnitin submissions in
2017-18

13. Variety of Assessment
Different online summative assessment types: students
0 1 2 3 4
9%
45%
33% 9%
3%
Number of different assessment types used on their courses
24%
41%
24%
10%
1%
First Year
Undergraduates
Mid Year
Undergraduates
Taught
Postgraduates
38% 38% 11%11% 3%
Photo by Sharon McCutcheon on Unsplash

14. Student perceptions
Online assessment are delivered & managed well
53%38%
9%
Neutral
Disagree
Agree
Photo by Lidya Nada on Unsplash

15. Student perceptions
Online assessment c.f. Digital Teaching & Learning
28%
15%
7% 2%
48%
45%
42%
15%
23%
40%
51%
83%
2. Overall, how would you rate the quality of digital
teaching & learning on your course?
1. Online assessments are delivered
& managed well
Agree
Neutral
Disagree
Poor/Awful/
Worst Imaginable
Average Good
Excellent/
Best Imaginable
Photo by Lidya Nada on Unsplash

16. What the students said:
paper vs online
2 students preferred paper assessment processes
Photo by STIL on Unsplash
I partly submit my work online but it is a wasted task
when you must print 2 copies off, staple them
individually, print a cover sheet, and finally the digital
receipt before paper clipping the whole thing together.
And we rely on the office open hours.
Second-year undergraduate in Arts & Humanities faculty

17. 2 students preferred paper assessment processes
29 students said that they preferred online assessment
and/or feedback
10 students didn’t want to submit both online and on paper
What the students said:
paper vs online
Photo by STIL on Unsplash
It would both make it easier for students, and for the
environment, to stop paper submissions altogether.
Second-year undergraduate in the Social Sciences and Health faculty

18. What students said: online submission
students complained about
technical issues with online
submission9
students cited issues with online
submission communications,
processes or settings13
12students said that online
submission was useful to their
learning
Photo by Daniel Cheung on Unsplash

19. Findings: types of assessment
Photo by John-Mark Smith on Unsplash

20. Online assessment
Types of assessment used: staff
78%
83%
11%
6%
17%
Word
PDF
Audio
Video
Computer programme
Photo by Patrick Tomasso on Unsplash

21. 80%
24%
10% 5% 6%
17%
11%
6%
3%
3%
3% 2%
3%
2%
Essay Presentation Portfolio Audio
recording
Video Quiz or game Other
Online assessment
Experience of current assessments: students
Have you submitted the following assessments online – and what did you think?
Yes – but I didn’t like it
Yes – and I liked it
Photo by Patrick Tomasso on Unsplash

22. 4%
20% 18% 13% 15% 14% 10%2%
16%
8% 16% 16% 10%
7%8%
37% 60%
63% 62%
56% 70%
Essay Presentation Portfolio Audio
recording
Video Quiz or game Other
Online assessment
Experience of current assessments: students
Have you submitted the following assessments online – and what did you think?
No – not appropriate to my subject
No – and wouldn’t like to
No – but would like to
Photo by Patrick Tomasso on Unsplash

23. Other assessment types
programming
report
laboratory
code
coursework
online
pdf
poster
problem
questions
science
tests
benchtest
computer
discussion
document
email
essay
exam
file
homework
lecture
matlab
python
Photo by Patrick Tomasso on Unsplash

24. Other assessment types
Photo by Patrick Tomasso on Unsplash
Was very useful for building an
understanding of python
2nd-year undergraduate in Science faculty

25. Online marks & feedback
Photo by arvin febry on Unsplash

26. Staff marking & feedback processes
33%
20%
40%
7%
27%
Offline marks list
Offline pro forma
Turnitin
Paper marks list
Paper pro forma
Photo derived from Anton Sukhinov on Unsplash

27. Photo by Tom Roberts on Unsplash
Staff would like to be able to identify
anonymous assignment non-submitters

28. Photo by MustangJoe on Pixabay
Staff would like marks from Blackboard to
feed into the student record system

29. How do students access marks and feedback?
44%
19%
19%
38%
6%
Paper
Email
Blackboard
Turnitin
In-house portal
Photo derived from Anton Sukhinov on Unsplash

30. Receipt of marks and feedback
Photo derived from Anton Sukhinov on Unsplash
scripts given feedback using
Turnitin in 2017-1832,103
students complained about
processes to access feedback5
students complained about
the quality of online feedback7

31. Student feedback on feedback
Photo derived from Anton Sukhinov on Unsplash
Feedback depends on Tutors, some are good
at giving [it] on-line and some are not
Master’s student in Arts & Humanities faculty
Rubrics or sample answers along with
marking would be useful.
2nd-year undergraduate in Science faculty

32. Student feedback on feedback
Photo derived from Anton Sukhinov on Unsplash
Online grading and feedback would be
useful for all modules
1st-year undergraduate in Arts & Humanities faculty
It would be helpful if…feedback on
assessments are given via voice recording for
better clarity
2nd-year undergraduate in Social Sciences & Health faculty

33. Tackling the challenges
Photo by Chinh Le Duc on Unsplash

34. Challenge 1
Making the process transparent
Photo by freddie marriage on Unsplash

35. Consistency for students
Where to find rules of submission
deadlines, late submissions, extensions, proof of
submission, originality
Where to find the assignment details
e.g. essay titles, what is expected for non-standard
submissions
Where to find instructions for submission
e.g. file format, file length/size, whom to contact if there’s
a problem
Where to submit
When marks and feedback will be released
Where to find marks and feedback
PhotobyfreddiemarriageonUnsplash

36. It can be very difficult to find where to submit
assignments for the first time. More guidance with this
would have been appreciated as it took me a long time at
first.
Master’s student in Science faculty
Photo by freddie marriage on Unsplash

37. How to create submission points
How to deal with anonymity
How to deal with student issues
e.g. non-submission, late submission,
extensions, claims of system failure
Consistency for administrative staff
PhotobyfreddiemarriageonUnsplash

38. My best friend had a massive argument with a staff
member who shouted at her for not knowing how to
access the assignment portal online. There is sometimes
not great help from admin staff. Its hit and miss.
Master’s student in Social Sciences & Health faculty
Photo by freddie marriage on Unsplash

39. How to help students
e.g. make it clear where to find information,
guidance, submit points, assessment policies
Methods of marking
e.g. how anonymous marking, double marking or
moderation will work, if they should use a rubric or
grading form, whether to annotate the script
What the students see
e.g. if the VLE automatically calculates a mark—
correct or otherwise, what a marked script looks
like online, whether students can see the
originality report
Consistency for markers
PhotobyfreddiemarriageonUnsplash

40. Essay titles and coversheets should be on the module's
DUO page, not random folders.
Master’s student in Social Sciences & Health faculty
Photo by freddie marriage on Unsplash

41. Challenge 2
Engaging students with
innovative assessments
Photo by rawpixel on Unsplash

42. Making a video to consolidate
learning [was a useful course digital
activity]
Master’s student in Arts & Humanities faculty
Photo by rawpixel on Unsplash

43. The creation of games or websites
[was a useful course digital activity]
1st-year undergraduate in Science faculty
Photo by rawpixel on Unsplash

44. Blog writing on DUO [was a useful course
digital activity]
Master’s student in Arts & Humanities faculty
Photo by rawpixel on Unsplash

45. Give students access to plenty of
guidance and support
When we are expected to use digital devices to
produce work, don’t assume we are comfortable
with the teachnology [sic]
2nd-year undergraduate in Science faculty
Photo by rawpixel on Unsplash

46. Give students access to plenty of
guidance and support
Photo by rawpixel on Unsplash
[DON’T] Make digital teaching condescending
or patronising.
2nd-year undergraduate in Science faculty

47. Research the best way for students
to submit
Because of the University guidelines on the
marking of coursework, submitted coursework
doesn't get processed through the duo.
1st-year undergraduate in Science faculty
Photo by rawpixel on Unsplash
No support to send large videos
Master’s student in Social Sciences & Health faculty

48. Consider digital group
projects?
52% of students preferred a mix
of individual & group work
Working together with the group on Google Docs
online is one of the best experiences.
Master’s student in Social Sciences & Health faculty
Photo by rawpixel on Unsplash

49. Take the pressure off students who have less
experience with the technologies being used.
Use digital tools to make it easy to see how
group work evolves throughout the course of
a project (Crisp, 2007).
Use digital tools for self and peer evaluation
at the end (Kao, 2013).
Digital group projects
Photo by rawpixel on Unsplash

50. Challenge 3
Providing
quality online
feedback
Photo by chuttersnap on Unsplash

51. Get experience with the tools,
including rubrics and recording
Photo by chuttersnap on Unsplash
Only some courses allow you to get your
feedback online (mine doesn't) and this is
frustrating particularly if you get feedback
during the holidays and you can't see your
feedback until you come back to uni
Master’s student in Science faculty

52. Consider ongoing assessment
students named in-class polling as a
useful course activity73
students named online quizzes and
tests as useful course activities32
Photo by chuttersnap on Unsplash

53. Consider iterative feedback
Spreads feedback duties out for staff and gives
students feedback that they can act on immediately.
Use digital tools that allow students to show the
growth and development of a project or a piece of
research over time.
Use digital tools to turn feedback into an ongoing
conversation about the student’s work.
See Hounsell et al, 2007; Boud & Soler, 2016; Bayne & Ross, 2013; McKenna
and McAvinia, 2011; Hatzipanagos & Warburton, 2009
Photo by chuttersnap on Unsplash

54. Questions?
Photo by Alejandro Alvarez on Unsplash

55. References
Bayne, S. and Ross, J. (2013) ‘Posthuman literacy in heterotopic space: a pedagogical proposal’, in
Goodfellow, R. and Lea, M. (eds.) Literacy in the Digital University. London: Routledge.
Boud, D. and Soler, R. (2016) ‘Sustainable assessment revisited’, Assessment & Evaluation in Higher
Education 41(3), pp. 400-413.
Crisp, G. (2007) The e-assessment handbook. London: Continuum.
Hatzipanagos, S. and Warburton, S. (2009) ‘Feedback as dialogue: exploring the links between
formative assessment and social software in distance learning’, Learning, Media and Technology
34(1), pp. 45-59.
Hounsell, D. et al (2007) Integrative Assessment: Balancing assessment of and assessment for
learning, Guide no 2. Mansfield: The Quality Assurance Agency for Higher Education. Available at:
http://www.enhancementthemes.ac.uk/pages/docdetail/docs/publications/guide-no-2---balancing-
assessment-of-and-assessment-for-learning.
Jisc Student Digital Experience Tracker available at: https://www.jisc.ac.uk/rd/projects/student-
digital-experience-tracker.
Kao, G. Y. M. (2013). ‘Enhancing the quality of peer review by reducing student “free riding”: Peer
assessment with positive interdependence’, British Journal of Educational Technology 44(1), pp.
112-124.
McKenna, C. and McAvinia, C. (2011) ‘Difference and discontinuity – making meaning through
hypertexts’, in Land, R. and Bayne, s. (eds.) Digital Difference: Perspectives on Online Learning.
Rotterdam: Sense Publishers.
Photo by Patrick Tomasso on Unsplash