The document summarizes the development and piloting of a cross-sector benchmarking tool called the Digital Experience Tracker. It was developed by Jisc to help education providers gather feedback from students on their digital experiences and skills. Over 10,000 students from 24 pilot institutions completed the tracker survey. Key findings included that students want more access to devices and digital resources, and that FE & Skills students reported receiving more guidance but had less access than HE students. The tracker is now available for other providers to use to understand and improve students' digital experiences.

1. Tracking students’ digital experience: development and use of a
cross sector benchmarking tool
Helen Beetham and Sarah Knight
ALT-C
07/09/16
#digitalstudent https://digitalstudent.jiscinvolve.org

2. Workshop outline
»Background: why a Digital ExperienceTracker?
»Insights from some pilot sites:
› University ofWestminster
› Barton Peverill 6th Form College
› University of Northampton
»Panel session: your chance to find out more
»Try it for yourself: live access using your own device
»Activity: help us make theTracker better
»Findings: using theTracker to build sector intelligence
»Round-up: where next?
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

3. http://digitalstudent.jiscinvolve.org
» Phase 1 study reviewed students’ expectations and experiences of the digital
environment at university and we spoke to 500 staff and students during our
consultation (2013-2014) http://ji.sc/Digital_ Student_Expectations
» We conducted a review of practice in schools to identify likely incoming expectations
(2014)
» In phase 2 we focused on FE speaking to 220 learners and 300 staff from colleges across
the UK (2014-2015) http://ji.sc/Digital_ Students_Expectations_FE
» Phase 3 Skills study speaking to adult and community learners, work based learners
including apprentices and offender learners (2015-2016)
http://ji.sc/Digital_Student_Skills
» Phase 4 Online learners study, speaking to students studying on online or partly online
courses (2016)
Tracking students’ digital experience http://digitalstudent.jiscinvolve.org13/09/2016

4. Digital student online learners study
Tracking students’ digital experience http://digitalstudent.jiscinvolve.org
»What makes a successful online learner?
»Findings of the Digital Student Online
learners’ expectations and experiences
of the digital environment
»Now available from
http://ji.sc/onlinesurveyreport
13/09/2016

5. Why a student digital experience tracker?
» The Student digital experienceTracker supports universities, colleges and skills
providers to:
› gather evidence from learners about their digital experience, and track changes
over time
› make better informed decisions about the digital environment
› target resources for improving digital provision
› plan other research, data gathering and student engagement around digital
issues
› demonstrate quality enhancement and student engagement to external bodies
and to students themselves
» See http://bit.ly/jisctracker
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

6. Piloting the tracker
» Survey implemented in BOS, a system
developed for UK HE/FE and already used by
many
» 50+ education providers applied: 24 chosen
based on a spread of institutional sizes, types
and locations
» Pilot sites required to involve stakeholders,
consider how they would engage learners, and
explore hoped-for benefits
» Teams at each pilot site responsible for:
customising the survey, releasing it to learners,
maximising completion rates and analysing
their own data
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

7. Outcomes from pilot
»10,753 unique data sets
»Case studies from 7 institutions
http://bit.ly/trackercasestudies
»Full report is available from:
http://bit.ly/student-tracker-
report
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

8. Insights from three of our pilot sites
»Yanna Nedelcheva, University ofWestminster
»AndyTaggart, Barton Peverill 6th Form College
»Rob Howe, University of Northampton
Each will present briefly on:
»Reasons for using theTracker
»Engaging students and other stakeholders
»Response to theTracker findings
»There will be a chance to ask questions in the panel
session immediately after!
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

9. Some feedback from pilot sites…
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org
“Thanks to the Student Digital
ExperienceTracker we can better
understand the needs and demands of
our students in terms of infrastructure,
support and pedagogy.”
“much needed baseline data that
should help the university measure
the impact of its new curriculum”
“a good starting point for
colleges to effectively evaluate
students skills and needs”
“really interesting.We would
definitely look forward to rolling
out across all our provision.”
“an extremely useful opportunity
to identify issues in relation to
benchmarked practice”
“allows us to have evidence
informed conversations with
the relevant stakeholders in the
institution.”
“a real insight into the
digital experience of
our students”
.
”The questions
are well written
and appropriate”
” has engaged people here in
a discussion about digital
skills development.”

10. Questions to the Panel
»Yanna Nedelcheva, University ofWestminster
»AndyTaggart, Barton Peverill College
»Rob Howe, University of Northampton
»Helen Beetham, project team
»(Chair) Sarah Knight, Jisc Co-Design Manager
»Ask a question viaTwitter use Q:
#digitalstudent
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

11. Try it for yourself
1. Access one of the twoTracker tools using the URL:
› HETracker 2016: bit.ly/TrackerHE16
› FE&SkillsTracker 2016 bit.ly/TrackerFE16
2. Browse the survey questions.You can customise
questions that are not locked. Download the full user
guide here: bit.ly/TrackerGuide16 (some actions will not
be relevant in the generic survey that you are using)
3. Discuss and be ready to feedback your thoughts about
the questions and how you might use the results.
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

12. Your ideas
»We are already planning a number of changes thanks to
the evidence we collected from the pilot process, e.g.
› More customisation options
› Optional extra questions e.g. on teaching and learning,
accessibility, attitudes to social media…
› Lose one or two free text questions
› New question set for principally online learners
»Now is your chance to tell us http://bit.ly/trackerpadlet
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

13. On post-it notes…
Please add to the A3 sheets:
»How might findings from the existing questions be used
at your institution?
»What new questions or question themes would you like to
see included in the future?
»Or add to padlet: http://bit.ly/trackerpadlet
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

14. Headline findings from theTracker pilot
»7,425 students from twelve HE and 3,328 students from
twelve FE & Skills institutions completed theTracker
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

15. Access and use
» Students are more likely to use personal laptops, smartphones and tablets than
desktops to support their learning
» 90% of HE students use personal laptops: only 66% of FE & Skills students
» 40% of FE & Skills students use personal desktops: only 30% of HE students
» FE & Skills students are significantly more likely to use institutional technologies
to support their learning
» Significantly more FE & Skills students could frequently access institutional
computers and printers in comparison with HE students
» Significantly more HE students could frequently access institutional WiFi
» A significantly higher % of FE & Skills students said they could never access
personal information online in comparison with HE students
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

16. Learning with technology
During the last 6 weeks of their course:
» 9 in every 10 students found information online
» 7 in 10 students produced work in a digital format
» 5 in every 10 students worked online with others
» 3 in every 10 students created a personal (digital) record of their learning
» HE students were significantly more likely to have carried out all of these
activities than FE students, with the exception of creating a personal record of
their learning (no significant difference)
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

17. Guidance and support
» Most students (63% HE, 82% FE) say they have received enough guidance on
behaving safely and respectfully online.
» Less than half say they have received enough guidance on modifying devices to
suit their individual needs, or creating a positive online presence.
» More than half but less than two thirds say they have received enough guidance
on bringing their own device, or developing digital skills relevant to their course.
» Significantly more FE and skills students than HE students say they had received
enough guidance on all of these issues, except creating a positive online presence
(about equal).
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

18. Guidance and support
» Most students (63% HE, 82% FE) say they have received enough guidance on
behaving safely and respectfully online.
» Less than half say they have received enough guidance on modifying devices to
suit their individual needs, or creating a positive online presence.
» More than half but less than two thirds say they have received enough guidance
on bringing their own device, or developing digital skills relevant to their course.
» Significantly more FE and skills students than HE students say they had received
enough guidance on all of these issues, except creating a positive online presence
(about equal).
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

19. Overall experience
» Approximately 7 in 10 students believe that when technology is used by teaching
staff it enhances their learning experience
» Around 6 in 10 students believe that digital assessments are delivered well
» Significantly less than half of all students know how their personal data is used
by their provider, or say that they feel involved in decisions about the digital
environment. In both cases the figure is significantly lower for HE students.
» Analysis of free text responses shows that most students want ‘more’ and better
access to digital devices, networks, services and resources.
» HE students also want: recorded lectures, better use ofVLE, less powerpoint,
better digital activities and assessments, 24/7 access to all content
» FE&Skills students also want: more guidance on effective use (staff and
students), no blocked sites, more accessible online resources, support for skills
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

20. Round-up and close
»Sign up now for open pilot from September 16:
http://bit.ly/jisctracker and join mailing list
http://www.jiscmail.ac.uk/jisc-digitalstudent-tracker
»Online digital student study http://ji.sc/onlinesurveyreport
»New online guide Enhancing the digital experience for skills
learners with case studies launched early October 2016
»Learners’ journeys with technology
13/09/2016 Tracking students’ digital experience http://digitalstudent.jiscinvolve.org

21. Find out more…
Digital Student project
Sarah.knight@jisc.ac.uk
Helen.Beetham@gmail.com
http://digitalstudent.jiscinvolve.org
#digitalstudent
Except where otherwise noted, this
work is licensed under CC-BY-NC-ND
Jisc Digital Student http://digitalstudent.jiscinvolve.org.uk