Slides from Jisc Student Experience Experts' meeting June 2016 introducing data from the Jisc Digital Student Experience Tracker pilot and findings about the Tracker process

1. Digital student experience Tracker:
an update from Jisc Helen Beetham on behalf of Jisc

2. 22/0 Student digital experience tracker pilot results 2016
Background
22/06 Student digital experience tracker pilot results 2016 23

3. 22/0 Student digital experience tracker pilot results 2016
Development of the Tracker
»Consultations 2014-2015: demand for
a better understanding of learners'
digital experiences and expectations
»No standard reliable instrument
»Tracker questions based on research by
the Jisc Digital student project, inc.
studies of learners in HE, FE and skills
»Consultation workshop with staff and
students refined the proposed question
set to 12 key questions
»Cognitive interviewing with FE and HE
students, and early beta testing further
refined the questions for piloting
22/06 Student digital experience tracker pilot results 2016 8

4. 22/0 Student digital experience tracker pilot results 2016
The Tracker questions
1.Access to digital devices, personal and institutional
2.Access to digital services
3.Digital activities in courses of study
4.Guidance and support
5.Overall digital experience
6.STOP
7.START
8.KEEP
9. (11) Value of VLE/LMS
22/06 Student digital experience tracker pilot results 2016 8

5. 22/0 Student digital experience tracker pilot results 2016
Aims of the Tracker
»gather evidence from learners about
their digital experience, and track
changes over time
»make better informed decisions about
the digital environment
»target resources for improving digital
provision
»plan other research, data gathering and
student engagement around digital
issues
»demonstrate quality enhancement and
student engagement to external bodies
and to students themselves
22/06 Student digital experience tracker pilot results 2016 8
Reasons for engaging
with the Tracker

6. 22/0 Student digital experience tracker pilot results 2016
Aims of the Tracker
Early findings:
»Big appetite for sector benchmarking
»Infrastructure, curriculum and
strategy development - plus
‘understand’ student experience
»Desire for a student-facing ‘digital
capabilities’ tool to use alongside the
Tracker
22/06 Student digital experience tracker pilot results 2016 8
Reasons for engaging
with the Tracker

7. 22/0 Student digital experience tracker pilot results 2016
Piloting the Tracker
»Survey implemented in BOS,
a system developed for UK HE/FE
and already used by many
»50+ education providers applied:
24 chosen based on a spread of
institutional sizes, types and
locations
»Pilot sites required to involve
stakeholders, consider how they
would engage learners, and explore
hoped-for benefits
»Teams at each pilot site responsible for: customising the survey,
releasing it to learners, maximising completion rates and
analysing their own data
22/06 Student digital experience tracker pilot results 2016 9

8. 22/0 Student digital experience tracker pilot results 2016
Support for the pilot
»Six separate guides produced to
support the pilot process and to
ensure parity of data collected
»Sites had only the guides and
collective email/blog postings to
reproduce service delivery
»Project negotiated for sector
benchmarking data to be made
available within BOS
»Pilot sites took different approaches
to engaging stakeholders and eliciting
learner responses
22/06 Student digital experience tracker pilot results 2016 10

9. 22/0 Student digital experience tracker pilot results 2016
Where we are today
»10,753 unique data sets
»Initial data report released,
press released
»Each site achieved a good
response rate: analysing,
understanding and
responding to their data
»Feedback about the pilot
process (18 out of 24 sites)
»The experience of six
contrasting sites is being
explored in more detail,
including feedback from learners and other stakeholders
22/06 Student digital experience tracker pilot results 2016 11

10. 22/0 Student digital experience tracker pilot results 2016
More information
22/06 Student digital experience tracker pilot results 2016
»Full report is available from:
http://bit.ly/student-tracker-report
»Further information about the pilot
and the process:
http://bit.ly/jiscdigidataservice
12

11. 22/0 Student digital experience tracker pilot results 2016
At your tables
22/06 Student digital experience tracker pilot results 2016
»How might the Tracker be useful at your
institution?
»What would you most want to find out from your
learners?
12

12. 22/0 Student digital experience tracker pilot results 2016
Headline findings
22/06 Student digital experience tracker pilot results 2016 23

13. 22/0 Student digital experience tracker pilot results 2016
Sample and response rates
22/06 Student digital experience tracker pilot results 2016 24

14. 22/0 Student digital experience tracker pilot results 2016
Headline 1: sector differences
There are significant differences between HE and FE, particularly in
the areas of:
»personal access to devices (generally higher in HE)
»use of institutional computing and printing (generally higher in FE)
»access to institutional digital services (generally better in HE)
»access to guidance on digital issues (generally better in FE)
»digital activities as part of a course of study (generally more widely
undertaken in HE)
22/06 Student digital experience tracker pilot results 2016 25

15. 22/0 Student digital experience tracker pilot results 2016
Headline 2: BYO is not a panacea
Despite high levels of personal device ownership (especially in
HE), learners continue to make extensive use of institutional
computing and printing facilities.
22/06 Student digital experience tracker pilot results 2016 27

16. 22/0 Student digital experience tracker pilot results 2016
Headline 3: digital guidance and support
Across the board, access to digital guidance is lower than we might
expect. For example:
»Less than half of all students say they have received guidance on:
modifying devices to suit their individual needs, or creating a
positive online presence.
»Less than two thirds of all students say they have received
guidance on: bringing your own device, or developing digital skills
relevant to their course.
22/06 Student digital experience tracker pilot results 2016 28

17. 22/0 Student digital experience tracker pilot results 2016
Headline 4: digital activities on course
»HE students are significantly more likely than FE/skills students to
have produced work in a digital format, and to have worked online
with others. Overall, during the last six weeks:
»9 in every 10 students found information online
»7 in every 10 students produced work in a digital format
»5 in every 10 students
worked online with others
»3 in every 10 students
created a personal record
of their learning
22/06 Student digital experience tracker pilot results 2016 51

18. 22/0 Student digital experience tracker pilot results 2016
Headline 5: overall digital experience
Should we be doing better?
»Around 7 in every 10 students say that when technology is used
by teaching staff it enhances their learning experience
»Just over half believe that digital assessments are delivered well
»31.5% of HE students and 44.5% of FE & Skills students agree
that they are told how their personal data is stored and used
»37.5% of HE students and 43.2% of FE & Skills students agree
that learners are involved in decisions about digital services
22/06 Student digital experience tracker pilot results 2016 30

19. 22/0 Student digital experience tracker pilot results 2016
At your tables
»What findings are most surprising/interesting to you?
»What (new) questions do you have that we could
explore in future iterations of the Tracker?
22/06 Student digital experience tracker pilot results 2016 32

20. 22/0 Student digital experience tracker pilot results 2016
About the pilot process
22/06 Student digital experience tracker pilot results 2016 23

21. 22/0 Student digital experience tracker pilot results 2016
Being a pilot institution
»A chance to ask questions of Rob Howe,
University of Northampton, and Mark Breedon,
Basingstoke College
22/06 Student digital experience tracker pilot results 2016 14

22. 22/0 Student digital experience tracker pilot results 2016
Findings about the process
»Pilot sites found the Tracker easy to customise and implement
»With some adjustments, especially for learners using assistive
technology, learners found the Tracker easy to complete
»Pilot sites valued and used the guidance materials
»Many sites are still analysing their data and formulating their
responses
»Most sites say that using the Tracker has been 'quite’ or ‘very
useful' – but with some caveats (we are exploring these)
22/06 Student digital experience tracker pilot results 2016 14

23. 22/0 Student digital experience tracker pilot results 2016
Quotes
»‘Thanks to the Tracker we can better understand the needs and
demands of our students...They have sent us a clear message
about the importance they attach to digital skills.’
»‘...particularly useful in being able to gather data locally and to then
benchmark more widely. We would also be interested in
comparative data over a number of years (to see if some of our
interventions are making a difference).’
»‘It has confirmed ... that learners want to use more types of digital
learning and resources.’
»‘I feel it is important for colleges to be aware of their students'
digital experience and there may not be any other mechanism to
monitor this. Also the questions are well written and appropriate.’
22/06 Student digital experience tracker pilot results 2016 15

24. 22/0 Student digital experience tracker pilot results 2016
Quotes
»‘I just wanted to say how valuable we found the survey data. Although we
didn’t get the number of responses we would have liked (this is partly to
do with some restrictions we have to avoid student “survey fatigue”) the
data we did gather has been evaluated and reported on at key Learning &
teaching committees. It has also helped us to prioritise some key plans
around the IT infrastructure programme and helped us to identify some
other areas where we might need to provide further support.’
» ‘I look forward to seeing the survey evolve and become a regular activity
(at least in our institution).’
»‘Yes please to the Tracker, with some core questions and the ability to
add more of our own”
»‘The Tracker has provided a real insight into the digital experience of our
students.’
»‘The results and comparisons have been really insightful.
22/06 Student digital experience tracker pilot results 2016 15

25. 22/0 Student digital experience tracker pilot results 2016
Changes suggested
»Additional questions (optional?) e.g. on
» inclusivity and accessibility,
» digital pedagogies (more about tech in the curriculum)
» digital learning and study habits (more about informal uses of tech)
» digital identity, digital literacies, digital capability and confidence
» wellbeing and safeguarding
»Explore more emergent issues, ‘things we don’t already know’
»Delivery to suit students using assistive technologies
»(Alongside the Tracker) instrument that assesses learners’ digital
capabilities and gives feedback to the learner...
22/06 Student digital experience tracker pilot results 2016 15

26. 22/0 Student digital experience tracker pilot results 2016
Over to you
Would your institution be interested in using the
Tracker? (Show of hands)
What would make you want to use the Tracker (more)?
(drivers and potential benefits on green post-its please)
What would be your concerns/barriers?
(barriers and potential costs on pink post-its please)
22/06 Student digital experience tracker pilot results 2016 16

27. 22/0 Student digital experience tracker pilot results 2016
Thank you for your contributions!
»Any further questions or contributions before we close?
22/06 Student digital experience tracker pilot results 2016 33

28. 22/0 Student digital experience tracker pilot results 2016
More information
22/06 Student digital experience tracker pilot results 2016
»Full report is available from:
http://bit.ly/student-tracker-report
»Further information about the pilot
and the process:
http://bit.ly/jiscdigidataservice
12

29. jisc.ac.uk
22/0 Student digital experience tracker pilot results 2016
»For more
information contact
Sarah Knight: sarah.knight@jisc.ac.uk
22/06 Student digital experience tracker pilot results 2016 34
Helen Beetham: helen.beetham@gmail.com