22 September 2020

1. Getting started with the
Digital experience
insights (DEI) surveys
2020/21
September 2020
Helen Beetham, Tabetha Newman
Sarah Knight and Ruth Drysdale

2. 1. The DEI surveys: background (SK)
2. New for 2020 / post-Covid (HB)
3. Using DEI as a ‘pulse’ survey (TN)
Questions and feedback (voice, chat)
3. Detail: demographics, participation (TN)
4. Detail: online learning/teaching/working (HB)
Questions and feedback (voice, chat)
5. Next steps (RD)
Outline

3. • Our insights surveys provide powerful data on how students,
teaching and professional services staff (and now
researchers) are using technology
• Designed to help to understand and improve the digital
experience in FE and HE
• Provide baseline and benchmarking data to inform your
digital strategy
• Annual summative report on each survey for each sector,
providing vision and leadership
• New question sets for 2020-1 surveys running from October
and will reflect requirements for the COVID-19 context
• https://digitalinsights.jisc.ac.uk/what-is-digital-insights-
experience/
Digital experience insights surveys

4. The survey questions
Around 30 questions in four thematic areas:
• You and your technology
• Technology at your university/college
• Technology in your learning (or teaching,
research or work role)
• Developing your digital skills
Questions equivalent/mapped across users
Easily visualised in real time via user dashboard
Response data shown to be relevant, actionable and
comparable
https://digitalinsights.jisc.ac.uk/what-is-digital-insights-experience/

5. New for 2020/21

6. Pre-Covid plans for 2020/21
• Develop and pilot a new question set for researchers
• Analyse existing and new questions
• Refine and update all question sets
• Map questions more closely across all four question
sets
• Maximise continuity for loyal users

7. Emergent findings on impact of COVID-19
 Pivot to new practice – rapid learning (opportunities and
stresses), training and workload issues
 Issues of mental health and wellbeing online
 Motivation and engagement
 New and widening inequalities e.g. hardware, networks,
data costs, home space
 Quality of and access to remote systems
 Quality of and access to resources/materials
 Safety and privacy issues

8. Post-Covid plans for 2020/21
• ‘Experience’ happens in a shorter time frame
• Mode of participation is key – but can change
• Iterative data collection (tracking change within the year)
• Regular reporting and sharing rather than summative
• Adapt Qs to focus on online and post-Covid issues
• Space for feelings/wellbeing issues and negative reactions
• Many surveys, panels, focus groups etc – how to add value?
• Build on everything we have learned - but continuity less critical
than supporting organisations through crisis and change

9. Post-Covid plans for 2020/21
• ‘Experience’ happens in a shorter time frame
• Mode of participation is key – but can change
• Iterative data collection (tracking change within the year)
• Regular reporting and sharing rather than summative
Adapting the service
Adapting the questions
• Adapt Qs to focus on online and post-Covid issues
• Space for feelings/wellbeing issues and negative reactions
• Many surveys, focus groups etc – how can DEI add value?
• Build on everything we have learned - but continuity now less
critical than supporting organisations through crisis and change

10. Adapting the DEI service to post-Covid
requirements

11. Participation, context, expectation, time frames
How people participate (online/onsite/blended) is crucial to
their whole experience
• We need to be able to partition or filter data using this factor
Context likely to change
• Personal circumstance (e.g. in quarantine, shielding)
• Shifting policies and priorities: course, organisation, national
• Online mode may be personal or group (cohort, team)
Expectation likely to drive attitudes
• Choosing online vs enforced/emergency online
Some questions are time-bound (e.g. ‘weekly’, ‘monthly’)
• Chose ‘last two weeks’ for all questions

12. Participation, context, expectation, time frames

13. What is a ‘pulse survey’
• One survey, run for short time periods, repeated soon after to
identify and resolve issues
• Very useful now that the learning, teaching and working experience
has changed and is changing unpredictably
• Questions refer to shorter time frame: “in the last two weeks”
• Can be used with the same set of people, or with different cohorts
• Can be run in discrete windows (e.g. start of year, just before
Christmas, just before Easter), or continuously

14. Using DEI surveys as ‘pulse surveys’
Oct.2020
30April
2021
Term 1 Term 3Term 2
2
weeks
2
weeks
2
weeks
College
A
Uni’ B 2
weeks
2
weeks
2
weeks
2
weeks
2
weeks
2
weeks

15. Practicalities: Using DEI as a pulse survey

16. Follow the ‘six steps’ as before…
1. Log in
2. Locate your surveys on your dashboard
3. Design/customise (add a logo and any questions of your own)
4. All surveys open (October 2020)
5. Check and launch
6. Promote and engage

17. But also…
1. Log in
2. Locate your surveys on your dashboard
3. Design/customise (add a logo and any questions of your own)
4. All surveys open (September 2020)
5. Check and launch
6. Promote and engage
7. Visualise your data at any time using the live ‘analyse’ window
8. Filter by date at any time
9. Promote and engage again to collect further data
10.Close, download and filter all data (all surveys close by 30 April 2021)

18. Online surveys allows you to easily filter by date

19. Issues to consider if you use DEI as ‘pulse surveys’
• Useful to see responses over time and track changes
• If surveying the same people at different times, analyse only within each
time frame – it is not valid to group data across the year
• If surveying different people in each date range, make sure your sample
is big enough to minimize random effects
• We expect so much change over the year that a summative view (adding
all the data across the year) is unlikely to be meaningful
• This means annual benchmarking may be of limited value (to discuss…)

20. Full ‘getting started’ guide: coming soon
https://digitalinsights.jisc.ac.uk/running
-insights-surveys/getting-started/

21. Questions, comments?

22. Adapting the DEI questions: focus on
learning/teaching/working online

23. Theme 1: You and your technology
 Focus on those issues we know (from DEI and other surveys)
create and widen digital inequalities

24. Theme 2: Technology in your organisation
 Physical infrastructure on site matters less than online
 Qual analysis 2019/20 found ‘learning environment’ was a
complex and evolving idea
 A lot of confusion about separate platforms and functions
 But respondents do know what qualities they want

25. Theme 3: Technology in your role
(learning/teaching/research/prof work)

26. Theme 3: Technology in your role
(wellbeing aspects)

27. Questions and feedback?

28. Next steps

29. Student digital experience insights survey 2020: UK
higher education (HE) survey findings:
https://www.jisc.ac.uk/reports/student-digital-
experience-insights-survey-2020-uk-higher-education
Learner digital experience insights survey 2020: UK
further education (FE) findings:
https://www.jisc.ac.uk/reports/learner-digital-
experience-insights-survey-2020-uk-further-education
Student digital experience insights survey 2020:
question by question analysis of findings from
students in UK further and higher education:
https://www.jisc.ac.uk/sites/default/files/dei-2020-
student-survey-question-by-question-analysis.pdf

30. Sign up to run the surveys https://digitalinsights.jisc.ac.uk/subscribe/find-out-more/
See our guidance https://digitalinsights.jisc.ac.uk/running-insights-surveys/
Join our Insights community: Click on jiscmail.ac.uk/JISC-DIGITALINSIGHTS-
COP and join by clicking on the 'Subscribe or Unsubscribe' button
Follow #digitalstudent
Date for your diary – 17th November for our online community of practice event
Get involved

31. Except where otherwise noted,
this work is licensed under CC-BY
Get in touch…
Digital experience insights
Sarah Knight
Sarah.knight@jisc.ac.uk
Ruth Drysdale
Ruth.drysdale@jisc.ac.uk
https://digitalinsights.jisc.ac.uk
#digitalstudent