1.
How to go about analysing your
survey data webinar
14 May 2020

2.
Agenda
• Introduction (Sarah Knight) – 5 mins
• Top tips for carrying out quantitative and qualitative analysis (Mark LC) – 10 mins
• COVID-19 – updates in relation to analysis (Mark LC) – 10 mins
• Presentations from institutions (10 mins each):
• Laura Hollingshead (University of Derby) - ‘My data analysis journey’
• Rod Cullen (Manchester Metropolitan University) – ‘The use of added free text
questions to allow richer insight of the data’
• Community of practice (Ruth Drysdale) – 5 mins
• Open questions – (10 mins)

3.
Introduction
Sarah Knight – Head of Data and Digital Capability (Jisc)

4.
Top tips for carrying out quantitative and
qualitative analysis
Mark Langer-Crame – Senior Data Intelligence Analyst
(Jisc)

5.
Top analysis tips – quantitative data
• Comparing across your different surveys really add value to your findings – use the
question mapping document!
• If you completed the same survey last year, also really add value to compare on same
questions - to measure progress
• Use data in Jisc online surveys (after national survey closes) to benchmark your scores
against your sector – will help get attention of senior management to the data
• Use the analysis templates we provide – they make it much easier to analyse and
present your data - they can be found here
• How well do your responses reflect your target population? – using confidence intervals
• Are the differences you see in response to a question statistically significant? – z-tests

6.
Top analysis tips – qualitative data
• Qualitative analysis software will allow you to code the responses –
but it won’t do the ‘thinking’ for you as to how and what you decide to
code!
• Many software packages, i.e. Nvivo (has to be paid for) and QDA
Miner Lite (which is free) – but specialist packages not necessary
• Will be overwhelmed if try to analyse all your qualitative data – look at
sample a random sample of the responses, i.e. around 10%
• Carry out word frequency analysis or Word clouds (using Wordsift,
Wordle or WordClouds)
• Free text responses allow you to use staff and students' own words to
illustrate and support key findings

7.
Our official online guidance
For more information you can
find our official guidance on
‘analysing and understanding
your insights survey data’ here

8.
COVID-19 – updates in relation to analysis
Mark Langer-Crame – Senior Data Intelligence Analyst
(Jisc)

9.
Running your insights surveys again
• Many of you may want to run your insights survey(s) again if you ran your previous
survey(s) pre-COVID
• You can run your survey(s) again so you can compare your student/staff responses pre
and post-COVID
• You can simply make a copy of your original survey in Jisc online surveys, reset
open/close dates (and maybe change your customised questions) and launch
• Please can you include in survey name for second surveys the words ‘POST C19
RERUN’ - this is essential
• Note the responses to these second surveys will not be included in the national reports

10.
Close dates for the second surveys
• For those who have already signed up for 2020-21, you can run your second
survey up to the end of August this year
• For those who have not signed up for 2020-21, you will need to ensure any second
survey is closed by the end of July this year

11.
For those who already have insights surveys
open
• For those who currently have your insights surveys open, once closed you can split the
responses by a specified date (pre and post-COVID) as all responses are date stamped
when they were submitted
• To get the date stamp download into Excel, when carrying out the survey export, you will
need to select the option ‘include date of response submission’ from the customisation
options on the export screen to include the date stamp data
• Once the data is separated out by date of submission, you can compare your responses
pre and post the date you choose

12.
New analysis template for those running
second surveys
• We will be creating an analysis templates in Excel so you can create charts to
visualise your survey responses from the original and any second survey you run
• These will be on the resources page of the website, alongside the other standard
analysis templates (which can be found here)
• These will be available by the end of this month (May)

13.
Laura Hollingshead – University of Derby
My data analysis journey

14.
My data
analysis
journey
Laura Hollinshead,
University of Derby

15.
Novice

16.
Simple
Approach

17.
Stages of
analysis
Match questions to last year
Identify areas where there is a
difference
Compare sector data in previous year
Identify key areas of priority for
organisation
In June, compare this year's sector
data and update report

18.
Looking at data
more closely

19.
Future plans

20.
Rod Cullen - Manchester Metropolitan University
The use of added free text questions to allow richer
insight of the data

21.
Custom Questions: Thematic Analysis and Use
Dr Rod Cullen
Engaging with staff and students: What are we trying to achieve?
• Gain a greater understanding of how digital technologies are employed in teaching and
how our teaching staff engage with key aspects of the digital environment provided by
the institution.
• Support ISDS’ ongoing aim to work collaboratively with our teaching staff to improve
and shape both their and our students’ digital experience.

22.
Staff Insights Survey: Custom Questions
21. How can Moodle, the university Virtual Leaning Environment,
be improved to better support your teaching and assessment
practice?
22. How can the universities formal teaching spaces be improved
(with or without technology) to better support your teaching?
23. Do you think student learning is enhanced by the use of
technology? Y/N
a. Please explain your answer?

23.
Thematic Analysis

24.
Overview of data sources
JISC Staff Digital Insights
Survey
22nd April – 12th May 2019
333 Responses
(21% response rate)
Insights Survey Follow-up
June 2019
4 - Focus Groups
Classroom Tech Review
June/July 2018
20 user scenarios from TELAs
Internal Student Survey
TEL free text comments
20k comments over 6 years
• A lot of data
• Common &
Conflicting
Narratives
• Complex
picture

25.
Main Narratives
Narrative 1
Overall
satisfaction
and reliability
of technology
Narrative 2
Teaching
spaces and
technology
Narrative 3
The
Moodle
Ecosystem
Narrative 4
The desire to
use
technology
Narrative 5
The need for
strategic
guidance
Narrative 6
The perceived
value of
technology to
support
learning

26.
Narrative 3 - Moodle
I rely on the VLE for my teaching
Man Met 78% Agree
Sector 73% Agree
Insights
• 83% consider access to Moodle
is reliable (sector 85%)
• 51% agree it is easy to design
and organise course materials
(sector 49%)
• 26% regularly use it for student
collaboration and only (sector
27%)
• 21% agree it encourages
experimentation (sector 24%)
Student perspective from
ISS:
• Moodle highly valued -
how it is used is more
important that how it
looks
• Materials in advance of
teaching
• Areas well organised
and clearly labelled
• Consistent approach
across all of their units.
Free text questions and follow-up focus
groups:
• Some academics are happy with Moodle
• Many consider it to be “Clunky”
• Inflexible
• Content driven/non-collaborative
• Too linear
• Difficult to navigate
• Unattractive to students/dated
• Requests for additional features
• Better integration with other infrastructure
• Office 365
• Teams (enhanced collaborative tools)
• Marks transfer to SRS
• Some features that already exist
• Assessment dates and deadlines

27.
Community of practice
Ruth Drysdale – Senior Consultant (Data and Digital Capability)
(Jisc)

28.
Digital experience insights community of practice
• Launched in May 2018 and runs twice
yearly
• Next event – 21st May online in
partnership with University of
Northampton
• Join our Community of Practice mailing
list: JISC-DIGITALINSIGHTS-COP@JISCMAIL.AC.UK
‘Valuable for inspiring new ideas and
making new contacts, for sharing
resources.’
Register: https://www.jisc.ac.uk/events/building-digital-capability-and-digital-experience-
insights-community-of-practice-21-may-2020
More details: https://digitalinsights.jisc.ac.uk/our-service/community-of-practice/

29.
Open Questions

30.
customerservices@jisc.ac.uk
jisc.ac.uk
help@jisc.ac.uk (putting
‘insights surveys’ in subject title)
Digital experience insights
https://digitalinsights.jisc.ac.uk