Successfully reported this slideshow.

Jisc - how to go about analysing your survey data webinar 14 May 2020

1 of 30
1 of 30

More Related Content

Jisc - how to go about analysing your survey data webinar 14 May 2020

  1. 1. How to go about analysing your survey data webinar 14 May 2020
  2. 2. Agenda • Introduction (Sarah Knight) – 5 mins • Top tips for carrying out quantitative and qualitative analysis (Mark LC) – 10 mins • COVID-19 – updates in relation to analysis (Mark LC) – 10 mins • Presentations from institutions (10 mins each): • Laura Hollingshead (University of Derby) - ‘My data analysis journey’ • Rod Cullen (Manchester Metropolitan University) – ‘The use of added free text questions to allow richer insight of the data’ • Community of practice (Ruth Drysdale) – 5 mins • Open questions – (10 mins)
  3. 3. Introduction Sarah Knight – Head of Data and Digital Capability (Jisc)
  4. 4. Top tips for carrying out quantitative and qualitative analysis Mark Langer-Crame – Senior Data Intelligence Analyst (Jisc)
  5. 5. Top analysis tips – quantitative data • Comparing across your different surveys really add value to your findings – use the question mapping document! • If you completed the same survey last year, also really add value to compare on same questions - to measure progress • Use data in Jisc online surveys (after national survey closes) to benchmark your scores against your sector – will help get attention of senior management to the data • Use the analysis templates we provide – they make it much easier to analyse and present your data - they can be found here • How well do your responses reflect your target population? – using confidence intervals • Are the differences you see in response to a question statistically significant? – z-tests
  6. 6. Top analysis tips – qualitative data • Qualitative analysis software will allow you to code the responses – but it won’t do the ‘thinking’ for you as to how and what you decide to code! • Many software packages, i.e. Nvivo (has to be paid for) and QDA Miner Lite (which is free) – but specialist packages not necessary • Will be overwhelmed if try to analyse all your qualitative data – look at sample a random sample of the responses, i.e. around 10% • Carry out word frequency analysis or Word clouds (using Wordsift, Wordle or WordClouds) • Free text responses allow you to use staff and students' own words to illustrate and support key findings
  7. 7. Our official online guidance For more information you can find our official guidance on ‘analysing and understanding your insights survey data’ here
  8. 8. COVID-19 – updates in relation to analysis Mark Langer-Crame – Senior Data Intelligence Analyst (Jisc)
  9. 9. Running your insights surveys again • Many of you may want to run your insights survey(s) again if you ran your previous survey(s) pre-COVID • You can run your survey(s) again so you can compare your student/staff responses pre and post-COVID • You can simply make a copy of your original survey in Jisc online surveys, reset open/close dates (and maybe change your customised questions) and launch • Please can you include in survey name for second surveys the words ‘POST C19 RERUN’ - this is essential • Note the responses to these second surveys will not be included in the national reports
  10. 10. Close dates for the second surveys • For those who have already signed up for 2020-21, you can run your second survey up to the end of August this year • For those who have not signed up for 2020-21, you will need to ensure any second survey is closed by the end of July this year
  11. 11. For those who already have insights surveys open • For those who currently have your insights surveys open, once closed you can split the responses by a specified date (pre and post-COVID) as all responses are date stamped when they were submitted • To get the date stamp download into Excel, when carrying out the survey export, you will need to select the option ‘include date of response submission’ from the customisation options on the export screen to include the date stamp data • Once the data is separated out by date of submission, you can compare your responses pre and post the date you choose
  12. 12. New analysis template for those running second surveys • We will be creating an analysis templates in Excel so you can create charts to visualise your survey responses from the original and any second survey you run • These will be on the resources page of the website, alongside the other standard analysis templates (which can be found here) • These will be available by the end of this month (May)
  13. 13. Laura Hollingshead – University of Derby My data analysis journey
  14. 14. My data analysis journey Laura Hollinshead, University of Derby
  15. 15. Novice
  16. 16. Simple Approach
  17. 17. Stages of analysis Match questions to last year Identify areas where there is a difference Compare sector data in previous year Identify key areas of priority for organisation In June, compare this year's sector data and update report
  18. 18. Looking at data more closely
  19. 19. Future plans
  20. 20. Rod Cullen - Manchester Metropolitan University The use of added free text questions to allow richer insight of the data
  21. 21. Custom Questions: Thematic Analysis and Use Dr Rod Cullen Engaging with staff and students: What are we trying to achieve? • Gain a greater understanding of how digital technologies are employed in teaching and how our teaching staff engage with key aspects of the digital environment provided by the institution. • Support ISDS’ ongoing aim to work collaboratively with our teaching staff to improve and shape both their and our students’ digital experience.
  22. 22. Staff Insights Survey: Custom Questions 21. How can Moodle, the university Virtual Leaning Environment, be improved to better support your teaching and assessment practice? 22. How can the universities formal teaching spaces be improved (with or without technology) to better support your teaching? 23. Do you think student learning is enhanced by the use of technology? Y/N a. Please explain your answer?
  23. 23. Thematic Analysis
  24. 24. Overview of data sources JISC Staff Digital Insights Survey 22nd April – 12th May 2019 333 Responses (21% response rate) Insights Survey Follow-up June 2019 4 - Focus Groups Classroom Tech Review June/July 2018 20 user scenarios from TELAs Internal Student Survey TEL free text comments 20k comments over 6 years • A lot of data • Common & Conflicting Narratives • Complex picture
  25. 25. Main Narratives Narrative 1 Overall satisfaction and reliability of technology Narrative 2 Teaching spaces and technology Narrative 3 The Moodle Ecosystem Narrative 4 The desire to use technology Narrative 5 The need for strategic guidance Narrative 6 The perceived value of technology to support learning
  26. 26. Narrative 3 - Moodle I rely on the VLE for my teaching Man Met 78% Agree Sector 73% Agree Insights • 83% consider access to Moodle is reliable (sector 85%) • 51% agree it is easy to design and organise course materials (sector 49%) • 26% regularly use it for student collaboration and only (sector 27%) • 21% agree it encourages experimentation (sector 24%) Student perspective from ISS: • Moodle highly valued - how it is used is more important that how it looks • Materials in advance of teaching • Areas well organised and clearly labelled • Consistent approach across all of their units. Free text questions and follow-up focus groups: • Some academics are happy with Moodle • Many consider it to be “Clunky” • Inflexible • Content driven/non-collaborative • Too linear • Difficult to navigate • Unattractive to students/dated • Requests for additional features • Better integration with other infrastructure • Office 365 • Teams (enhanced collaborative tools) • Marks transfer to SRS • Some features that already exist • Assessment dates and deadlines
  27. 27. Community of practice Ruth Drysdale – Senior Consultant (Data and Digital Capability) (Jisc)
  28. 28. Digital experience insights community of practice • Launched in May 2018 and runs twice yearly • Next event – 21st May online in partnership with University of Northampton • Join our Community of Practice mailing list: JISC-DIGITALINSIGHTS-COP@JISCMAIL.AC.UK ‘Valuable for inspiring new ideas and making new contacts, for sharing resources.’ Register: insights-community-of-practice-21-may-2020 More details:
  29. 29. Open Questions
  30. 30. (putting ‘insights surveys’ in subject title) Digital experience insights

Editor's Notes

  • Carrying out analysis since 2017
    Each year I have developed my approach further
  • Limited research background
    Math's anxiety
    Qualitative data in BSc and Masters
    Really interested in doing research but not sure where to start
  • No fancy spreadsheets
    Use the data provided in the survey tool via the PDF documents
    Have a draft report which I reuse every year where I transfer data into
  • Start analysis in January as survey is carried out December of previous year
    Highlight difference of more than 5%

  • Questions related to Covid-19
    Questions related to NSS
    Main report helps me repurpose this
  • Statistical analysis – comparisons between groups e.g. first years vs others
    Need to overcome more of my Maths anxiety
    This year and last year could show significant differences with digital learning opportunities increasing
    Would like to publish off the back of this work. Unsure what this might look like at the moment.
  • The JISC Teaching Staff Insight survey is a national survey to find out more about the experiences of teaching staff involved in the delivery of learning.
  • Our digital capabilities community of practice with over 100 active participants is an open community that welcomes both experienced participants and those who have not previously engaged in this field but who would like to find out more.

    So if this interests you and you are not already a participant, please do join our Community of Practice. We run events twice a year in partnership with universities and colleges with the focus on sharing experiences, solutions and challenges. This, along with the mailing list provides opportunities for members be kept up to date with news and events and to work together on all aspects of digital capability for staff and students

    Our next event will be taking place in November in Edinburgh, the date of which will be shared via the mailing list and on the community of practice website. We are always looking for volunteers to share their practice at these events - so do let us know if you are interested in being involved.
  • ×