Learning analytics FAQs


Published on

Presentation given at SCONUL 2014, the summer conference of The Society of College, National and University Libraries, Glasgow, June 2014. The presentation focuses on frequently asked questions (FAQs) about learning analytics, with the emphasis on the role and perspective of libraries in this area.

Published in: Education, Technology

Learning analytics FAQs

  1. 1. Learning analytics: answering the FAQs Rebecca Ferguson The Open University SCONUL: June 2014
  2. 2. Take a minute to write down or tweet the three questions that would head your list of FAQs about learning analytics #sconulfaqs
  3. 3. Who are you? • The Open University: largest in Europe • Making use of data for more than 40 years • Informal learning: iTunes, YouTube… • MOOCs on FutureLearn and OpenLearn • Learning analytics research / events • LACE project
  4. 4. What are analytics? High-level figures Brief overview for internal and external reports Academic analytics Figures on retention and success, for the institution to assess performance Educational data mining Searching for patterns in the data Learning analytics Use of [big] data to provide actionable intelligence for learners and teachers
  5. 5. Is there a precise definition? The measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.
  6. 6. Why do learners use analytics? • Monitor their own activities and interactions • Monitor the learning process • Compare their activity with that of others • Increase awareness, reflect and self reflect • Improve discussion participation • Improve learning behaviour • Improve performance • Become better learners • Learn!
  7. 7. Why might I use analytics? • Monitor the learning process • Explore student data • Identify problems • Discover patterns • Find early indicators for success • Find early indicators for poor marks or drop-out • Assess usefulness of learning materials • Increase awareness, reflect and self reflect • Increase understanding of learning environments • Intervene, advise and assist • Improve teaching, resources and the environment
  8. 8. Can you give me an example? • Aligned with clear aims • Huge and sustained effort • Agreed proxies for learning • Clear and standardised visualisation • Driving behaviour at every level Individual assessment within cohort
  9. 9. Is that an ideal example? Can we achieve this? • Aligned with clear aims • Huge and sustained effort • Agreed proxies for learning • Clear and standardised visualisation • Driving behaviour at every level Can we avoid this? • Instructivist approach • Stressed, unhappy learners • Analytics with little value for learners or teachers • Omission of key areas, such as collaboration
  10. 10. Don’t start with the data – start with the pedagogy How do people learn? How can I use data to facilitate that process? Social learning analytics: How do people learn socially & in social settings? How can I use data to facilitate that process? How could we achieve that?
  11. 11. What questions should I ask? • Which elements are learners struggling with? • Which sections engage them the most? • What prompts them to ask questions? • How are they navigating resources? • What misconceptions have they shown? • Are there any accessibility issues? How can analytics be used to achieve desired learning outcomes?
  12. 12. • Demographic data • Calendar information about assignments • VLE activity data (including forums). • Lists of required reading • Library resources usage data • Library helpdesk enquiries • Library website usage and analytics data • Assessment results • User survey results • Student retention and attainment data What data could I use?
  13. 13. Does HE use learning analytics? Purdue University
  14. 14. SNAPP: British Columbia Are all analytics like that?
  15. 15. Disparate data VLE (Moodle) data Survey data Help desk data Library data Assessment data Registration data 250,000 students around the world Faculty expertise Learning design Learning outcomes Assessment strategy Module connections Resource understanding Data Wranglers Do you have an OU example?
  16. 16. Resource page? Sub page? URL? Significance of dates? What does a Wrangler do?
  17. 17. Reach agreement on conventions 80% agreed or strongly agreed that…. Is that 80% of registered students? 80% of those who completed the survey? 80% of those who completed both survey and course? Or is it 80% of survey respondents who completed the course and did not select ‘not applicable’ What else do Wranglers do?
  18. 18. Relate analytics to learning design How many forums? What were their roles? How were they timetabled? Anything else?
  19. 19. Support, not surveillance Reporting at module level, not individual level Survey responses kept apart from activity data Making data accessible, not overwhelming Negotiating meaning, not dictating interpretation Are there any ethical issues?
  20. 20. Shared knowledge building Is it all about wrangling?
  21. 21. Shared knowledge building Does your library use analytics?
  22. 22. What about informal learning?
  23. 23. Social Network Status Domain Expert Reputation
  24. 24. Massive thousands may sign up Open no payment is required Online resources on the Internet Courses time-bounded cohorts Any links with MOOCs? (What are MOOCs?)
  25. 25. Introduction to Forensic Science University of Strathclyde You have been actively engaged in the discussions, which is excellent, thank you, but with more than 23,000 participants it means that our responses and comments risk getting lost. Analytics could help to filter discussions How could MOOCs use analytics?
  26. 26. Introduction to Forensic Science University of Strathclyde This will be primary school material for some of you and exactly the opposite for others. It is just not possible to tailor the material to each of you […] How could MOOCs use analytics? Analytics could help to group learners
  27. 27. Analytics could help to focus attention How could MOOCs use analytics?
  28. 28. Helping to identify useful analytics What do learners need to know in order to network, collaborate, browse or reflect? What do educators need to know to support them? Helping to identify gaps in the data Which data do we need to collect? Helping to identify gaps in our toolkit Which design elements can we look at easily? Which ones still pose problems? Any links with learning design?
  29. 29. Helping to frame and focus analytics questions What did they learn?… in relation to learning outcomes Were they social?... when they were collaborating Did they share links?... when encouraged to browse Did they return to steps?... when encouraged to reflect Helping to identify appropriate forms of analysis The same content, but with a focus on Number of visits if content Length, quality, number of comments if conversational Dwell time and repeat visits if reflection Any links with learning design?
  30. 30. How are analytics used in libraries? Mission To impact student measures of satisfaction To demonstrate value-added to users To inform and justify policy Teaching and learning To target and assess interventions To position library teaching effort To track user journeys Recommendations To promote use of content beyond reading list To link discovery, lists and recommendations To surface unused content jisclamp.mimas.ac.uk
  31. 31. How are analytics used in libraries? Collection management To deliver better / best value for money To promote evidence-led decision-making To identify how faculties use the library Data To merge data from multiple systems To incorporate web analytics To provide compelling visualisations Service To correlate feedback, enquiries and collection strength To understand access patterns To target skills development jisclamp.mimas.ac.uk
  32. 32. How are you using analytics? Collection management Mission Data Teaching and learning Recommendations Service jisclamp.mimas.ac.uk
  33. 33. bit.ly/1130R0Y bit.ly/VUGBfJ Where can I find out more? jisclamp.mimas.ac.uk bit.ly/1qulB06