Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Learning analytics, learning design and MOOCs


Published on

Workshop run at the European Conference for e-Learning 2015 (ECEL 2015) at the University of Hertfordshire, UK. The workshop included an introduction of both learning analytics and learning design, as well as an exploration of how these could be employed in MOOCs. Some of the group work was focused on the Agincourt MOOC run by the University of Southampton on the FutureLearn platform.

Published in: Education

Learning analytics, learning design and MOOCs

  1. 1. Learning design and learning analytics: building the links with MOOCs European Conference for e-Learning 2015
  2. 2. Rebecca Ferguson • The Open University (200,000 learners) • Informal learning: • YouTube (30 million views) FutureLearn (2.5 million learners) OpenLearn (39.3 million visits) • Making use of big data for 45 years • Learning analytics research / events • LACE project – learning analytics community exchange 2 Lead on MOOC evaluation at The Open University, UK
  3. 3. Workshop overview 14.00 Introduction Linking learning analytics, learning design and MOOCs 14.30 Initial group work Learning outcomes in MOOCs 15.00 Coffee break and discussion 15.10 Group work Learning design in MOOCs 15.30 Discussion 15.40 Group work Learning analytics in MOOCs 15.40 Final plenary 16.00 Workshop end 3 You can view and download a version of these slides at
  4. 4. Learning analytics What are they, and how are they used? 4
  5. 5. What are learning analytics? High-level figures Brief overviews for internal and external reports Academic analytics Figures on retention and success, for the institution to assess performance Learning analytics Use of big data to provide actionable intelligence for learners and educators 5
  6. 6. Educators use analytics to • Monitor the learning process • Explore student data • Identify problems • Discover patterns • Find early indicators for success • Find early indicators for poor marks or drop-out • Assess usefulness of learning materials • Increase awareness, reflect and self reflect • Increase understanding of learning environments • Intervene, supervise, advise and assist • Improve teaching, resources and the environment 6 Dyckhoff, A L, Lukarov, V, Muslim, A, Chatti, M A, & Schroeder, U. (2013). Supporting Action Research with Learning Analytics. Paper presented at LAK13.
  7. 7. Learners use analytics to • Monitor their own activities and interactions • Monitor the learning process • Compare their activity with that of others • Increase awareness, reflect and self reflect • Improve discussion participation • Improve learning behaviour • Improve performance • Become better learners • Learn! 7 Dyckhoff, A L, Lukarov, V, Muslim, A, Chatti, M A, & Schroeder, U. (2013). Supporting Action Research with Learning Analytics. Paper presented at LAK13.
  8. 8. Analytics example: UK schools 8 • Aligned with clear aims • Huge and sustained effort • Agreed proxies for learning • Clear and standardised visualisation • Driving behaviour at every level BUT • Stressed, unhappy learners • Analytics with little value for learners or educators • Omission of key areas, such as collaboration
  9. 9. Analytics example: Course Signals Developed at Purdue University 9 Arnold, K E, & Pistilli, M (2012). Course Signals at Purdue: Using Learning Analytics To Increase Student Success. Paper presented at LAK12, Vancouver, Canada.
  10. 10. Analytics example: SNAPP Network analysis 10
  11. 11. Analytics example: iSpot Heading 11
  12. 12. Making the links between learning analytics and learning design 12
  13. 13. Learning design in MOOCs ● Puts the learning journey at the heart of the design process ● Provides a set of tools and information to support a learner- activity based approach ● Helps to show the costs and performance outcomes of design decisions ● Enables the sharing of best practice ● Helps MOOC designers to choose and integrate a coherent range of media, technologies and pedagogies ● Enables a consistent and structured approach to review and analytics 13 Mor, Y, Ferguson, R, & Wasson, B. (2015). Editorial: learning design, teacher inquiry into student learning and learning analytics: a call for action. British Journal of Educational Technology, 46(2), 221-229.
  14. 14. MOOC learning design tools • MOOC design template • MOOC planner • MOOC map • Journey planner 14
  15. 15. Design template analytics 15 Learning outcome How this is assessed 1. Be able to define an ecosystem. 2. Have joined the iSpot community and obtained identifications for animals, plants or fungi. 1. Multiple choice. Week 1, question 5 2. Self report. Analytics 1. How many attempted that question? How many got it right 1st / 2nd / 3rd time? How many followed the link back to resources? 2. Access to iSpot data. Use of MOOC hashtag. Persistence over time. Ethical implications of tracking off-site. Short description of course and learning outcomes
  16. 16. MOOC planner • Delivered • Reflection • Collaboration • Conversation • Networking • Browsing • Assessment 16 Blocking out types of learning activity Conole, Gráinne. (2010). Learning design – making practice explicit. Paper presented at ConnectEd, Sydney, Australia.
  17. 17. MOOC planner analytics Delivered Content (reading, watching, listening and observing) Analytics: amount of content viewed, dwell time Reflection (thinking, considering and reflecting) Analytics: returns to the same material, reflection exercises completed, quality of reflection Collaboration (constructing, collaborating, defining and engaging) Analytics: collaboration exercises completed, quality of collaboration Conversation (debating, arguing, questioning, discussing…) Analytics: number and length of contributions, quality of discussion Browsing (exploring, searching, finding and discovering) Analytics: Number of click-throughs to external links, number of visits, number of resources Assessment (answering, presenting, demonstrating, critiquing…) Analytics: Assessments completed, scores, dwell time on hints, persistence in answering questions 17
  18. 18. MOOC map analytics ● How long did you expect learners to spend on these key elements? ● How long did learners actually spend on the key elements ● How many missed out these elements? ● How many jumped ahead to these elements? ● Which types of element are consistently (un)popular? ● How many left the MOOC at these points? 18 The MOOC map identifies key elements of the course 0 100 200 300 400 500 600 Assimilative InformationHandling Productive Experiential Adaptive Communicative Assessment Organisation Minutes
  19. 19. MOOC journey planner analytics 19 Relationships between tools, resources, activities & narrative A framework for data collection
  20. 20. Analytics to solve problems Analytics could filter discussions or group learners 20 You have been actively engaged in the discussions, which is excellent, thank you, but with more than 23,000 participants it means that our responses and comments risk getting lost. This will be primary school material for some of you and exactly the opposite for others. It is just not possible to tailor the material to each of you […] Introduction to Forensic Science: University of Strathclyde
  21. 21. Start with the pedagogy • How do people learn? • How can we use data to facilitate that process in our MOOC? • Which elements are learners struggling with? • Which sections engage them the most? • What prompts them to ask questions? • Are they finding assessment challenging? • What misconceptions have learners shown? • Are there any accessibility issues? • How can analytics be used to obtain desired learning outcomes? 21
  22. 22. Learning analytics and design Learning design – helping to identify useful analytics ● What do learners need to know in order to – network, collaborate, browse or reflect? ● What do educators need to know to support them? Learning design – helping to identify gaps in the data ● What data do we need to collect? Learning design – helping to identify gaps in our toolkit ● Which design elements can we look at easily? ● Which ones still pose problems? Learning design – helping to frame & focus analytics questions ● What did they learn?… in relation to learning outcomes ● Were they social?... when they were collaborating ● Did they share links?... when encouraged to browse ● Did they return to steps?... when encouraged to reflect 22 Making the links
  23. 23. Workshop activity 23
  24. 24. Learning outcomes 24 Make a note of the learning outcomes in these two videos Promotional video for The University of Southampton’s FutureLearn MOOC: Agincourt 1415: Myth and Reality You can currently access this without registering at
  25. 25. Learning outcomes By the end of the course, learners should know: ●Something about medieval weaponry from the period ●How important the long bow was at the battle of Agincourt ●How important gunpowder weaponry was at the time ●Why the battle still means so much today ●How historians operate ●What the Hundred Years War was ●Whether Henry V had a legitimate claim to the French crown ●Why Henry launched the campaign when he did ●How the soldiers were recruited, transported and organised ●What is meant by the ‘Southampton plot’ ●Whether their ancestors served on the campaign 25 FutureLearn MOOC from University of Southampton
  26. 26. Reflecting, discussing 26 Shakespeare is to blame Being French, we were taught of course about the battle of Agincourt although not in the same way as English students have been! :) it's the difference between myth and reality. I do Medieval re-creation in the SCA and my persona is that of a historian who lived only a hundred years after the Battle. For me it's like being interested in ancient sport reviews, as if the hundred years war was like the six nations tournament of medieval times! I am an archer My 17th great grandfather served as a man at arms in the retinue of the Duke of Gloucester at Agincourt. Agincourt was presented to us as the epitome of britishness
  27. 27. My ancestor was easily found Exploring, searching, finding 27 Screenshots from The University of Southampton’s FutureLearn MOOC: Agincourt 1415: Myth and Reality You can currently access this without registering at I think my family would have been on the other side!
  28. 28. Debating, arguing 28 Screenshots from The University of Southampton’s FutureLearn MOOC: Agincourt 1415: Myth and Reality You can currently access this without registering at
  29. 29. Thinking, considering, reflecting 29 Screenshots from The University of Southampton’s FutureLearn MOOC: Agincourt 1415: Myth and Reality You can currently access this without registering at
  30. 30. Why the battle still means so much Delivered Content (reading, watching, listening and observing) Ask learners to read a recent article on the subject Reflection (thinking, considering and reflecting) Ask ‘What reasons are given for the importance of Agincourt?’, and ask them to watch a video of the recent commemoration event Collaboration (constructing, collaborating, defining and engaging) Ask learners to put together their own reasons for considering that the battle is important Conversation (debating, arguing, questioning, discussing…) Ask learners to discuss an audio recording about the battle Browsing (exploring, searching, finding and discovering) Point learners to some key resources and ask them to develop a list of reasons for the importance of the battle Assessment (answering, presenting, demonstrating, critiquing…) Ask learner to post a list of the five main reasons for the battle’s importance, and then comment on another learner’s list 30
  31. 31. Group activity 15 minutes Take the learning outcomes of the Agincourt MOOC and briefly outline a different type of learning activity for each one (delivered, reflection, conversation, collaboration, browsing or assessment). 31
  32. 32. Types of learning behaviour Delivered Content reading, watching, listening and observing Reflection thinking, considering and reflecting Collaboration constructing, collaborating, defining and engaging Conversation debating, arguing, questioning, discussing… Browsing exploring, searching, finding and discovering Assessment answering, presenting, demonstrating, critiquing… 32
  33. 33. Example analytics questions Read a recent article Is the article engaging learners? How long did they spend on it? How many skipped it? Reflect on a video of a commemoration event Are learners engaging in behaviours that indicate they are reflecting? Did they return to the video? Did they pause the video? Put together reasons why the battle is important Does this work as a discusssion activity? How many engaged? How detailed were the comments? Were they key words in the discussion the same as key words in the previous video? Develop a list of reasons for the importance of the battle What misunderstandings do I need to address? What reasons did they identify? Did they miss the importance of any reasons? Post a list of five reasons, and then comment What type of general feedback can I provide as a model for this activity in future? What were common mistakes? What did they generally get right? 33
  34. 34. Group activity 15 minutes For each of your learning activities, identify questions you would like answered in order to improve learning and teaching on the MOOC What types of data would help you to answer these questions? 34