More Related Content

Slideshows for you(20)

Similar to EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links(20)


More from EUmoocs (20)


EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links

  1. Learning design and learning analytics: building the links EMMA Summer School, Ischia, July 2015
  2. Rebecca Ferguson • The Open University: largest in UK • Informal learning: iTunes, YouTube… • MOOCs on FutureLearn, OpenLearn and elsewhere • Making use of big data for more than 40 years • Learning analytics research / events • LACE project 2 Lead on MOOC evaluation at The Open University, UK
  3. Workshop overview 09.45 Introduction Linking learning analytics, learning design and MOOCs 10.05 Group work Focusing on learning outcomes in MOOCs 10.25 Plenary Discussing how these data might be used to support learning 10.40 Group work Analytics, step by step 11.00 Plenary What would MOOC learners and educators need to know to use these analytics? 11.15 Workshop end 3 You can view and download these slides at
  4. What are learning analytics? High-level figures Brief overviews for internal and external reports Academic analytics Figures on retention and success, for the institution to assess performance Learning analytics Use of big data to provide actionable intelligence for learners and educators 4
  5. Educators use analytics to • Monitor the learning process • Explore student data • Identify problems • Discover patterns • Find early indicators for success • Find early indicators for poor marks or drop-out • Assess usefulness of learning materials • Increase awareness, reflect and self reflect • Increase understanding of learning environments • Intervene, supervise, advise and assist • Improve teaching, resources and the environment 5 Dyckhoff, A L, Lukarov, V, Muslim, A, Chatti, M A, & Schroeder, U. (2013). Supporting Action Research with Learning Analytics. Paper presented at LAK13.
  6. Learners use analytics to • Monitor their own activities and interactions • Monitor the learning process • Compare their activity with that of others • Increase awareness, reflect and self reflect • Improve discussion participation • Improve learning behaviour • Improve performance • Become better learners • Learn! 6 Dyckhoff, A L, Lukarov, V, Muslim, A, Chatti, M A, & Schroeder, U. (2013). Supporting Action Research with Learning Analytics. Paper presented at LAK13.
  7. Analytics example: UK schools 7 • Aligned with clear aims • Huge and sustained effort • Agreed proxies for learning • Clear and standardised visualisation • Driving behaviour at every level BUT • Stressed, unhappy learners • Analytics with little value for learners or educators • Omission of key areas, such as collaboration
  8. Analytics example: Course Signals Developed at Purdue University 8 Arnold, K E, & Pistilli, M (2012). Course Signals at Purdue: Using Learning Analytics To Increase Student Success. Paper presented at LAK12, Vancouver, Canada.
  9. Analytics example: SNAPP Network analysis 9
  10. Analytics example: iSpot Heading 10
  11. Learning design in MOOCs ● Puts the learning journey at the heart of the design process ● Provides a set of tools and information to support a learner- activity based approach ● Helps to show the costs and performance outcomes of design decisions ● Enables the sharing of best practice ● Helps MOOC designers to choose and integrate a coherent range of media, technologies and pedagogies ● Enables a consistent and structured approach to review and analytics 11 Mor, Y, Ferguson, R, & Wasson, B. (2015). Editorial: learning design, teacher inquiry into student learning and learning analytics: a call for action. British Journal of Educational Technology, 46(2), 221-229.
  12. MOOC learning design tools • MOOC design template • MOOC planner • MOOC map • Journey planner 12
  13. Design template analytics 13 Learning outcome How this is assessed 1. Be able to define an ecosystem. 2. Have joined the iSpot community and obtained identifications for animals, plants or fungi. 1. Multiple choice. Week 1, question 5 2. Self report. Analytics 1. How many attempted that question? How many got it right 1st / 2nd / 3rd time? How many followed the link back to resources? 2. Access to iSpot data. Use of MOOC hashtag. Persistence over time. Ethical implications of tracking off-site. Short description of course and learning outcomes
  14. MOOC planner • Delivered • Reflection • Collaboration • Conversation • Networking • Browsing • Assessment 14 Blocking out types of learning activity Conole, Gráinne. (2010). Learning design – making practice explicit. Paper presented at ConnectEd, Sydney, Australia.
  15. MOOC planner analytics Delivered Content (reading, watching, listening and observing) Analytics: amount of content viewed, dwell time Reflection (thinking, considering and reflecting) Analytics: returns to the same material, reflection exercises completed, quality of reflection Collaboration (constructing, collaborating, defining and engaging) Analytics: collaboration exercises completed, quality of collaboration Conversation (debating, arguing, questioning, discussing…) Analytics: number and length of contributions, quality of discussion Browsing (exploring, searching, finding and discovering) Analytics: Number of click-throughs to external links, number of visits, number of resources Assessment (answering, presenting, demonstrating, critiquing…) Analytics: Assessments completed, scores, dwell time on hints, persistence in answering questions 15
  16. MOOC map analytics ● How long did you expect learners to spend on these key elements? ● How long did learners actually spend on the key elements ● How many missed out these elements? ● How many jumped ahead to these elements? ● Which types of element are consistently (un)popular? ● How many left the MOOC at these points? 16 The MOOC map identifies key elements of the course 0 100 200 300 400 500 600 Assimilative InformationHandling Productive Experiential Adaptive Communicative Assessment Organisation Minutes
  17. MOOC journey planner analytics 17 Relationships between tools, resources, activities & narrative A framework for data collection
  18. Analytics to solve problems Analytics could filter discussions or group learners 18 You have been actively engaged in the discussions, which is excellent, thank you, but with more than 23,000 participants it means that our responses and comments risk getting lost. This will be primary school material for some of you and exactly the opposite for others. It is just not possible to tailor the material to each of you […] Introduction to Forensic Science: University of Strathclyde
  19. Start with the pedagogy • How do people learn? • How can we use data to facilitate that process in our MOOC? • Which elements are learners struggling with? • Which sections engage them the most? • What prompts them to ask questions? • Are they finding assessment challenging? • What misconceptions have learners shown? • Are there any accessibility issues? • How can analytics be used to obtain desired learning outcomes? 19
  20. Learning analytics and design Learning design – helping to identify useful analytics ● What do learners need to know in order to network, collaborate, browse or reflect? ● What do educators need to know to support them? Learning design – helping to identify gaps in the data ● What data do we need to collect? Learning design – helping to identify gaps in our toolkit ● Which design elements can we look at easily? ● Which ones still pose problems? Learning design – helping to frame & focus analytics questions ● What did they learn?… in relation to learning outcomes ● Were they social?... when they were collaborating ● Did they share links?... when encouraged to browse ● Did they return to steps?... when encouraged to reflect 20 Making the links
  21. Group activity 20 minutes Visit FutureLearn and register if necessary Select a MOOC currently open for registration. Look at the introductory material and the first week – What are the expected learning outcomes? How will learners know they have achieved these? 21
  22. Plenary 15 minutes What types of learning outcome are specified? In what ways are learners assessed or could they be assessed? What sorts of data and analytics could be used to support learners? What sorts of data and analytics could be used to support educators? 22
  23. Group activity 20 minutes Take the first week of your chosen MOOC Classify each step in terms of learning activity: (delivered content, reflection, collaboration, conversation, browsing or assessment) Note how long the step would be likely to take Discuss which types of analytic would be useful to learners and to educators 23
  24. Plenary 15 minutes Share your findings Which types of data and analytic could be used to support these types of learning activity? What would the educators need to know? What would the learners need to know? 24