Learning Analytics for MOOCs: EMMA case

PhD Candidate, School of Digital Technologies,Tallinn University
Feb. 19, 2016

More Related Content

Similar to Learning Analytics for MOOCs: EMMA case(20)


Learning Analytics for MOOCs: EMMA case

  1. Maka Eradze, Kairit Tammets University of Naples Federico II/Tallinn University Data Science and Social Research conference, Naples, Italy, 19.02.2015
  2. DEFINITIONS • Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs. • LA as a technology-enhanced learning (TEL) research area that focuses on the development of methods for analyzing and detecting patterns within data collected from educational settings, and leverages those methods to support the learning experience. (Chatti et al 2015)
  3. LA, ITS CHALLENGES AND FUTURE DIRECTIONS Learning analytics is inherently intidisciplinary • Operates on the verge of many different fields Still in its infancy • It still has to establish itself as field with its own traditions and theoretical frameworks
  4. • Two systematic literature reviews found: • Nistor et al(2015): The main issue with LA research is the frequent lack of an explicit theoretical framework from educational perspective. The authors call on the educational and psychological theories for significant progress of upcoming LA research • Gasevic et al(2015): researchers from educational sciences tend to publish more mature research and show more rigor • Fortunately, the field is evolving quickly and one measure of this growth is the increasing breadth of methodologies being applied to learning‐related data sources (Pardo et al, 2014)
  5. • Big learning analytics that come from heterogeneous sources • open learning environments (MOOCs), with data coming from outside of platform. • Mobile learning analytics • Context modeling • Privacy-aware analytics. • Personalized learning analytics. • Life-long learner modeling • Learning analytics for open assessment. • Embedded Learning analytics • Learning analytics design patterns. • Learning analytics evaluation. Chatti, M. A., Lukarov, V., Thüs, H., Muslim, A., Yousef, A. M. F., Wahid, U., ... & Schroeder, U. (2014). Learning Analytics: Challenges and Future Research Directions.
  6. FACTORS Different factors are involved in LA Greller and Draschler
  7. From Chatti
  8. MOOCS AND LA • Big data makes it possible to scale up learning • MOOCs make it possible to open up learning • “Teaching crowds”* needs new pedagogical models and new ways to support the process • Learning analytics can make it possible to help design, teach and analyse the processes of “teaching crowds” * Dron and Anderson
  9. LEARNING ANALYTICS IN MOOCS: CHALLENGES AND OPPORTUNITIES Challenge of Effectiveness ● According to Clow (Clow 2013)* most of the MOOC analytics develop around the formal education context and questions surrounding predictive modeling, that is problematic in MOOCs. ● Chatti and Clow agree that LA analytics for MOOCs for now is rather limited ● A focus on the perspectives of learners in MOOCs has the potential to extend LA beyond completion rates and interventions to support personalization, feedback, assessment, recommendation, awareness, and self-reflection ● Facilitate individual decision making process *Clow, D. (2013, April). MOOCs and the funnel of participation. In Proceedings of the Third International Conference on Learning Analytics and Knowledge (pp. 185- 189). ACM.
  10. LEARNING ANALYTICS IN MOOCS: CHALLENGES AND OPPORTUNITIES Business Models – opportunities ● Another and less well publicized MOOC business model is dependent on “big data” or data analytics that can provide indirect financial advantage. * ● Also, the utilisation of data analytics can improve the accuracy of decision making, for instance by identifying markets in which to target specific advertising campaigns* Burd, E. L., Smith, S. P., & Reisman, S. (2015). Exploring business models for MOOCs in higher education. Innovative Higher Education, 40(1), 37-49. *Loukis, E., Pazalos, K., & Salagara, A. (2012). Transforming e-services evaluation data into business analytics using value models. Electronic Commerce Research and Applications, 11(2), 129-141.
  11. LEARNING ANALYTICS IN MOOCS: CHALLENGES AND OPPORTUNITIES Technology and pedagogy – challenges and opportunities (Learning Analytics in Open Learning Environments (e.g. MOOCs) ● Incredible amounts of data for retrospective analysis ● Much of this data goes beyond platforms ● xMOOCs are seen as centralized courses, with predetermined paths and closed systems - analytics would differ for these platforms ● cMOOCs are open ended, distributed and go beyond the platforms ● Learning analytics that focuses on learners, can provide them much needed personalized feedback
  12. EMMA is • EU project (30 month CiP) • It is a platform and an authoring system (LMS +CMS) to provide hosting space for universities and cultural institutions to experiment with MOOCs + (aggregation) • EMMA integrates existing technological elements to create a pan- European platform that offers opportunities for multicultural and multilingual learning: • UNINA Federica platform; UPV ASR + MT systems; CSP tracking; TLU methodology + dashboards; IPSOS surveys • Use learning analytics to create a feedback loop (for learners, providers and developers)
  13. EMMA LEARNING ANALYTICS FRAMEWORK LA in EMMA focuses: 1. On real time analytics through learning analytics dashboards for instructors and students – feedback loop 2. Retrospective analytics of the digital traces in EMMA platform – evidence based learning design 3. Is built on the Greller and Draschler framework that LA has a great potential as a powerful tool to inform learners and educators From Greller and Draschler
  14. XAPI Event-driven analytics <Actor (learner)> <verb> <object>, with <result>, in <context>
  15. EMMA LA AND XAPI ● The Experience API is a service that allows for statements of experience (typically learning experiences, but could be any experience) to be delivered to and stored securely in a Learning Record Store ● The Experience API is dependent on Learning Activity Providers to create and track learning. ● Learning Activity Provider is a software object that communicates with the LRS to record information about the learning experience. ● Learning activity is a unit of instruction, experience or performance that has to be tracked. ● A Statement consists of <Actor (learner)> <verb> <object>, with <result>, in <context> to track an aspect of a learning experience.
  16. EMMA LA AND XAPI ● It repeats the structure of the sentence in practically any language and is human readable • Actor data is a unique information that describes a specific subject, such as a student or group of students. • Verb data classifies the type of activity the actor participated in and often links to a human readable description of the event. • Object data will link to an artifact that is typically a byproduct of or related to the activity
  17. EMMA LA AND XAPI ● It is much more, than just a simple sentence-like structure. The context is very important and it makes possible to relate one activity to another* • Quoting Verbert, Suthers and Rosen, Kevan and Ryan believe that the issue of variety of educational data taxonomies and also the distributed learning events collection challenges can be solved by xAPI* ● Some researchers believe that it is aligned with constructivist approach and activity theory ● xAPI has a lot of potential for The xAPI offers a renewed opportunity to research, develop, and explore theories involving learning beyond academia’s digital environments. Kevan, J. M., & Ryan, P. R. Experience API: Flexible, Decentralized and Activity-Centric Data Collection.Technology, Knowledge and Learning, 1-7.
  18. EMMA learning analytics technical description built-in tracking system that collects the data about users’ interactions ● The set of users’ interactions making up learning experience statements in xAPI format are sent to Learning Record Store (LRS), which is used for storing learning experiences of EMMA courses ● Some of the data (e.g. the structure of the MOOC and start and end date of the course) to be visualized by the dashboard was retrieved from ● EMMA database with the support of web service ● implementation of visualizations is mainly based on Highcharts charts’ framework ● Social network analysis graph is developed by using Sigma.js
  19. Dashboards: Facilitator support: ● Overview of the course activities against the lessons of the MOOC ● Overview of the different interactions under the units and lessons ● Enrollments and unenrollments ● Activity streams ● Social network analysis
  20. Dashboards: Learner support ● Progress compared with different lessons - how many units under each lesson have been accessed by the learner, assignments submitted with what result and participated ● Enrollments ● Activity stream of the recent activities – 100 latest activities of the MOOC participants will be visualized as stream; ● Social network analysis based on comments and responses in conversation module of the MOOC ● Overview of the popular resources and suggestions to access materials that other participants have accessed, but this certain learner has not yet.
  21. Clustering the participants of MOOCs In the first phase, participants were clustered as: enrolled, observers and contributors. As in the second phase, there were a bit more interactions, the following clustering scheme was used: • Enrolled – participant entered the MOOC up to five times; • Observer – participant entered the MOOC more than five times, but did not interact with the content or other participants; • Contributor – participant contributed with the assignment, comment or post to the MOOC at least once; • Active – participant contributed with the assignment, comment or post to the MOOC more than once • Note: This sample does not include one course that had the greatest success and highest enrollment rates: Coding in your classroom, NOW!
  22. MOOC Enrolled Observer Contributor Active Computer support inquiry 29,3% 29,3% 40% 1,4% Business Intelligence 45,6% 22,2% 10% 22,2% Developing Blended Learning 44% 41,4% 8,9% 5,7% E-learning 68,2% 27,4% 3,2% 1,2% Lisbon and the Sea: a Story of Arrivals and Departures, 42,4% 28% 20,3% 9,3% The Organization of Cultural Enterprises 66% 10,7% 10,7% 12,6% General and Social Pedagogy 58,8% 27,5% 7,8% 5,9% Social Innovation and cultural Heritage 55,5% 28,7% 9,9% 5,9% Mobile devices in your everyday life 26,4% 59,5% 9% 5,1% Climate changes: The Context of Life Experience 46,9% 22,2% 16% 14,9% Open Wine University 36,4% 12,4% 21,4% 29,8% E-Portfolio self -development study 56,4% 34,9% 6,4% 2,3% Excel 62,8% 16,6% 7,5% 13,1% Search on the internet 62,5% 14,1% 9,3% 14,1%
  23. Some conclusions • This indicates that the majority of Mooc participants just enrolled in the courses or observed the content. • One MOOC seems like an outlier with over 50% participants who contributed to the course. • TLU course Computer-Supported Inquiry had more than 40% of participants who interacted with the platform • Reasons • technical – some participants did not find the functionalities or the course assumed using blogs for interactions which is not the part of learning analytics • Selected pedagogical design – no interaction require
  24. Social network analysis is based on the statements related with commenting responding and replying. Example 1 Based on the figure, it is possible to assume that 1. the course had more than one facilitator and that some of the participants were really active commentators. 2. the direction of the communication is not only from learner to facilitator, but participants interact with each other quite a lot.
  25. Example 2 We can see that 1. the course pedagogical design affects the interactions within the course. In that course the students were supposed to publish some of their tasks with the conversation tool, so they responded to the facilitator’s request with their contribution. 2. It seems that participants did not perceive others’ contributions relevant enough, so they did not comment or answer to nearly no comment. each other quite a lot.
  26. Interactions with content The aim of the current analysis was to find out how participants engage with the content during the course. indicates that the amount of interactions and minutes spent on content were slowly decreasing after each lesson.
  27. • illustrates how in every second lesson there was an increasing trend of interactions and also then people spent more time on the content. • Course design may indicate that the participants were supposed to do something more or extra within course settings, also it may
  28. CONCLUSIONS • Pedagogical neutrality - based on data that was analysed, it can be said that EMMA platform supports different pedagogical designs of MOOC. • MOOC could be xMOOC, which focuses on content consuming and could be also a cMOOC where participants actively communicate and construct new knowledge together.
  29. FUTURE WORK • Deeper insights into learning processes and uptake of knowledge - • Learning analytics dashboards evaluation finding out in which way it supports the learning experience in EMMA platform and also how MOOC facilitators plan their pedagogical interventions of MOOCs to next iterations of the course: how do they make sense of the data and how it is integrated to course design process. • Learning analytics data to be combined with the data of the participants surveys in the platform: what expectations do they have when they enter to the MOOC and what is their learning path during the course
  30. FUTURE WORK FROM WIDER PERSPECTIVE • The development of the EMMA learning analytics research, framework and implementation also depends on the developments in the field theory and shared experiences from different projects using similar approaches. • EMMA LA data translates into policy indicators for a common European MOOC model • The issues of data-standardization, data collection and analysis that goes beyond the platform, specification and use of particular verbs are the issues that also influence the EMMA platform LA developments.
  31. Contact: