Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How educators value data analytics about their moocs (1)


Published on

Michos, K., Hernández-Leo, D., Jiménez, M., (2017) How educators value data analytics about their MOOCs, CEUR Proceedings of Work in Progress Papers of the Experience and Research Tracks and Position Papers of the Policy Track at EMOOCs 2017 co-located with the EMOOCs 2017 Conference (Vol-1841), Madrid, Spain, 77-82.

Published in: Engineering
  • Be the first to comment

How educators value data analytics about their moocs (1)

  1. 1. • Divergent opinions about the usefulness and understandability of the course report: activity of students by step and responses in the quizzes and tests. • Students´ satisfaction and comments by step were among the most useful information for the presented monitor goals as in [2]. How educators value data analytics about their MOOCs Introduction Results Research focus Methods Conclusions and future work Konstantinos Michos, Davinia Hernández-Leo & Manel Jiménez {kostas.michos, davinia.hernandez-leo, manel.jimenez} ICT Department, Communication Department, Universitat Pompeu Fabra, Barcelona References 1. Dyckhoff, A. L., Lukarov, V., Muslim, A., Chatti, M. A., & Schroeder, U.: Supporting action research with learning analytics. In Proceedings of the Third International Conference on Learning Analytics and Knowledge (pp. 220-229). ACM (2013). 2. Stephens-Martinez, K., Hearst, M. A., & Fox, A.: Monitoring MOOCs: Which information sources do instructors value?. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 79-88). ACM (2014). • Data analytics provided to educators in MOOCs: e.g. students´ profiles, students´ online behavior, students´ satisfaction with the course. • Supporting the role of educators, their decisions and their actions based on the data analytics [1]. • Learning design & Data anaytics . • Survey study with closed and open-ended questions. • 4 Educators who developed two editions of three different MOOCs in UPF. RQ1: Which information sources and visualizations (from FutureLearn reports) are most useful for MOOC educators? RQ2: What information sources (from FutureLearn reports) help MOOC educators to identify problems and potential improvements in a redesign of the course? • Problem: How do educators value the different sources of data and how do they connect them with their course re-design? • FutureLearn case – Summary Reports about: a) pre-course survey b) course report c) satisfaction surveys. • Questions about the usefulness and understandability of the different provided data analytics from their own MOOCs. Connection with monitoring goals as in [2]: 1. Problems with the activities-tasks. 2. Struggling students and what they are struggling with. 3. Difficulty of the grading activities. 4. Appropriateness of course difficulty level for students. 5. Engaging content for students. 6. Most difficult part of the course. 7. Improving the presentation of a topic. 8. Least interesting content for students. 22 22 28 22 28 22 25 19 41 28 41 47 59 44 34 38 34 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% a. Previous experiences of students b. Enrollement cummulative growth c. Activity by step d. Comments by step e. Quizzes and tests f. Students' satisfaction Percent of educators’ responses (n = 32) Informationsources Usefulness of the information sources for all the monitoring goals Neutral Agree Strongly agree Not applicable 29 29 25 29 29 25 25 13 4 13 8 4 17 33 25 38 29 33 21 13 42 29 58 29 29 33 38 63 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1. Problems with the activities-tasks 2. With what students struggle 3. Difficulty of the grading activities 4. Appropriatness of course difficulty 5. Engaging content for students 6. Most difficult part of the course 7. Improving the presentation of a topic 8. Least interesting content Percent of educators’ responses (n= 24) Monitoringgoals Usefulness of the information sources for each of the monitoring goals Neutral Agree Strongly agree Not applicable • Εducators redesign actions based on their availble data: simplification of the content. Interventions in specific parts of the course. Redesigned quizzes and tests based on the levels of shown difficulty. • Final decissions were determined after discussion between the groups of educators and the FutureLearn platform. • Proposal for interaction analysis and words in sentiment analysis. • Preliminary results indicated that educators perceived specific data analytics particularly useful, with some divergences in their opinions. • Difficulty to connect the information sources with the monitoring goals. • Extend the study with more educators. • How topics, learning design and audience characteristics influence the educators. • Monitoring goals defined by educators before their course. • Study in detail the educators’ actions based on the presented data analytics. • Take in to account educators’ divergent needs in the design of MOOC dashboards.  During the course: monitoring, interventions, dynamic changes in the course design, prompts in the forums.  After the course: reflection on what happened to improve re- runs of the course. Decisions in design-time based on past cohorts of MOOC students