This document discusses the value and challenges of learning analytics in MOOCs. It notes that while learning analytics can provide useful insights if implemented carefully and ethically, there are also risks of drawing incorrect correlations without context or participant consent. The document advocates defining goals and metrics upfront to ensure privacy and addresses stakeholders' objectives of reflection and prediction. It provides examples of indicators and dashboards that could help optimize learning for both teachers and learners.

1. Is there value of learning analytics in
MOOCs?
Data Science & Social Research International Conference
Naples, 19 February 2016
Francis Brouns, Olga Firssova, Marco Kalz

2. Learning analytics: hype?
• Big data, learning analytics: HYPE
• Correlations between data without meaning,
cause or relation
DSSR conference, Naples, 19 February 2016
Page 2

3. Patterns and correlations
• Green, stem, leaf: tree, plant, cocktail,
book?
DSSR conference, Naples, 19 February 2016
Page 3

4. Ethical concerns
• Participant might not be aware that data is being used, nor
how and to what purpose
• Ownership
DSSR conference, Naples, 19 February 2016
Page 4

5. What is learning analytics?
Learning analytics is the measurement,
collection, analysis and reporting of data about
learners and their contexts, for purposes of
understanding and optimising learning and
the environments in which it occurs
https://solaresearch.org/about/
DSSR conference, Naples, 19 February 2016
Page 5

6. Need context
• Stakeholders
• Objectives: Prediction, Reflection
• Norms and procedures
• Protection of data
DSSR conference, Naples, 19 February 2016
Page 6

7. Need context
DSSR conference, Naples, 19 February 2016
Page 7
Greller, W, & Drachsler, H. (2012) Translating Learning into Numbers: Toward a Generic Framework for
Learning Analytics. ET&S, 15(3), 42-57

8. DSSR conference, Naples, 19 February 2016
Page 8

9. ECO MOOC
• Online courses designed for large numbers of
participants,
• That can be accessed by anyone
• Anywhere as long as they have an internet
connection,
• Are open to everyone without entry
qualifications,
• And offer a full/complete course experience
online for free
DSSR conference, Naples, 19 February 2016
Page 9

10. ECO sMOOC
• Social, networked learning
• Seamless, ubiquitous learning
• Inclusiveness
DSSR conference, Naples, 19 February 2016
Page 10

11. Learner is central
• Takes an active role
• Responsible for their own learning
• Support learning community
• Knowledge through reflection and practice
(creation, production) and dialogue in a
social collaborative context
• Success by learners’ goals and intentions
DSSR conference, Naples, 19 February 2016
Page 11

12. EMMA Learning analytics model
• Progress and performance
• Types of users
• Learning by engaging with product and/or
with others
DSSR conference, Naples, 19 February 2016
Page 12

13. MOOC success
• Drop-out?  NO
• Individual learning goals
• Interaction, conversations, community
DSSR conference, Naples, 19 February 2016
Page 13
Exploring
learners
Auditing
learners
Completing
learners

14. Learning analytics purpose
DSSR conference, Naples, 19 February 2016
Page 14
Reflection Prediction

15. Define
• Progress: movement towards the
achievement of a goal
• Performance: the degree of understanding
and mastery of course material
• Mastery: degree of improvements
DSSR conference, Naples, 19 February 2016
Page 15

16. Define
• Participation: process during which
participants are consulted about or have the
opportunity to become actively involved in a
course or activity of the course
• At risk of drop-out are those participants with
a remarkable low participation, progress and
performance with relation to those who have
got the same goals
DSSR conference, Naples, 19 February 2016
Page 16

17. Academic goals and metrics
• Define and describe academic and learner
goals
• Indicate learning design components
• Define metrics
DSSR conference, Naples, 19 February 2016
Page 17

18. Example
• Performance: variety of indicators
measuring ability
e.g. compulsary activities and assignments
passed successfully
• Metrics
– Number of assignments passed
– Grade achieved in each assignment
DSSR conference, Naples, 19 February 2016
Page 18

19. Algorithms
• Personal grade in module
• The final grade will be calculated as
DSSR conference, Naples, 19 February 2016
Page 19

20. Indicators
• Performance
• Mastery
• Progress
• Engagement
• Effort
• Satisfaction
• Social affiliation
• Social recognition
• Social responsibility
DSSR conference, Naples, 19 February 2016
Page 20

21. Learner dashboard
DSSR conference, Naples, 19 February 2016
Page 21

22. Teacher dashboard
DSSR conference, Naples, 19 February 2016
Page 22

23. Teacher’s view
DSSR conference, Naples, 19 February 2016
Page 23
Resources
viewed
No of
learners
accessed

24. Learner’s view
DSSR conference, Naples, 19 February 2016
Page 24
Activity per day
Performance compared
to others

25. Number and duration of interactions
DSSR conference, Naples, 19 February 2016
Page 25

26. Accessing learning content and
assigments
DSSR conference, Naples, 19 February 2016
Page 26

27. Enrollment history
DSSR conference, Naples, 19 February 2016
Page 27

28. Conclusion
• Value when carefully designed and used
• Define model first: what do you want to know and why
• Ensure privacy
• Ethical concerns
• Legal aspects
DSSR conference, Naples, 19 February 2016
Page 28

29. Further information
• Francis Brouns, Open Universiteit, Welten Institute,
http://welteninstitute.org
• Welten Institute digital repository: http://dspace.ou.nl
• ECO project: http://ecolearning.eu
• EMMA project: http://platform.europeanmoocs.eu
DSSR conference, Naples, 19 February 2016
Page 29