The keynote by Prof. Dr. Paul A. Kirschner explores the dichotomy of learning analytics as a utopia or dystopia, emphasizing the importance of integrating learning sciences with analytics. It discusses potential pitfalls such as myopic visions of learning, invalid variables, and spurious correlations, alongside the promise of personalized learning and improved educational environments. The document underscores the necessity of meaningful engagement with data to enhance student outcomes effectively.

1. Learning Analytics: Utopia or Dystopia
Prof. dr. Paul A. Kirschner
Distinguished University Professor
Open University of the Netherlands

2. Thanks

3. Disclaimer
This keynote is not about:
• Privacy issues / Data protection / Big Brother
• Ethical aspects / questions
• Legal aspects / questions
• Commercial interests / Business optimisation
• Technology, Techniques, Dashboards, etcetera
• …
To paraphrase Bill Clinton: It’s the learning, people!

11. Your task Mr. Phelps – Mission Impossible?

12. Learning

14. National Tsing Hua Univerity
Learning
Sciences

16. Learning Analytics Model (Siemens, 2013)
(Siemens, 2013)

19. Learning Analytics Model
(Siemens, 2013)
Dystopia 1: Myopic vision of what learning is

20. Learning Analytics Model
(Siemens, 2013)

21. Missing link?

22. Learning Sciences theory as missing link
• Variables to include in a model
• Potential confounds, subgroups, or covariates in the data
• Which results to attend to
• Framework for interpreting results
• How to make results actionable
• Generalisation of results to other contexts and populations

24. Not only true of the US elections
Dystopia 2: Theory Free / Theory Poor LA

25. What are we looking for?

26. Meaningful variables?
Dystopia 3: Looking at wrong or invalid variables

27. Dystopia 4: Seeing Correlation as Causal

28. Spurious correlations
http://tylervigen.com/spurious-correlations
http://tylervigen.com/discover

31. Unintended consequences
http://interactioninstitute.org/unintended-consequences/
Dystopia 5: Unintended and unwanted effects

32. Goal orientation
http://buehlereducation.com/pedagogy-assessment/goal-
orientation-theory-and-education/

33. Pigeonholing / Profiling / Stereotyping

39. Predict
Utopia 1: Knowing what will happen (and when and why)

40. Simple data
Dietz-Uhler & Hurn (2013). Using Learning Analytics to Predict
(and Improve) Student Success: A Faculty Perspective

42. Adapt / Personalise?
Utopia 2: Custom tailored learning and instruction

43. Learning / study strategies
Summarise
Highlight/underline
Elaborative questions
Restudy
Generate keywords
Distribute practise
Variable practise
Visualise
Explain
(Self) Testing
Dr. Gino Camp – Welten Institute, OUNL

44. Variability of practise
a‐a‐a‐b‐b‐b‐c‐c‐c‐d‐d‐d versus a‐b‐c‐d‐a‐b‐c‐d‐a‐b‐c‐d
Ten steps to complex learning – Van Merriënboer & Kirschner

47. Recommend / Advise / Intervene
Utopia 3: The right thing for the right learner at the right time

48. Yong Zheng, Center for Web Intelligence, DePaul University
Navisro Analytics: Collaborative Filtering and Recommender Systems

49. Classification system*
* Drachsler et al., Panorama of Recommender Systems to Support Learning

50. Feedback
Utopia 4: Enlightening the learner

52. A better learning environment
Utopia 5: Simply the best
• Course improvement
• Feedback for staff (instructors, tutors)
• Grouping students
• Planning and scheduling
• Resource allocation
• Etcetera

54. Multimodal data, LA ,
&
dashboards for (S)SRL)
http://www.slamproject.org/blog

57. JCAL special issue
“Learning analytics in
massively multi-user
virtual environments and
courses"
Available online
CODE: JCALLAK16

58. paul.kirschner@ou.nl
@P_A_Kirschner