Learning Analytics Dashboards

7,513 views
7,293 views

Published on

My presentation at the Learning Analytics day at VOV Lerend Netwerk in Brussels on May 22, 2014.

Published in: Education, Technology
1 Comment
31 Likes
Statistics
Notes
  • Great slides! We have our own Learning Analytics Dashboard for third parties and we'd love your opinion on it http://endermetrics.com/?slide=about-us
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
7,513
On SlideShare
0
From Embeds
0
Number of Embeds
463
Actions
Shares
0
Downloads
207
Comments
1
Likes
31
Embeds 0
No embeds

No notes for slide

Learning Analytics Dashboards

  1. 1. Learning Analytics dashboards Sten Govaerts EPFL, Zwitserland slides partially based on “Information Dashboard” by Katrien Verbert
  2. 2. about me
  3. 3. KULeuven team:
 Human-Computer Interaction ! •technology enhanced learning •music •research EPFL team:
 REACT - Real-time Coordination & Distributed Interaction Systems Group ! •technology enhanced learning •social media •transportation http://hci.cs.kuleuven.be/ http://react.epfl.ch/
  4. 4. •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014
  5. 5. •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014 information visualisation}
  6. 6. •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014 social recommendations } information visualisation}
  7. 7. •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014 social recommendations } information visualisation} learning analytics }
  8. 8. •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014 social recommendations } information visualisation} learning analytics }inquiry learning with online labs }
  9. 9. PhDstudent! @KULeuven •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014 social recommendations } information visualisation} learning analytics }inquiry learning with online labs }
  10. 10. PhDstudent! @KULeuven Postdoc
 @EPFL •2008: context-aware music information retrieval •2009 •2010 •2011 •2012 •2013 •2014 social recommendations } information visualisation} learning analytics }inquiry learning with online labs }
  11. 11. about you…
  12. 12. why are you interested in dashboards?
  13. 13. what I planned…
  14. 14. what I planned… what are dashboards?
  15. 15. what I planned… what are dashboards? dashboard examples
  16. 16. what I planned… what are dashboards? what is wrong with most dashboards? dashboard examples
  17. 17. what I planned… what are dashboards? what is wrong with most dashboards? how can you design better dashboards? dashboard examples
  18. 18. what I planned… what are dashboards? what is wrong with most dashboards? how can you design better dashboards? dashboard examples learning analytics 
 dashboard examples
  19. 19. what I planned… what are dashboards? what is wrong with most dashboards? how can you design better dashboards? dashboard examples how can you quickly get started 
 making your own dashboards? learning analytics 
 dashboard examples
  20. 20. Typical dashboards…
  21. 21. Typical dashboards…
  22. 22. Typical dashboards…
  23. 23. Typical dashboards…
  24. 24. Typical dashboards…
  25. 25. Typical dashboards…
  26. 26. What is an 
 information dashboard? A dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance. - Stephen Few Stephen Few, "Dashboard Confusion," Intelligent Enterprise, March 20, 2004.
  27. 27. Learning Analytics! dashboards 
 (& visualisations ;-)
  28. 28. http://research.uow.edu.au/learningnetworks/seeing/snapp/index.html SNAPP
  29. 29. GISMO http://www.noe-kaleidoscope.org/pub/researcher/resources/video/aied/mazza.html
  30. 30. http://www.cs.uu.nl/docs/vakken/uem/Janssen.pdfThe ParticipationTool
  31. 31. Backstage Pohl A, Bry F, Schwarz J, Gottstein M (2012) Sensing the classroom: improving awareness and self-awareness of students with backstage. In: Proceedings of the international conference on interactive and collaborative learning (ICL), Villach, Austria, 26–28 September 2012.
  32. 32. Dollár A, Steif PS (2012) Web-based statics course with learning dashboard for instructors. In: Uskov V (ed) Proceedings of computers and advanced technology in education (CATE 2012), June 25–27, 2012, Napoli, Italy OLI Dashboard
  33. 33. LARA
 Learning Analytics Reflection & Awareness environment
  34. 34. also on tabletop & large displays
  35. 35. J. L. Santos, K.Verbert, S. Govaerts, and E. Duval. Visualizing PLE Usage. Proceedings of EFEPLE11: 1st Workshop on Exploring the Fitness and Evolvability of Personal Learning Environments, CEUR Laclusaz, France, 30-31 March 2011
  36. 36. Student Activity Meter Govaerts, Sten;Verbert, Katrien, Duval, Erik; Pardo,Abelardo, The Student Activity Meter for Awareness and Self-reflection. http://ariadne.cs.kuleuven.be/monitorwidget-chi
  37. 37. Problem http://www.ypfp.org/classroom
  38. 38. Problem http://www.ypfp.org/classroom
  39. 39. Objectives • self-monitoring for learners http://free-extras.com/images/obama_hope_progress-2848.htm
  40. 40. Objectives • self-monitoring for learners • awareness for teachers http://free-extras.com/images/obama_hope_progress-2848.htm
  41. 41. Objectives • self-monitoring for learners • awareness for teachers • time tracking http://free-extras.com/images/obama_hope_progress-2848.htm
  42. 42. Objectives • self-monitoring for learners • awareness for teachers • time tracking • learning resource use and 
 recommendations http://free-extras.com/images/obama_hope_progress-2848.htm
  43. 43. STUDENT ACTIVITY METER
  44. 44. STUDENT ACTIVITY METER
  45. 45. STUDENT ACTIVITY METER
  46. 46. STUDENT ACTIVITY METER
  47. 47. Methodology. • design-based research [Obrenović 2011]: design a tool, give students & teachers and evaluate • iterative process of design & evaluation • paper mockups • usability studies • surveys on 
 usefulness http://funnytogo.com/pictures/evolution/timeline.htm
  48. 48. iter. evaluation goal methodology demo-graphics data I. usability, satisfaction, preliminary usefulness task-based interview with think-aloud, SUS & MSDT 12 CS students at KULeuven timetracking viaTwitter II. assessing teacher needs, use & usefulness online survey with Likert items, yes/no and open questions 19 teachers & TA’s with CGIAR courses Moodle logs III. assessing teacher needs, expert feedback, use & usefulness online survey with Likert items, yes/no and open questions 12 participants from LAK MOOC Moodle logs IV. use, usefulness & satisfaction structured face-to-face interview with tasks & open questions 11 teachers & TA’s at Carlos III Uni. Madrid VM with extensive tool tracking
  49. 49. Conclusion • SAM is perceived useful and contributes to better awareness.
  50. 50. Conclusion • SAM is perceived useful and contributes to better awareness. • SAM enables:
  51. 51. Conclusion • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk
  52. 52. Conclusion • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview
  53. 53. Conclusion • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview • understanding student time spending
  54. 54. Conclusion • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview • understanding student time spending • to find a wide variety of new insights
  55. 55. Conclusion • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview • understanding student time spending • to find a wide variety of new insights • almost all want to continue using SAM
  56. 56. Go-Lab Learning Analytics dashboard
  57. 57. STRUCTURE • explain main idea • science edu -> to labs -> lab fire -> need something safer • objectives golab • partner list • react’s job • portal • architecture
  58. 58. Orientation Discussion Investigation Conceptua- lisation Conclusion
  59. 59. An overview of where students are Currently there are 4 students in the orientation phase, 8 in the conceptualization phase, 3 in the investigation phase and one discussing. Student currently in phase 4 8 3 0 1
  60. 60. A detailed view of where each student is Tommy is in the discussion phase, Becky in the investigation phase, etc. Student current phase Tommy Becky Jessi Alex
  61. 61. An overview of the time spent in each space for all students There are currently students in the conclusion and the discussion phases, the average time spent in the discussion phase so far is 20 seconds and the average time spent in the investigation phase is 15:30 minutes Average time per phase 2:50 3:01 15:30 4:20 0:20
  62. 62. The number of time students change phases Alex has been changing 12 times between phases so far, Jessi only 5 times. Number of phase changes Tommy Becky Jessi Alex 4 8 5 12
  63. 63. An overview of who handed in their assignment Tommy, Jessi and Alex have already handed in their assignments. Student reports Tommy Becky Jessi Alex report.pdf Jessi.pdf Pat pat.pdf
  64. 64. Student reports Tommy Becky Jessi Alex report.pdf Jessi.pdf Pat pat.pdf Number of phase changes Tommy Becky Jessi Alex 4 8 5 12 Connected Students 11 Connected Students 4 Connected Stud 6 Student current phase Tommy Becky Jessi Alex
  65. 65. common issues with dashboards
  66. 66. multiple screens.
  67. 67. multiple screens.
  68. 68. multiple screens.
  69. 69. inadequate context 
 for the data.
  70. 70. inadequate context 
 for the data.
  71. 71. too much detail/precision.
  72. 72. choosing a deficient measure.
  73. 73. choosing a deficient measure.
  74. 74. inappropriate display media.
  75. 75. inappropriate display media.
  76. 76. inappropriate display media.
  77. 77. meaningless variety.
  78. 78. poor visualisations.
  79. 79. poor visualisations.
  80. 80. poor visualisations.
  81. 81. poor visualisations.
  82. 82. bad encoding of data
  83. 83. not highlighting 
 important data.
  84. 84. useless decoration.
  85. 85. too much colour.
  86. 86. dashboards require thoughtful design…
  87. 87. how to design better dashboards?
  88. 88. how to design better dashboards? 1. reduce the non-data pixels
  89. 89. how to design better dashboards? 1. reduce the non-data pixels 2. enhance the data pixels
  90. 90. how to design better dashboards? 1. reduce the non-data pixels 2. enhance the data pixels 3. design for usability
  91. 91. how to design better dashboards? 1. reduce the non-data pixels 2. enhance the data pixels 3. design for usability
  92. 92. reduce non-data pixels
  93. 93. reduce non-data pixels
  94. 94. reduce non-data pixels
  95. 95. reduce non-data pixels
  96. 96. reduce non-data pixels
  97. 97. Enhance data pixels
  98. 98. Enhance data pixels
  99. 99. Design for usability.
  100. 100. Design for usability. 1. organize information to support its meaning & use
  101. 101. Design for usability. 1. organize information to support its meaning & use 2. make it aesthetically pleasing!
  102. 102. Design for usability. 1. organize information to support its meaning & use 2. make it aesthetically pleasing! 3. design for a launch pad
  103. 103. Design for usability. 1. organize information to support its meaning & use 2. make it aesthetically pleasing! 3. design for a launch pad
  104. 104. Design for usability. 1. organize information to support its meaning & use 2. make it aesthetically pleasing! 3. design for a launch pad
  105. 105. How to evaluate? usability System Usability Scale (SUS)
  106. 106. How to evaluate? usability System Usability Scale (SUS)
  107. 107. How to evaluate? usability interviews: task list
  108. 108. How to evaluate? usability interviews: task list think aloud+
  109. 109. How to evaluate? usability interviews: task list think aloud+ observe+
  110. 110. How to evaluate? usability interviews: task list think aloud+ observe+ understand what the user does understand what the issues are +
  111. 111. How to evaluate? usefulness&impact
  112. 112. How to evaluate? usefulness&impact much HARDER…
  113. 113. How to evaluate? usefulness&impact interviews much HARDER…
  114. 114. How to evaluate? usefulness&impact interviews much HARDER… perceived usefulness surveys
  115. 115. How to evaluate? usefulness&impact interviews much HARDER… perceived usefulness surveys longitudinal studies
  116. 116. How to quickly build 
 your own dashboard?
  117. 117. How to quickly build 
 your own dashboard? get data
  118. 118. How to quickly build 
 your own dashboard? get data (clean data)
  119. 119. How to quickly build 
 your own dashboard? get data (clean data) visualise!
  120. 120. get data import.io
  121. 121. get data import.iokimonolabs.com
  122. 122. Organic Chemistry Organic chemistry Organical Chemestry OrganicChemistry Organ Chemistry (clean data)
  123. 123. Organic Chemistry Organic chemistry Organical Chemestry OrganicChemistry Organ Chemistry (clean data) Google Refine/OpenRefine http://openrefine.org/
  124. 124. Organic Chemistry Organic chemistry Organical Chemestry OrganicChemistry Organ Chemistry (clean data) Google Refine/OpenRefine http://openrefine.org/ DataWrangler http://vis.stanford.edu/wrangler/
  125. 125. visualise! ManyEyeshttp://www.manyeyes.com/software/analytics/manyeyes/
  126. 126. visualise! ManyEyeshttp://www.manyeyes.com/software/analytics/manyeyes/ RAW http://app.raw.densitydesign.org/#%2F
  127. 127. visualise! ManyEyeshttp://www.manyeyes.com/software/analytics/manyeyes/ RAW http://app.raw.densitydesign.org/#%2F video: http://www.ted.com/talks/ hans_rosling_shows_the_best_stats_you_ve_ever_seen http://www.gapminder.org/
  128. 128. visualise! ManyEyeshttp://www.manyeyes.com/software/analytics/manyeyes/ RAW http://app.raw.densitydesign.org/#%2F video: http://www.ted.com/talks/ hans_rosling_shows_the_best_stats_you_ve_ever_seen http://www.gapminder.org/
  129. 129. visualise! http://www.simile-widgets.org/exhibit/
  130. 130. visualise! http://www.simile-widgets.org/exhibit/ http://www.simile- widgets.org/timeline/
  131. 131. visualise! http://www.simile-widgets.org/exhibit/ http://www.simile- widgets.org/timeline/ Google Charts https://developers.google.com/chart/
  132. 132. there are also all-in- one solutions…
  133. 133. How many kids are repeating a year in elementary school?
  134. 134. • http://www.wolframalpha.com/input/?i=how+many+kids +are+repeating+primary+school+in+Belgium • http://www.wolframalpha.com/input/?i=university+with+the +largest+enrollment&lk=3 • http://www.wolframalpha.com/input/?i=university+with +the+largest+enrollment+in+Belgium • http://www.wolframalpha.com/input/?i=Belgium+vs+USA +education+expenditures+per+high+school+student
  135. 135. further reading… http://pdf.th7.cn/down/files/1312/information_dashboard_design.pdf
  136. 136. Edward Tufte.
  137. 137. Questions? slides on: http://www.slideshare.net/stengovaerts

×