Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Preliminary PhD Defence - Student-facing Dashboards

337 views

Published on

Slides of my 20 min presentation during my preliminary PhD defence: "Student-facing Dashboards"

Published in: Science
  • Be the first to comment

  • Be the first to like this

Preliminary PhD Defence - Student-facing Dashboards

  1. 1. Sven Charleer Student-facing Learning Dashboards Preliminary PhD Defence May 23, 2017
  2. 2. LEARNING ANALYTICS 2 “The measurement, collection, analysis, and reporting of data about learners and their contexts, for purpose of understanding and optimising learning and the environments in which it occurs” J. L. Santos. Learning Analytics and Learning Dashboards: a Human- Computer Interaction Perspective. PhD dissertation, KU Leuven, 2015. G. Siemens. “Learning analytics: envisioning a research discipline and a domain of practice”. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge . ACM. 2012, pp. 4–8. session course degree year Microlevel intro
  3. 3. 3 EMPOWER predictions recommendations LEARNING ANALYTICSintro
  4. 4. LEARNING DASHBOARDS 4 “A Learning Dashboard is a single display that aggregates different indicators about learner(s), learning process(es) and/or learning context(s) into one or multiple visualisations.” B. A. Schwendimann, M. J. Rodríguez-Triana, A. Vozniuk, L. P. Prieto, M. S. Boroujeni, A. Holzer, D. Gillet, and P. Dillenbourg. Understanding learning at a glance: An overview of learning dashboard studies. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, pages 532–533. ACM, 2016. K. Verbert, E. Duval, J. Klerkx, S. Govaerts, and J. L. Santos. Learning Analytics Dashboard Applications. American Behavioral Scientist, 57(10):1500–1509, 2013. Perceived benefits Design guidelines intro
  5. 5. External input Rapid prototyping Multiple dependent variables Inform 5 DESIGN-BASED RESEARCHintro
  6. 6. 6 CONTEXTintro ch2 ch3 ch4
  7. 7. RQ1: What are relevant learning traces, and how should we visualise these data to support students to explore the path from effort to outcomes? RQ2: How can we promote students, inside and outside the classroom, to actively explore this effort to outcomes path? 7 CREATING EFFECTIVE LEARNING DASHBOARDSch2 abundance of data - effort - outcome
  8. 8. 8 CONTRIBUTIONSch2
  9. 9. Abstract the LA data Provide access to the artefacts Augment the abstracted data Provide access to teacher and peer feedback 9 RESULTSch2 RQ1: What are relevant learning traces, and how should we visualise these data to support students to explore the path from effort to outcomes?
  10. 10. 10 RESULTSch2 RQ2: How can we promote students, inside and outside the classroom, to actively explore this effort to outcomes path? Visualise the learner path Integrate LA into the workflow Facilitate collaborative exploration of the LA data
  11. 11. 11 RESULTSch2 Integrate LA into the workflow Augment the abstracted data
  12. 12. 12 RESULTSch2
  13. 13. 13 CONTRIBUTIONSch2 Guidelines published at EC-TEL
 (25% acceptance)
 S. Charleer, J. Klerkx, E. Duval, T. De Laet, and K. Verbert. Creating effective learning analytics dashboards: Lessons learnt. Adaptive and Adaptable Learning: 11th European Conference on Technology Enhanced Learning, EC-TEL 2016, Lyon, France, September 13-16, 2016, Proceedings, pages 42–56, Cham, 2016. Springer International Publishing LARAe
 test beds across Europe, 461 students 14 papers
 Stanford University - University of Technology Sidney - University of the Basque Country - Murdoch University Perth - Curtin University Perth Visual Learning Analytics workshop
 Learning Analytics Summer Institute 2014, Harvard
  14. 14. 14 BALANCED DISCUSSION IN THE CLASSROOMch3 RQ3: What are the design challenges for ambient Learning Dashboards to promote balanced group participation in classrooms, and how can they be met? RQ4: Are ambient Learning Dashboards effective means for creating balanced group participation in classroom settings? over- and under-participation
  15. 15. oup 1 Group 2 oup 5 Group 3 EVALUATION SETUPch3 case study 1 # participants 12 students deployment 1 3h session with dashboard 1 3h session without dashboard evaluation class discussion
 questionnaires (perceived distraction/ awareness/usefulness)
 activity/quality logging case study 2 # participants 19 students deployment half 3h session without dashboard
 half 3h session with dashboard evaluation questionnaires (perceived importance feedback/motivation) activity/quality logging 15
  16. 16. Visualise balance in an abstract and neutral way Add the qualitative dimension to the visualisation Create a realistic picture of the classroom situation 16 RESULTS RQ3: What are the design challenges for ambient LDs to promote balanced group participation in classrooms, and how can they be met? ch3
  17. 17. Ambient dashboards as support for teacher/presenter Ambient dashboards raise awareness of the invisible Ambient feedback information can activate students 17 RESULTS RQ4: Are ambient LDs effective means for creating balanced group participation in classroom settings? ch3
  18. 18. Ambient dashboards as support for teacher/presenter Ambient dashboards raise awareness of the invisible Ambient feedback information can activate students 18 RESULTS RQ4: Are ambient LDs effective means for creating balanced group participation in classroom settings? ch3
  19. 19. EVALUATION SETUPch3 -0.2 0 0.2 0.4 0.6 0.8 %distancefromaverage -0.2 0 0.2 0.4 0.6 0.8 0 10.5 % of time of feedback session passed %ofdistancefromaverage without visualisation with visualisation group 1 group 2 group 3 group 4 group 5 group 6 19
  20. 20. 20 CONTRIBUTIONS Published Special Issue on Awareness and Reflection in Technology-Enhanced Learning, IJTEL
 (8/21 submissions accepted)
 Charleer, S., Klerkx, J., Duval, E., De Laet, T. and Verbert, K. (2017) ‘Towards balanced discussions in the classroom using ambient information visualisations’, Int. J. Technology Enhanced Learning, Vol. 9, Nos. 2/3, pp.227–253. Basis for new research collaboration
 with the University of Sidney ch3
  21. 21. SUPPORTING ADVISER-STUDENT DIALOGUEch4 RQ5: What are the design challenges for creating a Learning Dashboard to support study advice sessions, and how can they be met? RQ6: How does such a Learning Dashboard contribute to the role of the adviser, student, and dialogue? 21 lack of data-based feedback
  22. 22. EVALUATION SETUPch4 design # participants 17 study advisers (preliminary feedback)
 5 study advisers (iterative feedback) approach brainstorms/observations
 iterative design evaluation # participants 5 study advisors deployment Engineering Science, Engineering Science: Architecture
 97 sessions (15-30min per session) evaluation 15 sessions observed questionnaires perceived usefulness 22
  23. 23. Data Confidence Collaboration Adviser’s role 23 RESULTS RQ6: How does such a Learning Dashboard contribute to the role of the adviser, student, and dialogue? ch4 RQ5: What are the design challenges for creating a Learning Dashboard to support study advice sessions, and how can they be met? Authorship Visual Encoding Ethics
  24. 24. RESULTSch4 24
  25. 25. RESULTSch4 25
  26. 26. RESULTSch4 S. Claes, N. Wouters, K. Slegers, and A. V. Moere. Controlling In-the-Wild Evaluation Studies of Public Displays. pages 81–84, 2015. 26
  27. 27. 27 CONTRIBUTIONSch4 Conditionally accepted to IEEE Transactions on Learning Technologies with minor revisions
 11% acceptance rate, IF most recent: 1.129, 5-year IF: 1.608
 S. Charleer, A. Vande Moere, J. Klerkx, K. Verbert, and T. De Laet. Learning analytics dashboards to support adviser-student dialogue. IEEE Transaction on Learning Technologies, conditionally accepted with minor revisions, 18 pages Deployed at
 Engineering Science, Engineering Science: Architecture, Maths, Biology, Physics, Geology, Geography, Biochemistry, Informatics, Bio-engineering, Engineering Technology (3 campuses) 15 study advisers during 165 sessions
  28. 28. 28 SUMMARYwrp 3 learning settings 7 dashboards 100+ students, 20 instructors, 17 study advisers 19 publications
  29. 29. 29 Citation indices Since 2012 # citations 112 h-index 6 i10-index 3 CITATIONSwrp 29
  30. 30. 30 FUTURE (ONGOING) WORKwrp Ground work for long-term evaluations/deployments Leiden University Student union requests faculty deployment KU Leuven has shown interest in dashboard university-wide
  31. 31. Student-facing Learning Dashboards Sven Charleer Preliminary PhD Defence May 23, 2017

×