Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Ectel2017 paper schmitz_mjwm_presentation

Presentation on the Opportunities and Challenges paper as presented during EC-TEL in 2017

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

Ectel2017 paper schmitz_mjwm_presentation

  1. 1. Opportunities and Challenges in using Learning Analytics in Learning Design Marcel Schmitz1, Evelien van Limbeek1, Wolfgang Greller2, Peter Sloep3 and Hendrik Drachsler3,4,5 1 Zuyd University of Applied Sciences, NL 2 Vienna University of Education, AT 3 Open University, NL 4 Goethe University Frankfurt, DE 5 German Institute for International Educational Research (DIPF), DE
  2. 2. Marcel Schmitz 2
  3. 3. Structure presentation 1. Why do we want to connect LA and LD 1. An idea on how to bring this connection to practice 1. How are we going to study that idea 1. Elaboration on the first steps of the research 1. Limitations 1. Conclusions 3
  4. 4. Structure presentation 1. Why do we want to connect LA and LD 1. An idea on how to bring this connection to practice 1. How are we going to study that idea 1. Elaboration on the first steps of the research 1. Limitations 1. Conclusions 4
  5. 5. Learning Design Current situation Design Time Learning Design 5 Learning Activity Lecture Learning Activity Workshop Learning Activity Online Discussion Learning Activity Learning Activity
  6. 6. Learning Design Current situation Design Time Run Time Learning Design Behavior during Learning Action after analysis What we want to know! How does my student behave? (teacher) How do I perform? (student) How do I perform? (teacher) An action rate < 13+ weeks (teacher+student) 6 Learning Activity Lecture Learning Activity Workshop Learning Activity Online Discussion Learning Activity Learning Activity
  7. 7. Learning Analytics Current Situation Design Time Run Time Learning Design Behavior during Learning Action after analysis 7 Activity data, assessment data, attendance data, behavioral data, communication data, course evaluation data, student coach data, teacher observation data, … Learning Activity Lecture Learning Activity Workshop Learning Activity Online Discussion Learning Activity Learning Activity
  8. 8. Structure presentation 1. Why do we want to connect LA and LD 1. An idea on how to bring this connection to practice 1. How are we going to study that idea 1. Elaboration on the first steps of the research 1. Discussion 1. Conclusions 8
  9. 9. Metacognitive Competences Learning Activity Lecture Learning Activity Workshop Learning Activity Online Discussion Learning Activity Learning Activity Learning Activity Lecture Learning Activity Workshop Learning Activity Online Discussion Learning Activity Learning Activity Idea Desired Situation Design Time Run Time Learning Design Action after analysis 9 Behavior during Learning Analysis Learning Analytics Dashboards Actions after analysis
  10. 10. Structure presentation 1. Why do we want to connect LA and LD 1. An idea on how to bring this connection to practice 1. How are we going to research that idea 1. Elaboration on the first steps of the research 1. Limitations 1. Conclusions 10
  11. 11. Design based Research Main research Question: Can Learning Analytics, by use of Learning Dashboards, improve the Learning Design of a course during run-time and in that way make the learning process more efficient, effective or increase the satisfaction of teachers and students? Research Type: Design Based Research Substudy 1: State of the art, Identification context, users Substudy 2: Concept/Tool design – “Run Time” Substudy 3: Tool (re)design – “Metacognitive competences” Substudy 4: Interventions/actions? 11
  12. 12. Structure presentation 1. Why do we want to connect LA and LD 1. An idea on how to bring this connection to practice 1. How are we going to study that idea 1. Elaboration on the first steps of the research 1. Limitations 1. Conclusions 12
  13. 13. Substudy 1 State of the art, Identification context, users • View on state of the art from literature, focussed on key elements of our idea Future steps • Experiment with Learning Analytics Dashboard (SURF) • Analysis on students from multiple institutions doing the experiment of SURF in comparison with student results and motivation (REFLECTOR) • Focusgroups/Survey/Creative sessions on a data and a dashboard perspective for a LA4LD tool with students and teachers 13
  14. 14. Substudy 1 RQ: Which opportunities and challenges can be identified when designing a Learning Analytics for Learning Design solution Aim: Evidence from literature to inform design decisions Method: Literature review on solution related topics: - Learning Design - Learning Analytics - Learning Dashboards - Metacognitive competences 14
  15. 15. Results Three main opportunties Per opportunity 3 subopportunities and 2 subchallenges O1. Using on demand indicators for evidence based decisions on Learning Design O2. Intervening during run-time of a course O3. Increasing student learning outcome and satisfaction 15
  16. 16. Results O1. Using on demand indicators for evidence based decisions on Learning Design Subopportunities SO1. Observing effects 16 Avella, J., Kebritchi, M., Nunn, S., Kanai, T.: Learning analytics methods, benefits, and challenges in higher education: a systematic literature review. Online Learning Journal 20(2), 13–29 (2016). Mor, Y., Ferguson, R., Wasson, B.: Editorial: learning design, teacher inquiry into student learning and learning analytics: a call for action. British Journal of Educational Technology 46(2), 221–229 (2015).
  17. 17. Results O1. Using on demand indicators for evidence based decisions on Learning Design Subopportunities SO1. Observing effects 17 Avella, J., Kebritchi, M., Nunn, S., Kanai, T.: Learning analytics methods, benefits, and challenges in higher education: a systematic literature review. Online Learning Journal 20(2), 13–29 (2016). Mor, Y., Ferguson, R., Wasson, B.: Editorial: learning design, teacher inquiry into student learning and learning analytics: a call for action. British Journal of Educational Technology 46(2), 221–229 (2015).
  18. 18. Results O1. Using on demand indicators for evidence based decisions on Learning Design Subopportunities SO1. Observing effects SO2. Sharing knowledge 18 Avella, J., Kebritchi, M., Nunn, S., Kanai, T.: Learning analytics methods, benefits, and challenges in higher education: a systematic literature review. Online Learning Journal 20(2), 13–29 (2016). Mor, Y., Ferguson, R., Wasson, B.: Editorial: learning design, teacher inquiry into student learning and learning analytics: a call for action. British Journal of Educational Technology 46(2), 221–229 (2015).
  19. 19. Results O1. Using on demand indicators for evidence based decisions on Learning Design Subopportunities SO1. Observing effects SO2. Sharing knowledge SO3. Involving students 19 Avella, J., Kebritchi, M., Nunn, S., Kanai, T.: Learning analytics methods, benefits, and challenges in higher education: a systematic literature review. Online Learning Journal 20(2), 13–29 (2016). Mor, Y., Ferguson, R., Wasson, B.: Editorial: learning design, teacher inquiry into student learning and learning analytics: a call for action. British Journal of Educational Technology 46(2), 221–229 (2015).
  20. 20. Results O1. Using on demand indicators for evidence based decisions on Learning Design Subchallenges SC1. Interoperability Learning Design 20 Falconer, I., Beetham, H., Oliver, R., Littlejohn, A.: Mod4L final report: representing learning designs. Final report for the JISC-funded MOD4L project. Glasgow (2007). Verbert, K., Govaerts, S., Duval, E., Santos, J., Van Assche, F., Parra, G., Klerkx, J.: Learning dashboards: an overview and future research opportunities. Personal and Ubiquitous Computing 18(6), 1499–1514 (2014).
  21. 21. Results O1. Using on demand indicators for evidence based decisions on Learning Design Subchallenges SC1. Interoperability Learning Design SC2. Interoperability Learning Analytics 21 Bodily, R., Verbert, K.: Trends and issues in student-facing learning analytics reporting systems research. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp 309–318. ACM, New York (2017). Schwendimann, B., Rodriguez-Triana, M., Vozniuk, A., Prieto, L., Boroujeni, M., Holzer, A., Gillet, D., Dillenbourg, P.: Perceiving learning at a glance: a systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies 10(1), 30-41 (2017). Verbert, K., Govaerts, S., Duval, E., Santos, J., Van Assche, F., Parra, G., Klerkx, J.: Learning dashboards: an overview and future research opportunities. Personal and Ubiquitous Computing 18(6), 1499–1514 (2014).
  22. 22. Results O2. Intervening during run-time of a course Subopportunities SO1. Delivering information/feedback on demand 22 Avella, J., Kebritchi, M., Nunn, S., Kanai, T.: Learning analytics methods, benefits, and challenges in higher education: a systematic literature review. Online Learning Journal 20(2), 13–29 (2016). Mor, Y., Ferguson, R., Wasson, B.: Editorial: learning design, teacher inquiry into student learning and learning analytics: a call for action. British Journal of Educational Technology 46(2), 221–229 (2015).
  23. 23. Results O2. Intervening during run-time of a course Subopportunities SO1. Delivering information/feedback on demand SO2. Creating and using interventions 23 Rienties, B., Toetenel, L.: The impact of 151 learning designs on student satisfaction and performance: social learning (analytics) matters. In: Proceedings of LAK16 6th International Conference on Analytics and Knowledge, pp 339–343. (2016). Rienties, B., Boroowa, A., Cross, S., Kubiak, C., Mayles, K., Murphy, S.: Analytics4Action evaluation framework: a review of evidence-based learning analytics interventions at the Open University UK. Journal of Interactive Media in Education (1), 2 (2016). Verbert, K., Duval, E., Klerkx, J., Govaerts, S., Santos, J.: Learning analytics dashboard applications. American Behavioral Scientist 57(10), 1500–1509 (2013). Schwendimann, B., Rodriguez-Triana, M., Vozniuk, A., Prieto, L., Boroujeni, M., Holzer, A., Gillet, D., Dillenbourg, P.: Perceiving learning at a glance: a systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies 10(1), 30-41 (2017).
  24. 24. Results O2. Intervening during run-time of a course Subopportunities SO1. Delivering information/feedback on demand SO2. Creating and using interventions SO3. Change learning behavior 24 Few, S.: Information dashboard design: the effective visual communication of data. O'Reilly Media, Sebastopol (2006). Verbert, K., Duval, E., Klerkx, J., Govaerts, S., Santos, J.: Learning analytics dashboard applications. American Behavioral Scientist 57(10), 1500–1509 (2013). Schwendimann, B., Rodriguez-Triana, M., Vozniuk, A., Prieto, L., Boroujeni, M., Holzer, A., Gillet, D., Dillenbourg, P.: Perceiving learning at a glance: a systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies 10(1), 30-41 (2017).
  25. 25. Results O2. Intervening during run-time of a course Subchallenges SC1. Presenting the relevant information the right way 25 Schwendimann, B., Rodriguez-Triana, M., Vozniuk, A., Prieto, L., Boroujeni, M., Holzer, A., Gillet, D., Dillenbourg, P.: Perceiving learning at a glance: a systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies 10(1), 30-41 (2017).
  26. 26. Results O2. Intervening during run-time of a course Subchallenges SC1. Presenting the relevant information the right way SC2. Improving the ability to act on information 26 Rienties, B., Boroowa, A., Cross, S., Kubiak, C., Mayles, K., Murphy, S.: Analytics4Action evaluation framework: a review of evidence-based learning analytics interventions at the Open University UK. Journal of Interactive Media in Education (1), 2 (2016). Pardo, A., Martinez-Maldonado, R., Buckingham Shum, S., Schulte, J., McIntyre, S., Gašević, D., Gao, J., Siemens, G.: Connecting data with student support actions in a course: a hands-on tutorial. LAK '17 Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp 522-523 (2017).
  27. 27. Results O3. Increasing student learning outcome and satisfaction Subopportunities SO1. Making learning outcomes visible 27 Trigwell, K., Prosser, M.: Improving the quality of student learning: the influence of learning context and student approaches to learning on learning outcomes. Higher Education 22 (251) (1991). Gašević, D., Jovanović, J., Pardo, A., & Dawson, S.: Detecting learning strategies with analytics: Links with self-reported measures and academic performance. Journal of Learning Analytics 4(1) (2017).
  28. 28. Results O3. Increasing student learning outcome and satisfaction Subopportunities SO1. Making learning outcomes visible SO2. Making learning information accessible 28 Prinsloo, P., Slade, S.: An elephant in the learning analytics room: the obligation to act. LAK '17 Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp 46-55 (2017). Pardo, A., Martinez-Maldonado, R., Buckingham Shum, S., Schulte, J., McIntyre, S., Gašević, D., Gao, J., Siemens, G.: Connecting data with student support actions in a course: a hands-on tutorial. LAK '17 Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp 522-523 (2017).
  29. 29. Results O3. Increasing student learning outcome and satisfaction Subopportunities SO1. Making learning outcomes visible SO2. Making learning information accessible SO3. Improving learning to learn 29 De La Fuente, J., Sander, P., Martínez-Vicente, J. M., Vera, M., Garzon, A., & Fadda, S.: Combined Effect of Levels in Personal Self-Regulation and Regulatory Teaching on Meta-Cognitive, on Meta-Motivational, and on Academic Achievement Variables in Undergraduate Students. Frontiers in psychology 8 (2017). Gašević, D., Jovanović, J., Pardo, A., & Dawson, S.: Detecting learning strategies with analytics: Links with self-reported measures and academic performance. Journal of Learning Analytics 4(1) (2017).
  30. 30. Results O3. Increasing student learning outcome and satisfaction Subchallenges SC1. Understanding learning dashboards 30 Park, Y., Jo, I.: Development of the learning analytics dashboard to support students’ learning performance. Journal of Universal Computer Science 21(1), 110–133 (2015).
  31. 31. Results O3. Increasing student learning outcome and satisfaction Subchallenges SC1. Understanding learning dashboards SC2. Coping with the diversity of students 31 Altbach, P., Reisberg, L., Rumbley, L.: Tracking a global academic revolution. Change 42(2), 30–39 (2010). Nonis, S., Hudson, G.: Academic performance of college students: influence of time spent studying and working. Journal of Education for Business 81(3), 151–159 (2006). Tuononen, T., Parpala, A., Mattsson, M., Lindblom-Ylänne, S.: Work experience in relation to study pace and thesis grade: investigating the mediating role of student learning. Higher Education 72(1), 41–58 (2016). Ministerie van Onderwijs, Cultuur & Wetenschap: De waarde(n) van weten: strategische agenda hoger onderwijs en onderzoek 2015-2025. Den Haag (2015).
  32. 32. Results 32 Schmitz, M., Limbeek van, E., Greller, W., Sloep, P., Drachsler, H.: Opportunities and challenges in using learning analytics in learning design. In: Data Driven Approaches in Digital Education: Proceeding of 12th European Conference on Technology Enhanced Learning, EC-TEL pp. 209–223 (2017).
  33. 33. Structure presentation 1. Why do we want to connect LA and LD 1. An idea on how to bring this connection to practice 1. How are we going to study that idea 1. Elaboration on the first steps of the research 1. Limitations 1. Conclusions 33
  34. 34. Limitations 34 - No systematic literature review - Focus is both on student and teacher - What is needed to facilitate students and teachers to act upon information (metacognitive competences?)
  35. 35. Structure presentation 1. Why do we want to connect LA and LD 1. An idea on how to bring this connection to practice 1. How are we going to study that idea 1. Elaboration on the first steps of the research 1. Limitations 1. Conclusions 35
  36. 36. Conclusions 36 The first step in aligning and incorporating Learning Analytics into Learning Design is studying what the opportunities and challenges are that have to be addressed. The identification of opportunities and challenges we presented here will help in designing LA4LD solutions. By addressing those we, as everyone else can improve higher education and achieve a more personalized and just in time learning culture with on demand feedback mechanisms.
  37. 37. marcel.schmitz@zuyd.nl @marcelschmitz

×