Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The role of data in the provision of feedback at scale

707 views

Published on

Technology mediation allows to capture comprehensive data sets about interactions occurring in learning experiences. Although these data sets have the potential of increasing the insight on how learning occurs, their use strongly depends on two aspects: the data has to be properly situated in the learning design, and the insights derived need to be translated into actions. In this talk we will explore how to establish this connection for the case of the provision of feedback. We will approach the problem from the point of view of intelligence amplification, that is, how data can support instructors to provide better support to learners through feedback. The talk will discuss some preliminary results from the Ontasklearning.org project.

Published in: Education
  • Be the first to comment

The role of data in the provision of feedback at scale

  1. 1. Abelardo Pardo (@abelardopardo)
 Faculty of Engineering and IT slideshare.net/abelardo_pardo The role of data in the provision of feedback at scale WallyGobetzflickr.com ASCILITE Webinar 11 April 2017
  2. 2. Abelardo Pardo The role of data in the provision of feedback at scale 2 Learning Analytics Situated Actions Feedback Examples theilrflickr.com
  3. 3. Abelardo Pardo The role of data in the provision of feedback at scale 3 Learning Analytics Situated Actions theilrflickr.com
  4. 4. Abelardo Pardo The role of data in the provision of feedback at scale 4 DmitryGrigorievflickr.com Learning Analytics: measure, collect, analyse data about learners to understand and improve their learning and the environment in which it occurs
  5. 5. Abelardo Pardo The role of data in the provision of feedback at scale 5 DerekBruffflickr.com Understanding and Improving Learning Algorithms Dashboards Clickstreams Multimodal SNA Writing analytics Learning Designs
  6. 6. Abelardo Pardo The role of data in the provision of feedback at scale 6 JamesRhodesflickr.com Let’s not forget: Learning analytics are about learning Gašević, D., Dawson, S., & Siemens, G. (2015). Let's not forget: Learning analytics are about learning. TechTrends, 59(1), 64-75.
  7. 7. Abelardo Pardo The role of data in the provision of feedback at scale 7 DerekBruffflickr.com Learning is situated in interactions with others and relationships to the environment Ellis, R. A., & Goodyear, P. (2013). Students' Experiences of E-learning in Higher Education. New York and London: Routledge
  8. 8. Abelardo Pardo The role of data in the provision of feedback at scale 8 ChrisMooreflickr.com Learning environment has a profound effect on how data is analysed
  9. 9. Abelardo Pardo The role of data in the provision of feedback at scale 9 senscienceflickr.com Marzouk, Z., Rokovic, M., Liaqat, A., Vytasek, J., Samadi, D., Stewart-Alonso, J., . . . Nesbit, J. C. (2016). What if learning analytics were based on learning science? Australasian Journal of Educational Technology, 32(6). • Setting goals & monitoring progress • Distributed practice • Retrieval practice • Prior knowledge for reading • Comparative evaluation of writing • Collaborative Learning
  10. 10. Abelardo Pardo The role of data in the provision of feedback at scale 10 JoshuaRappenekerflickr.com Data Understanding Actions Improving
  11. 11. Abelardo Pardo The role of data in the provision of feedback at scale 11 Learning Analytics Model
  12. 12. Abelardo Pardo The role of data in the provision of feedback at scale 12 Learning Analytics Situated Actions Feedback theilrflickr.com
  13. 13. Abelardo Pardo The role of data in the provision of feedback at scale 13 Hattie, J. A. (1999). Influences on student learning. Inaugural professorial address, University of Auckland, New Zealand If you could choose one… • More than 500 meta-analyses of student achievements • 100 factors with potential influence • Feedback in top five • (74 meta-analyses) Most effective form: video, audio, computer-assisted instructional feedback, and/or related goals
  14. 14. Abelardo Pardo The role of data in the provision of feedback at scale 14 PaulaGrubbflickr.com "Feedback is a process whereby learners obtain information about their work in order to appreciate the similarities and differences between the appropriate standards for any given work, and the qualities of the work itself, in order to generate improved work" Boud, D., & Molloy, E. (Eds.). (2013). Feedback in Higher and Professional Education: Understanding it and doing it well. London and New York: Routledge.
  15. 15. Abelardo Pardo The role of data in the provision of feedback at scale 15 Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. doi:10.3102/003465430298487 ChrisBallardflickr.com Feedback Levels 1. Task Level (understanding, performance) 2. Process Level (what to do to understand, perform) 3. Self-regulation level (detecting and directing effort) 4. Self level (personal evaluation and affect)
  16. 16. Abelardo Pardo The role of data in the provision of feedback at scale 16 Nicol, D., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. doi:10.1080/03075070600572090
  17. 17. Abelardo Pardo The role of data in the provision of feedback at scale Good feedback practice 1. Clarify what good performance is 2. Facilitate self-assessment 3. Deliver high quality feedback information 4. Encourage teacher and peer dialogue 5. Encourage positive motivation and self-esteem 6. Provide opportunities to close the gap 7. Use information to improve teaching 17 Nicol, D., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. doi:10.1080/03075070600572090
  18. 18. Abelardo Pardo The role of data in the provision of feedback at scale 18 Baker, R. S. (2016). Stupid Tutoring Systems, Intelligent Humans. International Journal of Artificial Intelligence in Education. doi: 10.1007/s40593-016-0105-0 Rienties, B., Boroowa, A., Cross, S., Kubiak, C., Mayles, K., & Murphy, S. (2016). Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK. Journal of Interactive Media in Education, 2016(1). doi: 10.5334/jime.394 Change in focus, from artificial intelligence to
 intelligence amplification Tutors are informed about students struggling but it is up to them to decide if/when to act
  19. 19. Abelardo Pardo The role of data in the provision of feedback at scale 19 Learning Analytics Situated Actions Feedback Examples theilrflickr.com
  20. 20. Abelardo Pardo The role of data in the provision of feedback at scale 20 Tracking
  21. 21. Abelardo Pardo Can data help me learn? 21 Jovanovic, J., Gašević, D., Pardo, A., Dawson, S., & Mirriahi, N. (In Press. Accepted 20/Feb/2017). Learning Analytics to Unveil Learning Strategies in a Flipped Classroom. Internet and Higher Education. doi:10.1016/j.iheduc.2017.02.001 Analyse Learning Strategies
  22. 22. Abelardo Pardo The role of data in the provision of feedback at scale 22 Report to students Kahn, I., & Pardo, A. (2016). Data2U: Scalable Real time Student Feedback in Active Learning Environments. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh, UK. doi:10.1145/2883851.2883911
  23. 23. Abelardo Pardo The role of data in the provision of feedback at scale 23 No statistically significant difference in the rating of feedback (2013 edition, M=3.25, SD=0.97; 2014 edition, M=3.35, SD=1.03); t(389.78) = -0.97, p <0.17
  24. 24. Abelardo Pardo The role of data in the provision of feedback at scale 24 You should take a more careful look at how symbols are encoded in the video. Would you be able to encode/ decode UAL symbols without looking at the video? Good initial work. However, did you understand the trick to handle encoding with a variable number of bits? Would you be able to provide an example? Good work. Would you be able to come up with your own machine language and your encoding scheme? Remember that it has to be unambiguous. Thorough work with the task about machine language encoding. Give it a quick review before the midterm. Q1 Q2 Q3 Q4 Instructor
  25. 25. Abelardo Pardo The role of data in the provision of feedback at scale 25 Algorithm
  26. 26. Abelardo Pardo The role of data in the provision of feedback at scale 26 Automatic Email
  27. 27. Abelardo Pardo The role of data in the provision of feedback at scale 27 Helpful feedback Effect size (Cohen’s d) = 0.49. Medium positive effect Midterm Scores Effect size (Cohen’s d) = 0.21. Small positive effect Pardo, A., Jovanovic, J., Dawson, S., Gašević, D. & Mirriahi, N. (2017). Using Learning Analytics to Scale the Provision of Personalised Feedback. Manuscript in preparation
  28. 28. Abelardo Pardo The role of data in the provision of feedback at scale 28 What if I tell you…
  29. 29. Abelardo Pardo The role of data in the provision of feedback at scale 29 ontasklearning.org
  30. 30. Abelardo Pardo The role of data in the provision of feedback at scale 30 IF [video 3.9.5 not watched] THEN “…." ontasklearning.org
  31. 31. Abelardo Pardo The role of data in the provision of feedback at scale 31 • Support instructors to create personalised feedback • Simple rule-base knowledge encoding • Provide appropriate view of data sources • Scale to large and highly diverse cohorts • Will be released as open-source project Q3/4 2017 • First pilots in Q1/2 2017 • Tutorial in LAK 2017 • Contact us if interested ontasklearning.org
  32. 32. Abelardo Pardo The role of data in the provision of feedback at scale 32 BillTyneflickr.com • Data to understand and improve learning needs to be situated and connected to actions • Feedback is a powerful focal point to drive data- supported decisions • Explore the design of theory-driven learning analytics tools that connect data with support actions
  33. 33. Abelardo Pardo (@abelardopardo)
 Faculty of Engineering and IT slideshare.net/abelardo_pardo The role of data in the provision of feedback at scale WallyGobetzflickr.com ASCILITE Webinar 11 April 2017

×