Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Articulating the connection between Learning Design and Learning Analytics

3,883 views

Published on

Learning analytics is a discipline that uses data captured by technology during a learning experience to increase our level of understanding, increase its quality, and improve the environment in which it occurs. But these experiences need to be designed first. In this talk we start from the statement that there is no such thing as a neutral design. In the era of increasing technology mediation Learning experiences need to be designed considering the capacity to capture data, the possibility of making sense and derive knowledge from the data, and the need to act on that knowledge. In this talk we will explore some initiatives to make these connections explicit in a learning design. Using a flipped learning experience, we will explore how to embed data and data analysis as part of the design tasks.

Published in: Education
  • Be the first to comment

Articulating the connection between Learning Design and Learning Analytics

  1. 1. Abelardo Pardo (@abelardopardo)
 Faculty of Engineering and IT slideshare.net/abelardo_pardo Articulating the connection between Learning Analytics and Learning Design Faculty of Education University of Hong Kong 16/May/2017 MarianoMantelflickr.com
  2. 2. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 2 Ivanflickr.com Learning Analytics Data Journey Connection Model How to integrate
  3. 3. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 3 Ivanflickr.com Learning Analytics
  4. 4. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 4 DmitryGrigorievflickr.com Learning Analytics: measure, collect, analyse data about learners to understand and improve their learning and the environment in which it occurs 1st International Conference on Learning Analytics and Knowledge [Online]. https://tekri.athabascau.ca/analytics. [Accessed 2017].
  5. 5. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 5 EustaquioSantimanoflickr.com Collect Report Analyse Act Refine Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (2007). Academic Analytics. Louisville, CO: Educause White Paper.
  6. 6. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 6 Long, P., & Siemens, G. (2011). Penetrating the Fog: Analytics in Learning and Education. EDUCAUSE Review, 48(5), 31-40.
  7. 7. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 7 Learning Analytics Model (v1)
  8. 8. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 8 Ivanflickr.com Learning Analytics Data Journey
  9. 9. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 9 trepeluflickr.com Let’s explore clickstreams!
  10. 10. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 10 4.174.239.145 - user_2@bogus.com "user 2" [27/Jan/2016:00:00:26 +0000] "GET resource_6 HTTP/1.1" 302 1428 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:39.0) Gecko/20100101 Firefox/39.0" Clickstreams Machine address User id User name When date/time 4.174.239.145 - user_2@bogus.com "user 2" [27/Jan/2016:00:00:26 +0000] "GET resource_6 HTTP/1.1" 302 1428 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:39.0) Gecko/20100101 Firefox/39.0" Which resource One line written in a file for every click!
  11. 11. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 11 Clickstream - Video 4187, 2016-07-14 00:56:46.341946+00, {“time”":0,"id":"xEJtdMQMcrs","event":"PLAY"}, https://mycourse.com/Material/HLP/HLP_notes.html, embedded-video User id When
 date/time Video played
 from start PageEvent
 type
  12. 12. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 12 Clickstream - Question 7141, 2016-07-25 00:41:42.282416+00, {“question_id":"COD-baseencoding-eqt_1","answer":0}, https://mycourse.com/Material/COD/Activities/encoding_videoeqt.html, embedded-question User id When
 date/time Question
 answered
 incorrectly PageEvent
 type
  13. 13. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 13 Large number of events per user 2016-08-09 00:05:43.124199+00,989,129.78.56.144,"{""outcome"": ""incorrect"", ""assessment"": ""summative"", ""score"": 44.4444444444444, ""exercise"": ""/data2u/static/exco_exc/DRM/ problem_13_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196619,exco-answer 2016-08-09 00:05:44.140307+00,989,129.78.56.144,"{""score"": 44.4444444444444, ""exercise"": ""/data2u/ static/exco_exc/DRM/problem_03_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196620,exco-view 2016-08-09 00:05:47.76122+00,968,49.182.128.186,"{""question_id"":""COD-numberofwires-eqt_1"",""answer"": 1}",abelardopardo,196621,embedded-question 2016-08-09 00:05:48.861036+00,806,129.78.56.151,"{""question_id"":""COD-encodeintegers-section- eqt_4"",""answer"":0}",abelardopardo,196622,embedded-question 2016-08-09 00:05:48.959791+00,989,129.78.56.144,"{""outcome"": ""correct"", ""assessment"": ""summative"", ""score"": 50.0, ""exercise"": ""/data2u/static/exco_exc/DRM/problem_03_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196623,exco-answer 2016-08-09 00:05:49.539162+00,989,129.78.56.144,"{""score"": 50.0, ""exercise"": ""/data2u/static/ exco_exc/DRM/problem_14_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196624,exco-view 2016-08-09 00:05:49.571138+00,1311,129.78.56.159,"{""url"":""https://flip.ee.usyd.edu.au/elec1601/ Material/COD/index.html""}",abelardopardo,196625,resource-view 2016-08-09 00:05:50.050035+00,1069,129.78.56.199,"{""question_id"":""COD-integerencoding-videoeqt- eqt_3"",""answer"":0}",abelardopardo,196626,embedded-question 2016-08-09 00:05:51.709295+00,806,129.78.56.151,"{""question_id"":""COD-encodeintegers-section- eqt_4"",""answer"":0}",abelardopardo,196627,embedded-question 2016-08-09 00:05:51.962474+00,1069,129.78.56.199,"{""question_id"":""COD-integerencoding-videoeqt- eqt_3"",""answer"":0}",abelardopardo,196628,embedded-question 2016-08-09 00:05:52.0819+00,806,129.78.56.151,"{""question_id"":""COD-encodeintegers-section- eqt_4"",""answer"":""-1""}",abelardopardo,196629,embedded-question 2016-08-09 00:05:53.025356+00,1069,129.78.56.199,"{""question_id"":""COD-integerencoding-videoeqt- eqt_3"",""answer"":1}",abelardopardo,196630,embedded-question 2016-08-09 00:05:54.756229+00,1311,129.78.56.159,"{""url"":""https://flip.ee.usyd.edu.au/elec1601/ Material/COD/COD_notes.html#range-accuracy-and-precision-of-the-floating-point- representation""}",abelardopardo,196631,resource-view 2016-08-09 00:05:54.856333+00,989,129.78.56.144,"{""outcome"": ""incorrect"", ""assessment"": ""summative"", ""score"": 50.0, ""exercise"": “"/data2u/static/exco_exc/DRM/problem_03_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196623,exco-answer
  14. 14. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 14 Logs without design context • How many videos were played? • How many questions answered correctly/incorrectly? • How many pages were viewed? • How many posts in the forum? • Are posts and video plays together?* • What type of resources are being used?* • Correlation? Linear model? Student at risk? Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588-599. doi:10.1016/j.compedu.2009.09.008
  15. 15. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 15 4187, 2016-07-14 00:56:46.341946+00, {“time”":0,"id":"xEJtdMQMcrs","event":"PLAY"}, https://mycourse.com/Material/HLP/HLP_notes.html, embedded-video 4187, 2016-07-14 00:56:46.341946+00, VIDEO, xEJtdMQMcrs, PLAY, COD, W2 Combine logs with design Student played a video about topic COD during Week 2
  16. 16. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 16 Logs with design context • Is a student preparing the lecture? • Are study sessions related to schedule? • Reviewing or catching up? • Is there distributed practice? • Are there changes in strategies? ChrisLewisflickr.com
  17. 17. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 17 Detect use of resources in sessions Kahn, I., & Pardo, A. (2016). Data2U: Scalable Real time Student Feedback in Active Learning Environments. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh, UK. doi:10.1145/2883851.2883911
  18. 18. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 18 Jovanovic, J., Gašević, D., Pardo, A., Dawson, S., & Mirriahi, N. (In Press. Accepted 20/Feb/2017). Learning Analytics to Unveil Learning Strategies in a Flipped Classroom. Internet and Higher Education. doi:10.1016/j.iheduc.2017.02.001 Analyse Learning Strategies
  19. 19. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 19 Decision Trees • Week 10 • Predicted score at leaves (out of 40) • Conditions at nodes • If (EXC.in >=22) and (VID.PL < 8.5) then score = 6 Pardo, A., Mirriahi, N., Martínez-Maldonado, R., Jovanović, J., Dawson, S., & Gasevic, D. (2016). Generating Actionable Predictive Models of Academic Performance. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh, UK. doi: 10.1145/2883851.2883870
  20. 20. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 20 Ivanflickr.com Learning Analytics Data Journey Connection Model
  21. 21. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 21 Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439-1459. doi:10.1177/0002764213479367 Identifying and coding 
 learning and teaching contexts 
 is essential! Checkpoints Process Analytics JudithKleinflickr.com
  22. 22. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 22 Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gašević, D., Mulder, R., . . . Lockyer, L. (2016). A Conceptual Framework linking Learning Design with Learning Analytics. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh, UK. doi:10.1145/2883851.2883944 The3Bsflickr.com Comparative
 Analytics Cohort
 Dynamics Tool
 Specific Intervention
 tools Temporal
 Analytics
  23. 23. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 23 RobertBrookflickr.com Use actions to promote an objective 
 to drive the connection between
 design elements and analytics affordances Student Engagement
  24. 24. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 24 Bartimote-Aufflick, K., Reimann P., Pardo, A. The perspective theory brings to learning analytics in the classroom: A realist approach Manuscript in preparation. Learning Analytics Model v2
  25. 25. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 25 Questions guiding the design • What can/will be observed when the design is instantiated? • How can the data be interpreted in the design context? • What actions can be derived from the analysis? • During the enactment Instructors Students • After the enactment (design refinement)
  26. 26. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 26 4 layer connection 1. Click (low) level events 2. Process to engage with a set of tasks 3. Use effective learning strategies 4. Overall effectiveness of the design MichaelCoughlanflickr.com
  27. 27. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 27 Click/Low level Learning process with set of tasks Used learning strategies Overall effectiveness What can be observed? How to interpret data Immediate actions Students Instructor Future actions LA lense for LD
  28. 28. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 28 Example • Blended course • Large student cohort • Face-to-face sessions require preparation • Activities include: video, questions, readings, exercises (summative) • Pattern repeated weekly • Interaction through discussion forum • Three submissions throughout course • Midterm examination
  29. 29. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 29 2 Hours 2 Hours 3 Hours Pardo, A. & Mirriahi N. (in press). Design, Deployment and Evaluation of a Flipped Learning First Year Engineering Course. In C. Reidsema, Kavanagh, R. Hadgraft & N. Smith (Eds.), Flipping the Classroom: Practice and Practices. Singapore: Springer.
  30. 30. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 30 Click/Low level Learning process with set of tasks Used learning strategies Overall effectiveness What can be observed? Events in videos, questions, exercises How to interpret data If video viewed and questions correct, incorrect Immediate actions Students Suggestions to engage with resources Instructor Provide further information about topics with high failure rate Future actions Learning process
  31. 31. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 31 Click/Low level Learning process with set of tasks Used learning strategies Overall effectiveness What can be observed? Topics in each study session How to interpret data Obtain distribution, depth and effectiveness of session. Immediate actions Students Suggest plan Instructor Flag most popular strategies Future actions Where to include additional strategic support Learning strategies
  32. 32. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 32 Ivanflickr.com Learning Analytics Data Journey Connection Model How to integrate
  33. 33. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 33 Increasingly complex design space ThomasLeth-Olsenflickr.com Cognitive Social Material
 (Tangible) Material
 (Intangible)
  34. 34. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 34 “… teaching in higher education will necessarily shift the balance of its efforts towards a greater investment in design as a way of coping with otherwise intolerable pressures on staff and resources.” Goodyear, P. (2015). Teaching as Design. HERDSA Review of Higher Education, 2, 27-50. TobiasToftflickr.com
  35. 35. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics Design Patterns 35 Anneflickr.com general repeatable solution to a commonly occurring problem • Creational • Structural • Behavioural
  36. 36. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics Assume/Promote recording 36 NickLeeflickr.com
  37. 37. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics Use focal points for LA 37 EricWustenhagenflickr.com • Actions • Expected reactions • Rubrics
  38. 38. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 38 ontasklearning.org
  39. 39. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 39 • Support instructors to create personalised feedback • Simple rule-base knowledge encoding • Provide appropriate view of data sources • Scale to large and highly diverse cohorts • Will be released as open-source project Q3/4 2017 • First pilots in Q1/2 2017 • Tutorial in LAK 2017 • Contact us if interested ontasklearning.org
  40. 40. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 40 NASAJohnsonflickr.com • LA needs LD • Design may drive log analysis to meaningful questions • LA cues can be included in LD • Use of patterns and focal points Conclusions
  41. 41. Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 41
  42. 42. Abelardo Pardo (@abelardopardo)
 Faculty of Engineering and IT slideshare.net/abelardo_pardo Articulating the connection between Learning Analytics and Learning Design Faculty of Education University of Hong Kong 16/May/2017 MarianoMantelflickr.com

×