Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Learning Analytics and student feedback


Published on

Research Workshop by Prof. Denise Whitelock
Open University Institute of Educational Technology
7 May 2013

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Learning Analytics and student feedback

  1. 1. Learning Analytics and Student FeedbackProfessor Denise WhitelockThe Open University, Walton Hall,Milton Keynes MK7 6AA,
  2. 2. Learning Analytics and Student Feedback• What is LearningAnalytics?• Origins• Early work• Learning Analyticsand AssessmentDMW UOC May 2013
  3. 3. Definition“Learning Analytics are concerned with the measurement,collection, analysis and reporting of data about learnersand their contexts, for purposes of understanding andoptimising learning and the environments in which itoccurs”.ReferenceSoLAR, Open Learning Analytics: An Integrated &Modularised Platform, White Paper, Society for LearningAnalytics Research, 2011.DMW UOC May 2013
  4. 4. Interdisciplinary Research• Computer Sciences• Learning Sciences• AIED• Data MiningspecialistsDMW UOC May 2013
  5. 5. Middle Space• 3rdLAK Conference• Learning explicit• New analyticmethods• Computational• Representational• Statistical• VisualisationDMW UOC May 2013
  6. 6. Can LAK hold together for long?• Challenges• Differentmethodologies• Different theories• Different predjucies• Agreement on topics 3rdLAK 2013• Visualisation, social networkanalysis, communicationand collaboration, discourseanalytics, predictiveanalytics, sequenceanalytics, assessmentAfter Suthers & Verbert (2013)DMW UOC May 2013
  7. 7. Political and economic drivers• Educause Review (2007)• Academic Analytics,Campbell & Oblinger (2007)• Large data and stats =predictive modelling• Improve number ofgraduates in US• US finding now with schoolexam data• Hand code• M.L.• Apply whote set• Make predictionsDMW UOC May 2013
  8. 8. Formative Research towards separate field• Open Learner models (Bull & Kay, 2007)• Social Network analysis (De Laat et al, 2007)• Networks Adapting Pedagogical Practice (SNAPP),(Dawson et al, 2010)• Visualisation of large data sets, Honeycomb (van Hamet al, 2009)• Gephi: open source tool (Bastian et al, 2009)• Signals (Arnold, 2010)DMW UOC May 2013
  9. 9. Signals: Flagship Project• Moves data from VLE• Combines with predictionmodels• Real time red/amber/greentraffic lights• Pilot study (Arnold, 2010)showed• Students sought helpearlier• 12% more B/C grades• 14% less D/F gradesDMW UOC May 2013
  10. 10. Formative Assessment and LearningAnalytics (1)Tempelaar et al, 2013• 1styear Math & Statsundergraduates,Maastricht• Reason text book• Online questions• Practice andperformance tests• 92% higher practicepass• 51% lower practice passDMW UOC May 2013
  11. 11. Formative Assessment and LearningAnalytics (2)Important for SAFeSEA?• Learning styles (Vermunt, 1996)• Self regulation for deep learning• Practice for stepwise learning• Motivation and engagement wheel (Martin,2007)• Learning emotions• Pekrun’s control-value theory of learningemotionsDMW UOC May 2013
  12. 12. Performance of video lectures• Findings from Mirriahi &Dawson (2013)• Correlations betweenquizzes and lectures• Shows misalignmentbetween Assessmentand Teachingmaterials?DMW UOC May 2013
  13. 13. HOU2LEARN• PLE from Hellenic OpenUniversity• Social network analysisand final grades• Online collaboration isnot a predictor for finalgrade• Koulocheri & Xenos,2013DMW UOC May 2013
  14. 14. OpenEssayist: SAFeSEA Web application forsummarisation-based formative feedbackDMW UOC May 2013
  15. 15. Key words and phrases visualized in the essay context. Sentences inlight-grey (green) background are key sentences as extracted by theEssayAnalyser (the number at the start of the sentence indicates itskey-ness ranking); bigrams are indicated in bold (red) and boxed.DMW UOC May 2013
  16. 16. The structural elements of the essay can be used jointly withthe key word extraction to highlight relevant information withinspecific parts of the essay, here the introduction (and theassignment question)DMW UOC May 2013
  17. 17. Key words and phrases as separate listsDMW UOC May 2013
  18. 18. Dispersion of key words across the essay UOC May 2013
  19. 19. Can we find ways of using graph visualizationtechniques on the key words and key sentences, tomake them helpful and meaningful to students?DMW UOC May 2013
  20. 20. Final Thoughts• Instant machinefeedback not prevalent• Artificial Intelligenceanalysis to tutorsleading to Wizard of Ozresponses (Shaffer,2013)• Just in time feedback isthe ultimate goalDMW UOC May 2013
  21. 21. References (1)Arnold, K.E. (2010). Signals: applying academic analytics, EducauseQuarterly, 33(1), p10. 30 April 2013)Bastien, M., Heymann, S. & Jacomy, M. (2009). Gephi: an open sourcesoftware for exploring and manipulating networks. Paper presented atthe International AAAI Conference on Weblogs and Social Media.Bull, S & Kay, J. (2007). Student models that invite the learner in: theSMILI:-) open learner modelling framework, International Journal ofArtificial Intelligence in Education, 17(2).Campbell, J.P. & Oblinger, D.G. (2007). Academic Analytics,Educause.Dawson, S., Bakharia, A. & Heathcote, E. (2010). Snapp: Realising theaffordances of real-time SNA within networked learning environments.Paper presented at The 7thInternational Conference on NetworkedLearning, Aalborg, Denmark (3-4 May).DMW UOC May 2013
  22. 22. References (2)De Laat, M., Lally, V., Lipponen, L. & Simons, R.-J. (2007).Investigating patterns of interaction in networked learning andcomputer-supported collaborative learning: a role for social networkanalysis. International Journal of Computer Supported CollaborativeLearning, 2, pp.87-103.Koulocheri, E. & Xenos, M. (2013). Considering Formal Assessment inLearning Analytics within a PLE: The HOU2LEARN Case. Paperpresented at The International Learning Analytics & Knowledge (LAK)Conference, Leuven, Belgium, (8-12 April 2013)© 2013 ACM 978-1-4503-1785-6/13/04Martin, A.J. (2007). Examining a multidimensional model of studentmotivation and engagement using a construct validation approach.British Journal of Educational Psychology, 77, 413-440.DMW UOC May 2013
  23. 23. References (3)Mirriahi, N. & Dawson, S. (2013) The pairing of lecture recording datawith assessment scores: A method of discovering pedagogical impact.Paper presented at The International Learning Analytics & Knowledge(LAK) Conference, Leuven, Belgium, (8-12 April 2013)© 2013 ACM 978-1-4503-1785-6/13/04Pekrun, R. (2006). The control-value theory of achievement emotions:assumptions, corollaries and implications for educational research andpractice. Educational Psychology Review, 18, 315-34.SoLAR, Open Learning Analytics: An Integrated & ModularisedPlatform, White Paper, Society for Learning Analytics Research, 2011.Suthers, D. & Verbert, K. (2013) Learning Analytics as a “MiddleSpace”. Paper presented at The International Learning Analytics &Knowledge (LAK) Conference, Leuven, Belgium, (8-12 April 2013)© 2013 ACM 978-1-4503-1785-6/13/04DMW UOC May 2013
  24. 24. References (4)Tempelaar, D.T., Cuypers, H., van de Vrie, E., Heck, A. & van derKooij, H. (2013). Formative Assessment and Learning Analytics. Paperpresented at The International Learning Analytics & Knowledge (LAK)Conference, Leuven, Belgium, (8-12 April 2013)© 2013 ACM 978-1-4503-1785-6/13/04Van Ham, F., Schulz, H.-J. & Dimicco, J.M. (2009). Honeycomb: visualanalysis of large scale social networks, Ifip International Federation forInformation Processing, 429-442.Van Labeke, N., Whitelock, D., Field, D., Pulman, S. & Richardson, J.(2013) ‘OpenEssayist: Extractive Summarisation & FormativeAssessment of Free-Text Essays’. Workshop on Discourse-CentricLearning Analytics, 3rdConference on Learning Analytics andKnowledge (LAK 2013), Leuven, BelgiumVermunt, J.D. (1996). Leerstijlen en sturen van leerprocessen in hetHoger Onderwijs. Amsterdam/Lisse: Swets & Zeitlinger.DMW UOC May 2013