Your SlideShare is downloading. ×
0
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Learning Analytics: what are we optimizing for?
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Learning Analytics: what are we optimizing for?

1,949

Published on

edfuture.net MOOC on Current/Future State of HigherEd …

edfuture.net MOOC on Current/Future State of HigherEd

http://edfuture.mooc.ca/archive/12/10_29_newsletter.htm

Published in: Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,949
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
43
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. edfuture.net MOOC on Current/Future State of HigherEdLearning Analyticswhat are we optimizing for? Knowledge Media Institute Simon Buckingham Shum Knowledge Media Institute The Open University UK http://twitter.com/sbskmi simon.buckinghamshum.net @ 1
  • 2. edX: “this is big data, giving us the chanceto ask big questions about learning” Will the tomorrow’s educational researcher be as helpless without an analytics infrastructure, as a geneticist without genome databases, or a physicist without CERN? 2
  • 3. the plan…joined-up multi-layer analytics an analytics ecosystem are analytics (r)evolutionary? 3
  • 4. the convergence of analytics layers 4
  • 5. Macro/Meso/Micro Learning Analytics Macro: region/state/national/international Meso: institution-wide Micro: individual user actions (and hence cohort)
  • 6. Macro/Meso/Micro Learning Analytics Macro: region/state/national/international Meso: institution-wide Micro: individual user actions (and hence cohort) Will institutions be dazzled by the dashboards, or know what questions to ask at each level?
  • 7. For examples of each level of analytic…Buckingham Shum, S. 2012. Our Learning Analytics are Our Pedagogy. Keynote Address, Expanding Horizons 2012 Conference, 7Macquarie University, Sydney. http://www.slideshare.net/sbs/our-learning-analytics-are-our-pedagogy
  • 8. The VLE—BI—ITS convergence 8
  • 9. As data migrates up it enriches higherlayers, normally accustomed to sparse data Macro: region/state/national/international Meso: institution-wide Micro: individual user actions (and hence cohort) Aggregation of user tracesenriches meso + macro analytics with finer-grained process data
  • 10. …which in turn could enrich lower layers— local patterns can be cross-validated Macro: region/state/national/international Meso: institution-wide Micro: individual user actions (and hence cohort) Aggregation of user traces Breadth + depth from macroenriches meso + macro analytics + meso levels could add with finer-grained process data power to micro-analytics
  • 11. anatomy of ananalytics ecosystem 11
  • 12. A learning analytics ecosystemlearners educators 12
  • 13. A learning analytics ecosystemlearners educators 13
  • 14. A learning analytics ecosystemlearners ?!*?!* ?!*?!* educators 14
  • 15. A learning analytics ecosystem data capture design team dashboard design teamlearners ?!*?!* data curators/ translators ?!*?!* educators 15
  • 16. Where did the data come from? learners 16
  • 17. Where did the data come from? learners theories pedagogies assessments tools researchers / educators / instructional designers 17
  • 18. Where did the data come from? learners technologists theories pedagogies assessments tools researchers / educators / instructional designers 18
  • 19. The map is not the territoryAnalytics are not the end, but a meansThe goal is to optimize the whole system outcome feedback learners design Intent theories Data pedagogies assessments tools intent researchers / educators / instructional designers 19
  • 20. Optimize the system for what? 20
  • 21. Same outcomes,but higher scores? Learning Analytics as Evolutionary Technology • more engaging • better assessed • better outcomes • deliverable at scale 21
  • 22. New outcomes wecouldn’t assess before? Learning Analytics as Revolutionary Technology • learner behaviours quantifiable • interpersonal networks quantifiable • discourse quantifiable • moods and dispositions quantifiable 22
  • 23. Learning analytics for this?“We are preparing students for jobs that do not exist yet, that will use technologies that have not been invented yet, in order to solve problems that are not even problems yet.” “Shift Happens” http://shifthappens.wikispaces.com 23
  • 24. Learning analytics for this?“While employers continue to demand high academic standards, they also now want more. They want people who can adapt, see connections, innovate, communicate and work with others. This is true in many areas of work. The new knowledge-based economies in particular will increasingly depend on these abilities. Many businesses are paying for courses to promote creative abilities, to teach the skills and attitudes that are now essential for economic success…” All our Futures: Creativity, culture & education, May 1999 24
  • 25. Learning analytics for this?Think about the analytics products and initiativesreviewed above – where would you locate them on these dimensions?Creativity, Culture andEducation (2009)Changing Young Lives2012. Newcastle: CCE.http://www.creativitycultureeducation.org/changing-young-lives-2012 25
  • 26. Learning analytics for this? The Knowledge-Agency Window co-generation Expert-led enquiry Student-led enquiry Knowledge and use Teaching as Authenticity learning design Agency Identity Repetition, Pre-scribed Knowledge Abstraction Acquisition Expert-led teaching Student-led revision Teacher agency Student agencyRuth Deakin Crick, Univ. Bristol, Centre for Systems Learning & Leadership“Pedagogy of Hope”: http://learningemergence.net/2012/09/21/pedagogy-of-hope
  • 27. analytics grounded in the principles of good assessment for learning? (not summative assessment for grading pupils, teachers, institutions or nations) 27
  • 28. Assessment for Learning Few learning analytics arehttp://assessment-reform-group.org currently able to take o board the richness of this original conception of assessment for learning 28
  • 29. Assessment for Learninghttp://assessment-reform-group.org 29
  • 30. Assessment for Learninghttp://assessment-reform-group.org To what extent could automated feedback be designed and evaluated with emotional impact in mind? 30
  • 31. Assessment for Learninghttp://assessment-reform-group.org Can analytics identify proxies for such advanced qualities? 31
  • 32. Assessment for Learninghttp://assessment-reform-group.org How do we provide analytics feedback that does not disempower and de- motivate struggling learners? 32
  • 33. analytics forlearning conversations 33
  • 34. Socio-cultural discourse analysis(Mercer et al, OU)•  Disputational talk, characterised by disagreement and individualised decision making.•  Cumulative talk, in which speakers build positively but uncritically on what the others have said.•  Exploratory talk, in which partners engage critically but constructively with each others ideas.Mercer, N. (2004). Sociocultural discourse analysis: analysing classroom talk as a socialmode of thinking. Journal of Applied Linguistics, 1(2), 137-168. 34
  • 35. Socio-cultural discourse analysis(Mercer et al, OU)•  Exploratory talk, in which partners engage critically but constructively with each others ideas. •  Statements and suggestions are offered for joint consideration. •  These may be challenged and counter-challenged, but challenges are justified and alternative hypotheses are offered. •  Partners all actively participate and opinions are sought and considered before decisions are jointly made. •  Compared with the other two types, in Exploratory talk knowledge is made more publicly accountable and reasoning is more visible in the talk.Mercer, N. (2004). Sociocultural discourse analysis: analysing classroom talk as a socialmode of thinking. Journal of Applied Linguistics, 1(2), 137-168. 35
  • 36. Analytics for identifying Exploratory talk Elluminate sessions can be very long – lasting for hours or even covering days of a conference It would be useful if we could identify where quality learning conversations seem to be taking place, so we can recommend those sessions, and not have to sit through online chat about virtual biscuitsFerguson, R. and Buckingham Shum, S. Learning analytics to identify exploratory dialogue within synchronous text chat. 361st International Conference on Learning Analytics & Knowledge (Banff, Canada, 27 Mar-1 Apr, 2011)
  • 37. Defining indicators of Exploratory Talk Category Indicator Challenge But if, have to respond, my view Critique However, I’m not sure, maybe Discussion of Have you read, more links resources Evaluation Good example, good point Explanation Means that, our goals Explicit reasoning Next step, relates to, that’s why Justification I mean, we learned, we observed Reflections of Agree, here is another, makes the perspectives of others point, take your point, your view 37
  • 38. Extract classified as Exploratory Talk Time Contribution 2:42 PM I hate talking. :-P My question was whether "gadgets" were just basically widgets and we could embed them in various web sites, like Netvibes, Google Desktop, etc. 2:42 PM Thanks, thats great! I am sure I understood everything, but looks inspiring! 2:43 PM Yes why OU tools not generic tools? 2:43 PM Issues of interoperability 2:43 PM The "new" SocialLearn site looks a lot like a corkboard where you can add various widgets, similar to those existing web start pages. 2:43 PM What if we end up with as many apps/gadgets as we have social networks and then we need a recommender for the apps! 2:43 PM My question was on the definition of the crowd in the wisdom of crowds we acsess in the service model? 2:43 PM there are various different flavours of widget e.g. Google gadgets, W3C widgets etc. SocialLearn has gone for Google gadgets 38
  • 39. Discourse analytics on webinar textchat Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations? Not at the start and end of a webinar Sheffield, UK not as sunny but if we zoom in on a peak… See you! as yesterday - still warm bye for now! Greetings from Hong Kong bye, and thank you Morning from Wiltshire, 80 sunny here! Bye all for now 60 40 20 0 9:28 9:32 10:13 11:48 12:00 12:05 12:04 9:36 9:40 9:41 9:46 9:50 9:53 9:56 10:00 10:05 10:07 10:07 10:09 10:17 10:23 10:27 10:31 10:35 10:40 10:45 10:52 10:55 11:04 11:08 11:11 11:17 11:20 11:24 11:26 11:28 11:31 11:32 11:35 11:36 11:38 11:39 11:41 11:44 11:46 11:52 11:54 12:03 -20 -40 Average Exploratory -60Extensions to: Ferguson, R. and Buckingham Shum, S. (2011). Learning Analytics to Identify Exploratory Dialogue within Synchronous Text Chat.Proc. 1st Int. Conf. Learning Analytics & Knowledge. Feb. 27-Mar 1, 2011, Banff. ACM Press. Eprint: http://oro.open.ac.uk/28955
  • 40. Discourse analytics on webinar textchat Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations? Classified as “exploratory talk” (more substantive 100 for learning) 50 0 9:28 “non- 9:40 9:50 10:00 10:07 10:17 10:31 10:45 11:04 11:17 11:26 11:32 11:38 11:44 11:52 12:03 -50 exploratory” Averag -100Wei & He extensions to: Ferguson, R. and Buckingham Shum, S. (2011). Learning Analytics to Identify Exploratory Dialogue within SynchronousText Chat. Proc. 1st Int. Conf. Learning Analytics & Knowledge. Feb. 27-Mar 1, 2011, Banff. ACM Press. Eprint: http://oro.open.ac.uk/28955
  • 41. KMi’s Cohere: a web deliberation platform enabling semantic social network and discourse network analytics Rebecca is playing the role of broker, connecting 2 peers’ contributions in meaningful waysDe Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. Discourse-centric learning analytics. 1stInternational Conference on Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr, 2011) http://oro.open.ac.uk/25829
  • 42. analytics forscholarly writing 42
  • 43. Discourse analysis (Xerox Incremental Parser)Detection of salient sentences in scholarly reports,based on the rhetorical signals authors use:BACKGROUND KNOWLEDGE: NOVELTY: OPEN QUESTION:Recent studies indicate … ... new insights provide direct evidence ... … little is known …… the previously proposed … ... we suggest a new ... approach ... … role … has been elusive Current data is insufficient …… is universally accepted ... ... results define a novel role ...CONRASTING IDEAS: SIGNIFICANCE: SUMMARIZING:… unorthodox view resolves … studies ... have provided important The goal of this study ...paradoxes … advances Here, we show ...In contrast with previous Knowledge ... is crucial for ... Altogether, our results ... indicatehypotheses ... understanding... inconsistent with past findings ... valuable information ... from studiesGENERALIZING: SURPRISE:... emerging as a promising approach We have recently observed ... surprisinglyOur understanding ... has grownexponentially ... We have identified ... unusual... growing recognition of the The recent discovery ... suggests Ágnes Sándor & OLnet Project: http://olnet.org/node/512 intriguing rolesimportance ...De Liddo, A., Sándor, Á. and Buckingham Shum, S., Contested Collective Intelligence: Rationale, Technologies, and a Human-MachineAnnotation Study. Computer Supported Cooperative Work, 21, 4-5, (2012), 417-448. http://oro.open.ac.uk/31052
  • 44. Human and machine analysis of a text for keycontributions Document 1 19 sentences annotated 22 sentences annotated 11 sentences same as human annotation Document 2 71 sentences annotated 59 sentences annotated 42 sentences same as human annotationhttp://technologies.kmi.open.ac.uk/cohere/2012/01/09/cohere-plus-automated-rhetorical-annotationDe Liddo, A., Sándor, Á. and Buckingham Shum, S., Contested Collective Intelligence: Rationale, Technologies, and a Human-MachineAnnotation Study. Computer Supported Cooperative Work, 21, 4-5, (2012), 417-448. http://oro.open.ac.uk/31052
  • 45. analytics forintepersonal networking 45
  • 46. Semantic Social Network AnalyticsDe Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. Discourse-centric learning analytics. 1stInternational Conference on Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr, 2011) http://oro.open.ac.uk/25829
  • 47. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  • 48. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  • 49. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  • 50. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  • 51. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  • 52. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  • 53. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  • 54. Closing thoughts 54
  • 55. “The basic question is not what can we measure? The basic question is what does a good education look like?” (Gardner Campbell)http://chronicle.com/blogs/techtherapy/2012/05/02/episode-95-learning-analytics-could-lead-to-wal-martification-of-collegehttp://lak12.wikispaces.com/Recordings 55
  • 56. Our analytics promote values, pedagogy and assessment regimes. Are we clear which master our analytics serve? Are wehappy to be judged by them? 56
  • 57. Will learning analytics merely turbocharge the current educational paradigm?— which is so often declared not fit for purpose… 57
  • 58. …or will learning analytics reflect what we now know about designing authentic,engaged learning, developing the new qualities that a complex society demands? 58

×