Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
BETT 2013, London — LearnLive HigherEdLearning Analytics:Unlocking student data for21st century learning?Simon Buckingham ...
70-strong lab prototyping next generationlearning / sensemaking / social web media    linked data / semantic web services ...
learning objective:      walk out withbetter questionsthan you can ask right now                             3
Why are seeing this?                       4
Why are seeing this?                       5
Why are seeing this?                       6
edX: “this is big data, giving us the chanceto ask big questions about learning”https://www.edx.org/about                 ...
A recent analytics product review…                                     8
A recent analytics product review…“Some have tried to argue thatthis technology doesnt work outcost effectively when compa...
Aquarium Analytics!                      10
11
How is your aquatic ecosystem?“This means that the keeper can be notified before waterconditions directly harm the fish—an...
How is your learning ecosystem?This means that the teacher can be notified beforelearning conditions directly harm the stu...
but you still need to know    what good looks like…and what to do when it drops…   14
15
fishlearners?         16
Purdue University Signals: real time traffic-lights for students based on predictive model                                ...
Purdue University Signals: real time traffic-  lights for students based on predictive model   MODEL:   •  ACT or SAT scor...
Purdue University Signals: real time traffic- lights for students based on predictive model       “Results thus far show t...
Enabling staff tomonitor courses                                             View profilesand student                     ...
Predictive model relates predictions to studentsuccess factors to help staff identify interventions                       ...
predictive models      are excitingbut there are many other    kinds of analytics                           22
Analytics in your VLE:Blackboard: feedback to studentshttp://www.blackboard.com/Platforms/Analytics/Products/Blackboard-An...
Adaptive platforms generate fine-grainedanalytics on curriculum masteryhttps://grockit.com/research                       ...
a data-centric culturedoesn’t have to involve advanced technology                          25
Emerging interest in learning analyticsProfessor Mark Stubbs | m.stubbs@mmu.ac.uk•  Why? Make better decisions            ...
analytics for lifelong,  lifewide learning?                          27
Why do dispositions matter?“Knowledge of methods alone will not suffice: there must be the desire, the will, to employ the...
Validated as loading onto7 dimensions of “Learning Power”        Being Stuck & Static                           Changing &...
ELLI: Effective Lifelong Learning InventoryWeb questionnaire 72 items (children and adult versions: usedin schools, univer...
Analytics for lifelong/lifewide learning dispositions: ELLIBuckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispo...
ELLI generates cohort data for eachdimension                                      32
EnquiryBlogger:Tuning Wordpress as an ELLI-based learning journalPiloting from Yr 5, to secondary, to Masters level       ...
EnquiryBlogger:Tuning Wordpress as an ELLI-based learning journalPiloting from Yr 5, to secondary, to Masters level       ...
EnquiryBlogger:Tuning Wordpress as an ELLI-based learning journalPiloting from Yr 5, to secondary, to Masters level       ...
EnquiryBlogger  dashboard – directnavigation to learner’s blogs from the visual        analytic
LearningEmergence.netmore on analytics for learning to learn, authenticenquiry, leadership and complex learning systems   ...
unpacking deeper learning           example:online student discourse    analytics that go beyond“number of forum posts”   ...
Social Network Analysis (SNAPP)                            What’s going on                      in these discussion forums...
Social Network Analysis (SNAPP)                                                                        40http://www.slides...
Social Network Analysis (SNAPP)                                                            2 learners connect             ...
Social Learning Analytics about to appear inproducts…http://www.desire2learn.com/products/analytics (this is from a beta d...
Discourse analytics: what intellectual contribution does this learner make?   Rebecca is playing   the role of broker,   c...
Semantic Social Network Analytics: shows if users agree or disagreeDe Liddo, A., Buckingham Shum, S., Quinto, I., Bachler,...
Discourse analytics on webinar textchat                                                                         Can we spo...
Discourse analytics on webinartextchat                             Given a 2.5 hour webinar, where in the live            ...
Discourse analytics on webinartextchat           Given a 2.5 hour webinar, where in the live           textchat were the m...
Discourse analytics on webinartextchat                  Given a 2.5 hour webinar, where in the live                  textc...
“Rhetorical parsing” to identify constructions signifying scholarly writing  OPEN QUESTION:  “… little is known …”  “… rol...
“What are the key contributions of this text?Human analyst                                                                ...
learning objective    – how are we doing?        walk out with better questionsthan you could ask 30mins ago              ...
How will my org. evolve from a digitalexoskeleton to a nervous system?Ed Dumbill: http://strata.oreilly.com/2012/08/digita...
The Wal-Martification of education?                                                                                       ...
Analytics provide maps  = systematic ways of distorting reality  in order to reduce complexity                   “A marker...
Will your staff know how toread and write analytics? This will become a key literacy.                                    55
What if you engaged yourlearners in the co-design of the analytics which will track             them?Think about the conve...
Are you ready foryour performance indicators to be computed from analytics?                                  57
Our analytics are our        pedagogyThey promote assessment regimes   — which drive (and strangle)      educational innov...
Join the community…SoLAResearch.org / @SoLAResearchLAKconference.org / @LAKconf                                   59
Learning Analytics Policy BriefExec Summary for UNESCO IITE         http://bit.ly/LearningAnalytics   60
BETT 2013, London — LearnLive HigherEdLearning Analytics:Unlocking student data for21st century learning?Simon Buckingham ...
Upcoming SlideShare
Loading in …5
×

Learning Analytics BETT2013

2,291 views

Published on

Published in: Education
  • Be the first to comment

Learning Analytics BETT2013

  1. 1. BETT 2013, London — LearnLive HigherEdLearning Analytics:Unlocking student data for21st century learning?Simon Buckingham ShumKnowledge Media InstituteThe Open University UKsimon.buckinghamshum.net @sbskmi #LearningAnalytics
  2. 2. 70-strong lab prototyping next generationlearning / sensemaking / social web media linked data / semantic web services 2
  3. 3. learning objective: walk out withbetter questionsthan you can ask right now 3
  4. 4. Why are seeing this? 4
  5. 5. Why are seeing this? 5
  6. 6. Why are seeing this? 6
  7. 7. edX: “this is big data, giving us the chanceto ask big questions about learning”https://www.edx.org/about 7
  8. 8. A recent analytics product review… 8
  9. 9. A recent analytics product review…“Some have tried to argue thatthis technology doesnt work outcost effectively when compared toconventional tests... but thismisses a huge point. More oftenthan not, we test after the eventand discover the problem — butthis is too late..” 9
  10. 10. Aquarium Analytics! 10
  11. 11. 11
  12. 12. How is your aquatic ecosystem?“This means that the keeper can be notified before waterconditions directly harm the fish—an assured outcome ofpredictive software that lets you know if it looks like thepH is due to drop, or the temperature is on its way up.This way, it’s a real fish saver, asopposed to a forensic examiner,post-wipeout.” (From a review of Seneye, in a hobbyist magazine) 12
  13. 13. How is your learning ecosystem?This means that the teacher can be notified beforelearning conditions directly harm the students — anassured outcome of predictive software that lets youknow if it looks like engagement is due to drop, ordistraction is on its way up.This way, it’s a real student saver,as opposed to a forensicexaminer, post-wipeout. 13
  14. 14. but you still need to know what good looks like…and what to do when it drops… 14
  15. 15. 15
  16. 16. fishlearners? 16
  17. 17. Purdue University Signals: real time traffic-lights for students based on predictive model 17
  18. 18. Purdue University Signals: real time traffic- lights for students based on predictive model MODEL: •  ACT or SAT score •  Overall grade-point average •  CMS usage composite •  CMS assessment composite •  CMS assignment composite •  CMS calendar composite Predicted 66%-80% of struggling students who needed helpCampbell et al (2007). Academic Analytics: A New Tool for a NewEra, EDUCAUSE Review, vol. 42, no. 4 (July/August 2007): 40– 1857. http://bit.ly/lmxG2x
  19. 19. Purdue University Signals: real time traffic- lights for students based on predictive model “Results thus far show that students who have engaged with Course Signals have higher average grades and seek out help resources at a higher rate than other students.”Pistilli, M. D., Arnold, K. and Bethune, M., Signals: Using AcademicAnalytics to Promote Student Success. EDUCAUSE ReviewOnline, July/Aug., (2012).http://www.educause.edu/ero/article/signals-using-academic- 19analytics-promote-student-success
  20. 20. Enabling staff tomonitor courses View profilesand student showing predictionsacademic of academic successsuccess in relation to successpredictions factors and cohort Chris Ballard, Tribal Labs / @chrisaballard / www.triballabs.net
  21. 21. Predictive model relates predictions to studentsuccess factors to help staff identify interventions Understand patterns of student activity and engagement with university services Chris Ballard, Tribal Labs / @chrisaballard / www.triballabs.net
  22. 22. predictive models are excitingbut there are many other kinds of analytics 22
  23. 23. Analytics in your VLE:Blackboard: feedback to studentshttp://www.blackboard.com/Platforms/Analytics/Products/Blackboard-Analytics-for-Learn.aspx 23
  24. 24. Adaptive platforms generate fine-grainedanalytics on curriculum masteryhttps://grockit.com/research 24
  25. 25. a data-centric culturedoesn’t have to involve advanced technology 25
  26. 26. Emerging interest in learning analyticsProfessor Mark Stubbs | m.stubbs@mmu.ac.uk•  Why? Make better decisions MMU Example: Choosing a new VLE: exploring since 2010 … VLE usage Learner patterns demographics Exam Entry results … planning wide qualifications institution- 2013 support for•  Seek to correlate variables with final success/failure•  Triangulate with extensive survey and focus groups•  Result: Critical Success Factors inform requirements for new VLE
  27. 27. analytics for lifelong, lifewide learning? 27
  28. 28. Why do dispositions matter?“Knowledge of methods alone will not suffice: there must be the desire, the will, to employ them. This desire is an affair of personal disposition.” John DeweyDewey, J. How We Think: A Restatement of the Relation of Reflective Thinkingto the Educative Process. Heath and Co, Boston, 1933 28
  29. 29. Validated as loading onto7 dimensions of “Learning Power” Being Stuck & Static Changing & Learning Data Accumulation Meaning Making Passivity Critical Curiosity Being Rule Bound Creativity Isolation & Dependence Learning Relationships Being Robotic Strategic Awareness Fragility & Dependence ResilienceUniv. Bristol and Vital Partnerships provides practitioner resources andtools to support their application in schools, HEIs and the workplace 29
  30. 30. ELLI: Effective Lifelong Learning InventoryWeb questionnaire 72 items (children and adult versions: usedin schools, universities and workplace) 30
  31. 31. Analytics for lifelong/lifewide learning dispositions: ELLIBuckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling andLearning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, Vancouver). Eprint: http://oro.open.ac.uk/32823
  32. 32. ELLI generates cohort data for eachdimension 32
  33. 33. EnquiryBlogger:Tuning Wordpress as an ELLI-based learning journalPiloting from Yr 5, to secondary, to Masters level Standard Wordpress editorhttp://learningemergence.net/tools/enquiryblogger 33
  34. 34. EnquiryBlogger:Tuning Wordpress as an ELLI-based learning journalPiloting from Yr 5, to secondary, to Masters level Categories from ELLIhttp://learningemergence.net/tools/enquiryblogger 34
  35. 35. EnquiryBlogger:Tuning Wordpress as an ELLI-based learning journalPiloting from Yr 5, to secondary, to Masters level Plugin visualizes blog categories, mirroring the ELLI spider. Direct navigation to blog posts from here 35
  36. 36. EnquiryBlogger dashboard – directnavigation to learner’s blogs from the visual analytic
  37. 37. LearningEmergence.netmore on analytics for learning to learn, authenticenquiry, leadership and complex learning systems 37
  38. 38. unpacking deeper learning example:online student discourse analytics that go beyond“number of forum posts” + “trending topics” 38
  39. 39. Social Network Analysis (SNAPP) What’s going on in these discussion forums?Bakharia, A. and Dawson, S., SNAPP: a birds-eye view of temporal participant interaction. In: Proceedings of the 1st 39International Conference on Learning Analytics and Knowledge (Banff, Alberta, Canada, 2011). ACM. pp.168-173
  40. 40. Social Network Analysis (SNAPP) 40http://www.slideshare.net/aneeshabakharia/snapp-20minute-presentation
  41. 41. Social Network Analysis (SNAPP) 2 learners connect otherwise separate clusters tutor only engaging with active students, ignoring disengaged ones on the edge 41http://www.slideshare.net/aneeshabakharia/snapp-20minute-presentation
  42. 42. Social Learning Analytics about to appear inproducts…http://www.desire2learn.com/products/analytics (this is from a beta demo) 42
  43. 43. Discourse analytics: what intellectual contribution does this learner make? Rebecca is playing the role of broker, connecting peers’ contributions in meaningful waysDe Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. Discourse-centric learning analytics. 1st InternationalConference on Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr, 2011), ACM: New York. pp.22-33 http://oro.open.ac.uk/25829
  44. 44. Semantic Social Network Analytics: shows if users agree or disagreeDe Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. Discourse-centric learning analytics. 1st InternationalConference on Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr, 2011), ACM: New York. pp.22-33 http://oro.open.ac.uk/25829
  45. 45. Discourse analytics on webinar textchat Can we spot the quality learning conversations in a 2.5 hr webinar?Ferguson, R. and Buckingham Shum, S., Learning analytics to identify exploratory dialogue within synchronous text chat. In: 1stInternational Conference on Learning Analytics and Knowledge (Banff, Canada, 2011). ACM
  46. 46. Discourse analytics on webinartextchat Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations? Not at the start and end of a webinar…Sheffield, UK not as sunny See you!as yesterday - still warm bye for now!Greetings from Hong Kong bye, and thank youMorning from Wiltshire, 80sunny here! Bye all for now 60 40 20 0 9:28 9:32 10:13 11:48 12:00 12:05 12:04 9:36 9:40 9:41 9:46 9:50 9:53 9:56 10:00 10:05 10:07 10:07 10:09 10:17 10:23 10:27 10:31 10:35 10:40 10:45 10:52 10:55 11:04 11:08 11:11 11:17 11:20 11:24 11:26 11:28 11:31 11:32 11:35 11:36 11:38 11:39 11:41 11:44 11:46 11:52 11:54 12:03 -20 -40 Average Exploratory -60
  47. 47. Discourse analytics on webinartextchat Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations? Not at the start and end of a webinar but if we zoom in on a peak… 80 60 40 20 0 9:28 9:32 10:13 11:48 12:00 12:05 12:04 9:36 9:40 9:41 9:46 9:50 9:53 9:56 10:00 10:05 10:07 10:07 10:09 10:17 10:23 10:27 10:31 10:35 10:40 10:45 10:52 10:55 11:04 11:08 11:11 11:17 11:20 11:24 11:26 11:28 11:31 11:32 11:35 11:36 11:38 11:39 11:41 11:44 11:46 11:52 11:54 12:03 -20 -40 Average Exploratory -60
  48. 48. Discourse analytics on webinartextchat Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations? Not at the start and end of a webinar but if we zoom in on a peak… Classified as “exploratory talk” (more substantive100 for learning) 50 0 9:28 “non- 9:40 9:50 10:00 10:07 10:17 10:31 10:45 11:04 11:17 11:26 11:32 11:38 11:44 11:52 12:03 -50 exploratory” Averag-100
  49. 49. “Rhetorical parsing” to identify constructions signifying scholarly writing OPEN QUESTION: “… little is known …” “… role … has been elusive” “Current data is insufficient …” CONTRASTING IDEAS: “… unorthodox view resolves …” “In contrast with previousSURPRISE: hypotheses ...”“We have recently observed ... “... inconsistent with pastsurprisingly” findings ...”“We have identified ... unusual”“The recent discovery ... suggestsintriguing roles”http://technologies.kmi.open.ac.uk/cohere/2012/01/09/cohere-plus-automated-rhetorical-annotationDe Liddo, A., Sándor, Á. and Buckingham Shum, S., Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine AnnotationStudy. Computer Supported Cooperative Work, 21, 4-5, (2012), 417-448. http://oro.open.ac.uk/31052
  50. 50. “What are the key contributions of this text?Human analyst Computational analysthttp://technologies.kmi.open.ac.uk/cohere/2012/01/09/cohere-plus-automated-rhetorical-annotationDe Liddo, A., Sándor, Á. and Buckingham Shum, S., Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine AnnotationStudy. Computer Supported Cooperative Work, 21, 4-5, (2012), 417-448. http://oro.open.ac.uk/31052
  51. 51. learning objective – how are we doing? walk out with better questionsthan you could ask 30mins ago 51
  52. 52. How will my org. evolve from a digitalexoskeleton to a nervous system?Ed Dumbill: http://strata.oreilly.com/2012/08/digital-nervous-system-big-data.html 52
  53. 53. The Wal-Martification of education? “What counts as data, how do you get it, and what does it actually mean?” “The basic question is not what can we measure? The basic question is “data narrowness” what does a good “instrumental learning” education look like? “students with no curiosity” Big questions.http://chronicle.com/blogs/techtherapy/2012/05/02/episode-95-learning-analytics-could-lead-to-wal-martification-of-college 53http://lak12.wikispaces.com/Recordings
  54. 54. Analytics provide maps = systematic ways of distorting reality in order to reduce complexity “A marker of the health of the learning analytics field will be the quality of debate around what the technology renders visible and leaves invisible.”Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies:Pedagogy, Modelling and Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29Apr-2 May, 2012, Vancouver, BC). ACM: New York. Eprint: http://oro.open.ac.uk/32823
  55. 55. Will your staff know how toread and write analytics? This will become a key literacy. 55
  56. 56. What if you engaged yourlearners in the co-design of the analytics which will track them?Think about the conversations you’d need to have… 56
  57. 57. Are you ready foryour performance indicators to be computed from analytics? 57
  58. 58. Our analytics are our pedagogyThey promote assessment regimes — which drive (and strangle) educational innovation 58
  59. 59. Join the community…SoLAResearch.org / @SoLAResearchLAKconference.org / @LAKconf 59
  60. 60. Learning Analytics Policy BriefExec Summary for UNESCO IITE http://bit.ly/LearningAnalytics 60
  61. 61. BETT 2013, London — LearnLive HigherEdLearning Analytics:Unlocking student data for21st century learning?Simon Buckingham ShumKnowledge Media InstituteThe Open University UKsimon.buckinghamshum.net @sbskmi #LearningAnalytics

×