ICO Fall School 2012, Santuari de Santa Maria del Collell, Gironahttps://sites.google.com/site/icofallschool2012Learning A...
What does ‘the cloud’ know about you?                        2
The plan today   1.  Introductions + intro lecture2.  designing your own analytics v1                                     ...
Introducing a newAnalytics Platform…                      4
From a recent review…“Some have tried to argue thatthis technology doesnt work outcost effectively when compared toconvent...
Aquarium Analytics!                      6
Aquarium Analytics!                      7
How is your aquatic ecosystem?“This means that the keeper can be notified before waterconditions directly harm the fish—an...
How is your learning ecosystem?This means that the teacher can be notified beforelearning conditions directly harm the stu...
First-to-market immaturity, tricky install process…                 …but when it’s done calibrating and the dashboard     ...
is educationpoised to become a data-driven enterprise and science    ?    11
Possibly 90% of the digital data we havetoday was generated in the last 2 years Volume        outstrips old infrastructure...
edX: “this is big data, giving us the chanceto ask big questions about learning”                                     Will ...
Lifelogging: explosion of data captureand sharing about personal activities                                            htt...
Educational Data Mining research community
Learning Analytics research community
What do we mean byLearning Analytics?                      17
Learning analytics“Learning Analytics is concernedwith the collection, analysis andreporting of data about learning in ara...
A learning analytics ecosystemlearners                                 educators                                          ...
A learning analytics ecosystemlearners                                 educators                                          ...
learning analytics data    collection cycle                          21
Analytics cycle (Doug Clow)                                                                                               ...
Analytics cycle (George Siemens)                                                                                    	  h"p...
A learning analytics ecosystemlearners      ?!*?!*                       ?!*?!*    educators                              ...
A learning analytics ecosystem                                 dashboard                                 design teamlearne...
Where did the data come from?             learners                                26
Where did the data come from?                 learners                 theories              pedagogies             assess...
Where did the data come from?                 learners                                                  technologists     ...
The map is not the territoryAnalytics are not the end, but a meansThe goal is to optimize the whole system                ...
Optimize the system     for what?                      30
Same outcomes,but higher scores?  Learning Analytics asEvolutionary Technology        • more engaging       • better asses...
New outcomes wecouldn’t assess before?      Learning Analytics as Revolutionary Technology      • learner behaviours quant...
different levels   of analytic                   33
‘Learning Analytics’ and‘Academic Analytics’Long, P. and Siemens, G. (2011), Penetrating the fog: analytics in learning an...
Macro/Meso/Micro Learning Analytics                    Macro:      region/state/national/international
Macro/Meso/Micro Learning Analytics                    Macro:      region/state/national/international                    ...
Macro/Meso/Micro Learning Analytics                    Macro:      region/state/national/international                    ...
Macro/Meso/Micro Learning Analytics                    Macro:      region/state/national/international
US states are getting the infrastructurein placedataqualitycampaign.org                                           39
National league tables for English schools 40
Macro/Meso/Micro Learning Analytics                   Meso:             institution-wide
Analytics-savvy Leaders are the future?Parr-Rud, O. (2012). Drive Your Business with Predictive Analytics. SAS White Paper...
Business Intelligence companies see aneducation market opening up                                                 These ar...
Business Intelligence companies see aneducation market opening up                            …but do they know anything ab...
BI+HigherEd communities of practice                                      45
Business Intelligence         ≠ Learning Analytics
Macro/Meso/Micro Learning Analytics                   Micro:           individual user actions               (and hence co...
Analytics in your VLE:Blackboard: feedback to studentshttp://www.blackboard.com/Platforms/Analytics/Overview.aspx         ...
Socrato: train for SATshttp://www.socrato.com    49
Khan Academy: more data to teachers,finer-grained feedback to studentshttp://www.thegatesnotes.com/Topics/Education/Sal-Kh...
Adaptive platforms generate fine-grained analyticshttps://grockit.com/research                                    51
Adaptive platforms generate fine-grainedanalytics          http://knewton.com
Adaptive platforms generate fine-grainedanalyticshttp://oli.cmu.edu
Purdue University Signals: real time traffic-lights for students based on predictive model                  Premise: acade...
Desire2Learn visual analytics & predictive modelswhich can be interrogated on different dimensionshttp://www.desire2learn....
Desire2Learn visual analytics & predictive modelswhich can be interrogated on different dimensionshttp://www.desire2learn....
The VLE—BI convergence                         57
Hard distinctions between Learning +Academic analytics may dissolve…as they get joined up, each level enriches the others ...
Hard distinctions between Learning +Academic analytics may dissolve…as they get joined up, each level enriches the others ...
…so everybody’s happy?dawn of a new data-driven  enterprise + science?                         60
wrong.a very healthy debate     is brewing…                        61
data (indeed technology)    is not neutraldata does not wholly  ‘speak for itself’                             62
Measurement tools are not neutral“accounting tools...do not simply aid the measurement of economic activity, they shape th...
Analytics provide maps = systematicways of distorting reality“A marker of the health of the learning analytics field will ...
course completion  is only one proxy   for good learning and what’s easy tomeasure isn’t alwayswhat’s most important      ...
The Wal-Martification of education?http://chronicle.com/blogs/techtherapy/2012/05/02/episode-95-learning-analytics-could-l...
The Wal-Martification of education?                                                                                       ...
contextcontextcontext          68
Video conferencing analyticsOU KMi’s FlashmeetingVideo conference spoken foreign language tutorials                   — wh...
Video conferencing analyticsOU KMi’s FlashmeetingVideo conference spoken foreign language tutorials                   — wh...
Video conferencing analyticsOU KMi’s FlashmeetingVideo conference spoken foreign language tutorials                   — wh...
Video conferencing analyticsOU KMi’s FlashmeetingVideo conference spoken foreign language tutorials                   — wh...
contextcontextcontext          73
Learning analytics in English schools 74
Learning analytics in English schools 75
Will our analytics reflect the progress that‘Joe’ has made on so many other fronts –but not his sats? ?76
let’s just pretend that learning analytics took seriously the revolution going on outside the              university fron...
Learning analytics for this?“We are preparing students for jobs that do not exist yet, that will use technologies that hav...
Learning analytics for this?“While employers continue to demand high academic standards, they also now want more. They wan...
Learning analytics for this?  “Knowledge of methods alone   will not suffice: there must be   the desire, the will, to emp...
Learning analytics for this?“The test of successful education is not the amount of knowledge that pupils take away from sc...
Learning analytics for this?                         The Knowledge-Agency Window co-generation                   Expert-le...
consider    assessment    for learning(not summative assessment for   grading pupils, teachers,     institutions or nation...
Assessment for Learninghttp://assessment-reform-group.org                                     84
Assessment for Learninghttp://assessment-reform-group.org                                     85
Assessment for Learninghttp://assessment-reform-group.org        To what extent       could automated         feedback be ...
Assessment for Learninghttp://assessment-reform-group.org                                      Can analytics              ...
Assessment for Learninghttp://assessment-reform-group.org                                     Do analytics provide        ...
Assessment for Learninghttp://assessment-reform-group.org                      How do we provide                       ana...
analytics for…                  dispositions                   discourse                 social networks                  ...
Social Learning Analytics      §  Analytics focused on social learning theories,          practices and platforms, e.g.  ...
Socio-cultural discourse analysis(Mercer et al, OU)•  Disputational talk, characterised by disagreement and   individualis...
Socio-cultural discourse analysis(Mercer et al, OU)•  Exploratory talk, in which partners engage critically but   construc...
Analytics for identifying Exploratory talk        Elluminate sessions can        be very long – lasting for        hours o...
Defining indicators of Exploratory Talk  Category               Indicator  Challenge              But if, have to respond,...
Extract classified as Exploratory Talk  Time     Contribution 2:42 PM I hate talking. :-P My question was whether "gadgets...
Discourse analytics on webinar textchat                                         Given a 2.5 hour webinar, where in the liv...
Discourse analytics on webinar textchat     Given a 2.5 hour     webinar, where in the     live textchat were the     most...
KMi’s Cohere: a web deliberation platform enabling semantic social network and discourse network analytics   Rebecca is pl...
analytics forscholarly writing                    100
Discourse analysis (Xerox Incremental Parser)Detection of salient sentences in scholarly reports,based on the rhetorical s...
Human and machine analysis of a text for keycontributions             Document 1           19 sentences annotated         ...
analytics for reflecting on  “networked expertise”    (a key skill for our times)                                  103
Semantic Social Network AnalyticsDe Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. Discour...
Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Envir...
Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Envir...
Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Envir...
Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Envir...
Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Envir...
Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Envir...
Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Envir...
DispositionalLearning Analytics                     112
Dispositions are beginning to registerwithin the learning analytics communityBrown, M., Learning Analytics: Moving from Co...
In your experience, what are the qualitiesshown by the most effective learners?   Think about the most effective learners ...
A ‘visual learning analytic’  7-dimensional spider diagram of how the learner sees themself                               ...
ELLI: Effective Lifelong Learning InventoryWeb questionnaire 72 items (children and adult versions: usedin schools, univer...
Validated as loading onto7 dimensions of “Learning Power”       Being Stuck & Static                        Changing & Lea...
Learning to Learn: 7 Dimensions of Learning PowerFactor analysis of the literature plus expert interviews: identified seve...
Learning to Learn: 7 Dimensions of Learning PowerFactor analysis of the literature plus expert interviews: identified seve...
Learning Warehouse 2.0 analytics platform                User experience:       Research-validated assessment tools       ...
Adding imagery to ELLI dimensions toconnect with learner identity                                       121
Working with Gappuwiyak School, N. Territory AUS (Ruth Deakin Crick, University of Bristol)   http://bit.ly/srUSHE Changin...
Cohort analytics foreducators andorganizational leaders                         123
EnquiryBlogger:Tuning Wordpress as an ELLI-based learning journal                                   Standard Wordpress edi...
Primary School EnquiryBloggersBushfield School, Wolverton, UKEnquiryBlogger: blogging for Learning Power & Authentic Enqui...
EnquiryBlogger  dashboard
Could a platform generate an ELLI profile from user traces?                                                               ...
SocialLearn provides new possibilities of      looking at learners learning      ELLI works from what                     ...
ELLI feedbacks inform development of learning                                                                             ...
Dream? Student’s analytics dashboard                                       1                                              ...
Closingthoughts           131
“The basic question is not                              what can we measure?                       The basic question is  ...
Will learning analytics merely  turbocharge the current   educational paradigm?— which is so often declared   not fit for ...
…or will learning analytics  reflect what we now know about designing authentic,engaged learning, developing   the new qua...
Learning Analytics is becoming a newdiscipline and research fieldwww.SoLAResearch.orgFollow: @SoLAResearch             Lea...
Invent your own Analytics cyclebased on your research interests…                         What kinds of learners?          ...
Learning Analytics    workshop      Day 2                     137
day 2 plan1.    Post-it affinity mapping2.    Team dashboard design3.    Plenary presentations4.    LAnoirblanc photo shoo...
What are we interested in?(Affinity Mapping exercise)     Focus     Subject      Subject    e.g. maths     e.g. maths     ...
DIY AnalyticsElaborated version of figure from Doug Clow:h"p://www.slideshare.net/dougclow/the-­‐learning-­‐analy7cs-­‐cyc...
design your own   analytics   dashboard                  141
LAnoirblancreactions to Learning Analytics in image and story                                  LAnoirblanc.tumblr.com     ...
Emailing yourphoto…                143
LAnoirblanc        Add your photo and story to the website1.  Take a photo or choose an image from the web2.  Email it + y...
Upcoming SlideShare
Loading in …5
×

ico-fallschool2012-learninganalyticswkshp

2,404 views

Published on

ICO Fall School 2012, Santuari de Santa Maria del Collell, Girona https://sites.google.com/site/icofallschool2012

A week long PhD training school for educational and ed-tech researchers

Published in: Education
1 Comment
2 Likes
Statistics
Notes
  • Excellent workshop. I really liked it.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
2,404
On SlideShare
0
From Embeds
0
Number of Embeds
205
Actions
Shares
0
Downloads
69
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

ico-fallschool2012-learninganalyticswkshp

  1. 1. ICO Fall School 2012, Santuari de Santa Maria del Collell, Gironahttps://sites.google.com/site/icofallschool2012Learning AnalyticsWorkshop (8-9 Nov. 2012) Knowledge Media InstituteSimon Buckingham ShumKnowledge Media Institute, The Open University UKsimon.buckinghamshum.net @sbskmi linkedin.com/in/simon 1
  2. 2. What does ‘the cloud’ know about you? 2
  3. 3. The plan today 1.  Introductions + intro lecture2.  designing your own analytics v1 3
  4. 4. Introducing a newAnalytics Platform… 4
  5. 5. From a recent review…“Some have tried to argue thatthis technology doesnt work outcost effectively when compared toconventional tests... but thismisses a huge point. More oftenthan not, we test after the eventand discover the problem — butthis is too late..” 5
  6. 6. Aquarium Analytics! 6
  7. 7. Aquarium Analytics! 7
  8. 8. How is your aquatic ecosystem?“This means that the keeper can be notified before waterconditions directly harm the fish—an assured outcome ofpredictive software that lets you know if it looks like thepH is due to drop, or the temperature is on its way up.This way, it’s a real fish saver, asopposed to a forensic examiner,post-wipeout.” (From a review of Seneye, in a hobbyist magazine) 8
  9. 9. How is your learning ecosystem?This means that the teacher can be notified beforelearning conditions directly harm the students — anassured outcome of predictive software that lets youknow if it looks like engagement is due to drop, orattainment is on its way up.This way, it’s a real student saver,as opposed to a forensicexaminer, post-wipeout. 9
  10. 10. First-to-market immaturity, tricky install process… …but when it’s done calibrating and the dashboard springs to life, there’s an exciting sense of control – BUT you still need to know what ‘good’ looks like 10
  11. 11. is educationpoised to become a data-driven enterprise and science ? 11
  12. 12. Possibly 90% of the digital data we havetoday was generated in the last 2 years Volume outstrips old infrastructure Variety Internet of things, e-business transactions, environmental sensors, social media, audio, video, mobile… Velocity The speed of data access and analysis is exploding A quantitative shift on this scale is in fact a qualitative shift, requiring new ways of thinking about societal phenomena 12
  13. 13. edX: “this is big data, giving us the chanceto ask big questions about learning” Will the tomorrow’s educational researcher be as helpless without an analytics infrastructure, as a geneticist without genome databases, or a physicist without CERN? 13
  14. 14. Lifelogging: explosion of data captureand sharing about personal activities http://www.mirror-project.eu http://quantifiedself.com/guide 14
  15. 15. Educational Data Mining research community
  16. 16. Learning Analytics research community
  17. 17. What do we mean byLearning Analytics? 17
  18. 18. Learning analytics“Learning Analytics is concernedwith the collection, analysis andreporting of data about learning in arange of contexts, including informallearning, academic institutions, andthe workplace.It informs and provides input foraction to support and enhancelearning experiences, and thesuccess of learners.”2nd Int. Conf. Learning Analytics & Knowledge 2012
  19. 19. A learning analytics ecosystemlearners educators 19
  20. 20. A learning analytics ecosystemlearners educators 20
  21. 21. learning analytics data collection cycle 21
  22. 22. Analytics cycle (Doug Clow)  h"p://www.slideshare.net/dougclow/the-­‐learning-­‐analy7cs-­‐cycle-­‐closing-­‐the-­‐loop-­‐effec7vely  (slide  5) 22
  23. 23. Analytics cycle (George Siemens)  h"p://www.slideshare.net/gsiemens/eli-­‐2012-­‐sensemaking-­‐analy7cs  (slide  7) 23
  24. 24. A learning analytics ecosystemlearners ?!*?!* ?!*?!* educators 24
  25. 25. A learning analytics ecosystem dashboard design teamlearners ?!*?!* data curators/ translators ?!*?!* educators 25
  26. 26. Where did the data come from? learners 26
  27. 27. Where did the data come from? learners theories pedagogies assessments tools researchers / educators / instructional designers 27
  28. 28. Where did the data come from? learners technologists theories pedagogies assessments tools researchers / educators / instructional designers 28
  29. 29. The map is not the territoryAnalytics are not the end, but a meansThe goal is to optimize the whole system outcome feedback learners design Intent theories Data pedagogies assessments tools intent researchers / educators / instructional designers 29
  30. 30. Optimize the system for what? 30
  31. 31. Same outcomes,but higher scores? Learning Analytics asEvolutionary Technology • more engaging • better assessed • better outcomes • deliverable at scale 31
  32. 32. New outcomes wecouldn’t assess before? Learning Analytics as Revolutionary Technology • learner behaviours quantifiable • interpersonal networks quantifiable • discourse quantifiable • moods and dispositions quantifiable 32
  33. 33. different levels of analytic 33
  34. 34. ‘Learning Analytics’ and‘Academic Analytics’Long, P. and Siemens, G. (2011), Penetrating the fog: analytics in learning and education. Educause Review Online,46, 5, pp.31-40. http://www.educause.edu/ero/article/penetrating-fog-analytics-learning-and-education 34
  35. 35. Macro/Meso/Micro Learning Analytics Macro: region/state/national/international
  36. 36. Macro/Meso/Micro Learning Analytics Macro: region/state/national/international Meso: institution-wide
  37. 37. Macro/Meso/Micro Learning Analytics Macro: region/state/national/international Meso: institution-wide Micro: individual user actions (and hence cohort)
  38. 38. Macro/Meso/Micro Learning Analytics Macro: region/state/national/international
  39. 39. US states are getting the infrastructurein placedataqualitycampaign.org 39
  40. 40. National league tables for English schools 40
  41. 41. Macro/Meso/Micro Learning Analytics Meso: institution-wide
  42. 42. Analytics-savvy Leaders are the future?Parr-Rud, O. (2012). Drive Your Business with Predictive Analytics. SAS White Paperhttp://www.sas.com/reg/gen/corp/1800392 42
  43. 43. Business Intelligence companies see aneducation market opening up These are pedagogically agnostic: they seek to optimize operational efficiency whatever the sector These may make pedagogical assumptions: how will learning design and assessment regimes shape the analytics they offer?http://www.sas.com/industry/education/highered 43
  44. 44. Business Intelligence companies see aneducation market opening up …but do they know anything about the roles that language plays in learning and knowledge construction? 44
  45. 45. BI+HigherEd communities of practice 45
  46. 46. Business Intelligence ≠ Learning Analytics
  47. 47. Macro/Meso/Micro Learning Analytics Micro: individual user actions (and hence cohort)
  48. 48. Analytics in your VLE:Blackboard: feedback to studentshttp://www.blackboard.com/Platforms/Analytics/Overview.aspx 48
  49. 49. Socrato: train for SATshttp://www.socrato.com 49
  50. 50. Khan Academy: more data to teachers,finer-grained feedback to studentshttp://www.thegatesnotes.com/Topics/Education/Sal-Khan-Analytics-Khan-Academy 50
  51. 51. Adaptive platforms generate fine-grained analyticshttps://grockit.com/research 51
  52. 52. Adaptive platforms generate fine-grainedanalytics http://knewton.com
  53. 53. Adaptive platforms generate fine-grainedanalyticshttp://oli.cmu.edu
  54. 54. Purdue University Signals: real time traffic-lights for students based on predictive model Premise: academic success is defined as a function of aptitude (as measured by standardized test scores and similar information) and effort (as measured by participation within the online learning environment). Using factor analysis and logistic regression, a model was tested to predict student success based on: •  ACT or SAT score •  Overall grade-point average Predicted 66%-80% •  CMS usage composite of struggling •  CMS assessment composite students who •  CMS assignment composite needed help •  CMS calendar composite Campbell et al (2007). Academic Analytics: A New Tool for a New Era, EDUCAUSE Review, vol. 42, no. 4 (July/August 2007): 40–57. http://bit.ly/lmxG2x 54
  55. 55. Desire2Learn visual analytics & predictive modelswhich can be interrogated on different dimensionshttp://www.desire2learn.com/products/analytics 55
  56. 56. Desire2Learn visual analytics & predictive modelswhich can be interrogated on different dimensionshttp://www.desire2learn.com/products/analytics 56
  57. 57. The VLE—BI convergence 57
  58. 58. Hard distinctions between Learning +Academic analytics may dissolve…as they get joined up, each level enriches the others Macro: region/state/national/international Meso: institution-wide Micro: individual user actions (and hence cohort) Aggregation of user tracesenriches meso + macro analytics with finer-grained process data
  59. 59. Hard distinctions between Learning +Academic analytics may dissolve…as they get joined up, each level enriches the others Macro: region/state/national/international Meso: institution-wide Micro: individual user actions (and hence cohort) Aggregation of user traces Breadth + depth from macroenriches meso + macro analytics + meso levels add power to with finer-grained process data micro analytics
  60. 60. …so everybody’s happy?dawn of a new data-driven enterprise + science? 60
  61. 61. wrong.a very healthy debate is brewing… 61
  62. 62. data (indeed technology) is not neutraldata does not wholly ‘speak for itself’ 62
  63. 63. Measurement tools are not neutral“accounting tools...do not simply aid the measurement of economic activity, they shape the reality they measure” Du Gay, P. and Pryke, M. (2002) Cultural Economy: Cultural Analysis and Commercial Life Sage, London. pp. 12-13
  64. 64. Analytics provide maps = systematicways of distorting reality“A marker of the health of the learning analytics field will be the quality of debate around what the technology renders visible and leaves invisible.” Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, 2012, Vancouver, BC). ACM: New York. Eprint: http://oro.open.ac.uk/32823
  65. 65. course completion is only one proxy for good learning and what’s easy tomeasure isn’t alwayswhat’s most important 65
  66. 66. The Wal-Martification of education?http://chronicle.com/blogs/techtherapy/2012/05/02/episode-95-learning-analytics-could-lead-to-wal-martification-of-college 66http://lak12.wikispaces.com/Recordings
  67. 67. The Wal-Martification of education? “What counts as data, how do you get it, and what does it actually mean?” “The basic question is not what can we measure? The basic question is “data narrowness” what does a good “instrumental learning” education look like? “students with no curiosity” Big questions.http://chronicle.com/blogs/techtherapy/2012/05/02/episode-95-learning-analytics-could-lead-to-wal-martification-of-college 67http://lak12.wikispaces.com/Recordings
  68. 68. contextcontextcontext 68
  69. 69. Video conferencing analyticsOU KMi’s FlashmeetingVideo conference spoken foreign language tutorials — which mentor would you want to have?... Mentor 1 Mentor 2 AV Chat AV ChatSession 2 3 69
  70. 70. Video conferencing analyticsOU KMi’s FlashmeetingVideo conference spoken foreign language tutorials — which mentor would you want to have?... Mentor 1 Mentor 2 AV Chat AV Chat 1Session 2 3 70
  71. 71. Video conferencing analyticsOU KMi’s FlashmeetingVideo conference spoken foreign language tutorials — which mentor would you want to have?... Mentor 1 Mentor 2 AV Chat AV Chat 1Session 2 3 71
  72. 72. Video conferencing analyticsOU KMi’s FlashmeetingVideo conference spoken foreign language tutorials — which mentor would you want to have?... Mentor 1 Mentor 2 AV Chat AV Chat 1Session Mentor 1 is doing the best job: at this introductory 2 level, students need intensive input and flounder if left 3 72
  73. 73. contextcontextcontext 73
  74. 74. Learning analytics in English schools 74
  75. 75. Learning analytics in English schools 75
  76. 76. Will our analytics reflect the progress that‘Joe’ has made on so many other fronts –but not his sats? ?76
  77. 77. let’s just pretend that learning analytics took seriously the revolution going on outside the university front door…We need to devise learning analytics for this?... 77
  78. 78. Learning analytics for this?“We are preparing students for jobs that do not exist yet, that will use technologies that have not been invented yet, in order to solve problems that are not even problems yet.” “Shift Happens” http://shifthappens.wikispaces.com 78
  79. 79. Learning analytics for this?“While employers continue to demand high academic standards, they also now want more. They want people who can adapt, see connections, innovate, communicate and work with others. This is true in many areas of work. The new knowledge-based economies in particular will increasingly depend on these abilities. Many businesses are paying for courses to promote creative abilities, to teach the skills and attitudes that are now essential for economic success…” All our Futures: Creativity, culture & education, May 1999 79
  80. 80. Learning analytics for this? “Knowledge of methods alone will not suffice: there must be the desire, the will, to employ them. This desire is an affair of personal disposition.” John Dewey, 1933Dewey, J. How We Think: A Restatement of the Relation of Reflective Thinking to theEducative Process. Heath and Co, Boston, 1933 80
  81. 81. Learning analytics for this?“The test of successful education is not the amount of knowledge that pupils take away from school, but their appetite to know and their capacity to learn.” Sir Richard Livingstone, 1941 81
  82. 82. Learning analytics for this? The Knowledge-Agency Window co-generation Expert-led enquiry Student-led enquiry Knowledge and use Teaching as Authenticity learning design Agency Identity Repetition, Pre-scribed Knowledge Abstraction Acquisition Expert-led teaching Student-led revision Teacher agency Student agency
  83. 83. consider assessment for learning(not summative assessment for grading pupils, teachers, institutions or nations) 83
  84. 84. Assessment for Learninghttp://assessment-reform-group.org 84
  85. 85. Assessment for Learninghttp://assessment-reform-group.org 85
  86. 86. Assessment for Learninghttp://assessment-reform-group.org To what extent could automated feedback be designed and evaluated with emotional impact in mind? 86
  87. 87. Assessment for Learninghttp://assessment-reform-group.org Can analytics identify proxies for such advanced qualities? 87
  88. 88. Assessment for Learninghttp://assessment-reform-group.org Do analytics provide constructive next steps? 88
  89. 89. Assessment for Learninghttp://assessment-reform-group.org How do we provide analytics feedback that does not disempower and de- motivate struggling learners? 89
  90. 90. analytics for… dispositions discourse social networks See SoLAR Storm: Social Learning Analytics symposium 90http://www.open.ac.uk/blogs/SocialLearnResearch/2012/06/20/social-learning-analytics-symposium
  91. 91. Social Learning Analytics §  Analytics focused on social learning theories, practices and platforms, e.g. §  Discourse analytics: beyond quantitative summaries of online writing, to qualitative analysis §  Social network analytics: visualizing effective social ties for collective learning §  Dispositional analytics: measuring students’ readiness to engage in lifelong, lifewide learningFerguson R and Buckingham Shum S. (2012) Social Learning Analytics: Five Approaches. Proc. 2nd International Conference on Learning Analytics & Knowledge. Vancouver, 29 Apr-2 May: ACM Press. Eprint: http://oro.open.ac.uk/32910Buckingham Shum, S. and Ferguson, R., Social Learning Analytics. Educational Technology & Society (Special Issue on Learning & Knowledge Analytics, Eds. G. Siemens & D. Gašević), 15, 3, (2012), 3-26. http://www.ifets.info Open Access Eprint: http://oro.open.ac.uk/34092
  92. 92. Socio-cultural discourse analysis(Mercer et al, OU)•  Disputational talk, characterised by disagreement and individualised decision making.•  Cumulative talk, in which speakers build positively but uncritically on what the others have said.•  Exploratory talk, in which partners engage critically but constructively with each others ideas.Mercer, N. (2004). Sociocultural discourse analysis: analysing classroom talk as a socialmode of thinking. Journal of Applied Linguistics, 1(2), 137-168. 92
  93. 93. Socio-cultural discourse analysis(Mercer et al, OU)•  Exploratory talk, in which partners engage critically but constructively with each others ideas. •  Statements and suggestions are offered for joint consideration. •  These may be challenged and counter-challenged, but challenges are justified and alternative hypotheses are offered. •  Partners all actively participate and opinions are sought and considered before decisions are jointly made. •  Compared with the other two types, in Exploratory talk knowledge is made more publicly accountable and reasoning is more visible in the talk.Mercer, N. (2004). Sociocultural discourse analysis: analysing classroom talk as a socialmode of thinking. Journal of Applied Linguistics, 1(2), 137-168. 93
  94. 94. Analytics for identifying Exploratory talk Elluminate sessions can be very long – lasting for hours or even covering days of a conference It would be useful if we could identify where quality learning conversations seem to be taking place, so we can recommend those sessions, and not have to sit through online chat about virtual biscuitsFerguson, R. and Buckingham Shum, S. Learning analytics to identify exploratory dialogue within synchronous text chat. 941st International Conference on Learning Analytics & Knowledge (Banff, Canada, 27 Mar-1 Apr, 2011)
  95. 95. Defining indicators of Exploratory Talk Category Indicator Challenge But if, have to respond, my view Critique However, I’m not sure, maybe Discussion of Have you read, more links resources Evaluation Good example, good point Explanation Means that, our goals Explicit reasoning Next step, relates to, that’s why Justification I mean, we learned, we observed Reflections of Agree, here is another, makes the perspectives of others point, take your point, your view 95
  96. 96. Extract classified as Exploratory Talk Time Contribution 2:42 PM I hate talking. :-P My question was whether "gadgets" were just basically widgets and we could embed them in various web sites, like Netvibes, Google Desktop, etc. 2:42 PM Thanks, thats great! I am sure I understood everything, but looks inspiring! 2:43 PM Yes why OU tools not generic tools? 2:43 PM Issues of interoperability 2:43 PM The "new" SocialLearn site looks a lot like a corkboard where you can add various widgets, similar to those existing web start pages. 2:43 PM What if we end up with as many apps/gadgets as we have social networks and then we need a recommender for the apps! 2:43 PM My question was on the definition of the crowd in the wisdom of crowds we acsess in the service model? 2:43 PM there are various different flavours of widget e.g. Google gadgets, W3C widgets etc. SocialLearn has gone for Google gadgets 96
  97. 97. Discourse analytics on webinar textchat Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations? Not at the start and end of a webinar Sheffield, UK not as sunny but if we zoom in on a peak… See you! as yesterday - still warm bye for now! Greetings from Hong Kong bye, and thank you Morning from Wiltshire, 80 sunny here! Bye all for now 60 40 20 0 9:28 9:32 10:13 11:48 12:00 12:05 12:04 9:36 9:40 9:41 9:46 9:50 9:53 9:56 10:00 10:05 10:07 10:07 10:09 10:17 10:23 10:27 10:31 10:35 10:40 10:45 10:52 10:55 11:04 11:08 11:11 11:17 11:20 11:24 11:26 11:28 11:31 11:32 11:35 11:36 11:38 11:39 11:41 11:44 11:46 11:52 11:54 12:03 -20 -40 Average Exploratory -60Extensions to: Ferguson, R. and Buckingham Shum, S. (2011). Learning Analytics to Identify Exploratory Dialogue within Synchronous Text Chat.Proc. 1st Int. Conf. Learning Analytics & Knowledge. Feb. 27-Mar 1, 2011, Banff. ACM Press. Eprint: http://oro.open.ac.uk/28955
  98. 98. Discourse analytics on webinar textchat Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations? Classified as “exploratory talk” (more substantive 100 for learning) 50 0 9:28 “non- 9:40 9:50 10:00 10:07 10:17 10:31 10:45 11:04 11:17 11:26 11:32 11:38 11:44 11:52 12:03 -50 exploratory” Averag -100Wei & He extensions to: Ferguson, R. and Buckingham Shum, S. (2011). Learning Analytics to Identify Exploratory Dialogue within SynchronousText Chat. Proc. 1st Int. Conf. Learning Analytics & Knowledge. Feb. 27-Mar 1, 2011, Banff. ACM Press. Eprint: http://oro.open.ac.uk/28955
  99. 99. KMi’s Cohere: a web deliberation platform enabling semantic social network and discourse network analytics Rebecca is playing the role of broker, connecting 2 peers’ contributions in meaningful waysDe Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. Discourse-centric learning analytics. 1stInternational Conference on Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr, 2011) http://oro.open.ac.uk/25829
  100. 100. analytics forscholarly writing 100
  101. 101. Discourse analysis (Xerox Incremental Parser)Detection of salient sentences in scholarly reports,based on the rhetorical signals authors use:BACKGROUND KNOWLEDGE: NOVELTY: OPEN QUESTION:Recent studies indicate … ... new insights provide direct evidence ... … little is known …… the previously proposed … ... we suggest a new ... approach ... … role … has been elusive Current data is insufficient …… is universally accepted ... ... results define a novel role ...CONRASTING IDEAS: SIGNIFICANCE: SUMMARIZING:… unorthodox view resolves … studies ... have provided important The goal of this study ...paradoxes … advances Here, we show ...In contrast with previous Knowledge ... is crucial for ... Altogether, our results ... indicatehypotheses ... understanding... inconsistent with past findings ... valuable information ... from studiesGENERALIZING: SURPRISE:... emerging as a promising approach We have recently observed ... surprisinglyOur understanding ... has grownexponentially ... We have identified ... unusual... growing recognition of the The recent discovery ... suggests Ágnes Sándor & OLnet Project: http://olnet.org/node/512 intriguing rolesimportance ...De Liddo, A., Sándor, Á. and Buckingham Shum, S., Contested Collective Intelligence: Rationale, Technologies, and a Human-MachineAnnotation Study. Computer Supported Cooperative Work, 21, 4-5, (2012), 417-448. http://oro.open.ac.uk/31052
  102. 102. Human and machine analysis of a text for keycontributions Document 1 19 sentences annotated 22 sentences annotated 11 sentences same as human annotation Document 2 71 sentences annotated 59 sentences annotated 42 sentences same as human annotationhttp://technologies.kmi.open.ac.uk/cohere/2012/01/09/cohere-plus-automated-rhetorical-annotationDe Liddo, A., Sándor, Á. and Buckingham Shum, S., Contested Collective Intelligence: Rationale, Technologies, and a Human-MachineAnnotation Study. Computer Supported Cooperative Work, 21, 4-5, (2012), 417-448. http://oro.open.ac.uk/31052
  103. 103. analytics for reflecting on “networked expertise” (a key skill for our times) 103
  104. 104. Semantic Social Network AnalyticsDe Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. Discourse-centric learning analytics. 1stInternational Conference on Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr, 2011) http://oro.open.ac.uk/25829
  105. 105. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  106. 106. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  107. 107. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  108. 108. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  109. 109. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  110. 110. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  111. 111. Visualizing and filtering social ties in SocialLearn by topic and typeVisualising Social Learning in the SocialLearn Environment. Bieke Schreurs and Maarten de Laat (Open University, TheNetherlands), Chris Teplovs (Problemshift Inc. and University of Windsor), Rebecca Ferguson and Simon BuckinghamShum (Open University UK), SoLAR Storm webinar, Open University UK. http://bit.ly/UaFhbL
  112. 112. DispositionalLearning Analytics 112
  113. 113. Dispositions are beginning to registerwithin the learning analytics communityBrown, M., Learning Analytics: Moving from Concept to Practice. EDUCAUSE Learning InitiativeBriefing, 2012. http://www.educause.edu/library/resources/learning-analytics-moving-concept-practice 113
  114. 114. In your experience, what are the qualitiesshown by the most effective learners? Think about the most effective learners you’ve met/ mentored/taught Not necessarily the highest grade scorers, but the ones who showed a sustained appetite for learning What qualities/dispositions/attitudes did they bring? 114
  115. 115. A ‘visual learning analytic’ 7-dimensional spider diagram of how the learner sees themself Basis for a mentored- discussion on how the learner sees him/herself, and strategies for strengthening the profile 115Bristol and Open University are now embedding ELLI in learning software.
  116. 116. ELLI: Effective Lifelong Learning InventoryWeb questionnaire 72 items (children and adult versions: usedin schools, universities and workplace) 116
  117. 117. Validated as loading onto7 dimensions of “Learning Power” Being Stuck & Static Changing & Learning Data Accumulation Meaning Making Passivity Critical Curiosity Being Rule Bound Creativity Isolation & Dependence Learning Relationships Being Robotic Strategic Awareness Fragility & Dependence ResilienceUniv. Bristol and Vital Partnerships provides practitioner resourcesand tools to support their application in schools and the workplace 117
  118. 118. Learning to Learn: 7 Dimensions of Learning PowerFactor analysis of the literature plus expert interviews: identified sevendimensions of effective learning power , since validated empirically withlearners at many levels. (Deakin Crick, Broadfoot and Claxton, 2004)
  119. 119. Learning to Learn: 7 Dimensions of Learning PowerFactor analysis of the literature plus expert interviews: identified sevendimensions of effective learning power , since validated empirically withlearners at many levels. (Deakin Crick, Broadfoot and Claxton, 2004) 119
  120. 120. Learning Warehouse 2.0 analytics platform User experience: Research-validated assessment tools Researcher interface Learning Communities Analytics: Real time ELLI Analytics reports Bespoke research reports Datasets: >40,000 ELLI profiles (data from other hosted apps) 120
  121. 121. Adding imagery to ELLI dimensions toconnect with learner identity 121
  122. 122. Working with Gappuwiyak School, N. Territory AUS (Ruth Deakin Crick, University of Bristol) http://bit.ly/srUSHE Changing & Learning: Strategic Awareness: The Drongo - Guwak Emu - Wurrpan Meaning Making: The Pigeon - Nabalawal Critical Curiosity: Sea Eagle - Djert Resilience: Brolga - GudurrkuLearning Relationships: Creativity: The Cockatoo - Ngerrk Bower Bird - Djurwirr 122
  123. 123. Cohort analytics foreducators andorganizational leaders 123
  124. 124. EnquiryBlogger:Tuning Wordpress as an ELLI-based learning journal Standard Wordpress editor Categories from ELLI Plugin visualizes blog categories, mirroring the ELLI spider 124
  125. 125. Primary School EnquiryBloggersBushfield School, Wolverton, UKEnquiryBlogger: blogging for Learning Power & Authentic Enquiryhttp://learningemergence.net/2012/06/20/enquiryblogger-for-learning-power-authentic-enquiry
  126. 126. EnquiryBlogger dashboard
  127. 127. Could a platform generate an ELLI profile from user traces? Different social network patterns Questioning and in different challenging may contexts may load onto Critical load onto Curiosity Learning Relationships Repeated Sharing relevant attempts to pass resources from an online test other contexts may load onto may load onto Resilience Meaning MakingShaofu Huang: Prototyping Learning Power Modelling in SocialLearnhttp://www.open.ac.uk/blogs/SocialLearnResearch/2012/06/20/social-learning-analytics-symposium
  128. 128. SocialLearn provides new possibilities of looking at learners learning ELLI works from what Now we can observe what learners say they do they actually do…Shaofu Huang: Prototyping Learning Power Modelling in SocialLearn 128http://www.open.ac.uk/blogs/SocialLearnResearch/2012/06/20/social-learning-analytics-symposium
  129. 129. ELLI feedbacks inform development of learning Educator or leader s interventions Mentored discussionsShaofu Huang: Prototyping Learning Power Modelling in SocialLearn 129http://www.open.ac.uk/blogs/SocialLearnResearch/2012/06/20/social-learning-analytics-symposium
  130. 130. Dream? Student’s analytics dashboard 1 2 3 In your last discussion with your mentor, you decided to work on your resilience by taking on more learning challenges Your ELLI Spider 5 shows that you have made a start 4 Your most on working on recent mood comment: your resilience, “Great, at and that you are last I have also beginning to found all the resources work on your that I have creativity, which been you identified as looking for, another area to thanks to" Steve and work on. Ellen."Based on: Buckingham Shum, S. and Ferguson, R. (2011). Social Learning Analytics. Available as: Technical ReportKMI-11-01, Knowledge Media Institute, The Open University, UK. http://kmi.open.ac.uk/publications/pdf/kmi-11-01.pdf
  131. 131. Closingthoughts 131
  132. 132. “The basic question is not what can we measure? The basic question is what does a good education look like?” (Gardner Campbell)http://chronicle.com/blogs/techtherapy/2012/05/02/episode-95-learning-analytics-could-lead-to-wal-martification-of-collegehttp://lak12.wikispaces.com/Recordings 132
  133. 133. Will learning analytics merely turbocharge the current educational paradigm?— which is so often declared not fit for purpose… 133
  134. 134. …or will learning analytics reflect what we now know about designing authentic,engaged learning, developing the new qualities that a complex society demands? 134
  135. 135. Learning Analytics is becoming a newdiscipline and research fieldwww.SoLAResearch.orgFollow: @SoLAResearch Learning Analytics conference April 2012, Leuven: lakconference.org 135
  136. 136. Invent your own Analytics cyclebased on your research interests… What kinds of learners? What kinds of learning? What human +/or What data could besoftware interventions captured digitally in might be triggered? the use context? What data patterns might be proxies for good/poor learning? 136
  137. 137. Learning Analytics workshop Day 2 137
  138. 138. day 2 plan1.  Post-it affinity mapping2.  Team dashboard design3.  Plenary presentations4.  LAnoirblanc photo shoot5.  Sharing images + stories 138
  139. 139. What are we interested in?(Affinity Mapping exercise) Focus Subject Subject e.g. maths e.g. maths e.g. maths reading argumentation argumentation essay structure essay structure essay structure dispositions social networks Data Data social networks argumentation dispositions Data e.g. discourse e.g. discourse dispositions e.g. discourse social ties graphical graphical essays video uservideo logs user logs Pedagogy Pedagogy user logs survey survey Pedagogy ++Context survey +Context Context e.g. face-face e.g. face-face specialface-face e.g. needs special needs special needs constructivist constructivist constructivist PBL PBL PBLwrite 1 post-it per interest 139
  140. 140. DIY AnalyticsElaborated version of figure from Doug Clow:h"p://www.slideshare.net/dougclow/the-­‐learning-­‐analy7cs-­‐cycle-­‐closing-­‐the-­‐loop-­‐effec7vely  (slide  5) What kinds of learners? purpose ethics What kinds of learning? users What human +/or What data could be software generated digitally interventions / from the use context? (you can invent future recommendations? technologies if need) Does your theory What analytical tools How to render the analytics, predict patterns could be used to find for whom, and will they signifying learning? such patterns? understand them? 140
  141. 141. design your own analytics dashboard 141
  142. 142. LAnoirblancreactions to Learning Analytics in image and story LAnoirblanc.tumblr.com Choose an image and email it to the site with your story…
  143. 143. Emailing yourphoto… 143
  144. 144. LAnoirblanc Add your photo and story to the website1.  Take a photo or choose an image from the web2.  Email it + your tags, and a story or comments: To: semtaur2@tumblr.com Subject: (no title needed) Message: #dream #nightmare #fairydust #yourtag #yourtag (choose your tags: each must be a single word) text of your story... (add your name if you wish) Attachment: the photo3.  It will appear on LAnoirblanc.tumblr.com

×