0
How do Learning Analytics
“act” in Education?
Simon Buckingham Shum
Knowledge Media Institute, Open University UK
simon.bu...
Siri is smart…
“Find code acts in
education”
Siri is smart…
“Find code acts in
education”
h.ps://twi.er.com/Wiswijzer2/status/414055472451575808	
  
“Note:	
  check	
  the	
  
huge	
  difference	
  
between	
  kno...
A word of
introduction…
What does
this tell you?
Learning	
  
Technology	
  
KMi,	
  OU	
  
AI	
  &	
  
Argumenta7on	
  
Learning	
  
Disposi7ons	
  
Human-­‐Centred	
  	
...
Critical scepticism to economic modelling
Critical scepticism to economic modelling
The conversation you don’t want to have
with your learning analytics wonk
LAK13 Panel: Educational Data Scientists: A Scar...
Data ethics and societal readiness
ny7.ms/1k5rY72	
  
Data ethics and societal readiness
ny7.ms/1k5rY72	
  
In	
  what	
  senses	
  do	
  	
  
learning	
  analy1cs	
  	
  
“act”	
  in	
  educa1on?	
  
make	
  the	
  invisible	
  visible	
  
make	
  solids	
  permeable	
  
make	
  the	
  ephemeral	
  persistent	
  
Educa7o...
make	
  the	
  invisible	
  visible	
  
make	
  solids	
  permeable	
  
make	
  the	
  ephemeral	
  persistent	
  
Learnin...
outcome	
  
In	
  what	
  senses	
  do	
  learning	
  analy7cs	
  “act”	
  in	
  
educa7on?	
  
By	
  changing	
  
the	
  ...
outcome	
  
In	
  what	
  senses	
  do	
  learning	
  analy7cs	
  “act”	
  in	
  
educa7on?	
  
By	
  changing	
  
the	
  ...
Delega7on	
  of	
  	
  
authority	
  to	
  define	
  
goals,	
  analy7cs,	
  	
  
and	
  implica7ons	
  
In	
  what	
  sens...
In	
  what	
  senses	
  do	
  learning	
  analy7cs	
  “act”	
  in	
  
educa7on?	
  
What data, concepts
and relationships ...
Bowker, G. C. and Star, L. S. (1999). Sorting Things Out: Classification and Its Consequences. MIT Press, Cambridge, MA, p...
Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling ...
In	
  what	
  senses	
  do	
  learning	
  analy7cs	
  “act”	
  in	
  
educa7on?	
  
What thresholds, samples,
relationship...
governingalgorithms.org	
  
Learning	
  Analy7cs	
  
In	
  an	
  increasingly	
  algorithmic	
  
world	
  […]	
  What,	
  ...
governingalgorithms.org	
  
A	
  technology	
  or	
  an	
  epistemology?	
  
Barocas,	
  S.,	
  Hood,	
  S.	
  and	
  Ziew...
In	
  what	
  senses	
  do	
  learning	
  analy7cs	
  “act”	
  in	
  
educa7on?	
  
What meaning-making does
the represent...
In	
  what	
  senses	
  do	
  learning	
  analy7cs	
  “act”	
  in	
  
educa7on?	
  
epistemology
pedagogyassessment
Knight...
In	
  what	
  senses	
  do	
  learning	
  analy7cs	
  “act”	
  in	
  
educa7on?	
  
Analytics reports at the
organisationa...
Figure	
  from	
  Doug	
  Clow:	
  h.p://www.slideshare.net/dougclow/the-­‐learning-­‐analy7cs-­‐cycle-­‐closing-­‐the-­‐l...
28	
  
What	
  kinds	
  of	
  learners?	
  
What	
  kinds	
  of	
  learning?	
  
What	
  data	
  could	
  be	
  
generated...
Learning	
  Analy7cs	
  approaches	
  act	
  in	
  educa7on…	
  
Ontologically	
   Algorithmically	
   Semio7cally	
  
Sys...
>>>	
  Learning	
  Analy7cs	
  cri7cal	
  template…	
  
<Learning Analytics Approach>
Ontology …
Algorithms …
Semiotics …
...
examples	
  
English	
  primary	
  school	
  data	
  
RAISEonline	
  sta7s7cs	
  
for	
  a	
  school,	
  on	
  ‘core’	
  subjects	
  (m...
Visualising “attainment” and “progress”
Which analytics could reflect the progress that ‘Joe’
has made on so many other fronts other than his SATS?
34
student	
  modelling	
  by	
  an	
  
adap1ve	
  tutor	
  
Intelligent tutoring for skills mastery
http://oli.cmu.edu
37
https://grockit.com/research
Adaptive platforms generate fine-grained
analytics on curriculum mastery
analy1cs	
  for	
  	
  
learning	
  disposi1ons	
  
Qualities of effective learners
Factor analysis of the literature plus expert interviews: identified seven
dimensions of e...
Qualities of effective learners
40
ELLI: Effective Lifelong Learning Inventory
Quantifying learning dispositions
agency; identity; motivation; responsibility
Buckingham Shum, S. and Deakin Crick, R. (2...
Quantifying learning dispositions
agency; identity; motivation; responsibility
http://learningemergence.net/2014/03/01/ass...
http://learningemergence.net/2014/03/01/assessing-learning-dispositions-academic-mindsets
2020? personal data cloud genera...
Open source approach to analytics:
towards true transparency and emergence for complex systems?
http://www.solaresearch.or...
conclusion	
  
	
  
analy1cs	
  act	
  in	
  educa1on	
  
—	
  on	
  mul1ple	
  dimensions	
  an	
  analy1cs	
  approach	
...
Upcoming SlideShare
Loading in...5
×

How do Learning Analytics “act” in Education?

2,382

Published on

ESRC Code Acts in Education seminar series http://codeactsineducation.wordpress.com University of Edinburgh, 9th May 2014

Published in: Education, Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,382
On Slideshare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
31
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "How do Learning Analytics “act” in Education?"

  1. 1. How do Learning Analytics “act” in Education? Simon Buckingham Shum Knowledge Media Institute, Open University UK simon.buckinghamshum.net twitter @sbskmi ESRC  Code  Acts  in  Educa/on  seminar  series   h.p://codeactsineduca7on.wordpress.com   University  of  Edinburgh,  9th  May  2014   h.ps://twi.er.com/Wiswijzer2/status/414055472451575808  
  2. 2. Siri is smart… “Find code acts in education”
  3. 3. Siri is smart… “Find code acts in education”
  4. 4. h.ps://twi.er.com/Wiswijzer2/status/414055472451575808   “Note:  check  the   huge  difference   between  knowing   and  measuring…”  
  5. 5. A word of introduction… What does this tell you?
  6. 6. Learning   Technology   KMi,  OU   AI  &   Argumenta7on   Learning   Disposi7ons   Human-­‐Centred     Informa7cs   Learning   Analy7cs   Seman7c  Scien7fic     Publishing   Dialogue  /  Issue  /   Argument  Mapping   A word of introduction… What does this tell you?
  7. 7. Critical scepticism to economic modelling
  8. 8. Critical scepticism to economic modelling
  9. 9. The conversation you don’t want to have with your learning analytics wonk LAK13 Panel: Educational Data Scientists: A Scarce Breed http://people.kmi.open.ac.uk/sbs/2013/03/lak13-edu-data-scientists-scarce-breed John Behrens (Pearson)
  10. 10. Data ethics and societal readiness ny7.ms/1k5rY72  
  11. 11. Data ethics and societal readiness ny7.ms/1k5rY72  
  12. 12. In  what  senses  do     learning  analy1cs     “act”  in  educa1on?  
  13. 13. make  the  invisible  visible   make  solids  permeable   make  the  ephemeral  persistent   Educa7onal   (indeed  all)   Researchers   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?  
  14. 14. make  the  invisible  visible   make  solids  permeable   make  the  ephemeral  persistent   Learning   Analy7cs   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   Educa/onal  researchers  have  always  done  this  —  what’s  new  is  that  these   tools  are  now  in  the  hands  of  those  who  actually  produce  the  ac/vity  
  15. 15. outcome   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   By  changing   the  system   dynamics   researchers  /  educators  /  instruc7onal  designers   administrators  /  leaders  /  policymakers   intent  
  16. 16. outcome   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   By  changing   the  system   dynamics   Faster  feedback  loops   could  enable  more  rapid   adapta7on:  of  agents’   behaviour,  and  of  learning   resources  and  designs   researchers  /  educators  /  instruc7onal  designers   administrators  /  leaders  /  policymakers   intent  
  17. 17. Delega7on  of     authority  to  define   goals,  analy7cs,     and  implica7ons   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   Distribution of power between educators, learners, leaders, community…? ?   ?  
  18. 18. In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   What data, concepts and relationships do the analytics designers seek to model? Ontologically  
  19. 19. Bowker, G. C. and Star, L. S. (1999). Sorting Things Out: Classification and Its Consequences. MIT Press, Cambridge, MA, pp. 277, 278, 281 “Classification systems provide both a warrant and a tool for forgetting [...] what to forget and how to forget it [...] The argument comes down to asking not only what gets coded in but what gets coded out of a given scheme.”
  20. 20. Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, 2012, Vancouver, BC). ACM. Eprint: http://oro.open.ac.uk/32823 “A marker of the health of the learning analytics field will be the quality of debate around what the technology renders visible and leaves invisible.”
  21. 21. In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   What thresholds, samples, relationships, patterns, etc. do the algorithms encode and seek? On what basis is a recommendation engine proposing interventions? Algorithmically  
  22. 22. governingalgorithms.org   Learning  Analy7cs   In  an  increasingly  algorithmic   world  […]  What,  then,  do  we   talk  about  when  we  talk  about   “governing  algorithms”?   
  23. 23. governingalgorithms.org   A  technology  or  an  epistemology?   Barocas,  S.,  Hood,  S.  and  Ziewitz,  M.  (2013).   Governing  Algorithms:  A  Provoca1on  Piece.   Social  Science  Research  Network  Paper  2245322.   DOI:    h.p://dx.doi.org/10.2139/ssrn.2245322     Secrecy,  obscurity,  inscrutability   Agency,  automa7on,  accountabili7es   A  typology  of  algorithms  by  genre?   The  inscrutability  of  algorithms   Norma7vity,  bias,  values  
  24. 24. In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   What meaning-making does the representation and interaction design encourage? Semio7cally  
  25. 25. In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   epistemology pedagogyassessment Knight, S., Buckingham Shum, S. and Littleton, K. (In Press, 2014). Epistemology, Assessment, Pedagogy: Where Learning Meets Analytics in the Middle Space. Journal of Learning Analytics. Open Access Eprint: http://oro.open.ac.uk/39226 the ‘middle space’ of learning analytics in the context of computation... What epistemological assumptions are shaping the assessment regime, and hence the pedagogy? What questions are analytics used to help answer?
  26. 26. In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   Analytics reports at the organisational and national levels come with consequences at different scales — sometimes punitive, often impacting millions of people. Poli7cally  
  27. 27. Figure  from  Doug  Clow:  h.p://www.slideshare.net/dougclow/the-­‐learning-­‐analy7cs-­‐cycle-­‐closing-­‐the-­‐loop-­‐effec7vely  (slide  5)   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   All  of  the  above  are   encapsulated  in  any   learning  analy7cs   deployment  
  28. 28. 28   What  kinds  of  learners?   What  kinds  of  learning?   What  data  could  be   generated  digitally  from   the  use  context?     How  is  it  ‘cleaned’?   Does  your  theory  predict   pa.erns  signifying   learning?   What  human  +/or   sooware  interven7ons  / recommenda7ons?   How  to  render  the  analy7cs,   for  whom,  and  will  they   understand  them?   What  analy7cal  tools   could  be  used  to  find  such   pa.erns?   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?  
  29. 29. Learning  Analy7cs  approaches  act  in  educa7on…   Ontologically   Algorithmically   Semio7cally   Systemically   Poli7cally   Authority?  
  30. 30. >>>  Learning  Analy7cs  cri7cal  template…   <Learning Analytics Approach> Ontology … Algorithms … Semiotics … Epistemology … Pedagogy … Assessment regime … System Dynamics … Authority … Politics …
  31. 31. examples  
  32. 32. English  primary  school  data   RAISEonline  sta7s7cs   for  a  school,  on  ‘core’  subjects  (maths,  literacy),   a.endance  and  demographics  
  33. 33. Visualising “attainment” and “progress”
  34. 34. Which analytics could reflect the progress that ‘Joe’ has made on so many other fronts other than his SATS? 34
  35. 35. student  modelling  by  an   adap1ve  tutor  
  36. 36. Intelligent tutoring for skills mastery http://oli.cmu.edu
  37. 37. 37 https://grockit.com/research Adaptive platforms generate fine-grained analytics on curriculum mastery
  38. 38. analy1cs  for     learning  disposi1ons  
  39. 39. Qualities of effective learners Factor analysis of the literature plus expert interviews: identified seven dimensions of effective “learning power”, since validated empirically with learners at many levels. (Deakin Crick, Broadfoot and Claxton, 2004)
  40. 40. Qualities of effective learners 40
  41. 41. ELLI: Effective Lifelong Learning Inventory
  42. 42. Quantifying learning dispositions agency; identity; motivation; responsibility Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, Vancouver). Eprint: http://oro.open.ac.uk/32823 http://learningemergence.net/2012/04/30/learning-powered-learning-analytics A  wholis7c  visual,  intended  to   build  intrinsic  mo7va7on,   invi7ng  stretch,  providing  a  new   language,  provoking   conversa7on  that  7es  to  the   learner’s  iden7ty  
  43. 43. Quantifying learning dispositions agency; identity; motivation; responsibility http://learningemergence.net/2014/03/01/assessing-learning-dispositions-academic-mindsets
  44. 44. http://learningemergence.net/2014/03/01/assessing-learning-dispositions-academic-mindsets 2020? personal data cloud generates my dispositional profile for reflection >>> help me take responsibility for my own learning Shaofu Huang: Prototyping Learning Power Modelling in SocialLearn http://www.open.ac.uk/blogs/SocialLearnResearch/2012/06/20/social-learning-analytics-symposium Simon Knight: http://people.kmi.open.ac.uk/knight/2014/02/knowledge-in-search Social network patterns, teamwork effectiveness and initiation of relationships Questioning, arguing and search behaviours reveal intrinsic curiosity and epistemic commitments Tagging/sharing/ blogging/social patterns reveal how you see connections between ideas Perseverance, grit, tenacity; overcoming panic/stress when challenged — through behavioural and somatic traces
  45. 45. Open source approach to analytics: towards true transparency and emergence for complex systems? http://www.solaresearch.org/mission/ola
  46. 46. conclusion     analy1cs  act  in  educa1on   —  on  mul1ple  dimensions  an  analy1cs  approach     perpetuates  an  educa1onal  worldview     —  so  let’s  ensure  this  is   inten1onal  –  not  accidental...  
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×