How do Learning Analytics “act” in Education?

3,687 views

Published on

ESRC Code Acts in Education seminar series http://codeactsineducation.wordpress.com University of Edinburgh, 9th May 2014

Published in: Education, Technology
  • Be the first to comment

How do Learning Analytics “act” in Education?

  1. 1. How do Learning Analytics “act” in Education? Simon Buckingham Shum Knowledge Media Institute, Open University UK simon.buckinghamshum.net twitter @sbskmi ESRC  Code  Acts  in  Educa/on  seminar  series   h.p://codeactsineduca7on.wordpress.com   University  of  Edinburgh,  9th  May  2014   h.ps://twi.er.com/Wiswijzer2/status/414055472451575808  
  2. 2. Siri is smart… “Find code acts in education”
  3. 3. Siri is smart… “Find code acts in education”
  4. 4. h.ps://twi.er.com/Wiswijzer2/status/414055472451575808   “Note:  check  the   huge  difference   between  knowing   and  measuring…”  
  5. 5. A word of introduction… What does this tell you?
  6. 6. Learning   Technology   KMi,  OU   AI  &   Argumenta7on   Learning   Disposi7ons   Human-­‐Centred     Informa7cs   Learning   Analy7cs   Seman7c  Scien7fic     Publishing   Dialogue  /  Issue  /   Argument  Mapping   A word of introduction… What does this tell you?
  7. 7. Critical scepticism to economic modelling
  8. 8. Critical scepticism to economic modelling
  9. 9. The conversation you don’t want to have with your learning analytics wonk LAK13 Panel: Educational Data Scientists: A Scarce Breed http://people.kmi.open.ac.uk/sbs/2013/03/lak13-edu-data-scientists-scarce-breed John Behrens (Pearson)
  10. 10. Data ethics and societal readiness ny7.ms/1k5rY72  
  11. 11. Data ethics and societal readiness ny7.ms/1k5rY72  
  12. 12. In  what  senses  do     learning  analy1cs     “act”  in  educa1on?  
  13. 13. make  the  invisible  visible   make  solids  permeable   make  the  ephemeral  persistent   Educa7onal   (indeed  all)   Researchers   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?  
  14. 14. make  the  invisible  visible   make  solids  permeable   make  the  ephemeral  persistent   Learning   Analy7cs   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   Educa/onal  researchers  have  always  done  this  —  what’s  new  is  that  these   tools  are  now  in  the  hands  of  those  who  actually  produce  the  ac/vity  
  15. 15. outcome   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   By  changing   the  system   dynamics   researchers  /  educators  /  instruc7onal  designers   administrators  /  leaders  /  policymakers   intent  
  16. 16. outcome   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   By  changing   the  system   dynamics   Faster  feedback  loops   could  enable  more  rapid   adapta7on:  of  agents’   behaviour,  and  of  learning   resources  and  designs   researchers  /  educators  /  instruc7onal  designers   administrators  /  leaders  /  policymakers   intent  
  17. 17. Delega7on  of     authority  to  define   goals,  analy7cs,     and  implica7ons   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   Distribution of power between educators, learners, leaders, community…? ?   ?  
  18. 18. In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   What data, concepts and relationships do the analytics designers seek to model? Ontologically  
  19. 19. Bowker, G. C. and Star, L. S. (1999). Sorting Things Out: Classification and Its Consequences. MIT Press, Cambridge, MA, pp. 277, 278, 281 “Classification systems provide both a warrant and a tool for forgetting [...] what to forget and how to forget it [...] The argument comes down to asking not only what gets coded in but what gets coded out of a given scheme.”
  20. 20. Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, 2012, Vancouver, BC). ACM. Eprint: http://oro.open.ac.uk/32823 “A marker of the health of the learning analytics field will be the quality of debate around what the technology renders visible and leaves invisible.”
  21. 21. In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   What thresholds, samples, relationships, patterns, etc. do the algorithms encode and seek? On what basis is a recommendation engine proposing interventions? Algorithmically  
  22. 22. governingalgorithms.org   Learning  Analy7cs   In  an  increasingly  algorithmic   world  […]  What,  then,  do  we   talk  about  when  we  talk  about   “governing  algorithms”?   
  23. 23. governingalgorithms.org   A  technology  or  an  epistemology?   Barocas,  S.,  Hood,  S.  and  Ziewitz,  M.  (2013).   Governing  Algorithms:  A  Provoca1on  Piece.   Social  Science  Research  Network  Paper  2245322.   DOI:    h.p://dx.doi.org/10.2139/ssrn.2245322     Secrecy,  obscurity,  inscrutability   Agency,  automa7on,  accountabili7es   A  typology  of  algorithms  by  genre?   The  inscrutability  of  algorithms   Norma7vity,  bias,  values  
  24. 24. In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   What meaning-making does the representation and interaction design encourage? Semio7cally  
  25. 25. In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   epistemology pedagogyassessment Knight, S., Buckingham Shum, S. and Littleton, K. (In Press, 2014). Epistemology, Assessment, Pedagogy: Where Learning Meets Analytics in the Middle Space. Journal of Learning Analytics. Open Access Eprint: http://oro.open.ac.uk/39226 the ‘middle space’ of learning analytics in the context of computation... What epistemological assumptions are shaping the assessment regime, and hence the pedagogy? What questions are analytics used to help answer?
  26. 26. In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   Analytics reports at the organisational and national levels come with consequences at different scales — sometimes punitive, often impacting millions of people. Poli7cally  
  27. 27. Figure  from  Doug  Clow:  h.p://www.slideshare.net/dougclow/the-­‐learning-­‐analy7cs-­‐cycle-­‐closing-­‐the-­‐loop-­‐effec7vely  (slide  5)   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?   All  of  the  above  are   encapsulated  in  any   learning  analy7cs   deployment  
  28. 28. 28   What  kinds  of  learners?   What  kinds  of  learning?   What  data  could  be   generated  digitally  from   the  use  context?     How  is  it  ‘cleaned’?   Does  your  theory  predict   pa.erns  signifying   learning?   What  human  +/or   sooware  interven7ons  / recommenda7ons?   How  to  render  the  analy7cs,   for  whom,  and  will  they   understand  them?   What  analy7cal  tools   could  be  used  to  find  such   pa.erns?   In  what  senses  do  learning  analy7cs  “act”  in   educa7on?  
  29. 29. Learning  Analy7cs  approaches  act  in  educa7on…   Ontologically   Algorithmically   Semio7cally   Systemically   Poli7cally   Authority?  
  30. 30. >>>  Learning  Analy7cs  cri7cal  template…   <Learning Analytics Approach> Ontology … Algorithms … Semiotics … Epistemology … Pedagogy … Assessment regime … System Dynamics … Authority … Politics …
  31. 31. examples  
  32. 32. English  primary  school  data   RAISEonline  sta7s7cs   for  a  school,  on  ‘core’  subjects  (maths,  literacy),   a.endance  and  demographics  
  33. 33. Visualising “attainment” and “progress”
  34. 34. Which analytics could reflect the progress that ‘Joe’ has made on so many other fronts other than his SATS? 34
  35. 35. student  modelling  by  an   adap1ve  tutor  
  36. 36. Intelligent tutoring for skills mastery http://oli.cmu.edu
  37. 37. 37 https://grockit.com/research Adaptive platforms generate fine-grained analytics on curriculum mastery
  38. 38. analy1cs  for     learning  disposi1ons  
  39. 39. Qualities of effective learners Factor analysis of the literature plus expert interviews: identified seven dimensions of effective “learning power”, since validated empirically with learners at many levels. (Deakin Crick, Broadfoot and Claxton, 2004)
  40. 40. Qualities of effective learners 40
  41. 41. ELLI: Effective Lifelong Learning Inventory
  42. 42. Quantifying learning dispositions agency; identity; motivation; responsibility Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, Vancouver). Eprint: http://oro.open.ac.uk/32823 http://learningemergence.net/2012/04/30/learning-powered-learning-analytics A  wholis7c  visual,  intended  to   build  intrinsic  mo7va7on,   invi7ng  stretch,  providing  a  new   language,  provoking   conversa7on  that  7es  to  the   learner’s  iden7ty  
  43. 43. Quantifying learning dispositions agency; identity; motivation; responsibility http://learningemergence.net/2014/03/01/assessing-learning-dispositions-academic-mindsets
  44. 44. http://learningemergence.net/2014/03/01/assessing-learning-dispositions-academic-mindsets 2020? personal data cloud generates my dispositional profile for reflection >>> help me take responsibility for my own learning Shaofu Huang: Prototyping Learning Power Modelling in SocialLearn http://www.open.ac.uk/blogs/SocialLearnResearch/2012/06/20/social-learning-analytics-symposium Simon Knight: http://people.kmi.open.ac.uk/knight/2014/02/knowledge-in-search Social network patterns, teamwork effectiveness and initiation of relationships Questioning, arguing and search behaviours reveal intrinsic curiosity and epistemic commitments Tagging/sharing/ blogging/social patterns reveal how you see connections between ideas Perseverance, grit, tenacity; overcoming panic/stress when challenged — through behavioural and somatic traces
  45. 45. Open source approach to analytics: towards true transparency and emergence for complex systems? http://www.solaresearch.org/mission/ola
  46. 46. conclusion     analy1cs  act  in  educa1on   —  on  mul1ple  dimensions  an  analy1cs  approach     perpetuates  an  educa1onal  worldview     —  so  let’s  ensure  this  is   inten1onal  –  not  accidental...  

×