Learning and analytics – where do the two
meet?
Simon Knight @sjgknight
Image from http://xkcd.com/903/ licensed under a C...
Interpretive flexibility
• The basic question is not
what can we measure? The
basic question is what does a
good education...
Introduction to
Epistemology, Pedagogy & Assessment
The Triad: Bounding the middle space
Foreground relationships between:
• epistemology (the nature of knowledge)
• assessme...
Key questions
• What does it mean to
know?
• How do we decide
(assess) if someone
knows or not?
• How do we get people
to ...
Bounding the middle space
1. LA ‘buy in’ to ways of
thinking about
epistemology, assessment
and pedagogy
2. For theoretica...
Why does epistemology matter?
“…assessment is one area where notions
of truth, accuracy and fairness have a
very practical...
Why does epistemology matter?
“…assessment is one area where notions of
truth, accuracy and fairness have a very
practical...
Learning Analytics
• Data mining
• Digital trace –
including linguistic
data
• Deployed for
pedagogic purposes
• Big and r...
Epistemology & Assessment
an example
Danish exams with internet access
http://news.bbc.co.uk/1/hi/education/8341589.stm
Danish exams with internet access
• Allows testing of problem-solving and analysis - sifting
information
• "if you allow c...
Danish exams with internet access
• Potential for auto-grading – LA role, a new model?
http://www.timeshighereducation.co....
Epistemic assumptions
- It isn’t “knowledge” to recall facts
- Knowledge lies in the use
Assessment…
Formative or summative?
What is being assessed?
Pedagogic assumptions
• Assessment is used in teaching (but doesn’t drive it)
• Pedagogy should involve knowledge practice...
Technological determinism?
• Technology is situated
• Interpretive flexibility
The other side of the coin
Multiple perspectives
• One side: Direct technological affordances
• The other: The practices around technologies
– Educat...
Educator Practices: LA potential
• E.g. at LAK14: Wise, Piety & Hickey
Policy: LA potential
• Accountability systems, educator support,
investment in AfL, setting priorities (e.g.
Denmark)
Student Practices: LA potential
• Analytics give unprecedented (?) access to
“What students do” - & assumptions around
thi...
LA in Structured Knowledge Building
• Strong CSCL tradition to ‘make explicit’ in
structured environments (Knowledge Forum...
Natural Language Processing based LA
• Automated essay feedback
• Dialogue scaffolds and automated tutors
• Social functio...
Hot Topics
• Analytics for student ‘dispositions’
• ‘Personalisation’ through community building
– Social networks
– Disco...
Conclusions
1. Context of LA as assessment,
and pedagogic aid, in context
of policy considerations –
Danish example
2. Edu...
Thank you
@sjgknight
sjgknight@gmail.com
http://people.kmi.open.ac.uk/knight/
Our papers in this area:
• http://oro.open.a...
Acknowledgements
Thanks to my co-authors and supervisors Simon
Buckingham Shum and Karen Littleton for their
work on this ...
Upcoming SlideShare
Loading in …5
×

Learning and analytics – where do the two meet? #HEABigData summit day

1,868 views

Published on

I was an invited speaker at the Higher Education Academy Big Data summit where I talked about our Journal of Learning Analytics (2014) paper.

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,868
On SlideShare
0
From Embeds
0
Number of Embeds
828
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • http://wellcomeimages.org/indexplus/result.html?_IXMAXHITS_=1&_IXACTION_=query&_IXFIRST_=544&_IXSR_=CCCpq8yDLM6&_IXSS_=_IXFPFX_%3dtemplates%252ft%26_IXMAXHITS_%3d250%26c%3d%2522historical%2bimages%2522%2bOR%2b%2522contemporary%2bimages%2522%2bOR%2b%2522corporate%2bimages%2522%2bOR%2b%2522contemporary%2bclinical%2bimages%2522%2bOR%2bfooooooo%253b%26_IXFIRST_%3d1%26%252asform%3dwellcome%252dimages%26%2524%2b%2528%2528with%2bwi_sfgu%2bis%2bY%2529%2band%2bnot%2b%2528%2522contemporary%2bclinical%2bimages%2522%2bindex%2bwi_collection%2bor%2b%2522corporate%2bimages%2522%2bindex%2bwi_collection%2529%2529%2band%2bnot%2bwith%2bsys_deleted%3d%252e%26%2524%253dsi%3dtext%26_IXACTION_%3dquery%26%3d%26_IXrescount%3d553%26_IXINITSR_%3dy%26%2524%253dsort%3dsort%2bsortexpr%2bimage_sort%26_IXshc%3dy%26%2524%253ds%3dmeasurement&_IXSPFX_=templates%2ft&_IXFPFX_=templates%2ft
  • http://wellcomeimages.org/indexplus/result.html?_IXMAXHITS_=1&_IXACTION_=query&_IXFIRST_=544&_IXSR_=CCCpq8yDLM6&_IXSS_=_IXFPFX_%3dtemplates%252ft%26_IXMAXHITS_%3d250%26c%3d%2522historical%2bimages%2522%2bOR%2b%2522contemporary%2bimages%2522%2bOR%2b%2522corporate%2bimages%2522%2bOR%2b%2522contemporary%2bclinical%2bimages%2522%2bOR%2bfooooooo%253b%26_IXFIRST_%3d1%26%252asform%3dwellcome%252dimages%26%2524%2b%2528%2528with%2bwi_sfgu%2bis%2bY%2529%2band%2bnot%2b%2528%2522contemporary%2bclinical%2bimages%2522%2bindex%2bwi_collection%2bor%2b%2522corporate%2bimages%2522%2bindex%2bwi_collection%2529%2529%2band%2bnot%2bwith%2bsys_deleted%3d%252e%26%2524%253dsi%3dtext%26_IXACTION_%3dquery%26%3d%26_IXrescount%3d553%26_IXINITSR_%3dy%26%2524%253dsort%3dsort%2bsortexpr%2bimage_sort%26_IXshc%3dy%26%2524%253ds%3dmeasurement&_IXSPFX_=templates%2ft&_IXFPFX_=templates%2ft
  • CC licence (although, logos are owned by respective companies…) http://www.flickr.com/photos/damienbasile/3629544077/
  • http://wellcomeimages.org/indexplus/result.html?_IXMAXHITS_=1&_IXACTION_=query&_IXFIRST_=13&_IXSR_=FIz4ww6m3u_&_IXSS_=_IXFPFX_%3dtemplates%252ft%26_IXMAXHITS_%3d250%26c%3d%2522historical%2bimages%2522%2bOR%2b%2522contemporary%2bimages%2522%2bOR%2b%2522corporate%2bimages%2522%2bOR%2b%2522contemporary%2bclinical%2bimages%2522%2bOR%2bfooooooo%253b%26_IXFIRST_%3d1%26%252asform%3dwellcome%252dimages%26%2524%2b%2528%2528with%2bwi_sfgu%2bis%2bY%2529%2band%2bnot%2b%2528%2522contemporary%2bclinical%2bimages%2522%2bindex%2bwi_collection%2bor%2b%2522corporate%2bimages%2522%2bindex%2bwi_collection%2529%2529%2band%2bnot%2bwith%2bsys_deleted%3d%252e%26%2524%253dsi%3dtext%26_IXACTION_%3dquery%26%3d%26_IXrescount%3d128%26_IXINITSR_%3dy%26%2524%253dsort%3dsort%2bsortexpr%2bimage_sort%26_IXshc%3dy%26%2524%253ds%3dlens&_IXSPFX_=templates%2ft&_IXFPFX_=templates%2ft
  • http://wellcomeimages.org/indexplus/result.html?_IXMAXHITS_=1&_IXACTION_=query&_IXFIRST_=85&_IXSR_=n4w2a3w8NJi&_IXSS_=_IXMAXHITS_%3d250%26_IXFPFX_%3dtemplates%252ft%26_IXFIRST_%3d1%26c%3d%2522historical%2bimages%2522%2bOR%2b%2522contemporary%2bimages%2522%2bOR%2b%2522corporate%2bimages%2522%2bOR%2b%2522contemporary%2bclinical%2bimages%2522%2bOR%2bfooooooo%253b%26%252asform%3dwellcome%252dimages%26%2524%253dsi%3dtext%26_IXACTION_%3dquery%26i_pre%3d%26IXTO%3d%26t%3d%26_IXINITSR_%3dy%26i_num%3d%26%2524%253dsort%3dsort%2bsortexpr%2bimage_sort%26w%3d%26%2524%253ds%3dteaching%26IXFROM%3d%26_IXshc%3dy%26%2524%2b%2528%2528with%2bwi_sfgu%2bis%2bY%2529%2band%2bnot%2b%2528%2522contemporary%2bclinical%2bimages%2522%2bindex%2bwi_collection%2bor%2b%2522corporate%2bimages%2522%2bindex%2bwi_collection%2529%2529%2band%2bnot%2bwith%2bsys_deleted%3d%252e%26_IXrescount%3d793&_IXSPFX_=templates%2ft&_IXFPFX_=templates%2fthttp://wellcomeimages.org/indexplus/result.html?_IXMAXHITS_=1&_IXACTION_=query&_IXFIRST_=28&_IXSR_=n4w2a3w8NJi&_IXSS_=_IXMAXHITS_%253d250%26_IXFPFX_%253dtemplates%25252ft%26_IXFIRST_%253d1%26c%253d%252522historical%252bimages%252522%252bOR%252b%252522contemporary%252bimages%252522%252bOR%252b%252522corporate%252bimages%252522%252bOR%252b%252522contemporary%252bclinical%252bimages%252522%252bOR%252bfooooooo%25253b%26%25252asform%253dwellcome%25252dimages%26%252524%25253dsi%253dtext%26_IXACTION_%253dquery%26i_pre%253d%26IXTO%253d%26t%253d%26_IXINITSR_%253dy%26i_num%253d%26%252524%25253dsort%253dsort%252bsortexpr%252bimage_sort%26w%253d%26%252524%25253ds%253dteaching%26IXFROM%253d%26_IXshc%253dy%26%252524%252b%252528%252528with%252bwi_sfgu%252bis%252bY%252529%252band%252bnot%252b%252528%252522contemporary%252bclinical%252bimages%252522%252bindex%252bwi_collection%252bor%252b%252522corporate%252bimages%252522%252bindex%252bwi_collection%252529%252529%252band%252bnot%252bwith%252bsys_deleted%253d%25252e%26_IXrescount%253d793&_IXSPFX_=templates%2ft&_IXFPFX_=templates%2fthttp://wellcomeimages.org/indexplus/result.html?_IXMAXHITS_=1&_IXACTION_=query&_IXFIRST_=30&_IXSR_=n4w2a3w8NJi&_IXSS_=_IXMAXHITS_%253d250%26_IXFPFX_%253dtemplates%25252ft%26_IXFIRST_%253d1%26c%253d%252522historical%252bimages%252522%252bOR%252b%252522contemporary%252bimages%252522%252bOR%252b%252522corporate%252bimages%252522%252bOR%252b%252522contemporary%252bclinical%252bimages%252522%252bOR%252bfooooooo%25253b%26%25252asform%253dwellcome%25252dimages%26%252524%25253dsi%253dtext%26_IXACTION_%253dquery%26i_pre%253d%26IXTO%253d%26t%253d%26_IXINITSR_%253dy%26i_num%253d%26%252524%25253dsort%253dsort%252bsortexpr%252bimage_sort%26w%253d%26%252524%25253ds%253dteaching%26IXFROM%253d%26_IXshc%253dy%26%252524%252b%252528%252528with%252bwi_sfgu%252bis%252bY%252529%252band%252bnot%252b%252528%252522contemporary%252bclinical%252bimages%252522%252bindex%252bwi_collection%252bor%252b%252522corporate%252bimages%252522%252bindex%252bwi_collection%252529%252529%252band%252bnot%252bwith%252bsys_deleted%253d%25252e%26_IXrescount%253d793&_IXSPFX_=templates%2ft&_IXFPFX_=templates%2fthttp://wellcomeimages.org/indexplus/result.html?_IXMAXHITS_=1&_IXACTION_=query&_IXFIRST_=26&_IXSR_=n4w2a3w8NJi&_IXSS_=_IXMAXHITS_%253d250%26_IXFPFX_%253dtemplates%25252ft%26_IXFIRST_%253d1%26c%253d%252522historical%252bimages%252522%252bOR%252b%252522contemporary%252bimages%252522%252bOR%252b%252522corporate%252bimages%252522%252bOR%252b%252522contemporary%252bclinical%252bimages%252522%252bOR%252bfooooooo%25253b%26%25252asform%253dwellcome%25252dimages%26%252524%25253dsi%253dtext%26_IXACTION_%253dquery%26i_pre%253d%26IXTO%253d%26t%253d%26_IXINITSR_%253dy%26i_num%253d%26%252524%25253dsort%253dsort%252bsortexpr%252bimage_sort%26w%253d%26%252524%25253ds%253dteaching%26IXFROM%253d%26_IXshc%253dy%26%252524%252b%252528%252528with%252bwi_sfgu%252bis%252bY%252529%252band%252bnot%252b%252528%252522contemporary%252bclinical%252bimages%252522%252bindex%252bwi_collection%252bor%252b%252522corporate%252bimages%252522%252bindex%252bwi_collection%252529%252529%252band%252bnot%252bwith%252bsys_deleted%253d%25252e%26_IXrescount%253d793&_IXSPFX_=templates%2ft&_IXFPFX_=templates%2ft
  • Technologies are flexible, e.g. mini blackboards for AfL & collaboration v chalk n talkhttps://commons.wikimedia.org/wiki/File%3AX%2BY.jpeg By Quinchoa (Own work) [CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commonshttps://commons.wikimedia.org/wiki/File%3ABlackboard1257.jpg by alegri / 4freephotos.com [CC-BY-3.0 (http://creativecommons.org/licenses/by/3.0)], via Wikimedia Commons
  • http://commons.wikimedia.org/wiki/File:Janus_coin.png
  • Dumping IWBs in with no training or consideration of policy was a failing of earlier strategyBy svonog (http://flickr.com/photos/svonog/432774995/) [CC-BY-2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons
  • https://commons.wikimedia.org/wiki/File:Richard_Huish_College_Exam_Hall.jpg
  • http://mallet.cs.umass.edu/http://www.cs.waikato.ac.nz/~ml/weka/
  • http://mallet.cs.umass.edu/http://www.cs.waikato.ac.nz/~ml/weka/
  • http://wellcomeimages.org/indexplus/result.html?_IXMAXHITS_=1&_IXACTION_=query&_IXFIRST_=544&_IXSR_=CCCpq8yDLM6&_IXSS_=_IXFPFX_%3dtemplates%252ft%26_IXMAXHITS_%3d250%26c%3d%2522historical%2bimages%2522%2bOR%2b%2522contemporary%2bimages%2522%2bOR%2b%2522corporate%2bimages%2522%2bOR%2b%2522contemporary%2bclinical%2bimages%2522%2bOR%2bfooooooo%253b%26_IXFIRST_%3d1%26%252asform%3dwellcome%252dimages%26%2524%2b%2528%2528with%2bwi_sfgu%2bis%2bY%2529%2band%2bnot%2b%2528%2522contemporary%2bclinical%2bimages%2522%2bindex%2bwi_collection%2bor%2b%2522corporate%2bimages%2522%2bindex%2bwi_collection%2529%2529%2band%2bnot%2bwith%2bsys_deleted%3d%252e%26%2524%253dsi%3dtext%26_IXACTION_%3dquery%26%3d%26_IXrescount%3d553%26_IXINITSR_%3dy%26%2524%253dsort%3dsort%2bsortexpr%2bimage_sort%26_IXshc%3dy%26%2524%253ds%3dmeasurement&_IXSPFX_=templates%2ft&_IXFPFX_=templates%2ft
  • Learning and analytics – where do the two meet? #HEABigData summit day

    1. 1. Learning and analytics – where do the two meet? Simon Knight @sjgknight Image from http://xkcd.com/903/ licensed under a Creative Commons Attribution-NonCommercial 2.5 License.
    2. 2. Interpretive flexibility • The basic question is not what can we measure? The basic question is what does a good education look like? (Gardner Campbell) • Do we value what we can measure, or measure what we really value? (Guy Claxton, BBC Radio 4 Education Debate, Nov. 2012) 84
    3. 3. Introduction to Epistemology, Pedagogy & Assessment
    4. 4. The Triad: Bounding the middle space Foreground relationships between: • epistemology (the nature of knowledge) • assessment (of learnt knowledge?) • pedagogy (the nature of learning)
    5. 5. Key questions • What does it mean to know? • How do we decide (assess) if someone knows or not? • How do we get people to come to know (to learn)?
    6. 6. Bounding the middle space 1. LA ‘buy in’ to ways of thinking about epistemology, assessment and pedagogy 2. For theoretical, practical, ethical reasons we should engage in these debates 3. These considerations have practical implications – the middle ground
    7. 7. Why does epistemology matter? “…assessment is one area where notions of truth, accuracy and fairness have a very practical purchase in everyday life” (Williams, 1998, p. 221).
    8. 8. Why does epistemology matter? “…assessment is one area where notions of truth, accuracy and fairness have a very practical purchase in everyday life” (Williams, 1998, p. 221). • LA ‘buy in’ to particular ways of thinking about these issues, & are embedded in systems, but they might be flexible enough to move beyond the current impasse
    9. 9. Learning Analytics • Data mining • Digital trace – including linguistic data • Deployed for pedagogic purposes • Big and rich
    10. 10. Epistemology & Assessment an example
    11. 11. Danish exams with internet access http://news.bbc.co.uk/1/hi/education/8341589.stm
    12. 12. Danish exams with internet access • Allows testing of problem-solving and analysis - sifting information • "if you allow communication, discussions, searches and so on, you eliminate cheating because it's not cheating any more. That is the way we should think." Epistemological assumptions
    13. 13. Danish exams with internet access • Potential for auto-grading – LA role, a new model? http://www.timeshighereducation.co.uk/416090.article A role for LA?
    14. 14. Epistemic assumptions - It isn’t “knowledge” to recall facts - Knowledge lies in the use
    15. 15. Assessment… Formative or summative? What is being assessed?
    16. 16. Pedagogic assumptions • Assessment is used in teaching (but doesn’t drive it) • Pedagogy should involve knowledge practices – not assessment practices • Discourse is fundamental
    17. 17. Technological determinism? • Technology is situated • Interpretive flexibility
    18. 18. The other side of the coin
    19. 19. Multiple perspectives • One side: Direct technological affordances • The other: The practices around technologies – Educator practices – Wider policy/accountability context – Student practices
    20. 20. Educator Practices: LA potential • E.g. at LAK14: Wise, Piety & Hickey
    21. 21. Policy: LA potential • Accountability systems, educator support, investment in AfL, setting priorities (e.g. Denmark)
    22. 22. Student Practices: LA potential • Analytics give unprecedented (?) access to “What students do” - & assumptions around this • Potential shift, from standardised assessments – need for new (psychometric) models
    23. 23. LA in Structured Knowledge Building • Strong CSCL tradition to ‘make explicit’ in structured environments (Knowledge Forum, Belvedere, etc.)
    24. 24. Natural Language Processing based LA • Automated essay feedback • Dialogue scaffolds and automated tutors • Social functions – “if you’re interested in x, you should talk to …” • Tutor support “student a might need some help on…”
    25. 25. Hot Topics • Analytics for student ‘dispositions’ • ‘Personalisation’ through community building – Social networks – Discourse support – Community knowledge • Educator support, visualising data
    26. 26. Conclusions 1. Context of LA as assessment, and pedagogic aid, in context of policy considerations – Danish example 2. Educator practices matter 3. Hope to improve student practices 4. Use of data for particular conversations/assessment/ AfL, is key 5. Design implications – foreground particular facets of data & activity
    27. 27. Thank you @sjgknight sjgknight@gmail.com http://people.kmi.open.ac.uk/knight/ Our papers in this area: • http://oro.open.ac.uk/39226/ Epistemology, assessment, pedagogy: where learning meets analytics in the middle space (2014) • http://events.kmi.open.ac.uk/icls-analytics/ ICLS workshop on analytics for learning and becoming in practice (2014) • Discourse, computation and context – sociocultural DCLA revisited (2013) http://oro.open.ac.uk/36640/ • Tracking epistemic beliefs and sensemaking in collaborative information retrieval (2013) http://oro.open.ac.uk/36553/
    28. 28. Acknowledgements Thanks to my co-authors and supervisors Simon Buckingham Shum and Karen Littleton for their work on this and our workshop papers. Thanks to Cindy Kerawalla and anonymous reviewers for their helpful suggestions on the earlier (LAK13) version of this paper. Images – mostly from Wellcome Images http://wellcomeimages.org/ under CC licence

    ×