Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

UTSLTF2014 keynote - Learning.Analytics for Learning.Futures?

3,797 views

Published on

Learning.Analytics for Learning.Futures?

Simon Buckingham Shum and Ruth Deakin Crick
Centre for Connected Intelligence, UTS

The social, technical and political challenges we face as a society demand new ways of thinking and working which are collaborative, holistic and resilient. As we unpack what these words mean, the implications for a university – indeed any learning organisation – run deep. At the core of the paradigm shift we see the need for learners (at all levels) to take increasing responsibility for their learning in authentic contexts: to become resilient agents of their own learning trajectories; to think holistically and to make sense of complex data. Far from being solely ‘graduate attributes’, the same qualities are needed by us: if we can’t model these qualities ourselves, we can’t teach them; if we can’t assess them authentically, we have no evidence base and we can’t provide formative feedback. This line of argument shapes how CIC is conceiving learning analytics (computer-supported tools to help learners and educators gather, analyse, visualise and act on learners’ data) and collective intelligence (networking tools to build a learning community’s evidence-base). In this talk we will give glimpses of these approaches in action, we’ll hear from learners and educators on what this paradigm shift feels like, and through several activities, we invite you to imagine how we can collaborate to test these concepts across UTS, as we move into Learning.Futures.

Published in: Education

UTSLTF2014 keynote - Learning.Analytics for Learning.Futures?

  1. 1. Google Docs for interactive exercises: Learning.Analytics for Learning.Futures? Simon Buckingham Shum Director, Connected Intelligence Centre @sbuckshum #LearningAnalytics Ruth Deakin Crick UTS:CIC Visiting Professor & University of Bristol UK @ruthdeakincrick bit.ly/utsltf2014 cic.uts.edu.au
  2. 2. cic.uts.edu.au
  3. 3. cic.uts.edu.au
  4. 4. Learning.Futures @UTS 5 http://learningemergence.net/2014/04/07/learning-dispositions-authentic-inquiry-in-a-primary-school
  5. 5. cic.uts.edu.au Iden%ty Forma%on Personal Quali%es Knowledge & Skills Competence 6
  6. 6. we should be talking first about cic.uts.edu.au where we want to go then we can discuss engines, performance and dashboards 7
  7. 7. VUCA Volatile • Uncertain Complex • Ambiguous
  8. 8. 9 well-defined routes mapped by others routine performance success depends on following the rules http://weheartit.com/entry/14357348
  9. 9. 10 well-defined routes mapped by others routine performance success depends on following the rules unknown territory naturally complex each challenge unique success depends on improvisation http://awe-inspired.deviantart.com/art/White-Water-Rapids-45947012 http://weheartit.com/entry/14357348
  10. 10. From the known to the unknown Unknown Strange Uncomfortable What we know Familiar Comfortable
  11. 11. From the known to the unknown Unknown Strange Uncomfortable What we know Familiar Comfortable “liminal space”
  12. 12. From the known to the unknown Unknown Strange Uncomfortable What we know Familiar Comfortable “liminal space”
  13. 13. “Liminal Space… when you have left the tried and true but have not yet been able to replace it with anything else. …when you are between your old comfort zone and any possible new answer… If you are not trained in how to hold anxiety, how to live with ambiguity, how to entrust and wait, you will run… anything to flee this terrible cloud of unknowing.” Richard Rohr O.F.M. — on the spirituality of liminal space Limina is the Latin word for threshold, the space betwixt and between http://sojo.net/magazine/2002/01/grieving-sacred-space
  14. 14. So while we initially strive to make our students feel comfortable […] we then must help them balance their desire for security with the need to take risks and explore new ideas and possibilities. Rather than attempting to resolve the tension, a college should help students find their place on this precarious threshold, in the liminal space between the familiar and strange, the old and the new. Johansson & Felten, 2014
  15. 15. Our learners face this. So do we as staff. Unknown Strange Uncomfortable What we know Familiar Comfortable “liminal space”
  16. 16. Can we equip students for liminal space? Do we know how to teach for liminality?
  17. 17. The Knowledge – Agency Window Open-­‐ ended Enquiry Pre-­‐scribed curriculum Teacher-­‐directed enquiry Student led-­‐enquiry Content/Expert led teaching Student-­‐led revision Teacher Directed Learning Self-­‐directed Learning
  18. 18. The Knowledge – Agency Window Open-­‐ ended Enquiry Pre-­‐scribed curriculum Teacher-­‐directed enquiry Student led-­‐enquiry Content/Expert led teaching Student-­‐led revision Teacher Directed Learning Self-­‐directed Learning
  19. 19. The Knowledge – Agency Window Open-­‐ ended Enquiry Pre-­‐scribed curriculum Teacher-­‐directed enquiry Student led-­‐enquiry Content/Expert led teaching Student-­‐led revision Teacher Directed Learning Self-­‐directed Learning
  20. 20. The Knowledge – Agency Window Open-­‐ ended Enquiry Pre-­‐scribed curriculum Teacher-­‐directed enquiry Student led-­‐enquiry Content/Expert led teaching Student-­‐led revision Teacher Directed Learning Self-­‐directed Learning
  21. 21. Analytics depends on what space you’re in Deakin Crick et al (2011), Learning Futures Final Report. Paul Hamlyn Foundation. http://learningemergence.net/library/reports/learning-futures-evaluation-2011 cic.uts.edu.au Open-­‐ ended Enquiry Pre-­‐scribed curriculum Teacher-­‐directed enquiry Student led-­‐enquiry Content/Expert led teaching Student-­‐led revision Teacher Directed Learning Self-­‐directed Learning Teaching as Design Teaching as Script Authenticity Agency Identity
  22. 22. 2 different worldviews (500 school teachers) Ruth Deakin Crick, Chris Goldspink & Margot Foster (2013). Telling Identities: Learning as Script or Design? Learning Emergence Discussion Paper (June, 2013). PDF http://learningemergence.net/events/lasi-dla-wkshp cic.uts.edu.au Develop democratic relationships 1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1 Build a community of learners Negotiate learning Challenge and support learners Teach students how to learn Foster deep understanding Explore the construction of knowledge Communicate in multiple modes Apply and assess in authentic contexts Connect learning to lives and aspirations Build on learners understanding Promote dialogue as a means of learning Dynamic Design Script
  23. 23. Knowledge Agency Window exercise cic.uts.edu.au 24 Google Docs for interactive exercises: bit.ly/utsltf2014
  24. 24. scaffolding open-ended enquiry cic.uts.edu.au 25
  25. 25. The big challenge for 21st century educators “educational philosophy and theory cic.uts.edu.au face the unfamiliar and challenging task of theorising a formative process which is not guided from the start by the target form designed in advance” Zygmunt Bauman 26 Zygmunt Bauman, 2001: The Individualised Society, Cambridge, Polity Press.
  26. 26. Conveying new knowledge cic.uts.edu.au 27 Choosing carefully Collecting thoroughly Connecting deeply Constructing creatively
  27. 27. 28 Choosing and Deciding Observing and Describing Generating Questions Uncovering Stories Knowledge Mapping Applying in the World Connecting with existing knowledge Assessing and Validating Interface with success criteria Using Genuine Interest to Understand, Explore, Create and Improve BREAK-THROUGH INNOVATION LEARNING and PERFORMANCE Dynamics of Open- Ended Enquiry
  28. 28. Jess’s Choice cic.uts.edu.au 29
  29. 29. 30 What will be there in 15 years? What was there before? How many people have been there? How was the gorge made? Have any famous people been there? Why do relationships matter? What kind of people used to be there?
  30. 30. 31
  31. 31. ‘It’s made me not so scared to learn other things,’ ‘It was a tiny little project and it spiraled into all these other things that were connected.’
  32. 32. Anchored in personal interest…Starting with experience ….. So my choice was made….. the gypsy caravan, and my friend…. I looked at the questions generated from my initial mind-map cic.uts.edu.au and I began to see a way of grouping them.
  33. 33. Is there a thin line between the acceptance and rejection of gypsy travellers as a minority group within a wider community? 34 The outcome is derived from the process
  34. 34. 35 …I was digging deep, maybe I was experiencing a little of what we had talked about, “theory U”. An inward bound journey leading to an outward, letting go of preconceived ideas and experiencing change through the process.
  35. 35. cic.uts.edu.au anchored in personal interest (rather than didactic intent) starts with experience (rather than opinion or theory) creates its own dynamic (rather than following taught routines) relates and unites disciplines and domains (rather than fragmenting them) Integrates values with learning (rather than separating them) is reflexive on its process (rather than concerned only with outcome) allows the outcome to derive from the process (rather than pre-determine it) 36
  36. 36. assessing open-ended enquiry? cic.uts.edu.au 37 Google Docs for interactive exercises: bit.ly/utsltf2014
  37. 37. Equipping learners to tackle complexity 38 Competence in the world Knowledge skills & understanding Personal qualities, values attitudes & dispositions Identity story, desire, motivation, relationships, purpose Deakin Crick, R. (2012). Student Engagement: Identity, Learning Power and Enquiry - a complex systems approach. in: Christenson, S., Reschly, A. & Wylie, C. (eds.) The Handbook of Research on Student Engagement. New York: Springer Practical Propositional Presentational Experiential Public Personal
  38. 38. 39 ….if the UK wants to produce more engineers, it needs to redesign the education system so that these ENGINEERING HABITS OF MIND become the desired outcomes of engineering education…p3 Royal Academy of Engineering, (2014) Thinking Like an Engineer, London, RAE.
  39. 39. This material is based upon work supported, in whole or in part, by the U.S. Department of Defense through the Systems Engineering Research Center (SERC) under Contracts H98230-08-D-0171 and HQ0034-13-D-0004 . The SERC is a federally funded University Affiliated Research Center (UARC) managed by Stevens Institute of Technology consisting of a collaborative network of over 20 universities. See www.SERCuarc.org. cic.uts.edu.au 40 Systems Engineering Research Center (SERC) Stevens Institute of Technology, New Jersey, USA. .characteristics of effective engineers…
  40. 40. cic.uts.edu.au 1. Paradoxical Mindset ― Big Picture Thinking and Attention to Detail ― Strategic and Tactical ― Analytic and Synthetic ― Courageous and Humble ― Methodical and Creative 2. Effective Communication ― Modes (oral and written; good speakers and listeners) ― Audience (bridge between problem domain and solution domain) ― Content (social, managerial, technical) ― Purpose (understanding needs, negotiation, information brokering, technical arbitration, driving consensus) 41 3. Flexible Comfort Zone ― Open Minded ― Rational Risk Taking ― Multidisciplinary ― Enjoys Challenges 4. Smart Leadership ― Quick Learning and Abstraction ― Knowing when to stop ― Focused on ‘Vision’ for System ― Ability to Connect the Dots ― Patience 5. Self Starter ― Curiosity _ Passionate and Motivated ― Eager to Learn
  41. 41. where do learning analytics fit? cic.uts.edu.au 42
  42. 42. cic.uts.edu.au Academic Analytics: focus on data for senior educational leaders Macro: region/state/national/international Meso: institution-wide 43
  43. 43. cic.uts.edu.au Learning Analytics: Data explosion at the micro level enriching, and enriched by, the meso+macro levels Macro: region/state/national/international Meso: institution-wide Micro: individual student progress Learning Analytics UNESCO Policy Briefing — Simon Buckingham Shum 44
  44. 44. analytics for open-ended enquiry? cic.uts.edu.au 45
  45. 45. cic.uts.edu.au dispositional analytics 46
  46. 46. Survey-based analytics for learning dispositions cic.uts.edu.au Deakin Crick et al (In Press). Developing Resilient Agency in Learning: the Crick Learning for Resilient Agency Profile. 47
  47. 47. Survey-based analytics for learning dispositions cic.uts.edu.au Mindful Agency Sense making Creativity Hope and optimism Collaboration Belonging Curiosity Deakin Crick et al (In Press). Developing Resilient Agency in Learning: the Crick Learning for Resilient Agency Profile. Rapid Visual Feedback to SImulate Self-­‐Directed Change A framework for a coaching conversaIon 48
  48. 48. Using my intuition & imagination to generate new ideas & knowledge. Taking risks & playing with ideas and artefacts to arrive at new solutions. cic.uts.edu.au Taking responsibility for my own learning over time through defining my purposes, understanding and managing my feelings, knowing how I go about learning & planning my learning journey carefully. Mindful Agency Sense making Creativity Having the optimism & hope that I can learn & achieve over time. Having a growth mindset; believing I can generate my own new knowledge for what I need to achieve Hope and optimism Being able to work with others, to collaborate and co-generate new ideas and artefacts. Being able to listen and contribute productively to a team. Collaboration Being part of a learning community at work, at home, in education & in my social networks. Knowing I have social resources to draw on when I need them Belonging Curiosity Making connections between what I already know & new information & experience. Making meaning by linking my story, my new learning & my purpose. Wanting to get beneath the surface & find out more. Always wondering why and how. 1 An emotional orientation of being open & ready to invest in learning, having flexible self-belief, willing to persist & manage any self-doubt. A necessary pre-requisite for developing resilience in learning Dependent Openness to learning Closed
  49. 49. cic.uts.edu.au blogging + dispositional analytics 50
  50. 50. cic.uts.edu.au Blogging for learning Blogs offer learners opportunities to incorporate many perspectives, develop carefully crafted contributions, reflect and make considered responses to others (Ferguson et al., 2007). This medium provides an environment in which people can observe, articulate and refine practices (Efimova et al., 2004). At the same time, by making use of the comment facility, bloggers are able to share thoughts, ideas and opinions (Du and Wagner, 2005). In order for students to engage effectively with this emerging genre, they need to be able to experiment and take ownership of their writing, learning to develop a blog as a space for personal learning, reflection and interaction (Bryman and Burgess, 1994).
  51. 51. cic.uts.edu.au EnquiryBlogger (Wordpress Multisite plug-ins)
  52. 52. Composing and categorising a blog post cic.uts.edu.au Standard blog editor, including option to embed multimedia Categories relating to authentic enquiry, which are visualized by the plugins
  53. 53. cic.uts.edu.au Enquiry Spiral widget
  54. 54. cic.uts.edu.au Enquiry Spiral widget
  55. 55. cic.uts.edu.au ELLI Spider widget
  56. 56. cic.uts.edu.au ELLI Spider widget
  57. 57. cic.uts.edu.au Mood View widget
  58. 58. cic.uts.edu.au Dashboard view
  59. 59. cic.uts.edu.au Teacher’s dashboard for EnquiryBlogger
  60. 60. cic.uts.edu.au Masters level EnquiryBloggers Graduate School of Education, University of Bristol
  61. 61. cic.uts.edu.au Masters level EnquiryBloggers Graduate School of Education, University of Bristol
  62. 62. cic.uts.edu.au Masters level EnquiryBloggers Graduate School of Education, University of Bristol
  63. 63. cic.uts.edu.au 64 Primary School EnquiryBloggers Bushfield School, Milton Keynes
  64. 64. cic.uts.edu.au Primary School EnquiryBloggers Bushfield School, Milton Keynes
  65. 65. writing analytics cic.uts.edu.au 66
  66. 66. cic.uts.edu.au Towards Academic Writing Analytics Relevance Understanding & Knowledge Structure & Organisation Linguistic Accuracy Illustrations Referencing Argumentation 67
  67. 67. cic.uts.edu.au Rhetorical functions of metadiscourse identified by the Xerox Incremental Parser (XIP) BACKGROUND KNOWLEDGE Recent studies indicate … … the previously proposed … … is universally accepted ... NOVELTY ... new insights provide direct evidence ... ... we suggest a new ... approach ... ... results define a novel role ... OPEN QUESTION … little is known … … role … has been elusive Current data is insufficient … GENERALIZING ... emerging as a promising approach Our understanding ... has grown exponentially ... ... growing recognition of the importance ... CONTRASTING IDEAS … unorthodox view resolves … paradoxes … In contrast with previous hypotheses ... ... inconsistent with past findings ... SIGNIFICANCE studies ... have provided important advances Knowledge ... is crucial for ... understanding valuable information ... from studies SURPRISE We have recently observed ... surprisingly We have identified ... unusual The recent discovery ... suggests intriguing roles SUMMARIZING The goal of this study ... Here, we show ... Altogether, our results ... indicate 68
  68. 68. cic.uts.edu.au XIP rhetorical parser applied to student writing Simsek, D., S. Buckingham Shum, Á. Sándor, A. De Liddo and R. Ferguson (2013). XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scientific Metadiscourse. 1st International Workshop on Discourse-Centric Learning Analytics, at 3rd International Conference on Learning Analytics & Knowledge. Leuven, BE (Apr. 8-12, 2013). http://oro.open.ac.uk/37391 69
  69. 69. SUMMARY & CONTRIBUTION cic.uts.edu.au XIP rhetorical parser applied to student writing CONTRAST Simsek, D., S. Buckingham Shum, Á. Sándor, A. De Liddo and R. Ferguson (2013). XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scientific Metadiscourse. 1st International Workshop on Discourse-Centric Learning Analytics, at 3rd International Conference on Learning Analytics & Knowledge. Leuven, BE (Apr. 8-12, 2013). http://oro.open.ac.uk/37391 70
  70. 70. Buckingham Shum S.J., Uren V., Li G., Sereno B. and Mancini C. (2007). Modelling Naturalistic Argumentation in Research Literatures: Representation and Interaction Design Issues. Int. Jnl. Intelligent Systems, Special Issue on Computational Models of Natural Argument, Eds: C. Reed and F. Grasso, 22: 17-47. Sereno B., Buckingham Shum S. and Motta E. (2005). ClaimSpotter: an environment to support sensemaking with knowledge triples. Proceedings of the 10th international conference on Intelligent user interfaces. San Diego, California, USA: ACM, 199-206. cic.uts.edu.au Detailed video analysis of user experience 71 • how users engage with a text annotated by a rhetorical parser • whether users can annotate a text semantically
  71. 71. Behavioural analytics for learning dispositions? Tagging/sharing/blogging/social patterns reveal how you see connections between ideas? cic.uts.edu.au Mindful Agency Sense making Creativity Hope and optimism Collaboration Belonging Curiosity Social network patterns, teamwork effectiveness and initiation of relationships? Questioning, arguing and search behaviours reveal intrinsic curiosity and epistemic commitments? 72
  72. 72. dispositional analytics exercise cic.uts.edu.au 73 Google Docs for interactive exercises: bit.ly/utsltf2014
  73. 73. cic.uts.edu.au mapping our learning@UTS analytics for organisational learning 74
  74. 74. 1. Build a picture of the target problem and ‘what works’ — what’s known about solutions cic.uts.edu.au Coordinating Maps (of the system, problem, and strategies) 75
  75. 75. A “Driver Diagram” summarising what is known about Productive Persistence in mathematics cic.uts.edu.au Examples of coordinating Maps to guide a systemic improvement community A map of the “Challenge Space” for student mathematics http://www.carnegiefoundation.org/elibrary/getting-ideas-action-building-networked- improvement-communities-in-education http://www.carnegiealphalabs.org/persistence/ 76
  76. 76. cic.uts.edu.au 2. Coordinate to test solutions in different contexts in UTS Coordinating Maps (of the system, problem, and strategies) UTS learning cUonTtSe lxeta rning cUoTnSte lxeta rning context 77
  77. 77. cic.uts.edu.au 3. Pool the Data (quant. + qual.) Data & Evidence Coordinating Hub Maps (of the system, problem, and strategies) UTS learning cUonTtSe lxeta rning cUoTnSte lxeta rning context 78
  78. 78. cic.uts.edu.au An evidence hub shows who is working on which parts of the problem evidence-hub.net People / Organizations / Projects / Claims / Evidence Evidence Hub for Research by Children & Young People: http://rcyp.evidence-hub.net 79
  79. 79. cic.uts.edu.au 4. Analyse the data Data & Evidence Hub Human + Machine Analytics Coordinating Maps (of the system, problem, and strategies) UTS learning cUonTtSe lxeta rning cUoTnSte lxeta rning context 80
  80. 80. cic.uts.edu.au 5. interpret, narrate, debate the meaning Data & Evidence Hub Collective Sensemaking Coordinating Maps (of the system, problem, and strategies) Human + Machine Analytics UTS learning cUonTtSe lxeta rning cUoTnSte lxeta rning context 81
  81. 81. 6. Share insights in Hub, update your maps, and repeat… cic.uts.edu.au Data & Evidence Hub Collective Sensemaking Coordinating Maps (of the system, problem, and strategies) Human + Machine Analytics UTS learning cUonTtSe lxeta rning cUoTnSte lxeta rning context 82
  82. 82. Evidence Hub: indexing stories + evidence cic.uts.edu.au Systems Learning & Leadership Evidence Hub: http://sysll.evidence-hub.net A wizard guides the user through the submission of a structured story: • What’s the Issue? • What claim are you making/ addressing? • What kind of evidence supports/challenges this? • Link it to papers/data • Index it against the Hub’s core themes 83
  83. 83. cic.uts.edu.au …which generates a knowledge tree http://learningemergence.net/2013/07/17/deed-elli-ai-ci-systemic-school-learning Issue Potential Solution Supporting Evidence (practitioner story) 84
  84. 84. Impact Map: how much evidence is there to support each hypothesis? cic.uts.edu.au http://oermap.org/hypothesis/578/hypothesis-a-performance/ 85
  85. 85. exercise on improvement cycle cic.uts.edu.au 86 Google Docs for interactive exercises: bit.ly/utsltf2014
  86. 86. cic.uts.edu.au summary use of analytics is shaped by pedagogy, epistemology, assessment regime 87
  87. 87. analytics for the assessment of authentic enquiry / C21 skills are a frontier challenge ! formative, personalised feedback to learners cic.uts.edu.au 88
  88. 88. to learn more… cic.uts.edu.au 89
  89. 89. Designing holistic analytics for 21st century learners? cic.uts.edu.au http://reinventors.net/roundtables/new-metrics 90
  90. 90. LearningEmergence.net http://learningemergence.net cic.uts.edu.au 91
  91. 91. ALASI 2014 Australian Learning Analytics Summer Institute Thurs 20th–Fri 21st Nov, 2014, Sydney alasi2014.uts.edu.au twitter: #alasi2014 cic.uts.edu.au In association with Hosted by
  92. 92. cic.uts.edu.au cic.uts.edu.au CIC events coming up… Nov 14, 10am-2pm: Assessing Learning Dispositions using CLARA (Ruth Deakin Crick) Nov 17, 9.30-10.30am: Open Digital Badges (Dan Hickey, Univ. Indiana) Nov 20-21: Australian Learning Analytics Summer Institute

×