Learning analytics for learning and becoming in practice - #ICLS2014 workshop
1. Learning Analytics
Process vs Practice
Simon Knight, Alyssa Wise, Golnaz Arastoopour,
David Williamson Shaffer, Simon Buckingham Shum, Paul Kirschner, Wesley Collier
18. Process vs. Practice and Learning Analytics
BIG Data Mining
RICH Data Exploration
&
THEORY Informed Interpretation
19. Process vs. Practice and Learning Analytics
BIG Data Mining
RICH Data Exploration
&
THEORY Informed Interpretation
RECONSTRUCTION of a past civilization
30. How can learning analytics get at practices?
Our schedule for the day
31. How can learning analytics get at practices?
Our schedule for the day
Strategies Over Time as Leverage Towards Practices
32. How can learning analytics get at practices?
Our schedule for the day
Strategies Over Time as Leverage Towards Practices
Co-Occurrence as Leverage Towards Practices
33. How can learning analytics get at practices?
Our schedule for the day
Strategies Over Time as Leverage Towards Practices
Co-Occurrence as Leverage Towards Practices
Practices in Constrained Environments
34. How can learning analytics get at practices?
Our schedule for the day
Strategies Over Time as Leverage Towards Practices
Co-Occurrence as Leverage Towards Practices
Practices in Constrained Environments
Interactive Demos
35. How can learning analytics get at practices?
Our schedule for the day
Strategies Over Time as Leverage Towards Practices
Co-Occurrence as Leverage Towards Practices
Practices in Constrained Environments
Interactive Demos
How Far Can Bottom-Up Methods Get us Towards
Practices?
36. Special Issue
We invite you to consider these issues and
submit to the related special issue in JLA!
37. Journal of Learning Analytics special issue on
Learning Analytics and Learning Theory
Topics include, but are not limited to:
• How understanding of specific learning phenomenon (processes, practices) has helped
to shape particular approaches to learning analytics
• How learning analytics have helped to refine / improve our understanding of
particular learning phenomenon (processes, practices)
• Work contributing to our understanding of how people engage with learning analytics
as part of their learning processes or practices
For more information please see the CFP available at http://learning-analytics.info
This special issue focuses on the intersection of learning theory and analytics.
Specifically it will include papers demonstrating the ways in which learning theories
are being used to craft analytics, how analytics have helped us advance learning
theories, and the initial development of theories of learning with analytics
Guest Editors: Alyssa Friend Wise & David Williamson Shaffer
38. Question groups
Constraint, co-occurrence, & temporality
https://icls.etherpad.mozilla.org/1
Discuss:
1. the ways in which analytics relate to our
understanding of practice, for example:
A. What are the specific challenges for using analytics to
understand practices as opposed to processes?
B. What is the distinction between process and practice?
C. How have the work we’ve seen so far tried to get at
these issues through their theoretical models, working
definitions, tools and analytic approaches they’ve taken
39. Food Near the Millennium Hotel
(1 block) 28th St. and Arapahoe Ave.,
The Twenty Ninth Street mall anchors the intersection and is ideal for
quick meals at Chipotle or Noodles & Company (both founded in
Boulder), along with Pei Wei, Modmarket, and Smashburger, among
several others.
Additional recommendations within walking distance:
Larkburger ($) – Fresh, fast, delicious burgers paired with truffle & parm
fries.
Rincon del Sol ($) – Simple, affordable Mexican food and powerful
margaritas.
Tibet Kitchen ($) – Authentic momos and noodle dishes affordably
priced.
Il Pastaio ($$) – Voted best Italian in Boulder with delicious homemade
pastas.
Pizzeria Da Lupo ($$) – Boulder’s best pizza, wood fired pies and
amazing salads.
Zolo Grill ($$) – A local favorite for upscale Southwest cuisine and tasty
margs.
40. Demos
To what extent do the tools ‘get at’ practice?
How do they do so? (e.g. how do the day’s themes relate?)
How could you integrate this tool as part of your
teaching / learning practice?
Generality / localization
How does theory conect to learning analytic
tools and methods?
How much theory is embedded within the tool?
What’s the perspective of the tool?
Does it matter?
41. (Some of the) Burning questions
• Is data mining/geology/archeology the right
analogy? What else might we describe?
• Process data and outcome data--how are they
related in the learning analytics world?
• How do we deal with granularity, and
segmentation? How do we ‘theorise' which events
(or features) make up our data for analysis?
• Can we build practice models without a rich
theorized account of the domain?
• Multivocality – What role does it play? (Validation?)
42. Thanks for coming! Food/drinks (with
moocshop) @ Backcountry Pizza & Tap House
Evaluation: see http://goo.gl/suk0rL
The definition of practice entails it is constituted of processes.
Is it just a sum of processes or is there more?
Practices encompass more than isolated behaviors. They reflect identity, goals, and intentionality. There is a move in learning analytics away from simple observational variables towards more meaningful latent factors that these observational variables may reflect, like commitment, metacognitive awareness, or cognitive engagement. The concept of a practice fits within this class of more abstract notions.
The definition of practice entails it is constituted of processes.
Is it just a sum of processes or is there more?
Practices encompass more than isolated behaviors. They reflect identity, goals, and intentionality. There is a move in learning analytics away from simple observational variables towards more meaningful latent factors that these observational variables may reflect, like commitment, metacognitive awareness, or cognitive engagement. The concept of a practice fits within this class of more abstract notions.
Is practice / process defined by (a) what is done in the world [constrained enviro] (b) what data is collected [bottom up?] (c) how data processed [over time, co-occurrence, (d) how interpreted
Process neutral?? Practice context dependent?
Our insights suggest that the dominant focus of learning analytics’ practices on “big” automatically collected data is insufficient for informing leaning design. There is a need to integrate “big” data mining with “rich” data exploration and theory -Informed interpretation
Our insights suggest that the dominant focus of learning analytics’ practices on “big” automatically collected data is insufficient for informing leaning design. There is a need to integrate “big” data mining with “rich” data exploration and theory -Informed interpretation
Our insights suggest that the dominant focus of learning analytics’ practices on “big” automatically collected data is insufficient for informing leaning design. There is a need to integrate “big” data mining with “rich” data exploration and theory -Informed interpretation
Lots of people tracking data, but often seems to be ‘learners are doing this’ w/o a clear conceptualization of why ‘this’ is something important for learning (in this context).
For example in online discussions, lots of people seem to collect data on the number of posts made or viewed. The processes of making and viewing posts are clearly part of learning through online discussions, but it is less clear how to think about these variables as important as part of the practice of learning through online discussions.
Shaffer, D. W. (2013). How Research into Epistemic Network Analysis Might Inform DCLA (Keynote). In 1st International Workshop on Discourse-Centric Learning Analytics. Leuven, Belgium. Retrieved from http://www.solaresearch.org/events/lak/lak13/dcla13/
In my (Wise’s) work, we built up a theoretical framework about the value of attending to others’ ideas related to the breadth and depth with which it is done and then crafted ‘second generation’ analytics to track this. I.e. from [# of posts viewed -> % of posts read].
So one point here is that the metrics are driven by a theoretical frame (this is the geology / archeology point).
But also note that whether ‘breadth’ is important also depends on the specific learning context (i.e. is the discussion a problem solving exercise or a Q & A help forum). So I wonder if perhaps the notion of ‘practice’ is more localized than process….
Finally it is perhaps interesting to note that both processes and practices are things that happen over time - what do we make of other analytics that don’t have temporality [e.g. end of course outcomes]
Our insights suggest that the dominant focus of learning analytics’ practices on “big” automatically collected data is insufficient for informing leaning design. There is a need to integrate “big” data mining with “rich” data exploration and theory -Informed interpretation
Also need to consider “recreation of a past civilization” / putting the person back in temporal analyses….
Our insights suggest that the dominant focus of learning analytics’ practices on “big” automatically collected data is insufficient for informing leaning design. There is a need to integrate “big” data mining with “rich” data exploration and theory -Informed interpretation
Also need to consider “recreation of a past civilization” / putting the person back in temporal analyses….
Wise, A. F. (2013). Moving Beyond (Mere) Narrative. In Cyberinfrastructure for Design-Based Research Workshop at the International Conference on Computer-Supported Collaborative Learning (Invited Talk), Madison, WI.
Is practice / process defined by (a) what is done in the world [constrained enviro] (b) what data is collected [bottom up?] (c) how data processed [over time, co-occurrence, (d) how interpreted
Process neutral?? Practice context dependent?
Single sequences, cde and count strategies, and individual time slices problematic
Analysis of patterns over time, and of developing behaviours = practices?
Strategy = pr
ocess in context
Single sequences, cde and count strategies, and individual time slices problematic
Analysis of patterns over time, and of developing behaviours = practices?
Strategy = pr
ocess in context
Single sequences, cde and count strategies, and individual time slices problematic
Analysis of patterns over time, and of developing behaviours = practices?
Strategy = pr
ocess in context
Single sequences, cde and count strategies, and individual time slices problematic
Analysis of patterns over time, and of developing behaviours = practices?
Strategy = pr
ocess in context
The ways procsses and practices are embedded; processes become practices within particular cultural contexts?
Tension between desire to generalise the tools and contexts in learning analytics and local context, etc., how does this play into process/practice?
Temporal component
Co-occurrence of events with other events, tools, actors, etc.
Often we work in constrained tasks/technological environments/pedagogic contexts, etc. how does this relate to practice/process. E.g. maths learning games in a constrained environment, what practices are being assessed? To what extent are there (And should we worry about) environment specific v. content or domain oriented practices/processes?
To what extent do the tools ‘get at’ practice?
How do they do so? (e.g. how do the day’s themes relate?)
How could you integrate this tool as part of your teaching / learning practice?
Generality / localization
How does theory conect to learning analytic tools and methods?
How much theory is embedded within the tool?
What’s the perspective of the tool?
Does it matter?