Learning analytics (LA) has been touted as a game changer for education. The rapidly growing literature associated with the field serves to promote this fervour in citing the vast impact LA can and will play in the education space. From the detection of at-risk students to address retention and performance, building self-regulated learning, development and identification of 21st Century literacies to the realisation of personalised learning, there appears little that LA cannot contribute to within learning and teaching practice. However, if LA is such an impactful, desirable and worthy endeavour that can effectively improve learning, and our understanding of the learning process, why are there so few examples of institutional LA adoption?

1. A tale of two constructs:
Understanding learning
analytics adoption
Shane Dawson
Shane.dawson@unisa.edu.au
Twitter: @shaned07

2. Introduction
Your Phone knows your GPA
http://studentlife.cs.dartmouth.edu/smartgpa.pdf

3. Introduction
SmartGPA study
• Using smart phones to track student study
behaviour – lead indicators for academic
performance
• Class attendance
• Conversational interactions
• Mobility
• Studying time and
• Social (partying behaviour)
http://studentlife.cs.dartmouth.edu/smartgpa.pdf

4. Introduction
Massive interest in data and analytics.
Examples in education - diverse and rapidly
growing
• Academic performance
• Student retention
• Pastoral care
• Academic literacies
• Social networks – collaborations

5. Introduction
Yet in terms of wide-scale institutional adoption there
are few examples
Why?

6. • Brief overview of learning analytics
• Findings from a national study
• Discussion
• 2 trajectories
• Systems model
• An idealised model?
• Complexity of LA
Introduction

7. Algorithms – predictions
Introduction

8. Education is no different
• Huge investment in analytics
• Ease of access to learner data - LMS
• Growth in technical devices
• Growth in blended/ online learning models
Introduction

9. Get answers to your most important questions like:
• How can I easily find students who are at-risk?
Introduction

10. Definition
…is the collection, collation, analysis and
reporting of data about learners and
their contexts, for the purposes of
understanding and optimizing learning
Learning Analytics
• “game changer” for education

11. Large data sets – trends/ patterns or anomalies.
Learning Analytics

12. LA/EDM research:
• Predictions of learning success (early
alerts)/Performance and retention
• Indicators of on/off task attention (Baker)
• Carelessness and gaming (Baker)
• SRL proficiency (Gasevic; Winne)
• NLP (Rose; Gasevic)
• Learning dispositions (Deakin Crick)
• Graduate qualities/ 21st C literacies (Siemens)
LA Research

13. Great research BUT:
• Ignores the complexity of university wide practices
• Small scale and technology specific
• Tends to be institutional specific
• Lacks guidance to aid further adoption
• Frequently requires high level skills and capacities
LA Research

14. Innovative research BUT:
• Very few University wide examples of LA adoption
• Obviously an area of increasing need and
importance
LA Research

15. • Huge investments in LA
• Consultancy and services
• Technologies
• Research funding
• University priorities
Where is the broad scale impact?
What are the “ideal” processes and practices
for LA adoption?

16. National project to benchmark LA status, policy
and practices for Australian Universities
Many thanks to:
Cassandra Colvin, Alex Wade and Tim Rogers
National Project

17. • understand current LA practice in Australia
• unpack the challenges to institutional
adoption
• Identify practices that can aid the
implementation of LA
Aims

18. 2 complementary but separate studies
• Study 1 – interviews with senior institutional leaders
• Study 2 – concept mapping with LA expert panel
Approach

19. First study:
Interviews with 32 Universities:
• Identification of current practice, methods and
approaches
• Identification of key drivers for institutions, stage
of development, process for implementation,
project leads
Study 1

20. First study:
Coding:
• Development and application of coding protocol
• Cluster analyses performed (PAM clusters)
Study 1

21. • Much interest in LA
• Stated organisational priority
• LA projects were in the early phases of
implementation and small scale (at time of
interview July 2014)
• 2 distinct clusters across variables such as:
implementation, conceptualisation, readiness
Cluster 1 (n=15) – Solutions focused
Cluster 2 (n=17) – Process focused
Study 1

22. High interest – slow uptake
- predominantly at the stage of basic reporting
• Goldstein & Katz (2005) reviewed US universities
and noted vast majority were in Stage 1 or 2 (of
5 stages in maturity)
• Yanosky (2009)
• Bichsel (2012)
Benchmarking

23. Clearly there remain challenges with implementing LA
at scale
Benchmarking

24. 2 Distinct trajectories for implementation
• Differences in cluster variables such as:
• Conceptualisation of LA
• Readiness
• Implementation approach
Study 1

25. • Cluster 1
• focused on retention outcomes
• Limited mention of LA as a means to
improve learning
• Main driver is budget (cost savings)
• Cluster 2
• Broader view of learning analytics and its
application into learning and teaching
practice
Conceptualisation

26. • Cluster 1
• Limited to no articulated strategy
• Minimal capacity building activities
• Success is seen as staff access to information
• Technology infrastructure sound and
developing
• Cluster 2
• Defined strategy
• Developing capacity building activities
• Technology – less emphasis on development.
Readiness

27. • Cluster 1
• Minimal stakeholder engagement
• Leadership top down and siloed
• Vendor tools heavily integrated
• Cluster 2
• Extensive stakeholder engagements
• Leadership top down – but wide-spread.
• Multiple engaged units (IT, teaching, faculty)
• Where vendor tools/ processes adopted
maintained critical perspective
Implementation

28. Strategic capability
Context
Student
demographics
Student
retention and
performance
Govn
accreditation
University
group / ATN/
Rural/ Go8

29. Strategic capability

30. 1. Solutions focused
• LA to address a pressing need
• Time sensitive
2. Process focused
• Networked and integrated model
• Minimal time pressures
• Innovation and experimentation
Strategic capability

31. What are the ideal dimensions for long term
sustainable uptake of LA?
• Invite to Australian and international LA experts
• 28 completed the entire concept mapping phases
• 3 phases – brainstorming; sorting and ranking of
statements
• Following the final ranking phase – a 7 cluster
solution emerged.
Study 2

32. Study 2

33. Bringing it together
Study 1 – 2 clusters
Study 2 – 7 clusters
Essentially – how an organisation approaches its
conceptualisation of LA underpins (2 clusters) the
method for deployment and adoption (7 clusters)

34. Systems Model
Strategic
capability
Interested Implementing
Implementation
capability
Tool/Data
Quality
Research/
Learning
Educator uptake

35. Systems Model
Strategic
capability
Implementing
Tool/Data
Quality
Educator uptake

36. Systems Model
Strategic
capability
Interested Implementing
Implementation
capability
Tool/Data
Quality
Research/
Learning
Educator uptake

37. LA Maturity

38. LA Maturity

39. LA Maturity
Solutions focused
Address an immediate concern – e.g. retention

40. LA Maturity
Process focused
Broad view of LA

41. Merging Models
Merged model – responsive and agile

42. Challenges to be addressed:
• Leadership awareness
• Teams are seldom interdisciplinary
• IT driven and system focused
• Scale versus understanding
• Capabilities and skills deficit.
• Over reliance on current research – requires
further validation across different contexts to
demonstrate transportability of models
Bringing it together

43. Leveraging the outcomes of short term goals for
long term gain
• How do we merge both models to gain both
short and long term impact?
Complexity

44. Complexity
Law of requisite variety (Ashby 1958)
• To control a system – the number of problems
need to be matched by at least an equal number
of responses
• The more complex the system the more problems
Ashby W.R. (1958) Requisite variety and its implications for the control of complex systems, Cybernetica 1:2, p.
83-99. (available at http://pcp.vub.ac.be/Books/AshbyReqVar.pdf, republished on the web by F. Heylighen—
Principia Cybernetica Project)

45. Complexity
Law of requisite complexity (Boisot & McKelvey 2011)
• It take complexity to defeat complexity
• A system must possess a level of complexity that
at least matches that of its environment to
function effectively
Boisot, M., & McKelvey, B. (2011). Complexity and organization--Environment Relations: Revisiting Ashby's Law
of Requisite Variety. The Sage handbook of complexity and management, 279-298.

46. Education is a complex system
• Resilient to change
• Adaptive and self-organising clusters
• Change is non-linear and often unexpected
LA is complex – tendency towards simplification for
implementation
Complexity

47. LA often reduced to independent components
• Data
• Analysis
• Technology
• Dashboards/ visualisations
• Staff training
Does not adequately deal with the inter-relationships
nor the overarching complexity of the system
Complexity

48. • The view to simplify can lead to fixed boundaries
and organisational silos
• IT example
Complexity

49. Conclusion
• LA requires new models for implementation and
leadership
• Enabling leadership
• Whole of organisation
• Models that are agile and research informed
• Working in complexity creates friction
• Embrace the friction – generates innovation

50. Conclusion
• A solutions based model can drive change – but
need to be mindful of responding to changing
organsational needs
• Process based model can drive innovation and
interest – but need to be mindful of how to scale

51. Conclusion
• Combined model framed in the organisational
context
• Small, diffuse pockets of innovation to build
capacity and build interest
• View to scale adoption – demonstration of
impact (technical, pedagogical)
• Distributed enabling leadership (complexity
leadership)

52. Conclusion
• Any “successful” adoption of LA will be dependent
on an institution’s ability to rapidly recognise and
respond to the organisational culture and the
concerns of all stakeholders.

53. Thank you…
Shane.dawson@unisa.edu.au
Twitter: @shaned07