This document discusses learning analytics (LA) practices at the University of Technology Sydney (UTS). It describes UTS's goal of becoming a "data intensive university" to solve problems like student attrition, improve student engagement, enable personalized learning, and allocate resources more effectively. The university uses LA to identify "killer subjects" with high failure rates and understand factors contributing to student failure. UTS also utilizes a student dashboard in its learning management system and provides data literacy training for staff and students. The document is part of a larger OLT-commissioned research project examining LA practices across Australian universities and comparing them to international examples to develop best practice guidance.

1. Student retention
and learning
analytics:A snapshot of current
Australian practices and a
framework for advancement.
OLT commissioned research
project

2. UTS: a data
intensive
university
Overview
1. A case study of an
institutional
approach
2. The commissioned
project

3. “We haven’t fully figured out how to
put analytics to work pervasively
throughout higher education to
make a difference and resolve our
most pressing issues.”
Susan Grajek Vice President of EDUCAUSE in
lamenting the lack of broad scale impact
that analytics has made to date:

4. UTS: A data
intensive
university

5.  a university that knows about data,
regardless of its volume and diversity. It
knows about the use and reuse of data to
better inform teaching, learning and research,
as well as understand business, society and
the university itself, how to learn about and
research data, how to store and curate it, and
how to apply and develop analytical tools.

6. UTS: a data
intensive
university
Learning
UTS: a data intensive
university
1. Solve problems
• attrition
• preparation
• ‘killer’ subjects
2. Promote student
engagement
3. Personalised learning
4. Allocate resources

7. Event lifecycle analytics
Past Present Future
Information What happened?
(Reporting)
What is happening
now?
(Alerts)
What will happen?
(Extrapolation)
Insight
How and why did it
happen?
(Modelling,
experimental
design)
What’s the next
best action?
(Recommendations)
What’s the best/
worst that can
happen?
(Prediction,
optimisatioin)
Understanding Intervention
Source: Davenport et al (2010) Analytics at work

8. Problem
solving

9. i-Educator-
introductionHi – I’m calling
from Student
Services. We’re
calling all first
years just to see
how you’re going
First
year
student
list
(7000+)
UTS students
Student Systems
Outreach program
Outreach workflow

10. Decision tree model for attrition after two years of engineering degree study for the
2003 Domestic entry cohort at Institution D
Source: http://www.altc.edu.au/resource-engineering-qualification-curriculum-uts-2011

11. The UTS Model of
Learning
Why are pass rates for subject X < 20%%
Killer subjects

12. Basic Analysis
Failure rate of 5 killer subjects
48510 is the most stable subject in terms of failure rate
48530 is the most volatile
48540 was no longer offered in recent years
48521 just starts for 3 years
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Basic Analysis
• Failure rate in 5 “killer” subjects

13. Learning informed by data
analytics

14. Findings
• The following factors relate to failing:
– Interval between two adjacent subjects,
– the semester of the last year of student enrollment,
– average grade,
– student pathway.

15. Promoting
student
engagement

16. UTS: a data
intensive
university
Student dashboard

19. Centre for
Connected
Intelligence

20. UTS: a data
intensive
university
Policies and
processes - a
roadmapEnvironmental scanning and mapping
Policies
Data management
The role of information technology
Staff and organisational structure
Communication
Numeracy levels – staff and students

21. UTS: a data
intensive
university
Data literacy for staff
and students

23. Student retention
and learning
analytics:A snapshot of current
Australian practices and a
framework for advancement.
OLT commissioned research
project

24. http://he-analytics.com

25. Assoc. Prof Shane Dawson Project leader
Dr Tim Rogers co-leader
Project Partners
Associ Prof Dragan Gašević
Professor Lori Lockyer
Professor Shirley Alexander
Gabrielle Gardiner
Professor Gregor Kennedy
Ms Linda Corrin
Professor Karen Nelson

26. OLT Commission Project
• OLT commissioned project to identify learning analytics
technologies, processes and policies Australian
universities are employing to address student retention
• 2 funded teams (working collaboratively) -
– Target policy, future direction and international
comparisons (UniSA, UTS, Macquarie, UMelb, USC, UNE)
– Identify current technologies and practices (CDU, Griffith,
BIITE, UNewcastle, Murdoch)

27. Questions
Q1. What is the research evidence, the strengths and
limitations of learning analytics?
Q2. How are Australian universities planning and
utilising analytics to support their learning and teaching
goals; retention strategies; and identification of at-risk
students?
Q3. How are international peer universities using and
developing analytics to support learning and teaching
practice?
Q4 What are the future trends for learning analytics and
what implications do they have for student retention
and informing student learning outcomes?

28. Project aims
Specifically:
- map current (and proposed) LA activity in Australian institutions
- identify the drivers shaping, and factors affording and constraining, its adoption
- Comparatively situate Australian LA activity within the international context
Outcomes:
- produce a ‘roadmap’ of best practice, principles and resources that can inform and
assist institutions in their LA activity
- develop resources that provide an overview of LA including definitions,
applications and tools within Australian universities, the overarching barriers and
potential opportunities surrounding its deployment, future trends and ethical
issues
- A Good Practice Guide that will provide case studies of best practice in LA activity
that will assist institutions in future LA activity.

29. Project methods
1. Semi-structured one-on-one interviews with:
- internationally-recognised research experts around current and future trends
and developments in LA research (completed)
- higher education leaders in North America, UK and Europe with responsibility
for LA around their institution’s LA strategy, progress and goals (completed)
- DVC’s in Australia around their institution’s LA strategy, progress and goals
2. Delphi-method exercise
- - involving panel of international and national experts to explore and provide
consensus on themes arising from interviews and survey
3. Survey data
- Snapshot of current use and planning for LA at all Australian universities. The
survey will be forwarded to DVC’s for input and coordination
- Investigation of academic understanding and use of LA in Australian
institutions