Effective Learner Analytics: a senior leadership - staff - student informed approach

@PlymUniASTI /PlymUniASTIasti@plymouth.ac.ukwww.plymouth.ac.uk/asti
Effective Learner Analytics:
a senior leadership - staff - student
informed approach
Prof Neil Witt, Dr Anne McDermott, Prof Pauline Kneale
& Prof David Coslett

Academic Support, Technology & Innovation
 ASTI and Teaching and Learning
Support are investigating the potential
of using Learning Analytics as a means
of enhancing the student experience
 Grant from the HEA’s Strategic
Excellence Initiative for Vice
Chancellors
Background
Teaching and Learning Support

 The electronic footprint our students leave behind
when they interact with our digital systems
e.g. digital learning environment, electronic library, ePortfolio
 This and other data sources used to track student
engagement and to identify those who may be in
danger of failing
 Web pages and apps present various data
visualisations for personal tutors, students and
others
Definition of Learning Analytics

Definition of Learning Analytics
Institution Faculty School Programme Module Individual
Academic
Analytics
Learning
Analytics
Data
Granularity of Data

Talked to stakeholder groups about
concerns and issues
 What we have found and
recommendations for future work
 Poster
Stakeholder consultation
 Students
 Academic Staff
 Professional Support
Staff
 Senior Leaders
 University Governors
Stakeholders

What could LA look like?

 Mostly information already collected
in a range of ways, from a range of
systems for a range of purposes
 Array of challenges   
 Systematic identification of ‘at risk’
students may place an unsustainable
obligation to act on the University
Challenges:
 Ethical
 Legal
 Data
 Technical
 Policy
 Process
Issues and challenges

Defining the challenges via Stakeholder
Perspectives

 Transparency vital to maintain
trust
 High degree of confidence that
staff would deal with their data
in a ‘professional way’
 Varied in degree of comfort
with easy access to data about
their own performance - closely
monitored students seemed
least worried e.g. health areas
 Most, but not all, keen to compare
their profile with anonymised
cohort average or an ‘ideal’
student, so consider ‘opt out/in’
 Could motivate some but
discourage others
 University’s response to a ‘red flag’
should not be an automatic process
but the start of a conversation
Stakeholder Perspectives:
Students

 More concerns than students
over legal / ethical data use
 Support and training to
understand responsibilities
 Need for openness and
transparency
 Focus should be on benefitting
student not institution
 Questioned effect on retention
students leave for many reasons
 Policy changes needed, e.g.
attendance monitoring, triggers
 Much of this data is already
collected in disparate ways
 Need to ensure compliance with
legislation for current, retros-
pective and future use of data
Academic, Technical & Support Staff

 Need shared vision of what is
meant by Learning Analytics
 Culture of respect for inform-
ation and anonymity required
 Concerns about the scope and
quality of current data
 Analytics data should be
triangulated with other
information
 Opposing views about students having
own and cohort data
‘why would they want to know?’
‘what are they afraid of?’
 Some concerns it could be demoti-
vational, a distraction or encourage a
strategic approach to study
 Could give insight into characteristics of
a successful programme, trajectory of a
successful student, value added over the
course of a programme (Learning Gain)
Senior Leaders

 Potential to increase retention and enhance performance but
would need to show a return on investment
 Varied responses to having personal access
 With Analytics, data must become everyone’s responsibility
 Academic Analytics could aid institutional decision-making
 Enable Plymouth to offer something distinctive to its students
Governors

Recommendations

Recommendations: Policy
Define goal(s)
and specify
initial
measures
Audit policies to
identify
amendments and
gaps
LA owned at a
very high
level
Set and
resource
Institution-
wide
standards for
responding
Application of roles
to control access
Implement
single version
of truth for
data & polices

Recommendations: Policy
Consider
offering ‘opt
out’
Plan for success
e.g. fewer
withdrawals
Build in to
future
procure-
ments

Recommendations: Process
Choice of
Learning
Analytics
solution
Governance
requires a
multidisciplinary
team including
students
Implement
institution-wide
standards for
responding
Consent
agreements
and statements
in line with
planned use

Recommendations: Process
Be open and
transparent,
particularly
with students
Staff develop-
ment to make
responsibilities
clear & support
policy changes
Be aware
there will be
false
negatives/
false
positives
Digital
literacy and
training for
staff

Recommendations: Technology
Specify data
currently easily
accessible
Establish
ownership,
stewardship
and users of
data
Single version
of truth
needed for
chosen data
Bring silos
together (data
warehouse)
Unique
identifier work
required
Work out synergies
with other existing
projects (e.g. S3,
Mobile With
Plymouth app)
Agreements
with 3rd party
provider to
reflect new use

 Work together to develop the
right approach for Plymouth
 Technology is “easy”, the
challenge is to embed into
practice
 Build on existing data sources
(e.g. DLE) and practice (e.g. S3)
 Policies and data need to be
owned centrally (i.e. Academic
Registry)
 Single version of the truth is
essential
 If we trust our data, we will make
decisions based on our data
 We can develop a universal
learning analytics solution, do we
have a universal personal
tutoring system to support it?
Key findings: There is an appetite to use
Learning Analytics at Plymouth

Defined PU Learning Analytics Goals
Goal Organisational or
Technical
Timeline
(expected, if recommendations
implemented )
Establish what data would be
useful and practicable to collect
Technical Summer 2016
Improve student
success/engagement
Technical/Organisational 2017
Provide information to students
to help them succeed
academically
Technical/Organisational 2017
Improve retention Technical/Organisational 2017

 UEG Sponsorship is essential
 Single Version of the truth for all data
 Have an “owner” for data and
revised/updated/new policies
 Use available solutions (i.e. Jisc toolkit,
Mobile With Plymouth, S3)
 Use analytics to support personal tutoring
and institutional decision making
So, what do we need to do now?

Thank you.
Questions?

Effective Learner Analytics: a senior leadership - staff - student informed approach

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Effective Learner Analytics: a senior leadership - staff - student informed approach

Similar to Effective Learner Analytics: a senior leadership - staff - student informed approach (20)

More from Neil Witt

More from Neil Witt (9)

Recently uploaded

Recently uploaded (20)

Effective Learner Analytics: a senior leadership - staff - student informed approach