Learning Analytics - What Do Stakeholders Really Think?

@PlymUniASTI /PlymUniASTIasti@plymouth.ac.ukwww.plymouth.ac.uk/asti
Learning Analytics: What Do Stakeholders Really Think?
Prof Neil Witt, Dr Anne McDermott, Prof Pauline Kneale & Prof David Coslett

Academic Support, Technology & Innovation
 ASTI and Teaching and Learning Support
investigating the potential of using Learning
Analytics as a means of enhancing the student
experience
 Grant from the HEA’s Strategic Excellence Initiative
for Vice-Chancellors
Background
Teaching and Learning Support

 The electronic footprint our students leave behind when they
interact with our digital systems
e.g. digital learning environment, electronic library, ePortfolio
 This and other data sources used to track student engagement
and to identify those who may be in danger of failing
 Web pages and apps used to present various data visualisations
for personal tutors, students and others
Definition of Learning Analytics

Definition of Learning Analytics
Institution Faculty School Programme Module Individual
Academic
Analytics
Learning
Analytics
Data
Granularity of Data

A student perspective?

Understanding our Stakeholders’ Perspectives

 Mostly information already collected in a range of ways,
from a range of systems for a range of purposes
 Array of challenges           
 Systematic identification of ‘at risk’ students may place an
unsustainable obligation to act on the University
Challenges:
 Ethical
 Legal
 Data
 Technical
 Policy
 Process
Key issues, challenges and concerns

 Transparency vital to maintain trust
 High degree of confidence that staff
would deal with their data in a
‘professional way’
 Varied in degree of comfort with easy
access to data about their own
performance - closely monitored
students seemed least worried e.g. health
areas
 Most, but not all, keen to compare their
profile with anonymised cohort average or
an ‘ideal’ student, so consider ‘opt out/in’
 Could motivate some but discourage others
 University’s response to a ‘red flag’ should
not be an automatic process but the start of
a conversation
Stakeholder Perspectives:
Students

 More concerns than students over legal /
ethical data use
 Support and training to understand
responsibilities
 Need for openness and transparency
 Focus should be on benefitting student
not institution
 Questioned effect on retention students leave
for many reasons
 Policy changes needed, e.g. attendance
monitoring
 Much of this data is already collected in
disparate ways
 Need to ensure compliance with legislation
for current, retrospective and future use of
data
Academic, Technical & Support Staff

 Need shared vision of what is meant
by Learning Analytics
 Culture of respect for information and
anonymity required
 Concerns about the scope and quality
of current data
 Analytics data should be triangulated
with other information
 Opposing views about students having own and
cohort data
‘why would they want to know?’
‘what are they afraid of?’
 Some concerns it could be demotivational, a
distraction or encourage a strategic approach to
study
 Could give insight into characteristics of a
successful programme, trajectory of a successful
student, value added over the course of a
programme (Learning Gain)
Senior Leaders

 Potential to increase retention and enhance performance but would need to
show a return on investment
 Varied responses to having personal access
 With Analytics, data must become everyone’s responsibility
 Academic Analytics could aid institutional decision-making
 Enable Plymouth to offer something distinctive to its students
Governors

Recommendations

 LA owned at a very high level
 Plan for success e.g. fewer
withdrawals
 Define goal(s) and specify initial
measures
 Audit policies to identify
amendments and gaps
 Implement single version of
truth for data & policies
 Set and resource Institution-
wide standards for responding
 Build-in to future procurements
 Consider offering ‘opt-out’
Recommendations: Policy

 Consent agreements and
statements in line with planned use
 Choice of Learning Analytics
solution
 Implement institution-wide
standards for responding
 Governance requires a
multidisciplinary team including
students
 Digital literacy & training for staff
 Be open and transparent,
particularly with students
 Be aware there will be false
negatives/ false positives
 Staff development to make
responsibilities clear and
support policy changes
Recommendations: Process

 Bring silos together (e.g. data
warehouse)
 Single version of truth needed
for chosen data
 Specify data currently easily
accessible
 Establish ownership,
stewardship and users of data
 Agreements with 3rd party
provider to reflect new use
 Unique identifier work required
 Work out synergies with other
existing projects (e.g. S3, Mobile
With Plymouth app)
Recommendations: Technology

 A response to a ‘red flag’ should not be
an automatic process but the start of a
conversation
 Implement single version of truth for
data and policies
 Policy changes needed, e.g. attendance
monitoring
 Support and training to understand
responsibilities and support policy
changes
 Culture of respect for information
and anonymity required
 Establish ownership, stewardship and
users of data
 Policies and data need to be owned
centrally (i.e. Academic Registry)
 With Analytics, data must become
everyone’s responsibility
Institutional Checklist

 Senior Sponsorship is essential
 Single Version of the truth for all data and policies
 Have an “owner” for data and revised/updated/new policies
 Use available solutions (i.e. Jisc toolkit, Mobile With Plymouth, S3)
 Use analytics to support personal tutoring and institutional decision making
 Learning Analytics is about Culture Change, not technology
Making it so

Thank you
Questions?

Learning Analytics - What Do Stakeholders Really Think?

More Related Content

What's hot

Viewers also liked

Similar to Learning Analytics - What Do Stakeholders Really Think?

More from Neil Witt

Recently uploaded

Learning Analytics - What Do Stakeholders Really Think?