1
Learning analytics:
the way ahead
Rebecca Ferguson
The Open University
Nordic LASI, 2017
2
Learning analytics
The measurement, collection, analysis and
reporting of data about learners and their
contexts, for purposes of understanding
and optimizing learning and the
environments in which it occurs.
Priority areas for education and training
3
Open and innovative education and training,
fully embracing the digital era.
Strong support for teachers, trainers, school leaders
and other educational staff
Relevant and high-quality knowledge, skills and
competences developed throughout lifelong learning
Focus on learning outcomes for
employability, innovation, active
citizenship and well-being and inclusive
education, equality, equity,
non-discrimination and the
promotion of civic competences.
• Our researchers and students shall contribute
insight and communication of knowledge in
public discussions.
• We will challenge the knowledge front and
conventional notions through critical analysis
and knowledge made available to everyone.
• Through research and education, we shall
contribute towards challenging power
structures and promote a diversified and
sustainable society.
• We shall have the reputation of being a
national institution of culture and a crucible for
new ideas, innovation and new ways of
learning.
• We shall be a meeting place for staff, students
and society at large in an attractive arena for
lifelong learning.
• We shall have a strong and vibrant university
democracy characterized by generosity,
openness, diversity and dialogue.
Priority areas for education and training
5
• Bringing together different sectors: higher education, schools & workplace learning
• Building networks that have outlived the project’s funding period
• Helping to develop learning analytics capability
• Creating and sharing resources
• Developing visions of the future and agreeing how to work towards them
http://www.laceproject.eu/
6
http://careers2030.cst.org/jobs/
Preparing for the future
7
(Just over) a decade of change
2012: ‘Year of the MOOC’
2007: Launch of the iphone
2006: First tweets
Provocations
for visions
of the
future
Full report
bit.ly/28X5tq7
Provocation 1:
Learners are
monitored by
their learning
environments
Learners are monitored by
their learning environments
Just as in bioethics,
there are
fundamental human
factors at play here
Too much Big
Brother vision to
be appealing
I think it is a promising
line of work for learning
analytics. I think there
will be many advances
in the use of sensors to
identify aspects that
can be applied to
this vision
Provocation 2:
Learners’ personal
data are tracked
Learners’ personal
data are tracked
Wearable sensors are
already present, but in
the next future they
must be improved,
especially for health
purposes, such as
diabetes monitoring or
cardiovascular diseases
prevention
Quantified self strongly
builds on reflection and
self-organisation. This is
good and feasible
A reliable evidence base
for the effectiveness of
these measures and some
sort of safety control to
prevent irresponsible
recommendations
Provocation 3:
Analytics are
rarely used
Analytics are
rarely used
There should be strong
discussion about the
ethical concerns that
apply to each approach
to learning analytics
Focus less on analytics to
automate, but rather to be
student-centred, to inform the
learner and improve their
personal practice. Once analytics
provides real, tangible value to
the learner, you begin to both
alleviate concerns about privacy
AND develop faith that if you can
give control of data back to the
learners, they will opt-in to
learning analytics.
The research community
must concretely show the
benefits of the use of data
to improve educational
outcomes
Provocation 4:
Learners
control
their own
data
Learners
control
their own
data
Absolutely
essential.
Organisations
must not rely on
the data, but try
to build trust with
their learners so
that learners see
the benefit of
sharing
Can we assume
that ‘data
owners’ know
what to do with
their data? Some
people can’t even
manage the
money in their
pockets
Users should be entitled to
know how their data are
interrogated and used and
for what reasons. This
should be made explicit
and easy to understand
Greatly limits our ability to
effectively use learning
analytics to improve
learning for
ALL students
Provocation 5:
Open systems are
widely adopted
Open systems are
widely adopted
• Define standards for the
information exchange
• Define standards to exploit the
information stored
• Define guides about what
information is useful
• Define and publish different models
to represent the information
Align with corporate
and stop thinking that
academic is leading!
A multi-pronged approach
involving IT services, national
organisations like JISC and
ministry of education and
tertiary institutions would all
need to be involved
Provocation 6:
Learning analytics are essential tools
Learning analytics are essential tools
Very little credible research has
demonstrated any real large-scale
benefits to learners or institutions
Learning is not only about
success is about learning from
failure. So, yes, I think that it
would be desirable that the
prediction rates are very high,
however, I don't think it is the
final goal
What is needed is support
for reflection, discussion and
debate on the purpose of it
all, especially to curb the
excesses of those that see
learning as something
teachers do to students. We
need to nurture rich,
reflective communities
Provocation 7:
Analytics help learners
make the right choices
Analytics help learners
make the right choices
From bitter experience,
I'm aware of the very
slow pace of
institutional change
Companies will sell
politicians on the
budget savings and
lead us here
With learning there is an
element that learning is a
process – reducing it to
simply outcomes that can be
measured is dangerous
Limited as ignores learning
through interactions with
both peers and the wider
social environment
Provocation 8:
Analytics have largely
replaced teachers
Analytics have largely
replaced teachers
Autonomy begets engagement,
motivation, persistence, relevance
The collective is as
important as the
individual: it is not just
about how ‘I’ learn
but how ‘we’ learn
Self-directed learning can let
students improve a lot according to
their needs. But they also need the
instructors to guide them when they
are confusing and frustrated during
the learning process
25
Learning analytics for European educational policy
LAEP
• What is the current state of the art?
• What are the prospects for the
implementation of learning analytics?
• What is the potential for European
policy to be used to guide and
support the take-up and adaptation
of learning analytics to enhance
education in Europe?
Action for analytics
Strategy
Research and development
Infrastructure
Context
Standards
Skills
Outreach
27
Strategy
Example of a framework for learning analytics: Siemens, G., Gašević, D., Haythornthwaite, C., Dawson, S., Buckingham
Shum, S., Ferguson, R., Duval, E., Verbert, K. & Baker, R.S.J.d. (2011). Open Learning Analytics: An Integrated and
Modularized Platform (Concept Paper). Download from solaresearch.org
• Align work on learning analytics with strategic
objectives and priority areas for education and
training
• Develop a roadmap for learning analytics
• Assign responsibility for development of
learning analytics
• Identify and build on work in
related areas and other countries
• Build on learning analytics
work to develop new priorities
28
http://evidence.laceproject.eu
Infrastructure
●Increase data-handling
capability
●Create organisational
structures to support use
of learning analytics
●Use Evidence Hub
to identify areas for
development
●Develop methods of
sharing experience
and good practice
29
Context
●Align learning analytics
work with different
sectors of education
●Develop practices that
are appropriate to
different contexts
●Identify successful
financial models
30
Standards
●Adapt and employ
interoperability standards
●Develop and employ ethical
standards, including data
protection
●Align analytics with
assessment practices
●Develop a robust quality
assurance process
●Develop evaluation
frameworks
http://www.laceproject.eu/deliverables/d7-1-interoperability-studies/
http://www.open.ac.uk/students/charter/essential-documents/ethical-use-student-data-learning-analytics-policy
31
Skills
●Identify the skills required in different
areas
●Train and support educators to use
analytics to support achievement
●Train and support researchers and
developers to work in this field
●Develop and support educational
leaders to implement these changes
●Educate learners to use analytics to
support their own achievement
32
Outreach
●Engage stakeholders
throughout the learning
analytics process
●Support collaboration
between academics and
commercial organisations
●Promote awareness of
learning analytics
33
Research and development
Develop the evidence base
• Develop pedagogy that makes
good use of analytics
• Develop analytics that
address strategic objectives
and priorities
• Develop technology that
enables deployment of
analytics
• Develop frameworks that
enable development of
analytics
34https://xkcd.com/1739/
35
A problem that we can address
Very little hard evidence is currently
available that is based on anything other
than short-term studies.
Some positive work is cited in the LACE
Evidence Hub but, at this stage, there is
no overwhelming evidence that
learning analytics have fostered more
effective and efficient learning
processes and organisations.
However, there is convincing evidence in
the Inventory and Case Studies that
companies and organisations believe
they can do this in the future, and are
prepared to invest time and resources in
order to achieve this.
36
Clow, LAK12, http://oro.open.ac.uk/34330/
http://evidence.laceproject.eu/
http://evidence.laceproject.eu/evidence/evidence-flow-map/
45
Slides online at www.slideshare.net/R3beccaF
Rebecca Ferguson @R3beccaF
http://r3beccaf.wordpress.com/

Learning analytics: the way forward

  • 1.
    1 Learning analytics: the wayahead Rebecca Ferguson The Open University Nordic LASI, 2017
  • 2.
    2 Learning analytics The measurement,collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.
  • 3.
    Priority areas foreducation and training 3 Open and innovative education and training, fully embracing the digital era. Strong support for teachers, trainers, school leaders and other educational staff Relevant and high-quality knowledge, skills and competences developed throughout lifelong learning Focus on learning outcomes for employability, innovation, active citizenship and well-being and inclusive education, equality, equity, non-discrimination and the promotion of civic competences.
  • 4.
    • Our researchersand students shall contribute insight and communication of knowledge in public discussions. • We will challenge the knowledge front and conventional notions through critical analysis and knowledge made available to everyone. • Through research and education, we shall contribute towards challenging power structures and promote a diversified and sustainable society. • We shall have the reputation of being a national institution of culture and a crucible for new ideas, innovation and new ways of learning. • We shall be a meeting place for staff, students and society at large in an attractive arena for lifelong learning. • We shall have a strong and vibrant university democracy characterized by generosity, openness, diversity and dialogue.
  • 5.
    Priority areas foreducation and training 5 • Bringing together different sectors: higher education, schools & workplace learning • Building networks that have outlived the project’s funding period • Helping to develop learning analytics capability • Creating and sharing resources • Developing visions of the future and agreeing how to work towards them http://www.laceproject.eu/
  • 6.
  • 7.
    7 (Just over) adecade of change 2012: ‘Year of the MOOC’ 2007: Launch of the iphone 2006: First tweets
  • 8.
  • 9.
    Provocation 1: Learners are monitoredby their learning environments
  • 10.
    Learners are monitoredby their learning environments Just as in bioethics, there are fundamental human factors at play here Too much Big Brother vision to be appealing I think it is a promising line of work for learning analytics. I think there will be many advances in the use of sensors to identify aspects that can be applied to this vision
  • 11.
  • 12.
    Learners’ personal data aretracked Wearable sensors are already present, but in the next future they must be improved, especially for health purposes, such as diabetes monitoring or cardiovascular diseases prevention Quantified self strongly builds on reflection and self-organisation. This is good and feasible A reliable evidence base for the effectiveness of these measures and some sort of safety control to prevent irresponsible recommendations
  • 13.
  • 14.
    Analytics are rarely used Thereshould be strong discussion about the ethical concerns that apply to each approach to learning analytics Focus less on analytics to automate, but rather to be student-centred, to inform the learner and improve their personal practice. Once analytics provides real, tangible value to the learner, you begin to both alleviate concerns about privacy AND develop faith that if you can give control of data back to the learners, they will opt-in to learning analytics. The research community must concretely show the benefits of the use of data to improve educational outcomes
  • 15.
  • 16.
    Learners control their own data Absolutely essential. Organisations must notrely on the data, but try to build trust with their learners so that learners see the benefit of sharing Can we assume that ‘data owners’ know what to do with their data? Some people can’t even manage the money in their pockets Users should be entitled to know how their data are interrogated and used and for what reasons. This should be made explicit and easy to understand Greatly limits our ability to effectively use learning analytics to improve learning for ALL students
  • 17.
    Provocation 5: Open systemsare widely adopted
  • 18.
    Open systems are widelyadopted • Define standards for the information exchange • Define standards to exploit the information stored • Define guides about what information is useful • Define and publish different models to represent the information Align with corporate and stop thinking that academic is leading! A multi-pronged approach involving IT services, national organisations like JISC and ministry of education and tertiary institutions would all need to be involved
  • 19.
  • 20.
    Learning analytics areessential tools Very little credible research has demonstrated any real large-scale benefits to learners or institutions Learning is not only about success is about learning from failure. So, yes, I think that it would be desirable that the prediction rates are very high, however, I don't think it is the final goal What is needed is support for reflection, discussion and debate on the purpose of it all, especially to curb the excesses of those that see learning as something teachers do to students. We need to nurture rich, reflective communities
  • 21.
    Provocation 7: Analytics helplearners make the right choices
  • 22.
    Analytics help learners makethe right choices From bitter experience, I'm aware of the very slow pace of institutional change Companies will sell politicians on the budget savings and lead us here With learning there is an element that learning is a process – reducing it to simply outcomes that can be measured is dangerous Limited as ignores learning through interactions with both peers and the wider social environment
  • 23.
    Provocation 8: Analytics havelargely replaced teachers
  • 24.
    Analytics have largely replacedteachers Autonomy begets engagement, motivation, persistence, relevance The collective is as important as the individual: it is not just about how ‘I’ learn but how ‘we’ learn Self-directed learning can let students improve a lot according to their needs. But they also need the instructors to guide them when they are confusing and frustrated during the learning process
  • 25.
    25 Learning analytics forEuropean educational policy LAEP • What is the current state of the art? • What are the prospects for the implementation of learning analytics? • What is the potential for European policy to be used to guide and support the take-up and adaptation of learning analytics to enhance education in Europe?
  • 26.
    Action for analytics Strategy Researchand development Infrastructure Context Standards Skills Outreach
  • 27.
    27 Strategy Example of aframework for learning analytics: Siemens, G., Gašević, D., Haythornthwaite, C., Dawson, S., Buckingham Shum, S., Ferguson, R., Duval, E., Verbert, K. & Baker, R.S.J.d. (2011). Open Learning Analytics: An Integrated and Modularized Platform (Concept Paper). Download from solaresearch.org • Align work on learning analytics with strategic objectives and priority areas for education and training • Develop a roadmap for learning analytics • Assign responsibility for development of learning analytics • Identify and build on work in related areas and other countries • Build on learning analytics work to develop new priorities
  • 28.
    28 http://evidence.laceproject.eu Infrastructure ●Increase data-handling capability ●Create organisational structuresto support use of learning analytics ●Use Evidence Hub to identify areas for development ●Develop methods of sharing experience and good practice
  • 29.
    29 Context ●Align learning analytics workwith different sectors of education ●Develop practices that are appropriate to different contexts ●Identify successful financial models
  • 30.
    30 Standards ●Adapt and employ interoperabilitystandards ●Develop and employ ethical standards, including data protection ●Align analytics with assessment practices ●Develop a robust quality assurance process ●Develop evaluation frameworks http://www.laceproject.eu/deliverables/d7-1-interoperability-studies/ http://www.open.ac.uk/students/charter/essential-documents/ethical-use-student-data-learning-analytics-policy
  • 31.
    31 Skills ●Identify the skillsrequired in different areas ●Train and support educators to use analytics to support achievement ●Train and support researchers and developers to work in this field ●Develop and support educational leaders to implement these changes ●Educate learners to use analytics to support their own achievement
  • 32.
    32 Outreach ●Engage stakeholders throughout thelearning analytics process ●Support collaboration between academics and commercial organisations ●Promote awareness of learning analytics
  • 33.
    33 Research and development Developthe evidence base • Develop pedagogy that makes good use of analytics • Develop analytics that address strategic objectives and priorities • Develop technology that enables deployment of analytics • Develop frameworks that enable development of analytics
  • 34.
  • 35.
    35 A problem thatwe can address Very little hard evidence is currently available that is based on anything other than short-term studies. Some positive work is cited in the LACE Evidence Hub but, at this stage, there is no overwhelming evidence that learning analytics have fostered more effective and efficient learning processes and organisations. However, there is convincing evidence in the Inventory and Case Studies that companies and organisations believe they can do this in the future, and are prepared to invest time and resources in order to achieve this.
  • 36.
  • 37.
  • 38.
  • 45.
    45 Slides online atwww.slideshare.net/R3beccaF Rebecca Ferguson @R3beccaF http://r3beccaf.wordpress.com/

Editor's Notes

  • #10 Provocation 1: Learners are monitored by their learning environments Provocation 1 relates to a world in which almost anything a learner uses can be used to collect data about their activities People saw how this vision could be connected with sensor technology and the Internet of Things. They also raised the issue of Big Brother watching over learners and controlling what they do
  • #12 Provocation 2: Learners’ personal data are tracked Provocation 2 deals not with external data but with internal data. Information about where students are looking, how they are reacting to stimuli, what their heart rate is. The picture shows the Mindlfex game in which the blue ball is controlled by the user’s brainwaves – which suggests we are moving towards being able to detect thought patterns Respondents linked this to the notion of the ‘quantified self’, and to activity in the field of medicine. They called for a reliable evidence base, which is an idea found in many of the responses to diffferent visions.
  • #16 Provocation 4: Learners control their own data Provocation 4 is concerned with who owns the data. Should it be owned and controlled by individual learners? Divergent opinions here. Some people think learners should control their data and that organisations should make this possible and desirable. Others think that this will make the data unusable and that it just adds needless extra responsibilities to the work of students
  • #18 Provocation 5: Open systems are widely adopted Provocation 5 is to do with getting learning analytics, and their related systems, to talk to each other and to understand each other. It also ties in with building a developer community. In order for this open approach to be possible, a lot of work needs to be done at local and national levels.
  • #20 Provocation 6: Learning analytics are essential tools Provocation 6 sees the role for learning analytics increasing, so that all learners are supported by a mound of data However, this requires thought about what it means to learn, how learning takes place, and how learning analytics support that process
  • #22 Provocation 7: Analytics help learners make the right choices Provocation 7 sees a positive role for analytics, with control remaining in the hands of the learners. Although this sounds a positive future, respondents could see potential problems.
  • #24 Provocation 8: Analytics have largely replaced teachers The final provocation sees analytics replacing teachers Any teacher that can be replaced by a computer deserves to be. This is a rewording by David Thornburg of the original Arthur C Clarke quote (“Teachers that can be replaced by a machine should be.”) Again, this prompts consideration of how learning takes place and how it can best be supported
  • #38 Onthe LACE project, we set out to find the evidence that does exist about learning analytics We set up an evidence hub – grouping published work in terms of these four statements We asked partners from across Europe to contribute We looked at LAK conferences and the Journal of Learning Analytics We put a call out to the community We prompted people at last year’s LAK to add their papers. So it’s not all the evidence, but it is a lot of it (an dyou can add more, if you see a gap)
  • #39 We found three main things: There was no point classifying papers in terms of a hieracrchy of evidence, because most of the work was exploratory or think pieces or small scale There was relatively little evidence. Lots of papers have nothing to say in relation to our four propositions What evidence there was turned out to be overwhelmingly positive. Which seemed unlikely and prompted our Failathons