Presentation for the Danish Accreditation institute images on slide 33: CC: Luis Torro Some rights reserved and CC: Ryan Hyde Some rights reserved

1. The European Commission’s
science and knowledge service
Joint Research Centre
Learning Analytics –
Challenges arising from a current
review of LA use
Seville May 23 5 2017
For the Danish Accreditation Institution (by Skype)
Dr. Riina Vuorikari
DG JRC – Directorate Innovation and Growth
Unit B4 Human Capital and Employment

2. The European Commission’s
science and knowledge service
Joint Research Centre
. Part 1: Introduction to the
Report: aims, inventory of LA
(5 min)
. Part 2: Opportunities
(10 min)
. Part 3: Challenges arising from
the study
(10 min)
Outline:

3. 3

4. 4
Focus on the priorities of the
European Commission:
working for more than
20 policy DGs
Policy neutral:
has no policy agenda
of its own
Independent:
no private, commercial or
national interests
The Joint Research Centre (JRC)
Directorate
Growth &
Innovation
Seville

5. DigComp
(DG EMPL)
EntreComp
(DG EMPL)
DigCompConsumers
(DG JUST)
Openedu Policies(HE)
(DG EAC)
MOOCKnowledge
(DG EAC)
Blockchain
(DG JRC)
OPTEV
(DG JRC)
MOOCs4 inclusion
(DG EAC)
Learning Analytics
(DG JRC)
Anticipatory studies Policy & society
OrganisationsIndividuals
DigCompEdu
(DG EAC)
DigPolEdu
(DG EAC)
CPDmodels
(DG EAC)
ICTinPISA
(DG EAC)
CompuThink
(DG JRC)
DigCompOrg4Schools
(DG EAC)
OpenEdu (HE)
(DG EAC)
DigCompOrg
(DG EAC)
Current JRC research on Digital Age Learning
and 21st Century Skills

6. 6
The Study behind the JRC Report
• Goal: Provide research evidence on the use of
learning analytics and discuss their implications
for education policy
• Study conducted between September 2015-June
2016
• Design of the study: the JRC in Seville,
Unit of "Human Capital and Employment”
• Research: The Open University, UK under the
contract and supervision of the JRC

7. 7
Learning analytics involve
the measurement, collection, analysis and reporting
of data about learners and their contexts, for
purposes of understanding and optimizing
learning and the environments in which it occurs.
(source)
Learning analytics have their roots in many fields of
educational and technical research, including assessment,
personal learning and social learning, but also in business
intelligence and data mining.
The field draws on theory and methodologies from
disciplines as statistics, artificial intelligence and
computer science (Dawson et al., 2014).

8. 8
To access the Inventory
• Google “leap inventory learning analytics”
• http://cloudworks.ac.uk/cloudscape/view/2959
• Google “learning analytics JRC science hub”
To access the Report

9. 9
What does the Study contain?
• An inventory of recent implementations of learning
analytics:
• Tools, practices and policies (60 examples)
• 5 case studies
• Review of research literature on implementation
• To critically reflect on the impact, potential and limits
of using learning analytics in education
• To consider the implication for education policy:
“The Action List for Learning Analytics”

10. The European Commission’s
science and knowledge service
Joint Research Centre
. Part 1: Introduction to the
Report: aims, inventory of LA
(5 min)
. Part 2: Opportunities of tools
(10 min)
. Part 3: Challenges arising from
the study
(10 min)
Outline:

11. 12
Example 1: Inventory of Tools
1.
2.
3.
4.
5.
Specific models
of domain knowledge
(in math) and
on the learner
responses (cognitive
models )
Stand alone
application that
generates its own
data.
What does this
data tell us about
learning?
Is it actionable?

12. 13
http://cloudworks.ac.uk/cloud/view/9633
1.
2.
3.
4.
5.
6.
Example 2: Inventory of Tools
Data sources include
VLE; social media, “card
swipes” (e.g. using
student card to go to
library), libraries,
housing

13. 14
1.
2.
3.
4.
Example 3: Inventory of Tools

14. 15
• Tools in Inventory target
• compulsory education (13); HE (8);
• workplace (2); any (6)
• “stand-alone” tools; add on to an existing VLE;
custom-made solutions
• Different target beneficiaries of analytics:
• Learners, teachers, tutors, advisors, counsellors, school
heads/managers, decision-makers, policy-makers,..
The Inventory: Tools (1)

15. 16
The Inventory: Tools (2)
• Different data sources:
• Student digital traces from the platform or
outside of it, e.g. interaction data, social media, libraries
• Data from offline sources, e.g. evaluations by the learner,
demographic data, nation-wide test data/evaluations
• Methods to use data: Descriptive data and statistics, visualisation,
statistical interferences, dataminig, machine learning,..
• Actions on data: modelling of data based on which actions to, e.g.
adaptive systems to scaffold and support; to recommend, predict,
alert..
• Action based on based: past behaviour, similarity in grades,
domain knowledge, right answers, statistics, …

16. 17
Narrowing the attainment gap: Georgia State
University At the university, predictive analytics
have been used to tackle the achievement gap for
low income and first-generation students. GSU
graduation rate rose from 32% in 2003 to 54% in
2014.
In the process, the university claims to have
removed the achievement gap between
students from minority backgrounds or lower
socioeconomic status and their peers.
Inclusive
education,
equality!
Example 4: Inventory of Practices

17. 18
Do we see real improvements in
learning outcomes for learners? 37 examples

18. The European Commission’s
science and knowledge service
Joint Research Centre
. Part 1: Introduction to the
Report: aims, inventory of LA
(5 min)
. Part 2: Opportunities
(10 min)
. Part 3: Challenges arising from
the study
(10 min)
Outline:

19. 20
2.
1..
Example 5: Inventory of Tools

20. 21
Challenge (1)
• Evidence of Impact: The research evidence
documented in this study shows that there is little
formal validation of tools
• e.g. whether the tools fulfil their intended purpose
such as having a positive impact on learning;
encouraging more efficient learning; or more
effective learning,..

21. 22
Challenge (2)
• Impact: The research evidence documented in
this study shows that currently, most impact of
learning analytics in education and training seems
takes place around issues, little impact on
changing practices yet:
• E.g. Sensitive issues of personal data and privacy are
at the centre of discussion

22. 23
Examples of Open University, UK
• Tools: Open Essayist and OU Analyse
• Practices: Ethical use of student data policy
• Case study on “The process of developing an institutional ethics
policy ” (part of the Report)
Example 6: Inventory

23. 24
Europe’s General Data Protection Regulation
(GDPR)
• Europe has taken the position that individual privacy is important and
that changes to current practices in general analytics are needed
• Institutions will need to understand their responsibilities and
obligations with regard to data privacy and data protection and will
have to put procedures in place to ensure that they are compliant with
the legislation.
http://ec.europa.eu/justice/data-protection/reform/

24. 25

25. 26
Challenge (3)
• Impact: the implementation of learning analytics
seems to be a long-term process requiring a vision
and a strategy, policy and structure, but also
knowledge and skills in technology and pedagogy
• E.g.
• Case study in UTS (Au): vision of becoming a data-
intensive university in 2011 – strategy and a new centre
in 2014, tools are being developed and piloted now
• Kennisnet working w/schools and vendors since 2014
(products that have useful features!), now focus also on
standardisation of student data, etc.

26. 31
Challenge (4)
• The majority of current learning analytics
work is not strongly aligned with the
European Union’s priority areas for
education and training
• E.g. Strategic objectives for European
cooperation in education and training
(ET2020)

27. 32
Challenge (4): Create an European
vision for Learning Analytics
Strategic objectives for European cooperation in education and
training (ET2020)
1. Relevant and high-quality skills and competences for employability,
innovation, active citizenship
2. Inclusive education, equality, non-discrimination, civic competences
Indicator: reducing school drop-out rates to less than 10%
3. Open and innovative education and training, including by fully embracing
the digital era
4. Strong support for educators
5. Transparency and recognition of skills and qualifications
6. Sustainable investment, performance and efficiency of education and
training systems

28. 33
Do Learning Analytics
with dashboards and
half-hearted visions of
learning remain the
lower hanging fruit of
digital technology ? What about the visions
for empowering learning?

29. 34
Action List for Learning Analyitcs
• Action 1: Create common vision for LA in
European education and training
• Use the ET2020 Priority areas to make policy hooks
• Action 2: Build LA tools that help teachers and
learners
• Now too much focus on the supply side!
• Help generate demand: Talk to teachers and
learners to understand what they want
• Action 3: Conduct research that ends up in the
LACE Evidence Hub
• Validation of tools and their promises should lead
the research

30. The European Commission’s
science and knowledge service
Joint Research Centre
Check the research of our team
at the JRC Science Hub:
https://ec.europa.eu/jrc/
New skills agenda:
https://ec.europa.eu/education/
news/20160610-education-skills-
factsheet_en
Thank you!