The document discusses requirements for learning analytics based on a lecture and workshop at East China Normal University. It begins with introductions and then outlines the day's plan to discuss definitions of analytics, actors in learning analytics, framework models, and requirements. It emphasizes starting with pedagogy and poses questions about what data is available and how to build trust. Ethical challenges are noted around data protection, privacy, transparency, and purpose. The goal is to use analytics to facilitate learning while avoiding instructivist approaches and stress for learners.

1. Requirements for Learning Analytics
Tore Hoel
Oslo and Akershus University College of Applied Sciences,
Oslo, Norway
Lecture & Workshop for PhD students @ ECNU, Shanghai 2014-12-22
Course on Smart Education

2. 2
Largest state university college in Norway.
I work mainly with European projects
on Learning Analytics and Open Education
About
Tore

3. 3
This is more an interactive workshop
than a lecture
➔
You have to contribute!

4. Today’s plan
1. Your own projects on LA and Big Data (paper assignment)
2. Definitions of analytics, academic analytics, learning analytics, etc.
3. Actors in LA
4. Framework models
5. Requirements - the big picture
6. Data and Privacy
4

5. Learning Analytics and Big Data
– Mapping your interest
Related to your selected themes and research goals
for your papers on Smart Education
5

6. What is your concepts of
Learning Analytics?
Write down 3 concepts that would be on the top
of your list when you will explain what LA is 6
x
y
z
Write like a
Mind Map – in your
own language if
you want!

7. Huaihao Zhang
• Learning analytics: The influence of demographic of K6-9 SL
teacher on their engagement in an online teacher training
initiative
7
Demographics
Teacher training

8. Zhenyue Ding
• Subject knowledge bank construction based on Big Data:
Framework for describing; Subject Bank; Visualization
• Cloud service platform for K12
• Smart assessment – adaptive assessment for K12
8
Ontology
Visualization
Assessment bank Adaptive assessment

9. Peter Riezebos
• Understanding LA as educational instrument: methods, ethical
issues, optimize learning paths
• Smart assessment: Identify learning outcome, cognitive learning
preferences
9
Definition of LA
Ethical guidelines
Learning paths
Learning outcome
Learning
preferences

10. Huan Liu
• Understanding LA and EDM
• Gathering and coding data
• LA impact on adaptive learning
10
Definition of LA
Definition of EDM
Data
Data metrics
Adaptive

11. Liang Luo
• Smart pedagogy Instructional Design: Classification of learning
activities; learning activity design model
11
Learning Activity
Description
Learning Design

12. Concept map example
12Drawn with the Open Source Cmap tool cmap.ihmc.us
Student’s summary of
course in LA - work in progress

13. What is Learning Analytics?
See the LACE FAQ
13
Uploaded to
Sakai platform

14. What are analytics?
• High-level figures
• Brief overview for internal and external reports
• Academic Analytics
• Figures on retention and success, for the institution
to assess performance
• Educational Data Mining
• Searching for patterns in the data
• Learning Analytics
• Use of [big] data to provide actionable intelligence
for learners and teachers
14

15. Levels of Learning Analytics
(UNESCO Policy Brief, November 2012)
15

16. Learning Analytics defined
«The measurement, collection, analysis and
reporting of data about learners and their
contexts, for purposes of understanding and
optimizing learning and the environments in
which it occurs.»
Society for Learning
Analytics Research (SoLAR)
16
Actionable intelligence!
Not
Theoretical
Insights!
Not Reporting!

17. Clow, LAK12, 2012

18. When designing a LA system –
where should you start?
18

19. Actors in learning analytics
19

20. Why do learners use analytics?
• Monitor their own activities and interactions
• Monitor the learning process
• Compare their activity with that of others
• Increase awareness, reflect and self reflect
• Improve discussion participation
• Improve learning behaviour
• Improve performance
• Become better learners
• Learn!
20

21. Why do teachers use analytics?
• Monitor the learning process
• Explore student data
• Identify problems
• Discover patterns
• Find early indicators for success
• Find early indicators for poor marks or drop-out
• Assess usefulness of learning materials
• Increase awareness, reflect and self reflect
• Increase understanding of learning environments
• Intervene, advise and assist
• Improve teaching, resources and the environment
21

22. Why do learning designers use analytics?
• Helping to identify useful analytics
• What do learners need to know in order to
network, collaborate, browse or reflect?
• What do educators need to know to support
them?
• Helping to identify gaps in the data
• Which data do we need to collect?
• Helping to identify gaps in our toolkit
• Which design elements can we look at easily?
• Which ones still pose problems?
22

23. More learning design
• Helping to frame and focus analytics questions
• What did they learn?… in relation to learning
outcomes
• Were they social?... when they were collaborating
• Did they share links?... when encouraged to browse
• Did they return to steps?... when encouraged to
reflect
• Helping to identify appropriate forms of analysis
• The same content, but with a focus on
• Number of visits if content
• Length, quality, number of comments if conversational23

24. Framework Models of LA
24

25. More models to be found at
http://insulardrafts.tumblr.com
Pick your model and explain to the group what it is all about!
25

26. 26
Source: Siemens et al. (2011) Open Learning Anlytics Integrated Platform

27. 27Source: www.apereo.org

28. 28
Source: www.laceproject.eu/blog/learning-analytics-research-schools-owd/

29. 29Source: Slides from professor Wu Yonghe, ECNU / ESERC

30. Requirements for LA
31

31. The LA feedback loop
32
(Greller & Drachsler, 2012)

32. Critical dimensions of learning analytics (Greller & Drachsler, 2012) 33

33. 34

34. Data
35

35. What data are available for LA?
• Data sharing and Privacy Survey
36
The draft questionnaire
is uploaded to Sakai. I
would like your
comments to the
questions and ideas how
to proceeed!

36. What data could be used? – some ideas…
• Demographic data
• Calendar information about assignments
• VLE activity data (including forums).
• Lists of required reading
• Library resources usage data
• Library helpdesk enquiries
• Library website usage and analytics data
• Assessment results
• User survey results
• Student retention and attainment data
37Source: Rebecca Ferguson

37. The concept of Data Commons
38Source: KERIS, Korea

38. Ideals and Threats
39

39. What is the ideal use of LA?
• Can we achieve this?
• Aligned with clear aims
• Huge and sustained effort
• Agreed proxies for learning
• Clear and standardised visualisation
• Driving behaviour at every level
• Can we avoid this?
• Instructivist approach
• Stressed, unhappy learners
• Analytics with little value for learners or teachers
• Omission of key areas, such as collaboration
40Source: Rebecca Ferguson, OUUK

40. Don’t start with the data – start
with the pedagogy
How do people learn?
How can I use data to facilitate that process?
Social learning analytics:
How do people learn socially & in social settings?
How can I use data to facilitate that process?
How could we achieve ideal LA?
41
Source: Rebecca Ferguson, OUUK

41. What questions should I ask?
42
• Which elements are learners struggling with?
• Which sections engage them the most?
• What prompts them to ask questions?
• How are they navigating resources?
• What misconceptions have they shown?
• Are there any accessibility issues?
How can analytics be used to
achieve desired learning outcomes?
Source: Rebecca Ferguson, OUUK

42. How to build
trust?
43
http://shanghaidaily.com/metro/society/Mini-spies-in-the-classroom-strain-relations/shdaily.shtml
2014-12-22

43. • Data Protection
• Privacy
• Transparency (related to Subject
Access requests)
• Whether students should be able to
opt in/out
• De-identification of data
• Timeliness and Duty of Care
(keeping data up to date)
• Access to data (who should have
access to the data, etc.)
• Students abusing the system by
misinformation
Some ethical challenges
• The use of student data outside
university systems (Social Media)
• Analysis of the data and the methods
used (what assumptions are used to
create the algorithm for the predictive
model, should there be an independent
audit?)
• Purpose of applying a learning analytics
approach
• Profiling of students
• How will it be done?
• What do we tell students?
• Should we tell students? – Students may
feel ‘at-risk’/labelled
Glasswinged butterfly, ? Greta oro
cc licensed ( BY NC ND ) flickr photo by Greg Foster: http://www.flickr.com/photos/gregfoster/3365801458/

44. Thanks to:
• Rebecca Ferguson, OUUK @R3beccaF (for letting me use her slides)
• LACE project colleagues
Funders:
• LACE: European Commission 619424-FP7-ICT-2013-11

45. Hoel, T. (2014). «Requirements for Learning Analytics»
– lecture and workshop at East China Normal University, Shanghai,
China, December 2014
Twitter: @tore - WeChat: Tore_no
about.me/torehoel
tore.hoel@hioa.no
This work was undertaken as part of the LACE Project, supported by the European Commission Seventh
Framework Programme, grant 619424.
These slides are provided under the Creative Commons Attribution Licence:
http://creativecommons.org/licenses/by/4.0/. Some images used may have different licence terms.
www.laceproject.eu
@laceproject
46