AFEL: Towards Measuring Online Activities
Contributions to Self-Directed Learning
Alessandro Adamou, Stefan Dietze, Besnik Fetahu, Ujwal Gadiraju, Ilire
Hasani-Mavriqi, Peter Holtz, Joachim Kimmerle, Dominik Kowald, Elisabeth
Lex, Susana López Sola, Ricardo A. Maturana, Vedran Sabol, Pinelopi
Troullinou, Eduardo Veas
Mathieu d’Aquin | @mdaquin | @afelproject
Insight Centre for Data Analytics
National University of Ireland, Galway
Learnign Analytics
According to Wikipedia (and past LAK CFPs, and some papers from relevant people)
Learning analytics is the measurement, collection, analysis and
reporting of data about learners and their contexts, for purposes of
understanding and optimizing learning and the environments in which it
occurs.
Learnign Analytics
According to Wikipedia (and past LAK CFPs, and some papers from relevant people)
Learning analytics is the measurement, collection, analysis and
reporting of data about learners and their contexts, for purposes of
understanding and optimizing learning and the environments in which
it occurs.
Learnign Analytics
According to Wikipedia (and past LAK CFPs, and some papers from relevant people)
Learning analytics is the measurement, collection, analysis and
reporting of data about learners and their contexts, for purposes of
understanding and optimizing learning and the environments in which
it occurs.
not only students
not only the classroom,
university, library or VLE
However, typically, Learning Analytics is...
A university uses data on the
students and their activities
collected through the
institution's information
systems with the goal to
predict their success so they
can improve help them
improve….
… and improve their teaching,
offering, environments as well
But: Everyday Learning
But: Everyday Learning
Mais : Everyday Learning
Challenge #1: Collecting data
eLearning
platform (e.g.
moddle)
learner
analyst teacher
activities
generating traces
traces and
metadata
resources and
metadata
analyse
learner
identifier
activities generating traces, with
(sometimes) different identifiers
?
analyse
learner
browser
Challenge #1: Collecting data
Difficulté #1: D'où viennent les données
learner
activités générant des traces et
(des fois, différents) identifiants
browser
AFEL Data Platform
Extension
app
Tracker
Crawler
Crawler
Traces and
metadata
AFEL identifier
and local
identifier
AFEL identifier
and local
identifier
learner
activités générant des traces et
(des fois, différents) identifiants
browser
AFEL Data Platform
Analytics platform
VisualisationAFEL identifier
Analysis
Integrated
personal data
Extension
app
Tracker
Crawler
Crawler
AFEL Core Data
Model (based on
schema.org)
Learning
indicators
Traces and
metadata
AFEL identifier
and local
identifier
AFEL identifier
and local
identifier
Challenge #1: Collecting data
Example : Browser Extension
Same model used
for Facebook
application, Twitter,
Didactalia analytics..
Challenge #2: Indicators of learning?
Maximising what? Minimising what?
teacher analyst
Ratio :
students’ success
cost in effort/resources
(?)
learner
In the context of informal,
self-directed learning, what
is success?
What are the relevant
notions of effort and cost?
The dynamic processes of learning and knowledge construction from
Kimmerle, Moskaliuk, Oeberst, and Cress, 2015.
Understanding learning where it can’t be measured
Understanding learning where it can’t be measured
The dynamic processes of learning and knowledge construction from
Kimmerle, Moskaliuk, Oeberst, and Cress, 2015.
The dynamic processes of learning and knowledge construction from
Kimmerle, Moskaliuk, Oeberst, and Cress, 2015.
“constructive friction is the driving force behind
learning” -- AFEL Deliverable 4.1, [CK08]
Understanding learning where it can’t be measured
Indicators of learning, based on measuring friction!
So to favour activities that generate a constructing fiction..
What kinds of frictions?
- In topic: In what way the activity introduces topics/themes/concepts
that have not been seen before?
- In complexity: In what way the activity introduces a further level of
complexity which was not accessible before and requires further
efforts.
- In view: In what way the activity introduces new points of view on an
already encountered topic, covering a different aspect.
Example - Analysing topic coverage from traces in web
browsing
Text analysis Clustering
Progress
analysis
Browser history
Learning scopes
(topics)
Example - Analysing topic coverage from traces in web
browsing
Example - Analysing topic coverage from traces in web
browsing
Example - Analysing topic coverage from traces in web
browsing
web programming
british isles
Example - Analysing topic coverage from traces in web
browsing
Integration
Next steps
The AFEL platform is in place and working,
collecting data from the Didactalia platform
and from early (alpha) users of the extractor
tools (the browser extensions, facebook app,
etc), but the technical challenges are still
there.
Indicators need to be better understood:
Which ones are useful? How to calculate
them? How to present them?
Needs feedback from end-users.
AFEL Resources
Public data release Learning Analytics
Glossary
http://mdaquin.net - @mdaquin
http://afel-project.eu - @afelproject
http://afel-project.eu/questionnaire
Jane is 37 and works as an administrative assistant in a local medium-sized company. As a hobbies, she
enjoyed sewing and cycling in the local forests. She is also interested in business management, and is
considering either developing in her current job to a more senior level or making a career change.
Jane spends a lot of time online at home and at her job. She has friends on facebook with whom she shares
and discusses local places to go biking, and others with whom she discusses sewing techniques and
possible projects, often through sharing youtube videos.
Jane also follows MOOCs and forums related to business management, on different topics. She often uses
online resources such as Wikipedia and online magazine on the topics. At school, she was not very
interested in maths, which is needed if she want to progress in her job. She is therefore registered on
Didactalia, connecting to resources and communities on maths, especially statistics.
Jane has also decided to take her learning seriously: She has registered to use the AFEL dashboard through the
Didactalia interface. She has also installed the browser extension to include her browsing history, as well as
the facebook app. She has not included in her dashboard her emails, as they are mostly related to her
current job, or twitter, since she rarely uses it.
Jane looks at the dashboard more or less once a day, as she is prompted by a notification from the AFEL
smartphone application or from the facebook app, to see how she has been doing the previous day in her
online social learning. It might for example say “It looks like you progressed well with sewing yesterday! See
how you are doing on other topics…”
Jane, as she looks at the dashboard, realises that she has been focusing a lot on her hobbies and
procrastinated on the topics she enjoys less, especially statistics. Looking specifically at statistics, she
realises that she almost only works on it in Friday evenings, because she feels guilty of not having done
much during the week. She also sees that she is not putting as much effort into her learning of statistics as
other learners, and not making as much progress. She therefore makes a conscious decision to put more
focus on it. She adds the dashboard goals of the form “to work on statistics during my lunch break every
week day” or “to have achieved a 10% progress compared to now by the same time next week”. The
dashboard will remind her how she is doing against those goals as she go about her usual online social
learning activities. She also gets recommendation of things to do on Didactalia and Facebook based on the
indicators shown on the dashboard and her stated goals.

AFEL: Towards Measuring Online Activities Contributions to Self-Directed Learning

  • 1.
    AFEL: Towards MeasuringOnline Activities Contributions to Self-Directed Learning Alessandro Adamou, Stefan Dietze, Besnik Fetahu, Ujwal Gadiraju, Ilire Hasani-Mavriqi, Peter Holtz, Joachim Kimmerle, Dominik Kowald, Elisabeth Lex, Susana López Sola, Ricardo A. Maturana, Vedran Sabol, Pinelopi Troullinou, Eduardo Veas Mathieu d’Aquin | @mdaquin | @afelproject Insight Centre for Data Analytics National University of Ireland, Galway
  • 2.
    Learnign Analytics According toWikipedia (and past LAK CFPs, and some papers from relevant people) Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.
  • 3.
    Learnign Analytics According toWikipedia (and past LAK CFPs, and some papers from relevant people) Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.
  • 4.
    Learnign Analytics According toWikipedia (and past LAK CFPs, and some papers from relevant people) Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs. not only students not only the classroom, university, library or VLE
  • 5.
    However, typically, LearningAnalytics is... A university uses data on the students and their activities collected through the institution's information systems with the goal to predict their success so they can improve help them improve…. … and improve their teaching, offering, environments as well
  • 6.
  • 7.
  • 8.
  • 9.
    Challenge #1: Collectingdata eLearning platform (e.g. moddle) learner analyst teacher activities generating traces traces and metadata resources and metadata analyse learner identifier activities generating traces, with (sometimes) different identifiers ? analyse
  • 10.
  • 11.
    Difficulté #1: D'oùviennent les données learner activités générant des traces et (des fois, différents) identifiants browser AFEL Data Platform Extension app Tracker Crawler Crawler Traces and metadata AFEL identifier and local identifier AFEL identifier and local identifier
  • 12.
    learner activités générant destraces et (des fois, différents) identifiants browser AFEL Data Platform Analytics platform VisualisationAFEL identifier Analysis Integrated personal data Extension app Tracker Crawler Crawler AFEL Core Data Model (based on schema.org) Learning indicators Traces and metadata AFEL identifier and local identifier AFEL identifier and local identifier Challenge #1: Collecting data
  • 13.
    Example : BrowserExtension Same model used for Facebook application, Twitter, Didactalia analytics..
  • 14.
    Challenge #2: Indicatorsof learning? Maximising what? Minimising what? teacher analyst Ratio : students’ success cost in effort/resources (?) learner In the context of informal, self-directed learning, what is success? What are the relevant notions of effort and cost?
  • 15.
    The dynamic processesof learning and knowledge construction from Kimmerle, Moskaliuk, Oeberst, and Cress, 2015. Understanding learning where it can’t be measured
  • 16.
    Understanding learning whereit can’t be measured The dynamic processes of learning and knowledge construction from Kimmerle, Moskaliuk, Oeberst, and Cress, 2015.
  • 17.
    The dynamic processesof learning and knowledge construction from Kimmerle, Moskaliuk, Oeberst, and Cress, 2015. “constructive friction is the driving force behind learning” -- AFEL Deliverable 4.1, [CK08] Understanding learning where it can’t be measured
  • 18.
    Indicators of learning,based on measuring friction! So to favour activities that generate a constructing fiction.. What kinds of frictions? - In topic: In what way the activity introduces topics/themes/concepts that have not been seen before? - In complexity: In what way the activity introduces a further level of complexity which was not accessible before and requires further efforts. - In view: In what way the activity introduces new points of view on an already encountered topic, covering a different aspect.
  • 19.
    Example - Analysingtopic coverage from traces in web browsing Text analysis Clustering Progress analysis Browser history Learning scopes (topics)
  • 20.
    Example - Analysingtopic coverage from traces in web browsing
  • 21.
    Example - Analysingtopic coverage from traces in web browsing
  • 22.
    Example - Analysingtopic coverage from traces in web browsing
  • 23.
    web programming british isles Example- Analysing topic coverage from traces in web browsing
  • 24.
  • 27.
    Next steps The AFELplatform is in place and working, collecting data from the Didactalia platform and from early (alpha) users of the extractor tools (the browser extensions, facebook app, etc), but the technical challenges are still there. Indicators need to be better understood: Which ones are useful? How to calculate them? How to present them? Needs feedback from end-users.
  • 28.
    AFEL Resources Public datarelease Learning Analytics Glossary
  • 29.
    http://mdaquin.net - @mdaquin http://afel-project.eu- @afelproject http://afel-project.eu/questionnaire
  • 30.
    Jane is 37and works as an administrative assistant in a local medium-sized company. As a hobbies, she enjoyed sewing and cycling in the local forests. She is also interested in business management, and is considering either developing in her current job to a more senior level or making a career change. Jane spends a lot of time online at home and at her job. She has friends on facebook with whom she shares and discusses local places to go biking, and others with whom she discusses sewing techniques and possible projects, often through sharing youtube videos. Jane also follows MOOCs and forums related to business management, on different topics. She often uses online resources such as Wikipedia and online magazine on the topics. At school, she was not very interested in maths, which is needed if she want to progress in her job. She is therefore registered on Didactalia, connecting to resources and communities on maths, especially statistics. Jane has also decided to take her learning seriously: She has registered to use the AFEL dashboard through the Didactalia interface. She has also installed the browser extension to include her browsing history, as well as the facebook app. She has not included in her dashboard her emails, as they are mostly related to her current job, or twitter, since she rarely uses it. Jane looks at the dashboard more or less once a day, as she is prompted by a notification from the AFEL smartphone application or from the facebook app, to see how she has been doing the previous day in her online social learning. It might for example say “It looks like you progressed well with sewing yesterday! See how you are doing on other topics…” Jane, as she looks at the dashboard, realises that she has been focusing a lot on her hobbies and procrastinated on the topics she enjoys less, especially statistics. Looking specifically at statistics, she realises that she almost only works on it in Friday evenings, because she feels guilty of not having done much during the week. She also sees that she is not putting as much effort into her learning of statistics as other learners, and not making as much progress. She therefore makes a conscious decision to put more focus on it. She adds the dashboard goals of the form “to work on statistics during my lunch break every week day” or “to have achieved a 10% progress compared to now by the same time next week”. The dashboard will remind her how she is doing against those goals as she go about her usual online social learning activities. She also gets recommendation of things to do on Didactalia and Facebook based on the indicators shown on the dashboard and her stated goals.