SlideShare a Scribd company logo
Multimodal Learning
Analytics
Xavier Ochoa
Escuela Superior Politécnica del Litoral
#edumetricas, Montevideo, UY
15 April 2016
http://www.slideshare.net/xaoch
Download this presentation
(Multimodal)
Learning Analytics
Learning analytics is the measurement,
collection, analysis and reporting of data
about learners and their contexts, for
purposes of understanding and
optimising learning and the
environments in which it occurs.
Examining engagement: analysing
learner subpopulations in massive
open online courses (MOOCs)
Using transaction-level data to
diagnose knowledge gaps and
misconceptions
Likelihood analysis of student
enrollment outcomes using learning
environment variables: a case study
approach
Tracking student progress in a game-
like learning environment with a
Monte Carlo Bayesian knowledge
tracing model
Strong focus on online data
Based on the papers it should be called
Online-Learning Analytics
Streetlight effect
Where learning is
happening?
Why
Multimodal Learning Analytics?
We should be looking where it is useful to look,
not where it is easy
There is learning outside the
LMS
But it is very messy!
Who is learning?
Who is learning?
Who is learning?
Who is learning? – Traditional way
But there are better ways to
assess learning
At least theoretically
Who is learning? – Educational Research
How can we approach the
problem from a
Learning Analytics perspective
Measure, collect, analyze and report
to understand and optimize
We need to capture learning
traces from the real world
Look ma, no log files!
In the real world, humans
communicate (and leave traces)
in several modalities
What you say is as important as
how you say it 😀
We need to analyze the traces
with variable degrees of
sophistication
And we have to do it automatically as
humans are not scalable
We need to provide
feedback in the real world
Often in a multimodal way too
But…
Which modes are important to
understand the learning
process?
We do not know yet…
Possibilities
• What we see
• What we hear
• How we move
• How we write
• How we blink
• Our pulse
• Brain activity?
• Our hormones?
What are the relevant
features of those signals
We do not know yet…
Our current analysis tools are
good enough?
We do not know yet…
How to present the information
(and uncertainty)
in a way that is actually useful?
We do not know yet…
It is an open
(but very dark) field
One feels like an explorer
This particular flavor of Learning
Analytics is what we called
Multimodal Learning Analytics
Multimodal Learning Analytics is related to:
• Behaviorism
• Cognitive Science
• Multimodal Interaction (HCI)
• Educational Research (old school one)
• Computer Vision
• Natural Language Processing
• Biosignals Processing
• And as many fields as modes you can think of...
Examples
Math Data Corpus
How to (easily) obtain
multimodal features?
What is already there?
Three Approaches
• Literature-based features
• Common-sense-based features
• “Why not?”-based features
All approaches proved useful
Proof that we are in an early stage
Video: Calculator Use (NTCU)
et s of
s was
t , on
t hat
eived
so in-
n and
lving
ed by
udent
e was
score
given
diffi-
ssion,
n ex-
Mat h
at t et
direct ion in which it was point ing at using t he rigid t rans-
format ions capabilit ies provided by OpenCV . W hile t here
were some frames in which t his mat ching was not possible
due t o object occlusions or changes in t he illuminat ion of
t he calculat or, in general t he described det ect ion t echnique
was robust and provided useful posit ion and direct ion dat a.
F igur e 1: D et er m in at ion of w hich st udent is usin g
Video: Total Movement (TM)
fined as t he number of whit e pixels cont ained in t he binary
image out put by t he Codebook algorit hm. T his magnit ude,
when comput ed for t he ent ire problem solving session, re-
sult s from summing up it s individual values obt ained from
each frame t hat compose a problem recording.
(a) Original frame (b) Difference frame
F igur e 2: R esult s of t he C odeb ook algor it hm .
Video: Distance from center table
(DHT)
he de-
oblem
ed t o
calcu-
par-
a spe-
ed as
e pre-
Code-
ficant
small
ondi-
where
ained
s de-
inary
t ude,
n, re-
from
each head cent roid t o t he cent er of t he t able is calculat ed
and t hen, t he average of t hese dist ances is obt ained by prob-
lem (see Figure 3). Addit ionally, t he variance of t he average
dist ance head t o t able (SD-DHT ), was calculat ed t o det er-
mine if a part icipant remains most ly st at ic or varies his or
her dist ance t o t he t able.
F igur e 3: C alculat ion of t he dist ance of t he st udent ’s
Audio: Processing
Audio: Features
• Number of Interventions (NOI)
• Total Speech Duration (TSD)
• Times Numbers were Mentioned (TNM)
• Times Math Terms were Mentioned (TMTM)
• Times Commands were Pronounced (TCP)
Digital Pen: Basic Features
• Total Number of Strokes (TNS)
• Average Number of Points (ANP)
• Average Stroke Path Length (ASPL)
• Average Stroke Displacement (ASD)
• Average Stroke Pressure (ASP)
Digital Pen: Shape Recognition
Stronium – Sketch Recognition Libraries
Analysis at Problem level
Solving Problem Correctly
• Logistic Regression to model Student Solving Problem
Correctly
• Resulting model was significantly reliable
• 60,9% - the problem solving student was identified
• 71,8% - incorrectly solved problems were identified
Analysis at Group Level
Expertise Estimation
• Features were feed to a Classification Tree algorithm
• Several variables had a high discrimination power between
expert and non-experts
• Best discrimination result in 80% expert prediction and 90%
non-expert prediction
Expert Estimation over Problems
Plateau reached after
just 4 problems
Main conclusion: Simple
features could identify expertise
Faster Writer (Digital Pen)
Percentage of Calculator Use (Video)
Times Numbers were Mentioned (Audio)
Oral Presentation Quality Corpus
Video Features
• 66 facial features were
extracted using Luxand
software including both eyes
and nose tip to estimate the
presenter’s gaze.
Kinect features
• Identify Common postures
Kinect features
• Identify Common postures
Kinect Features
Laban’s theory helps to describe human movement using non-verbal
characteristics:
Spatial aspects of movement
Temporal aspects of movement
Fluency, smoothness, impulsivity
Energy and power
Overall activity
EVALUATION METHODOLOGY
Extracted
Features
Human coded
Criterion
Video
Kinect
Body and Posture
Language
Eye Contact
Results: less than 50%
accuracy
What we were measuring was not was
humans were measuring
What is next in MLA?
Mode integration framework
for MLA
Currently pioneered by Marcelo Worsley
Developing
Multimodal Measuring Devices
Our Fitbits
Record different learning
settings
And share them with the community
Conclusions
Multimodal Learning Analytics is
not a subset of Learning
Analytics
Current Learning Analytics is a subset of MLA

Some problems are easy,
some hard
But we do not know until we try to solve them
There is a lot of
exploring to do
And we need explorers
Gracias / Thank you
Questions?
Xavier Ochoa
xavier@cti.espol.edu.ec
http://ariadne.cti.espol.edu.ec/xavier
Twitter: @xaoch
http://www.sigmla.org

More Related Content

What's hot

Deep Generative Models
Deep Generative ModelsDeep Generative Models
Deep Generative Models
Mijung Kim
 
Diffusion models beat gans on image synthesis
Diffusion models beat gans on image synthesisDiffusion models beat gans on image synthesis
Diffusion models beat gans on image synthesis
BeerenSahu
 
Basic Generative Adversarial Networks
Basic Generative Adversarial NetworksBasic Generative Adversarial Networks
Basic Generative Adversarial Networks
Dong Heon Cho
 
A Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial NetworksA Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial Networks
Jong Wook Kim
 
Machine Learning lecture6(regularization)
Machine Learning lecture6(regularization)Machine Learning lecture6(regularization)
Machine Learning lecture6(regularization)
cairo university
 
Genetic algorithm raktim
Genetic algorithm raktimGenetic algorithm raktim
Genetic algorithm raktim
Raktim Halder
 
Deep learning lecture - part 1 (basics, CNN)
Deep learning lecture - part 1 (basics, CNN)Deep learning lecture - part 1 (basics, CNN)
Deep learning lecture - part 1 (basics, CNN)
SungminYou
 
Genetic algorithms
Genetic algorithmsGenetic algorithms
Genetic algorithms
zamakhan
 
Machine learning
Machine learningMachine learning
Machine learning
Dr Geetha Mohan
 
deep learning applications in medical image analysis brain tumor
deep learning applications in medical image analysis brain tumordeep learning applications in medical image analysis brain tumor
deep learning applications in medical image analysis brain tumor
Venkat Projects
 
Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks
Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks
Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks
Christopher Morris
 
PR-231: A Simple Framework for Contrastive Learning of Visual Representations
PR-231: A Simple Framework for Contrastive Learning of Visual RepresentationsPR-231: A Simple Framework for Contrastive Learning of Visual Representations
PR-231: A Simple Framework for Contrastive Learning of Visual Representations
Jinwon Lee
 
Speaker Recognition using Gaussian Mixture Model
Speaker Recognition using Gaussian Mixture Model Speaker Recognition using Gaussian Mixture Model
Speaker Recognition using Gaussian Mixture Model
Saurab Dulal
 
A Simple Framework for Contrastive Learning of Visual Representations
A Simple Framework for Contrastive Learning of Visual RepresentationsA Simple Framework for Contrastive Learning of Visual Representations
A Simple Framework for Contrastive Learning of Visual Representations
Seunghyun Hwang
 
Winning data science competitions, presented by Owen Zhang
Winning data science competitions, presented by Owen ZhangWinning data science competitions, presented by Owen Zhang
Winning data science competitions, presented by Owen Zhang
Vivian S. Zhang
 
AutoML lectures (ACDL 2019)
AutoML lectures (ACDL 2019)AutoML lectures (ACDL 2019)
AutoML lectures (ACDL 2019)
Joaquin Vanschoren
 
Deep learning - A Visual Introduction
Deep learning - A Visual IntroductionDeep learning - A Visual Introduction
Deep learning - A Visual Introduction
Lukas Masuch
 
Human Action Recognition
Human Action RecognitionHuman Action Recognition
Human Action Recognition
NAVER Engineering
 
Introduction to MAML (Model Agnostic Meta Learning) with Discussions
Introduction to MAML (Model Agnostic Meta Learning) with DiscussionsIntroduction to MAML (Model Agnostic Meta Learning) with Discussions
Introduction to MAML (Model Agnostic Meta Learning) with Discussions
Joonyoung Yi
 
[Paper] Multiscale Vision Transformers(MVit)
[Paper] Multiscale Vision Transformers(MVit)[Paper] Multiscale Vision Transformers(MVit)
[Paper] Multiscale Vision Transformers(MVit)
Susang Kim
 

What's hot (20)

Deep Generative Models
Deep Generative ModelsDeep Generative Models
Deep Generative Models
 
Diffusion models beat gans on image synthesis
Diffusion models beat gans on image synthesisDiffusion models beat gans on image synthesis
Diffusion models beat gans on image synthesis
 
Basic Generative Adversarial Networks
Basic Generative Adversarial NetworksBasic Generative Adversarial Networks
Basic Generative Adversarial Networks
 
A Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial NetworksA Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial Networks
 
Machine Learning lecture6(regularization)
Machine Learning lecture6(regularization)Machine Learning lecture6(regularization)
Machine Learning lecture6(regularization)
 
Genetic algorithm raktim
Genetic algorithm raktimGenetic algorithm raktim
Genetic algorithm raktim
 
Deep learning lecture - part 1 (basics, CNN)
Deep learning lecture - part 1 (basics, CNN)Deep learning lecture - part 1 (basics, CNN)
Deep learning lecture - part 1 (basics, CNN)
 
Genetic algorithms
Genetic algorithmsGenetic algorithms
Genetic algorithms
 
Machine learning
Machine learningMachine learning
Machine learning
 
deep learning applications in medical image analysis brain tumor
deep learning applications in medical image analysis brain tumordeep learning applications in medical image analysis brain tumor
deep learning applications in medical image analysis brain tumor
 
Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks
Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks
Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks
 
PR-231: A Simple Framework for Contrastive Learning of Visual Representations
PR-231: A Simple Framework for Contrastive Learning of Visual RepresentationsPR-231: A Simple Framework for Contrastive Learning of Visual Representations
PR-231: A Simple Framework for Contrastive Learning of Visual Representations
 
Speaker Recognition using Gaussian Mixture Model
Speaker Recognition using Gaussian Mixture Model Speaker Recognition using Gaussian Mixture Model
Speaker Recognition using Gaussian Mixture Model
 
A Simple Framework for Contrastive Learning of Visual Representations
A Simple Framework for Contrastive Learning of Visual RepresentationsA Simple Framework for Contrastive Learning of Visual Representations
A Simple Framework for Contrastive Learning of Visual Representations
 
Winning data science competitions, presented by Owen Zhang
Winning data science competitions, presented by Owen ZhangWinning data science competitions, presented by Owen Zhang
Winning data science competitions, presented by Owen Zhang
 
AutoML lectures (ACDL 2019)
AutoML lectures (ACDL 2019)AutoML lectures (ACDL 2019)
AutoML lectures (ACDL 2019)
 
Deep learning - A Visual Introduction
Deep learning - A Visual IntroductionDeep learning - A Visual Introduction
Deep learning - A Visual Introduction
 
Human Action Recognition
Human Action RecognitionHuman Action Recognition
Human Action Recognition
 
Introduction to MAML (Model Agnostic Meta Learning) with Discussions
Introduction to MAML (Model Agnostic Meta Learning) with DiscussionsIntroduction to MAML (Model Agnostic Meta Learning) with Discussions
Introduction to MAML (Model Agnostic Meta Learning) with Discussions
 
[Paper] Multiscale Vision Transformers(MVit)
[Paper] Multiscale Vision Transformers(MVit)[Paper] Multiscale Vision Transformers(MVit)
[Paper] Multiscale Vision Transformers(MVit)
 

Viewers also liked

Education as the meta-problem: Opportunities for Technology R&D
Education as the meta-problem: Opportunities for Technology R&DEducation as the meta-problem: Opportunities for Technology R&D
Education as the meta-problem: Opportunities for Technology R&D
Xavier Ochoa
 
Game theory for neural networks
Game theory for neural networksGame theory for neural networks
Game theory for neural networks
David Balduzzi
 
Analitica del Aprendizaje
Analitica del AprendizajeAnalitica del Aprendizaje
Analitica del Aprendizaje
Xavier Ochoa
 
Latin American Open Textbook Initiative (LATIn Project)
Latin American Open Textbook Initiative (LATIn Project)Latin American Open Textbook Initiative (LATIn Project)
Latin American Open Textbook Initiative (LATIn Project)
Xavier Ochoa
 
Globe seminar
Globe seminarGlobe seminar
Globe seminar
Xavier Ochoa
 
Expert estimation from Multimodal Features
Expert estimation from Multimodal FeaturesExpert estimation from Multimodal Features
Expert estimation from Multimodal Features
Xavier Ochoa
 
Openness as a Tool
Openness as a ToolOpenness as a Tool
Openness as a Tool
Xavier Ochoa
 
Multimodal Learning Analytics
Multimodal Learning AnalyticsMultimodal Learning Analytics
Multimodal Learning Analytics
Xavier Ochoa
 
Inclusive & open learning analytics
Inclusive & open learning analyticsInclusive & open learning analytics
Inclusive & open learning analytics
Daniele Di Mitri
 
Open Data Repositories
Open Data RepositoriesOpen Data Repositories
Open Data Repositories
Xavier Ochoa
 
Towards Mobile Multimodal Learning Analytics
Towards Mobile Multimodal Learning AnalyticsTowards Mobile Multimodal Learning Analytics
Towards Mobile Multimodal Learning Analytics
Laila Shoukry
 
Multimodal Learning Analytics
Multimodal Learning AnalyticsMultimodal Learning Analytics
Multimodal Learning Analytics
Xavier Ochoa
 
Competências para aprender e competências para ensinar COM TIC
Competências para aprender e competências para ensinar COM TICCompetências para aprender e competências para ensinar COM TIC
Competências para aprender e competências para ensinar COM TIC
Fernando Albuquerque Costa
 
Medir para Entender y Mejorar: la Analítica del Aprendizaje como nuevo paradi...
Medir para Entender y Mejorar: la Analítica del Aprendizaje como nuevo paradi...Medir para Entender y Mejorar: la Analítica del Aprendizaje como nuevo paradi...
Medir para Entender y Mejorar: la Analítica del Aprendizaje como nuevo paradi...
Xavier Ochoa
 
SEO for Ecommerce: A Comprehensive Guide
SEO for Ecommerce: A Comprehensive GuideSEO for Ecommerce: A Comprehensive Guide
SEO for Ecommerce: A Comprehensive Guide
Adam Audette
 
Analisis de Algoritmos Multihilo
Analisis de Algoritmos MultihiloAnalisis de Algoritmos Multihilo
Analisis de Algoritmos Multihilo
Xavier Ochoa
 

Viewers also liked (16)

Education as the meta-problem: Opportunities for Technology R&D
Education as the meta-problem: Opportunities for Technology R&DEducation as the meta-problem: Opportunities for Technology R&D
Education as the meta-problem: Opportunities for Technology R&D
 
Game theory for neural networks
Game theory for neural networksGame theory for neural networks
Game theory for neural networks
 
Analitica del Aprendizaje
Analitica del AprendizajeAnalitica del Aprendizaje
Analitica del Aprendizaje
 
Latin American Open Textbook Initiative (LATIn Project)
Latin American Open Textbook Initiative (LATIn Project)Latin American Open Textbook Initiative (LATIn Project)
Latin American Open Textbook Initiative (LATIn Project)
 
Globe seminar
Globe seminarGlobe seminar
Globe seminar
 
Expert estimation from Multimodal Features
Expert estimation from Multimodal FeaturesExpert estimation from Multimodal Features
Expert estimation from Multimodal Features
 
Openness as a Tool
Openness as a ToolOpenness as a Tool
Openness as a Tool
 
Multimodal Learning Analytics
Multimodal Learning AnalyticsMultimodal Learning Analytics
Multimodal Learning Analytics
 
Inclusive & open learning analytics
Inclusive & open learning analyticsInclusive & open learning analytics
Inclusive & open learning analytics
 
Open Data Repositories
Open Data RepositoriesOpen Data Repositories
Open Data Repositories
 
Towards Mobile Multimodal Learning Analytics
Towards Mobile Multimodal Learning AnalyticsTowards Mobile Multimodal Learning Analytics
Towards Mobile Multimodal Learning Analytics
 
Multimodal Learning Analytics
Multimodal Learning AnalyticsMultimodal Learning Analytics
Multimodal Learning Analytics
 
Competências para aprender e competências para ensinar COM TIC
Competências para aprender e competências para ensinar COM TICCompetências para aprender e competências para ensinar COM TIC
Competências para aprender e competências para ensinar COM TIC
 
Medir para Entender y Mejorar: la Analítica del Aprendizaje como nuevo paradi...
Medir para Entender y Mejorar: la Analítica del Aprendizaje como nuevo paradi...Medir para Entender y Mejorar: la Analítica del Aprendizaje como nuevo paradi...
Medir para Entender y Mejorar: la Analítica del Aprendizaje como nuevo paradi...
 
SEO for Ecommerce: A Comprehensive Guide
SEO for Ecommerce: A Comprehensive GuideSEO for Ecommerce: A Comprehensive Guide
SEO for Ecommerce: A Comprehensive Guide
 
Analisis de Algoritmos Multihilo
Analisis de Algoritmos MultihiloAnalisis de Algoritmos Multihilo
Analisis de Algoritmos Multihilo
 

Similar to Multimodal Learning Analytics

Bring your own idea - Visual learning analytics
Bring your own idea - Visual learning analyticsBring your own idea - Visual learning analytics
Bring your own idea - Visual learning analytics
Joris Klerkx
 
Machinr Learning and artificial_Lect1.pdf
Machinr Learning and artificial_Lect1.pdfMachinr Learning and artificial_Lect1.pdf
Machinr Learning and artificial_Lect1.pdf
SaketBansal9
 
Knowledge base system appl. p 1,2-ver1
Knowledge base system appl.  p 1,2-ver1Knowledge base system appl.  p 1,2-ver1
Knowledge base system appl. p 1,2-ver1
Taymoor Nazmy
 
An analysis of_machine_and_human_analytics_in_classification
An analysis of_machine_and_human_analytics_in_classificationAn analysis of_machine_and_human_analytics_in_classification
An analysis of_machine_and_human_analytics_in_classification
Subhashis Hazarika
 
ML crash course
ML crash courseML crash course
ML crash course
mikaelhuss
 
Data Science and Analysis.pptx
Data Science and Analysis.pptxData Science and Analysis.pptx
Data Science and Analysis.pptx
PrashantYadav931011
 
1. Intro DS.pptx
1. Intro DS.pptx1. Intro DS.pptx
1. Intro DS.pptx
Anusuya123
 
LearningAG.ppt
LearningAG.pptLearningAG.ppt
LearningAG.ppt
butest
 
Topic Pages. From articles to answers.
Topic Pages. From articles to answers.Topic Pages. From articles to answers.
Topic Pages. From articles to answers.
Deep Kayal
 
Deep learning introduction
Deep learning introductionDeep learning introduction
Deep learning introduction
Adwait Bhave
 
IRJET - Twitter Sentiment Analysis using Machine Learning
IRJET -  	  Twitter Sentiment Analysis using Machine LearningIRJET -  	  Twitter Sentiment Analysis using Machine Learning
IRJET - Twitter Sentiment Analysis using Machine Learning
IRJET Journal
 
EssentialsOfMachineLearning.pdf
EssentialsOfMachineLearning.pdfEssentialsOfMachineLearning.pdf
EssentialsOfMachineLearning.pdf
Ankita Tiwari
 
Empirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an OverviewEmpirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an Overview
alessio_ferrari
 
Deep Learning, an interactive introduction for NLP-ers
Deep Learning, an interactive introduction for NLP-ersDeep Learning, an interactive introduction for NLP-ers
Deep Learning, an interactive introduction for NLP-ers
Roelof Pieters
 
Sangeetha seminar (1)
Sangeetha  seminar (1)Sangeetha  seminar (1)
Sangeetha seminar (1)
Sangeetha Mathew
 
Experimenting with eXtreme Design (EKAW2010)
Experimenting with eXtreme Design (EKAW2010)Experimenting with eXtreme Design (EKAW2010)
Experimenting with eXtreme Design (EKAW2010)
evabl444
 
Artificial Intelligence by B. Ravikumar
Artificial Intelligence by B. RavikumarArtificial Intelligence by B. Ravikumar
Artificial Intelligence by B. Ravikumar
Garry D. Lasaga
 
RING panel discussion, Coling 2010 ( E. Hovy + M. Zock)
RING panel discussion, Coling 2010 ( E. Hovy + M. Zock)RING panel discussion, Coling 2010 ( E. Hovy + M. Zock)
RING panel discussion, Coling 2010 ( E. Hovy + M. Zock)
Michael Zock
 
REVIEW PPT.pptx
REVIEW PPT.pptxREVIEW PPT.pptx
REVIEW PPT.pptx
SaravanaD2
 
Artificial Intelligence for Automated Software Testing
Artificial Intelligence for Automated Software TestingArtificial Intelligence for Automated Software Testing
Artificial Intelligence for Automated Software Testing
Lionel Briand
 

Similar to Multimodal Learning Analytics (20)

Bring your own idea - Visual learning analytics
Bring your own idea - Visual learning analyticsBring your own idea - Visual learning analytics
Bring your own idea - Visual learning analytics
 
Machinr Learning and artificial_Lect1.pdf
Machinr Learning and artificial_Lect1.pdfMachinr Learning and artificial_Lect1.pdf
Machinr Learning and artificial_Lect1.pdf
 
Knowledge base system appl. p 1,2-ver1
Knowledge base system appl.  p 1,2-ver1Knowledge base system appl.  p 1,2-ver1
Knowledge base system appl. p 1,2-ver1
 
An analysis of_machine_and_human_analytics_in_classification
An analysis of_machine_and_human_analytics_in_classificationAn analysis of_machine_and_human_analytics_in_classification
An analysis of_machine_and_human_analytics_in_classification
 
ML crash course
ML crash courseML crash course
ML crash course
 
Data Science and Analysis.pptx
Data Science and Analysis.pptxData Science and Analysis.pptx
Data Science and Analysis.pptx
 
1. Intro DS.pptx
1. Intro DS.pptx1. Intro DS.pptx
1. Intro DS.pptx
 
LearningAG.ppt
LearningAG.pptLearningAG.ppt
LearningAG.ppt
 
Topic Pages. From articles to answers.
Topic Pages. From articles to answers.Topic Pages. From articles to answers.
Topic Pages. From articles to answers.
 
Deep learning introduction
Deep learning introductionDeep learning introduction
Deep learning introduction
 
IRJET - Twitter Sentiment Analysis using Machine Learning
IRJET -  	  Twitter Sentiment Analysis using Machine LearningIRJET -  	  Twitter Sentiment Analysis using Machine Learning
IRJET - Twitter Sentiment Analysis using Machine Learning
 
EssentialsOfMachineLearning.pdf
EssentialsOfMachineLearning.pdfEssentialsOfMachineLearning.pdf
EssentialsOfMachineLearning.pdf
 
Empirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an OverviewEmpirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an Overview
 
Deep Learning, an interactive introduction for NLP-ers
Deep Learning, an interactive introduction for NLP-ersDeep Learning, an interactive introduction for NLP-ers
Deep Learning, an interactive introduction for NLP-ers
 
Sangeetha seminar (1)
Sangeetha  seminar (1)Sangeetha  seminar (1)
Sangeetha seminar (1)
 
Experimenting with eXtreme Design (EKAW2010)
Experimenting with eXtreme Design (EKAW2010)Experimenting with eXtreme Design (EKAW2010)
Experimenting with eXtreme Design (EKAW2010)
 
Artificial Intelligence by B. Ravikumar
Artificial Intelligence by B. RavikumarArtificial Intelligence by B. Ravikumar
Artificial Intelligence by B. Ravikumar
 
RING panel discussion, Coling 2010 ( E. Hovy + M. Zock)
RING panel discussion, Coling 2010 ( E. Hovy + M. Zock)RING panel discussion, Coling 2010 ( E. Hovy + M. Zock)
RING panel discussion, Coling 2010 ( E. Hovy + M. Zock)
 
REVIEW PPT.pptx
REVIEW PPT.pptxREVIEW PPT.pptx
REVIEW PPT.pptx
 
Artificial Intelligence for Automated Software Testing
Artificial Intelligence for Automated Software TestingArtificial Intelligence for Automated Software Testing
Artificial Intelligence for Automated Software Testing
 

More from Xavier Ochoa

Multimodal Learning Analytics
Multimodal Learning AnalyticsMultimodal Learning Analytics
Multimodal Learning Analytics
Xavier Ochoa
 
Supporting the Acquisition of 21st Century Skills through Multimodal Learning...
Supporting the Acquisition of 21st Century Skills through Multimodal Learning...Supporting the Acquisition of 21st Century Skills through Multimodal Learning...
Supporting the Acquisition of 21st Century Skills through Multimodal Learning...
Xavier Ochoa
 
Educational Technologies
Educational TechnologiesEducational Technologies
Educational Technologies
Xavier Ochoa
 
Developing 21st-Century Skills with  Multimodal Learning Analytics
Developing 21st-Century Skills with Multimodal Learning AnalyticsDeveloping 21st-Century Skills with Multimodal Learning Analytics
Developing 21st-Century Skills with  Multimodal Learning Analytics
Xavier Ochoa
 
Educational Technologies: Learning Analytics and Artificial Intelligence
Educational Technologies: Learning Analytics and Artificial IntelligenceEducational Technologies: Learning Analytics and Artificial Intelligence
Educational Technologies: Learning Analytics and Artificial Intelligence
Xavier Ochoa
 
Analiticas de Aprendizaje: Nuevo paradigma en la investigación educativa
Analiticas de Aprendizaje: Nuevo paradigma en la investigación educativaAnaliticas de Aprendizaje: Nuevo paradigma en la investigación educativa
Analiticas de Aprendizaje: Nuevo paradigma en la investigación educativa
Xavier Ochoa
 
Analítica del Aprendizaje como Nuevo Paradigma de la Investigación Educativa
Analítica del Aprendizaje como Nuevo Paradigma de la Investigación EducativaAnalítica del Aprendizaje como Nuevo Paradigma de la Investigación Educativa
Analítica del Aprendizaje como Nuevo Paradigma de la Investigación Educativa
Xavier Ochoa
 
Automatic Feedback for Oral Presentations
Automatic Feedback for Oral PresentationsAutomatic Feedback for Oral Presentations
Automatic Feedback for Oral Presentations
Xavier Ochoa
 
LAK-18 Program in Numbers
LAK-18 Program in NumbersLAK-18 Program in Numbers
LAK-18 Program in Numbers
Xavier Ochoa
 
Adaptive Multilevel Clustering Model for the Prediction of Academic Risk
Adaptive Multilevel Clustering Model for the Prediction of Academic RiskAdaptive Multilevel Clustering Model for the Prediction of Academic Risk
Adaptive Multilevel Clustering Model for the Prediction of Academic Risk
Xavier Ochoa
 
Simple metrics for Curricular Analytics
Simple metrics for Curricular AnalyticsSimple metrics for Curricular Analytics
Simple metrics for Curricular Analytics
Xavier Ochoa
 
Introduccion Algoritmos Multihilo
Introduccion Algoritmos MultihiloIntroduccion Algoritmos Multihilo
Introduccion Algoritmos Multihilo
Xavier Ochoa
 
Analitica aprendizaje
Analitica aprendizajeAnalitica aprendizaje
Analitica aprendizaje
Xavier Ochoa
 
Visualizing Uncertainty in the Prediction of Academic Risk
Visualizing Uncertainty in the Prediction of Academic RiskVisualizing Uncertainty in the Prediction of Academic Risk
Visualizing Uncertainty in the Prediction of Academic Risk
Xavier Ochoa
 
Openness as a Tool
Openness as a ToolOpenness as a Tool
Openness as a Tool
Xavier Ochoa
 
Techniques for Data-Driven Curriculum Analysis
Techniques for Data-Driven Curriculum AnalysisTechniques for Data-Driven Curriculum Analysis
Techniques for Data-Driven Curriculum Analysis
Xavier Ochoa
 
Pilot analysis
Pilot analysisPilot analysis
Pilot analysis
Xavier Ochoa
 
Medidas de Repositorios y Tendencias Actuales
Medidas de Repositorios y Tendencias ActualesMedidas de Repositorios y Tendencias Actuales
Medidas de Repositorios y Tendencias Actuales
Xavier Ochoa
 
Normas para la Creación de Recursos Educativos Abiertos
Normas para la Creación de Recursos Educativos AbiertosNormas para la Creación de Recursos Educativos Abiertos
Normas para la Creación de Recursos Educativos Abiertos
Xavier Ochoa
 

More from Xavier Ochoa (19)

Multimodal Learning Analytics
Multimodal Learning AnalyticsMultimodal Learning Analytics
Multimodal Learning Analytics
 
Supporting the Acquisition of 21st Century Skills through Multimodal Learning...
Supporting the Acquisition of 21st Century Skills through Multimodal Learning...Supporting the Acquisition of 21st Century Skills through Multimodal Learning...
Supporting the Acquisition of 21st Century Skills through Multimodal Learning...
 
Educational Technologies
Educational TechnologiesEducational Technologies
Educational Technologies
 
Developing 21st-Century Skills with  Multimodal Learning Analytics
Developing 21st-Century Skills with Multimodal Learning AnalyticsDeveloping 21st-Century Skills with Multimodal Learning Analytics
Developing 21st-Century Skills with  Multimodal Learning Analytics
 
Educational Technologies: Learning Analytics and Artificial Intelligence
Educational Technologies: Learning Analytics and Artificial IntelligenceEducational Technologies: Learning Analytics and Artificial Intelligence
Educational Technologies: Learning Analytics and Artificial Intelligence
 
Analiticas de Aprendizaje: Nuevo paradigma en la investigación educativa
Analiticas de Aprendizaje: Nuevo paradigma en la investigación educativaAnaliticas de Aprendizaje: Nuevo paradigma en la investigación educativa
Analiticas de Aprendizaje: Nuevo paradigma en la investigación educativa
 
Analítica del Aprendizaje como Nuevo Paradigma de la Investigación Educativa
Analítica del Aprendizaje como Nuevo Paradigma de la Investigación EducativaAnalítica del Aprendizaje como Nuevo Paradigma de la Investigación Educativa
Analítica del Aprendizaje como Nuevo Paradigma de la Investigación Educativa
 
Automatic Feedback for Oral Presentations
Automatic Feedback for Oral PresentationsAutomatic Feedback for Oral Presentations
Automatic Feedback for Oral Presentations
 
LAK-18 Program in Numbers
LAK-18 Program in NumbersLAK-18 Program in Numbers
LAK-18 Program in Numbers
 
Adaptive Multilevel Clustering Model for the Prediction of Academic Risk
Adaptive Multilevel Clustering Model for the Prediction of Academic RiskAdaptive Multilevel Clustering Model for the Prediction of Academic Risk
Adaptive Multilevel Clustering Model for the Prediction of Academic Risk
 
Simple metrics for Curricular Analytics
Simple metrics for Curricular AnalyticsSimple metrics for Curricular Analytics
Simple metrics for Curricular Analytics
 
Introduccion Algoritmos Multihilo
Introduccion Algoritmos MultihiloIntroduccion Algoritmos Multihilo
Introduccion Algoritmos Multihilo
 
Analitica aprendizaje
Analitica aprendizajeAnalitica aprendizaje
Analitica aprendizaje
 
Visualizing Uncertainty in the Prediction of Academic Risk
Visualizing Uncertainty in the Prediction of Academic RiskVisualizing Uncertainty in the Prediction of Academic Risk
Visualizing Uncertainty in the Prediction of Academic Risk
 
Openness as a Tool
Openness as a ToolOpenness as a Tool
Openness as a Tool
 
Techniques for Data-Driven Curriculum Analysis
Techniques for Data-Driven Curriculum AnalysisTechniques for Data-Driven Curriculum Analysis
Techniques for Data-Driven Curriculum Analysis
 
Pilot analysis
Pilot analysisPilot analysis
Pilot analysis
 
Medidas de Repositorios y Tendencias Actuales
Medidas de Repositorios y Tendencias ActualesMedidas de Repositorios y Tendencias Actuales
Medidas de Repositorios y Tendencias Actuales
 
Normas para la Creación de Recursos Educativos Abiertos
Normas para la Creación de Recursos Educativos AbiertosNormas para la Creación de Recursos Educativos Abiertos
Normas para la Creación de Recursos Educativos Abiertos
 

Recently uploaded

C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptxC1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
mulvey2
 
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
National Information Standards Organization (NISO)
 
Digital Artefact 1 - Tiny Home Environmental Design
Digital Artefact 1 - Tiny Home Environmental DesignDigital Artefact 1 - Tiny Home Environmental Design
Digital Artefact 1 - Tiny Home Environmental Design
amberjdewit93
 
MARY JANE WILSON, A “BOA MÃE” .
MARY JANE WILSON, A “BOA MÃE”           .MARY JANE WILSON, A “BOA MÃE”           .
MARY JANE WILSON, A “BOA MÃE” .
Colégio Santa Teresinha
 
Hindi varnamala | hindi alphabet PPT.pdf
Hindi varnamala | hindi alphabet PPT.pdfHindi varnamala | hindi alphabet PPT.pdf
Hindi varnamala | hindi alphabet PPT.pdf
Dr. Mulla Adam Ali
 
Your Skill Boost Masterclass: Strategies for Effective Upskilling
Your Skill Boost Masterclass: Strategies for Effective UpskillingYour Skill Boost Masterclass: Strategies for Effective Upskilling
Your Skill Boost Masterclass: Strategies for Effective Upskilling
Excellence Foundation for South Sudan
 
Liberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdfLiberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdf
WaniBasim
 
PCOS corelations and management through Ayurveda.
PCOS corelations and management through Ayurveda.PCOS corelations and management through Ayurveda.
PCOS corelations and management through Ayurveda.
Dr. Shivangi Singh Parihar
 
คำศัพท์ คำพื้นฐานการอ่าน ภาษาอังกฤษ ระดับชั้น ม.1
คำศัพท์ คำพื้นฐานการอ่าน ภาษาอังกฤษ ระดับชั้น ม.1คำศัพท์ คำพื้นฐานการอ่าน ภาษาอังกฤษ ระดับชั้น ม.1
คำศัพท์ คำพื้นฐานการอ่าน ภาษาอังกฤษ ระดับชั้น ม.1
สมใจ จันสุกสี
 
The Diamonds of 2023-2024 in the IGRA collection
The Diamonds of 2023-2024 in the IGRA collectionThe Diamonds of 2023-2024 in the IGRA collection
The Diamonds of 2023-2024 in the IGRA collection
Israel Genealogy Research Association
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
Jean Carlos Nunes Paixão
 
Life upper-Intermediate B2 Workbook for student
Life upper-Intermediate B2 Workbook for studentLife upper-Intermediate B2 Workbook for student
Life upper-Intermediate B2 Workbook for student
NgcHiNguyn25
 
Walmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdfWalmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdf
TechSoup
 
Wound healing PPT
Wound healing PPTWound healing PPT
Wound healing PPT
Jyoti Chand
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
Katrina Pritchard
 
How to deliver Powerpoint Presentations.pptx
How to deliver Powerpoint  Presentations.pptxHow to deliver Powerpoint  Presentations.pptx
How to deliver Powerpoint Presentations.pptx
HajraNaeem15
 
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Dr. Vinod Kumar Kanvaria
 
How to Make a Field Mandatory in Odoo 17
How to Make a Field Mandatory in Odoo 17How to Make a Field Mandatory in Odoo 17
How to Make a Field Mandatory in Odoo 17
Celine George
 
clinical examination of hip joint (1).pdf
clinical examination of hip joint (1).pdfclinical examination of hip joint (1).pdf
clinical examination of hip joint (1).pdf
Priyankaranawat4
 
Leveraging Generative AI to Drive Nonprofit Innovation
Leveraging Generative AI to Drive Nonprofit InnovationLeveraging Generative AI to Drive Nonprofit Innovation
Leveraging Generative AI to Drive Nonprofit Innovation
TechSoup
 

Recently uploaded (20)

C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptxC1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
 
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
 
Digital Artefact 1 - Tiny Home Environmental Design
Digital Artefact 1 - Tiny Home Environmental DesignDigital Artefact 1 - Tiny Home Environmental Design
Digital Artefact 1 - Tiny Home Environmental Design
 
MARY JANE WILSON, A “BOA MÃE” .
MARY JANE WILSON, A “BOA MÃE”           .MARY JANE WILSON, A “BOA MÃE”           .
MARY JANE WILSON, A “BOA MÃE” .
 
Hindi varnamala | hindi alphabet PPT.pdf
Hindi varnamala | hindi alphabet PPT.pdfHindi varnamala | hindi alphabet PPT.pdf
Hindi varnamala | hindi alphabet PPT.pdf
 
Your Skill Boost Masterclass: Strategies for Effective Upskilling
Your Skill Boost Masterclass: Strategies for Effective UpskillingYour Skill Boost Masterclass: Strategies for Effective Upskilling
Your Skill Boost Masterclass: Strategies for Effective Upskilling
 
Liberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdfLiberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdf
 
PCOS corelations and management through Ayurveda.
PCOS corelations and management through Ayurveda.PCOS corelations and management through Ayurveda.
PCOS corelations and management through Ayurveda.
 
คำศัพท์ คำพื้นฐานการอ่าน ภาษาอังกฤษ ระดับชั้น ม.1
คำศัพท์ คำพื้นฐานการอ่าน ภาษาอังกฤษ ระดับชั้น ม.1คำศัพท์ คำพื้นฐานการอ่าน ภาษาอังกฤษ ระดับชั้น ม.1
คำศัพท์ คำพื้นฐานการอ่าน ภาษาอังกฤษ ระดับชั้น ม.1
 
The Diamonds of 2023-2024 in the IGRA collection
The Diamonds of 2023-2024 in the IGRA collectionThe Diamonds of 2023-2024 in the IGRA collection
The Diamonds of 2023-2024 in the IGRA collection
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
 
Life upper-Intermediate B2 Workbook for student
Life upper-Intermediate B2 Workbook for studentLife upper-Intermediate B2 Workbook for student
Life upper-Intermediate B2 Workbook for student
 
Walmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdfWalmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdf
 
Wound healing PPT
Wound healing PPTWound healing PPT
Wound healing PPT
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
 
How to deliver Powerpoint Presentations.pptx
How to deliver Powerpoint  Presentations.pptxHow to deliver Powerpoint  Presentations.pptx
How to deliver Powerpoint Presentations.pptx
 
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
 
How to Make a Field Mandatory in Odoo 17
How to Make a Field Mandatory in Odoo 17How to Make a Field Mandatory in Odoo 17
How to Make a Field Mandatory in Odoo 17
 
clinical examination of hip joint (1).pdf
clinical examination of hip joint (1).pdfclinical examination of hip joint (1).pdf
clinical examination of hip joint (1).pdf
 
Leveraging Generative AI to Drive Nonprofit Innovation
Leveraging Generative AI to Drive Nonprofit InnovationLeveraging Generative AI to Drive Nonprofit Innovation
Leveraging Generative AI to Drive Nonprofit Innovation
 

Multimodal Learning Analytics

  • 1. Multimodal Learning Analytics Xavier Ochoa Escuela Superior Politécnica del Litoral #edumetricas, Montevideo, UY 15 April 2016
  • 4. Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.
  • 5. Examining engagement: analysing learner subpopulations in massive open online courses (MOOCs) Using transaction-level data to diagnose knowledge gaps and misconceptions Likelihood analysis of student enrollment outcomes using learning environment variables: a case study approach Tracking student progress in a game- like learning environment with a Monte Carlo Bayesian knowledge tracing model
  • 6. Strong focus on online data Based on the papers it should be called Online-Learning Analytics
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15. Why Multimodal Learning Analytics? We should be looking where it is useful to look, not where it is easy
  • 16. There is learning outside the LMS But it is very messy!
  • 20. Who is learning? – Traditional way
  • 21. But there are better ways to assess learning At least theoretically
  • 22. Who is learning? – Educational Research
  • 23.
  • 24. How can we approach the problem from a Learning Analytics perspective Measure, collect, analyze and report to understand and optimize
  • 25. We need to capture learning traces from the real world Look ma, no log files!
  • 26. In the real world, humans communicate (and leave traces) in several modalities What you say is as important as how you say it 😀
  • 27. We need to analyze the traces with variable degrees of sophistication And we have to do it automatically as humans are not scalable
  • 28. We need to provide feedback in the real world Often in a multimodal way too
  • 30. Which modes are important to understand the learning process? We do not know yet…
  • 31. Possibilities • What we see • What we hear • How we move • How we write • How we blink • Our pulse • Brain activity? • Our hormones?
  • 32. What are the relevant features of those signals We do not know yet…
  • 33. Our current analysis tools are good enough? We do not know yet…
  • 34. How to present the information (and uncertainty) in a way that is actually useful? We do not know yet…
  • 35. It is an open (but very dark) field One feels like an explorer
  • 36. This particular flavor of Learning Analytics is what we called Multimodal Learning Analytics
  • 37. Multimodal Learning Analytics is related to: • Behaviorism • Cognitive Science • Multimodal Interaction (HCI) • Educational Research (old school one) • Computer Vision • Natural Language Processing • Biosignals Processing • And as many fields as modes you can think of...
  • 39.
  • 41. How to (easily) obtain multimodal features? What is already there?
  • 42. Three Approaches • Literature-based features • Common-sense-based features • “Why not?”-based features
  • 43. All approaches proved useful Proof that we are in an early stage
  • 44. Video: Calculator Use (NTCU) et s of s was t , on t hat eived so in- n and lving ed by udent e was score given diffi- ssion, n ex- Mat h at t et direct ion in which it was point ing at using t he rigid t rans- format ions capabilit ies provided by OpenCV . W hile t here were some frames in which t his mat ching was not possible due t o object occlusions or changes in t he illuminat ion of t he calculat or, in general t he described det ect ion t echnique was robust and provided useful posit ion and direct ion dat a. F igur e 1: D et er m in at ion of w hich st udent is usin g
  • 45. Video: Total Movement (TM) fined as t he number of whit e pixels cont ained in t he binary image out put by t he Codebook algorit hm. T his magnit ude, when comput ed for t he ent ire problem solving session, re- sult s from summing up it s individual values obt ained from each frame t hat compose a problem recording. (a) Original frame (b) Difference frame F igur e 2: R esult s of t he C odeb ook algor it hm .
  • 46. Video: Distance from center table (DHT) he de- oblem ed t o calcu- par- a spe- ed as e pre- Code- ficant small ondi- where ained s de- inary t ude, n, re- from each head cent roid t o t he cent er of t he t able is calculat ed and t hen, t he average of t hese dist ances is obt ained by prob- lem (see Figure 3). Addit ionally, t he variance of t he average dist ance head t o t able (SD-DHT ), was calculat ed t o det er- mine if a part icipant remains most ly st at ic or varies his or her dist ance t o t he t able. F igur e 3: C alculat ion of t he dist ance of t he st udent ’s
  • 48. Audio: Features • Number of Interventions (NOI) • Total Speech Duration (TSD) • Times Numbers were Mentioned (TNM) • Times Math Terms were Mentioned (TMTM) • Times Commands were Pronounced (TCP)
  • 49. Digital Pen: Basic Features • Total Number of Strokes (TNS) • Average Number of Points (ANP) • Average Stroke Path Length (ASPL) • Average Stroke Displacement (ASD) • Average Stroke Pressure (ASP)
  • 50. Digital Pen: Shape Recognition Stronium – Sketch Recognition Libraries
  • 51. Analysis at Problem level Solving Problem Correctly • Logistic Regression to model Student Solving Problem Correctly • Resulting model was significantly reliable • 60,9% - the problem solving student was identified • 71,8% - incorrectly solved problems were identified
  • 52. Analysis at Group Level Expertise Estimation • Features were feed to a Classification Tree algorithm • Several variables had a high discrimination power between expert and non-experts • Best discrimination result in 80% expert prediction and 90% non-expert prediction
  • 53. Expert Estimation over Problems Plateau reached after just 4 problems
  • 54. Main conclusion: Simple features could identify expertise Faster Writer (Digital Pen) Percentage of Calculator Use (Video) Times Numbers were Mentioned (Audio)
  • 55.
  • 57. Video Features • 66 facial features were extracted using Luxand software including both eyes and nose tip to estimate the presenter’s gaze.
  • 58. Kinect features • Identify Common postures
  • 59. Kinect features • Identify Common postures
  • 60. Kinect Features Laban’s theory helps to describe human movement using non-verbal characteristics: Spatial aspects of movement Temporal aspects of movement Fluency, smoothness, impulsivity Energy and power Overall activity
  • 62. Results: less than 50% accuracy What we were measuring was not was humans were measuring
  • 63. What is next in MLA?
  • 64. Mode integration framework for MLA Currently pioneered by Marcelo Worsley
  • 66.
  • 67. Record different learning settings And share them with the community
  • 69. Multimodal Learning Analytics is not a subset of Learning Analytics Current Learning Analytics is a subset of MLA 
  • 70. Some problems are easy, some hard But we do not know until we try to solve them
  • 71. There is a lot of exploring to do And we need explorers
  • 72. Gracias / Thank you Questions? Xavier Ochoa xavier@cti.espol.edu.ec http://ariadne.cti.espol.edu.ec/xavier Twitter: @xaoch http://www.sigmla.org