SlideShare a Scribd company logo

The Multimodal Tutor - short pitch presentation at JTELSS 2018 in Durrës, Albania

These two slides were prepared to present the Multimodal Tutor, my PhD research project which I presented at the JTEL Summer School 2018 http://ea-tel.eu/jtelss/jtelss2018/ The slides summarise concisely the topic of my research project, the main innovation point and the major obstacles that I am facing. This presentation was awarded the EATEL doctoral academy award 2018 at the EC-TEL conference 2018 in Leeds.

1 of 2
Download to read offline
How to use multimodal
data to generate
feedback in learning?
RQ
multimodal data collection
Human
judge
Video
recordings
Sensor
data
human annotation
Digital support for learning across physical spaces
what’s
NEW?
Multimodal
Tutordata processing & analysis
ddm@ou.nl
@dimstudi0
Multimodal
Tutor
ddm@ou.nl
@dimstudi0
Ad

Recommended

SenseTheClassroom Live at EC-TEL 2022
SenseTheClassroom Live at EC-TEL 2022SenseTheClassroom Live at EC-TEL 2022
SenseTheClassroom Live at EC-TEL 2022Daniele Di Mitri
 
Guest Lecture: Restoring Context in Distance Learning with Artificial Intelli...
Guest Lecture: Restoring Context in Distance Learning with Artificial Intelli...Guest Lecture: Restoring Context in Distance Learning with Artificial Intelli...
Guest Lecture: Restoring Context in Distance Learning with Artificial Intelli...Daniele Di Mitri
 
SITE Interactive kenyote 2021
SITE Interactive kenyote 2021SITE Interactive kenyote 2021
SITE Interactive kenyote 2021Daniele Di Mitri
 
MOBIUS: Smart Mobility Tracking with Smartphone Sensors
MOBIUS: Smart Mobility Tracking with Smartphone SensorsMOBIUS: Smart Mobility Tracking with Smartphone Sensors
MOBIUS: Smart Mobility Tracking with Smartphone SensorsDaniele Di Mitri
 
The Multimodal Tutor - Presentation PhD defence
The Multimodal Tutor - Presentation PhD defenceThe Multimodal Tutor - Presentation PhD defence
The Multimodal Tutor - Presentation PhD defenceDaniele Di Mitri
 
Real-time Multimodal Feedback with the CPR Tutor
Real-time Multimodal Feedback with the CPR TutorReal-time Multimodal Feedback with the CPR Tutor
Real-time Multimodal Feedback with the CPR TutorDaniele Di Mitri
 
Multimodal Tutor for CPR presented at AIME'19
Multimodal Tutor for CPR presented at AIME'19Multimodal Tutor for CPR presented at AIME'19
Multimodal Tutor for CPR presented at AIME'19Daniele Di Mitri
 
The Multimodal Learning Analytics Pipeline
The Multimodal Learning Analytics PipelineThe Multimodal Learning Analytics Pipeline
The Multimodal Learning Analytics PipelineDaniele Di Mitri
 

More Related Content

More from Daniele Di Mitri

Workshop: Multimodal Tutor
Workshop: Multimodal TutorWorkshop: Multimodal Tutor
Workshop: Multimodal TutorDaniele Di Mitri
 
Read Between The Lines: an Annotation Tool for Multimodal Data
Read Between The Lines: an Annotation Tool for Multimodal DataRead Between The Lines: an Annotation Tool for Multimodal Data
Read Between The Lines: an Annotation Tool for Multimodal DataDaniele Di Mitri
 
Sensors for Learning workshop
Sensors for Learning workshopSensors for Learning workshop
Sensors for Learning workshopDaniele Di Mitri
 
Multimodal Machines #JTELSS17 workshop
Multimodal Machines #JTELSS17 workshopMultimodal Machines #JTELSS17 workshop
Multimodal Machines #JTELSS17 workshopDaniele Di Mitri
 
Multimodal Tutor - Adaptive feedback from multimodal experience capturing
Multimodal Tutor - Adaptive feedback from multimodal experience capturingMultimodal Tutor - Adaptive feedback from multimodal experience capturing
Multimodal Tutor - Adaptive feedback from multimodal experience capturingDaniele Di Mitri
 
Digital Learning Projection - poster for #LAK17
Digital Learning Projection - poster for #LAK17Digital Learning Projection - poster for #LAK17
Digital Learning Projection - poster for #LAK17Daniele Di Mitri
 
Learning Pulse - paper presentation at LAK17
Learning Pulse - paper presentation at LAK17Learning Pulse - paper presentation at LAK17
Learning Pulse - paper presentation at LAK17Daniele Di Mitri
 
Digital Learning Projection - Learning state estimation from multimodal learn...
Digital Learning Projection - Learning state estimation from multimodal learn...Digital Learning Projection - Learning state estimation from multimodal learn...
Digital Learning Projection - Learning state estimation from multimodal learn...Daniele Di Mitri
 
Academic writing in LaTeX
Academic writing in LaTeX Academic writing in LaTeX
Academic writing in LaTeX Daniele Di Mitri
 
Visual Learning Pulse - Final Thesis presentation
Visual Learning Pulse - Final Thesis presentationVisual Learning Pulse - Final Thesis presentation
Visual Learning Pulse - Final Thesis presentationDaniele Di Mitri
 
Research project MAI2 - Final Presentation Group 4
Research project MAI2  - Final Presentation Group 4Research project MAI2  - Final Presentation Group 4
Research project MAI2 - Final Presentation Group 4Daniele Di Mitri
 
Lifelong Topic Modelling presentation
Lifelong Topic Modelling presentation Lifelong Topic Modelling presentation
Lifelong Topic Modelling presentation Daniele Di Mitri
 
Word2Vec: Learning of word representations in a vector space - Di Mitri & Her...
Word2Vec: Learning of word representations in a vector space - Di Mitri & Her...Word2Vec: Learning of word representations in a vector space - Di Mitri & Her...
Word2Vec: Learning of word representations in a vector space - Di Mitri & Her...Daniele Di Mitri
 
Battlecode2014 - final presentation - group n.2
Battlecode2014 - final presentation - group n.2Battlecode2014 - final presentation - group n.2
Battlecode2014 - final presentation - group n.2Daniele Di Mitri
 
Inclusive & open learning analytics
Inclusive & open learning analyticsInclusive & open learning analytics
Inclusive & open learning analyticsDaniele Di Mitri
 
(IT) Slides della presentazione della tesi di Laurea
(IT) Slides della presentazione della tesi di Laurea(IT) Slides della presentazione della tesi di Laurea
(IT) Slides della presentazione della tesi di LaureaDaniele Di Mitri
 
Claim Your Voice - Presentation of the results of the campign for VET students
Claim Your Voice - Presentation of the results of the campign for VET studentsClaim Your Voice - Presentation of the results of the campign for VET students
Claim Your Voice - Presentation of the results of the campign for VET studentsDaniele Di Mitri
 
Obessu views on school resource
Obessu views on school resourceObessu views on school resource
Obessu views on school resourceDaniele Di Mitri
 
Obessu’s inputs on «opening up education»
Obessu’s inputs on «opening up education»Obessu’s inputs on «opening up education»
Obessu’s inputs on «opening up education»Daniele Di Mitri
 

More from Daniele Di Mitri (20)

Workshop: Multimodal Tutor
Workshop: Multimodal TutorWorkshop: Multimodal Tutor
Workshop: Multimodal Tutor
 
Read Between The Lines: an Annotation Tool for Multimodal Data
Read Between The Lines: an Annotation Tool for Multimodal DataRead Between The Lines: an Annotation Tool for Multimodal Data
Read Between The Lines: an Annotation Tool for Multimodal Data
 
Sensors for Learning workshop
Sensors for Learning workshopSensors for Learning workshop
Sensors for Learning workshop
 
Multimodal Machines #JTELSS17 workshop
Multimodal Machines #JTELSS17 workshopMultimodal Machines #JTELSS17 workshop
Multimodal Machines #JTELSS17 workshop
 
Multimodal Tutor - Adaptive feedback from multimodal experience capturing
Multimodal Tutor - Adaptive feedback from multimodal experience capturingMultimodal Tutor - Adaptive feedback from multimodal experience capturing
Multimodal Tutor - Adaptive feedback from multimodal experience capturing
 
Visual Learning Pulse
Visual Learning PulseVisual Learning Pulse
Visual Learning Pulse
 
Digital Learning Projection - poster for #LAK17
Digital Learning Projection - poster for #LAK17Digital Learning Projection - poster for #LAK17
Digital Learning Projection - poster for #LAK17
 
Learning Pulse - paper presentation at LAK17
Learning Pulse - paper presentation at LAK17Learning Pulse - paper presentation at LAK17
Learning Pulse - paper presentation at LAK17
 
Digital Learning Projection - Learning state estimation from multimodal learn...
Digital Learning Projection - Learning state estimation from multimodal learn...Digital Learning Projection - Learning state estimation from multimodal learn...
Digital Learning Projection - Learning state estimation from multimodal learn...
 
Academic writing in LaTeX
Academic writing in LaTeX Academic writing in LaTeX
Academic writing in LaTeX
 
Visual Learning Pulse - Final Thesis presentation
Visual Learning Pulse - Final Thesis presentationVisual Learning Pulse - Final Thesis presentation
Visual Learning Pulse - Final Thesis presentation
 
Research project MAI2 - Final Presentation Group 4
Research project MAI2  - Final Presentation Group 4Research project MAI2  - Final Presentation Group 4
Research project MAI2 - Final Presentation Group 4
 
Lifelong Topic Modelling presentation
Lifelong Topic Modelling presentation Lifelong Topic Modelling presentation
Lifelong Topic Modelling presentation
 
Word2Vec: Learning of word representations in a vector space - Di Mitri & Her...
Word2Vec: Learning of word representations in a vector space - Di Mitri & Her...Word2Vec: Learning of word representations in a vector space - Di Mitri & Her...
Word2Vec: Learning of word representations in a vector space - Di Mitri & Her...
 
Battlecode2014 - final presentation - group n.2
Battlecode2014 - final presentation - group n.2Battlecode2014 - final presentation - group n.2
Battlecode2014 - final presentation - group n.2
 
Inclusive & open learning analytics
Inclusive & open learning analyticsInclusive & open learning analytics
Inclusive & open learning analytics
 
(IT) Slides della presentazione della tesi di Laurea
(IT) Slides della presentazione della tesi di Laurea(IT) Slides della presentazione della tesi di Laurea
(IT) Slides della presentazione della tesi di Laurea
 
Claim Your Voice - Presentation of the results of the campign for VET students
Claim Your Voice - Presentation of the results of the campign for VET studentsClaim Your Voice - Presentation of the results of the campign for VET students
Claim Your Voice - Presentation of the results of the campign for VET students
 
Obessu views on school resource
Obessu views on school resourceObessu views on school resource
Obessu views on school resource
 
Obessu’s inputs on «opening up education»
Obessu’s inputs on «opening up education»Obessu’s inputs on «opening up education»
Obessu’s inputs on «opening up education»
 

Recently uploaded

UniSC Fraser Coast library self-guided tour
UniSC Fraser Coast library self-guided tourUniSC Fraser Coast library self-guided tour
UniSC Fraser Coast library self-guided tourUSC_Library
 
Chromatography-Gas chromatography-Principle
Chromatography-Gas chromatography-PrincipleChromatography-Gas chromatography-Principle
Chromatography-Gas chromatography-Principleblessipriyanka
 
2.15.24 Making Whiteness -- Baldwin.pptx
2.15.24 Making Whiteness -- Baldwin.pptx2.15.24 Making Whiteness -- Baldwin.pptx
2.15.24 Making Whiteness -- Baldwin.pptxMaryPotorti1
 
EDL 290F Week 1 - Meet Me at the Start Line.pdf
EDL 290F Week 1 - Meet Me at the Start Line.pdfEDL 290F Week 1 - Meet Me at the Start Line.pdf
EDL 290F Week 1 - Meet Me at the Start Line.pdfElizabeth Walsh
 
ACTIVIDAD DE CLASE No 1 sopa de letras.docx
ACTIVIDAD DE CLASE No 1 sopa de letras.docxACTIVIDAD DE CLASE No 1 sopa de letras.docx
ACTIVIDAD DE CLASE No 1 sopa de letras.docxMaria Lucia Céspedes
 
2.22.24 Black Nationalism and the Nation of Islam.pptx
2.22.24 Black Nationalism and the Nation of Islam.pptx2.22.24 Black Nationalism and the Nation of Islam.pptx
2.22.24 Black Nationalism and the Nation of Islam.pptxMaryPotorti1
 
Overview of Databases and Data Modelling-2.pdf
Overview of Databases and Data Modelling-2.pdfOverview of Databases and Data Modelling-2.pdf
Overview of Databases and Data Modelling-2.pdfChristalin Nelson
 
Appendicular SkeletonSystem PPT.....pptx
Appendicular SkeletonSystem PPT.....pptxAppendicular SkeletonSystem PPT.....pptx
Appendicular SkeletonSystem PPT.....pptxRenuka N Sunagad
 
Intuition behind Monte Carlo Markov Chains
Intuition behind Monte Carlo Markov ChainsIntuition behind Monte Carlo Markov Chains
Intuition behind Monte Carlo Markov ChainsTushar Tank
 
IR introduction Introduction, Principle & Theory
IR introduction Introduction, Principle & TheoryIR introduction Introduction, Principle & Theory
IR introduction Introduction, Principle & Theorynivedithag131
 
BEZA or Bangladesh Economic Zone Authority recruitment exam question solution...
BEZA or Bangladesh Economic Zone Authority recruitment exam question solution...BEZA or Bangladesh Economic Zone Authority recruitment exam question solution...
BEZA or Bangladesh Economic Zone Authority recruitment exam question solution...MohonDas
 
ICSE English Literature Class X Handwritten Notes
ICSE English Literature Class X Handwritten NotesICSE English Literature Class X Handwritten Notes
ICSE English Literature Class X Handwritten NotesGauri S
 
Exit Essay - Save the Filipino Language by Renz Perez.docx
Exit Essay - Save the Filipino Language by Renz Perez.docxExit Essay - Save the Filipino Language by Renz Perez.docx
Exit Essay - Save the Filipino Language by Renz Perez.docxMYDA ANGELICA SUAN
 
catch-up-friday-ARALING PNLIPUNAN SOCIAL JUSTICE AND HUMAN RIGHTS
catch-up-friday-ARALING PNLIPUNAN SOCIAL JUSTICE AND HUMAN RIGHTScatch-up-friday-ARALING PNLIPUNAN SOCIAL JUSTICE AND HUMAN RIGHTS
catch-up-friday-ARALING PNLIPUNAN SOCIAL JUSTICE AND HUMAN RIGHTSCarlaNicolas7
 
Andreas Schleicher - 20 Feb 2024 - How pop music, podcasts, and Tik Tok are i...
Andreas Schleicher - 20 Feb 2024 - How pop music, podcasts, and Tik Tok are i...Andreas Schleicher - 20 Feb 2024 - How pop music, podcasts, and Tik Tok are i...
Andreas Schleicher - 20 Feb 2024 - How pop music, podcasts, and Tik Tok are i...EduSkills OECD
 
Writing Agony Letter & If type O+1 & Diphthongs + Text “Arab Science”.pdf
Writing Agony Letter & If type O+1 & Diphthongs + Text “Arab Science”.pdfWriting Agony Letter & If type O+1 & Diphthongs + Text “Arab Science”.pdf
Writing Agony Letter & If type O+1 & Diphthongs + Text “Arab Science”.pdfMr Bounab Samir
 
LIPIDS UNIT ONE BIOMOLECULE PRESENTATION
LIPIDS UNIT ONE BIOMOLECULE PRESENTATIONLIPIDS UNIT ONE BIOMOLECULE PRESENTATION
LIPIDS UNIT ONE BIOMOLECULE PRESENTATIONSayali Powar
 
skeletal system details with joints and its types
skeletal system details with joints and its typesskeletal system details with joints and its types
skeletal system details with joints and its typesMinaxi patil. CATALLYST
 

Recently uploaded (20)

UniSC Fraser Coast library self-guided tour
UniSC Fraser Coast library self-guided tourUniSC Fraser Coast library self-guided tour
UniSC Fraser Coast library self-guided tour
 
Chromatography-Gas chromatography-Principle
Chromatography-Gas chromatography-PrincipleChromatography-Gas chromatography-Principle
Chromatography-Gas chromatography-Principle
 
2.15.24 Making Whiteness -- Baldwin.pptx
2.15.24 Making Whiteness -- Baldwin.pptx2.15.24 Making Whiteness -- Baldwin.pptx
2.15.24 Making Whiteness -- Baldwin.pptx
 
EDL 290F Week 1 - Meet Me at the Start Line.pdf
EDL 290F Week 1 - Meet Me at the Start Line.pdfEDL 290F Week 1 - Meet Me at the Start Line.pdf
EDL 290F Week 1 - Meet Me at the Start Line.pdf
 
ACTIVIDAD DE CLASE No 1 sopa de letras.docx
ACTIVIDAD DE CLASE No 1 sopa de letras.docxACTIVIDAD DE CLASE No 1 sopa de letras.docx
ACTIVIDAD DE CLASE No 1 sopa de letras.docx
 
2.22.24 Black Nationalism and the Nation of Islam.pptx
2.22.24 Black Nationalism and the Nation of Islam.pptx2.22.24 Black Nationalism and the Nation of Islam.pptx
2.22.24 Black Nationalism and the Nation of Islam.pptx
 
Overview of Databases and Data Modelling-2.pdf
Overview of Databases and Data Modelling-2.pdfOverview of Databases and Data Modelling-2.pdf
Overview of Databases and Data Modelling-2.pdf
 
Appendicular SkeletonSystem PPT.....pptx
Appendicular SkeletonSystem PPT.....pptxAppendicular SkeletonSystem PPT.....pptx
Appendicular SkeletonSystem PPT.....pptx
 
Intuition behind Monte Carlo Markov Chains
Intuition behind Monte Carlo Markov ChainsIntuition behind Monte Carlo Markov Chains
Intuition behind Monte Carlo Markov Chains
 
IR introduction Introduction, Principle & Theory
IR introduction Introduction, Principle & TheoryIR introduction Introduction, Principle & Theory
IR introduction Introduction, Principle & Theory
 
BEZA or Bangladesh Economic Zone Authority recruitment exam question solution...
BEZA or Bangladesh Economic Zone Authority recruitment exam question solution...BEZA or Bangladesh Economic Zone Authority recruitment exam question solution...
BEZA or Bangladesh Economic Zone Authority recruitment exam question solution...
 
ICSE English Literature Class X Handwritten Notes
ICSE English Literature Class X Handwritten NotesICSE English Literature Class X Handwritten Notes
ICSE English Literature Class X Handwritten Notes
 
first section physiology laboratory.pptx
first section physiology laboratory.pptxfirst section physiology laboratory.pptx
first section physiology laboratory.pptx
 
Exit Essay - Save the Filipino Language by Renz Perez.docx
Exit Essay - Save the Filipino Language by Renz Perez.docxExit Essay - Save the Filipino Language by Renz Perez.docx
Exit Essay - Save the Filipino Language by Renz Perez.docx
 
catch-up-friday-ARALING PNLIPUNAN SOCIAL JUSTICE AND HUMAN RIGHTS
catch-up-friday-ARALING PNLIPUNAN SOCIAL JUSTICE AND HUMAN RIGHTScatch-up-friday-ARALING PNLIPUNAN SOCIAL JUSTICE AND HUMAN RIGHTS
catch-up-friday-ARALING PNLIPUNAN SOCIAL JUSTICE AND HUMAN RIGHTS
 
Lipids as Biopolymer
Lipids as Biopolymer Lipids as Biopolymer
Lipids as Biopolymer
 
Andreas Schleicher - 20 Feb 2024 - How pop music, podcasts, and Tik Tok are i...
Andreas Schleicher - 20 Feb 2024 - How pop music, podcasts, and Tik Tok are i...Andreas Schleicher - 20 Feb 2024 - How pop music, podcasts, and Tik Tok are i...
Andreas Schleicher - 20 Feb 2024 - How pop music, podcasts, and Tik Tok are i...
 
Writing Agony Letter & If type O+1 & Diphthongs + Text “Arab Science”.pdf
Writing Agony Letter & If type O+1 & Diphthongs + Text “Arab Science”.pdfWriting Agony Letter & If type O+1 & Diphthongs + Text “Arab Science”.pdf
Writing Agony Letter & If type O+1 & Diphthongs + Text “Arab Science”.pdf
 
LIPIDS UNIT ONE BIOMOLECULE PRESENTATION
LIPIDS UNIT ONE BIOMOLECULE PRESENTATIONLIPIDS UNIT ONE BIOMOLECULE PRESENTATION
LIPIDS UNIT ONE BIOMOLECULE PRESENTATION
 
skeletal system details with joints and its types
skeletal system details with joints and its typesskeletal system details with joints and its types
skeletal system details with joints and its types
 

The Multimodal Tutor - short pitch presentation at JTELSS 2018 in Durrës, Albania

  • 1. How to use multimodal data to generate feedback in learning? RQ multimodal data collection Human judge Video recordings Sensor data human annotation Digital support for learning across physical spaces what’s NEW? Multimodal Tutordata processing & analysis ddm@ou.nl @dimstudi0

Editor's Notes

  1. My phd project is the Multimodal Tutor The problem I want to solve is how to use multimodal data to generate feedback in learning The originality is a new digital support for learning across physical spaces The Multimodal Tutor can be seen as a cycle (that we also call Multimodal pipeline) Green box Collecting data from multiple modalities using multiple sensors and data sources Red box using human judges to interpret and annotate multimodal data recordings Yellow box using the machine learning and data processing to combine multimodal data to human interpretations Blue box is how to enhance feedback
  2. The Multimodal Tutor now looks like the wall of the PinkFloyd Complex cycle made by multiple steps Few datasets Architecture