SlideShare a Scribd company logo
Analysis of Educational Robotics activities
using a machine learning approach
L. Cesaretti*,**, L. Screpanti*, D. Scaradozzi*,***, E. Mangina****
* Department of Information Engineering, Università Politecnica delle Marche, Ancona, Italy
** Talent srl, Osimo, Italy
*** LSIS- umr CNRS 6168, Laboratoire des Sciences de l'Information et des Systèmes Equipe I&M (ESIL)
**** School of Computer Science, University College Dublin, Dublin, Ireland
Why this research project?
In the Educational Robotics (ER) field
researchers have identified lack of
quantitative analysis on how robotics can
improve skills and increase learning
achievements in students (Benitti 2012;
Alimisis, 2013).
Alimisis, D. (2013). Educational robotics: Open questions and new challenges. Themes in Science and Technology Education, 6(1), 63-71.
Benitti, F. B. V. (2012). Exploring the educational potential of robotics in schools: A systematic review. Computers & Education, 58(3), 978-988.
Machine learning and Data mining are everywhere!
Made for me
That's a prediction “of what Netflix thinks
you may enjoy watching, based on your
own unique tastes”.
Real time monitoring of students’ activity
(Registered Patent: n. 102018000009636, «APPARATO DI MONITORAGGIO DI OPERAZIONI DI ASSEMBLAGGIO E
PROGRAMMAZIONE»)
Case Study: Research Questions
1. identification of different patterns in the students’ problem-solving trajectories;
2. accurate prediction of students’ team final performance; and
3. correlation of the discovered patterns of students’ problem-solving with the
evaluation given by the educators
Applying data mining and machine learning methods to data collected from the
educational environments can allow to predict and classify students’ behaviours and
discover latent structural regularities to large educational dataset.
Berland, M., Baker, R. S., & Blikstein, P. (2014). Educational data mining and learning analytics: Applications to constructionist
research. Technology, Knowledge and Learning, 19(1-2), 205-220.
Case Study: Participants and Procedure
Students from seven Italian lower and
higher secondary schools, located in
the Emilia Romagna and Marche
regions.
The total number of students involved
in this study is 197.
The experimentation was carried out
from March 2018 to March 2019.
STUDENT’S ACTIVITY
(Think – Make – Improve approach)
Program the robot so that it covers a given
distance (1 m), trying to be as precise as
possible.
Students’ teams involved in the research project
had to take into account some constraints:
• the amount of time within they had to design
and test their solution (15 - 20 minutes);
• the teams could test the programming
sequence as many times as they wanted;
• they were allowed to use measuring
instruments only to measure some robot’s
parameters (for example the radius of the
wheel).
Case Study: the Introductory Exercise
EVALUATION
• if the error was < 4 cm, the educator considered the
challenge completed;
• if the error was >= 4 cm the educator considered the
challenge not completed.
Case Study: how to represent the participants' activities in the
robot programming activity
Students’ teams designed 1113 programming sequences to solve the introductory Exercise.
Each programming test realised by the students’ team can be represented as a vector composed by these 12
elements:
● Motors: the n° of Motor blocks in the sequence
● Loops: the n° of Loop blocks in the sequence.
● Conditionals: the n° of Conditional and Sensors blocks in the sequence.
● Others: the n° of blocks in the sequence belonging to different categories than Motors, Loops and Conditionals.
● Added: the n° of blocks added, compared to the previous sequence;
● Deleted: the n° of blocks deleted, compared to the previous sequence;
● Changed: the n° of blocks changed, compared to the previous sequence;
● Equal: the n° of the same blocks, compared to the previous sequence;
● Delta Motors: amount of change in Motor blocks parameters (first, second or third parameter), compared to the
previous sequence (calculated only for blocks of the “Changed” category);
● Delta Loops: amount of change in Loop blocks parameters, compared to the previous sequence;
● Delta Conditionals: amount of change in Conditional blocks parameters, compared to the previous sequence;
● Delta Others: amount of change in Other blocks parameters, compared to the previous sequence.
SUPERVISED APPROACH
The feature matrix was created by calculating the
mean value and the standard deviation for each
indicator presented in the previous section, taking into
account all the trials performed by a students’ team to
solve an exercise; then, the authors compared the
performances of four different machine learning
algorithms:
• Logistic Regression
• Support Vector Machine
• k-nearest neighbors
• Random Forest classifier
in the prediction of the students’ teams final result.
MIXED APPROACH
Characterized by the benefits of both supervised
and unsupervised methods: a k-means algorithm
was applied to calculate in which clusters the
programming sequences could be divided. Following
the clustering, the percentage of each cluster in the
programming activity of the students’ groups was
calculated and these new features were then used
to create a feature matrix as an input for the
previously cited four supervised algorithms.
Case Study: Results
Case Study: Results
− Accuracy
− Mean Precision (calculated considering the average
value between precision in the prediction of students’
positive performance and negative performance)
− Mean Recall (calculated considering the average value
between the recall in the prediction of students’ positive
performance and negative performance)
− Mean F1 – Score (calculated considering the average
value between the F1-score in the prediction of
students’ positive performance and negative
performance)
To obtain these parameters a repeated 10-fold cross
validation was performed, so that the average value and
standard deviation of the previous four parameters
repeating the 10-fold validation multiple times were
calculated.
Case Study: Clustering Results
Case Study: Problem solving patterns
0
5
10
15
20
25
30
Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6 Cluster 7 Cluster 8 Trials
#ofprogrammingsequences
Different teams' behaviours
Mathematical / Planning Tinkering (with prevalent refining behaviuor)
Tinkering (with significant high changes)
Pearson correlation coefficient
(between the features extracted with
the k-means algorithms and the final
results obtained by students’ teams)
Two features show a statistically
significant negative correlation:
• number of trials (PCC = -0.48, p-
value < 0.001)
• percentage of sequences of the
cluster named “HIGH MOTORS
PARAMETERS CHANGE” (PCC = -
0.39, p-value < 0.01);
High values of these features (typical
for the “Tinkering with significantly
high changes” teams) indicate higher
probability of a negative
performance.
Case Study: Mathematical behaviour
FINAL ERROR = 1 CM
Case Study: Tinkering behaviour (with refining)
FINAL ERROR = 1,5 CM
0
5
10
15
20
25
30
35
40
45
50
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
DeltaMotors
Test number
Case Study: Tinkering behaviour (with large changes)
FINAL ERROR = 50 CM
Conclusions
• Compare the performance of the machine learning techniques already applied (SVM, Logistic
Regression, KNN, Random Forest) with MLP Neural Network → Larger Dataset
• Another improvement for future development of this study will be time tracking in the log files
generated by the system.
• Authors intend also to utilise recurrent neural network, in particular the long short-term memory
autoencoders (a structure specifically designed to support sequences of input data), in order to
translate the programming sequences created by students into fixed-length vectors (compress
representation of the input data), maintaining high level of information content.
• Another planned development is the update of the current system design with a personalised e-
learning system: an educational recommender system could give real-time feedback to teachers
and students involved in Educational Robotics activities, or propose personalised learning path to
learners.
Thank you!
Questions?
Contacts:
Lorenzo Cesaretti
l.cesaretti@pm.univpm.it
David Scaradozzi
d.scaradozzi@univpm.it
Laura Screpanti
l.screpanti@pm.univpm.it

More Related Content

Similar to Analysis of Educational Robotics activities using a machine learning approach

IRJET- Analysis of PV Fed Vector Controlled Induction Motor Drive
IRJET- Analysis of PV Fed Vector Controlled Induction Motor DriveIRJET- Analysis of PV Fed Vector Controlled Induction Motor Drive
IRJET- Analysis of PV Fed Vector Controlled Induction Motor Drive
IRJET Journal
 
IRJET- Deep Learning Model to Predict Hardware Performance
IRJET- Deep Learning Model to Predict Hardware PerformanceIRJET- Deep Learning Model to Predict Hardware Performance
IRJET- Deep Learning Model to Predict Hardware Performance
IRJET Journal
 
IRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various CoursesIRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various Courses
IRJET Journal
 
Autonomy Incubator Seminar Series: Tractable Robust Planning and Model Learni...
Autonomy Incubator Seminar Series: Tractable Robust Planning and Model Learni...Autonomy Incubator Seminar Series: Tractable Robust Planning and Model Learni...
Autonomy Incubator Seminar Series: Tractable Robust Planning and Model Learni...
AutonomyIncubator
 
Harnessing deep learning algorithms to predict software refactoring
Harnessing deep learning algorithms to predict software refactoringHarnessing deep learning algorithms to predict software refactoring
Harnessing deep learning algorithms to predict software refactoring
TELKOMNIKA JOURNAL
 
IRJET- Machine Learning Techniques for Code Optimization
IRJET-  	  Machine Learning Techniques for Code OptimizationIRJET-  	  Machine Learning Techniques for Code Optimization
IRJET- Machine Learning Techniques for Code Optimization
IRJET Journal
 
Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt
Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.pptProto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt
Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt
AnirbanBhar3
 
career guidance using ml and python for college students projects
career guidance using ml and python for college students projectscareer guidance using ml and python for college students projects
career guidance using ml and python for college students projects
Hamed Raza
 
EMPIRICAL APPLICATION OF SIMULATED ANNEALING USING OBJECT-ORIENTED METRICS TO...
EMPIRICAL APPLICATION OF SIMULATED ANNEALING USING OBJECT-ORIENTED METRICS TO...EMPIRICAL APPLICATION OF SIMULATED ANNEALING USING OBJECT-ORIENTED METRICS TO...
EMPIRICAL APPLICATION OF SIMULATED ANNEALING USING OBJECT-ORIENTED METRICS TO...
ijcsa
 
Intelligent Career Guidance System.pptx
Intelligent Career Guidance System.pptxIntelligent Career Guidance System.pptx
Intelligent Career Guidance System.pptx
Anonymous366406
 
IRJET - Comparative Analysis of GUI based Prediction of Parkinson Disease usi...
IRJET - Comparative Analysis of GUI based Prediction of Parkinson Disease usi...IRJET - Comparative Analysis of GUI based Prediction of Parkinson Disease usi...
IRJET - Comparative Analysis of GUI based Prediction of Parkinson Disease usi...
IRJET Journal
 
Bangla Hand Written Digit Recognition presentation slide .pptx
Bangla Hand Written Digit Recognition presentation slide .pptxBangla Hand Written Digit Recognition presentation slide .pptx
Bangla Hand Written Digit Recognition presentation slide .pptx
KhondokerAbuNaim
 
Classification of Machine Learning Algorithms
Classification of Machine Learning AlgorithmsClassification of Machine Learning Algorithms
Classification of Machine Learning Algorithms
AM Publications
 
A Survey of Machine Learning Methods Applied to Computer ...
A Survey of Machine Learning Methods Applied to Computer ...A Survey of Machine Learning Methods Applied to Computer ...
A Survey of Machine Learning Methods Applied to Computer ...
butest
 
Water Quality Index Calculation of River Ganga using Decision Tree Algorithm
Water Quality Index Calculation of River Ganga using Decision Tree AlgorithmWater Quality Index Calculation of River Ganga using Decision Tree Algorithm
Water Quality Index Calculation of River Ganga using Decision Tree Algorithm
IRJET Journal
 
Recuriter Recommendation System
Recuriter Recommendation SystemRecuriter Recommendation System
Recuriter Recommendation System
IRJET Journal
 
Combinatorial optimization and deep reinforcement learning
Combinatorial optimization and deep reinforcement learningCombinatorial optimization and deep reinforcement learning
Combinatorial optimization and deep reinforcement learning
민재 정
 
Automation and robotics ii
Automation and robotics iiAutomation and robotics ii
Automation and robotics ii
HCS
 
Performance Comparision of Machine Learning Algorithms
Performance Comparision of Machine Learning AlgorithmsPerformance Comparision of Machine Learning Algorithms
Performance Comparision of Machine Learning Algorithms
Dinusha Dilanka
 
Testing Machine Learning-enabled Systems: A Personal Perspective
Testing Machine Learning-enabled Systems: A Personal PerspectiveTesting Machine Learning-enabled Systems: A Personal Perspective
Testing Machine Learning-enabled Systems: A Personal Perspective
Lionel Briand
 

Similar to Analysis of Educational Robotics activities using a machine learning approach (20)

IRJET- Analysis of PV Fed Vector Controlled Induction Motor Drive
IRJET- Analysis of PV Fed Vector Controlled Induction Motor DriveIRJET- Analysis of PV Fed Vector Controlled Induction Motor Drive
IRJET- Analysis of PV Fed Vector Controlled Induction Motor Drive
 
IRJET- Deep Learning Model to Predict Hardware Performance
IRJET- Deep Learning Model to Predict Hardware PerformanceIRJET- Deep Learning Model to Predict Hardware Performance
IRJET- Deep Learning Model to Predict Hardware Performance
 
IRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various CoursesIRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various Courses
 
Autonomy Incubator Seminar Series: Tractable Robust Planning and Model Learni...
Autonomy Incubator Seminar Series: Tractable Robust Planning and Model Learni...Autonomy Incubator Seminar Series: Tractable Robust Planning and Model Learni...
Autonomy Incubator Seminar Series: Tractable Robust Planning and Model Learni...
 
Harnessing deep learning algorithms to predict software refactoring
Harnessing deep learning algorithms to predict software refactoringHarnessing deep learning algorithms to predict software refactoring
Harnessing deep learning algorithms to predict software refactoring
 
IRJET- Machine Learning Techniques for Code Optimization
IRJET-  	  Machine Learning Techniques for Code OptimizationIRJET-  	  Machine Learning Techniques for Code Optimization
IRJET- Machine Learning Techniques for Code Optimization
 
Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt
Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.pptProto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt
Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt Proto Spiral.ppt
 
career guidance using ml and python for college students projects
career guidance using ml and python for college students projectscareer guidance using ml and python for college students projects
career guidance using ml and python for college students projects
 
EMPIRICAL APPLICATION OF SIMULATED ANNEALING USING OBJECT-ORIENTED METRICS TO...
EMPIRICAL APPLICATION OF SIMULATED ANNEALING USING OBJECT-ORIENTED METRICS TO...EMPIRICAL APPLICATION OF SIMULATED ANNEALING USING OBJECT-ORIENTED METRICS TO...
EMPIRICAL APPLICATION OF SIMULATED ANNEALING USING OBJECT-ORIENTED METRICS TO...
 
Intelligent Career Guidance System.pptx
Intelligent Career Guidance System.pptxIntelligent Career Guidance System.pptx
Intelligent Career Guidance System.pptx
 
IRJET - Comparative Analysis of GUI based Prediction of Parkinson Disease usi...
IRJET - Comparative Analysis of GUI based Prediction of Parkinson Disease usi...IRJET - Comparative Analysis of GUI based Prediction of Parkinson Disease usi...
IRJET - Comparative Analysis of GUI based Prediction of Parkinson Disease usi...
 
Bangla Hand Written Digit Recognition presentation slide .pptx
Bangla Hand Written Digit Recognition presentation slide .pptxBangla Hand Written Digit Recognition presentation slide .pptx
Bangla Hand Written Digit Recognition presentation slide .pptx
 
Classification of Machine Learning Algorithms
Classification of Machine Learning AlgorithmsClassification of Machine Learning Algorithms
Classification of Machine Learning Algorithms
 
A Survey of Machine Learning Methods Applied to Computer ...
A Survey of Machine Learning Methods Applied to Computer ...A Survey of Machine Learning Methods Applied to Computer ...
A Survey of Machine Learning Methods Applied to Computer ...
 
Water Quality Index Calculation of River Ganga using Decision Tree Algorithm
Water Quality Index Calculation of River Ganga using Decision Tree AlgorithmWater Quality Index Calculation of River Ganga using Decision Tree Algorithm
Water Quality Index Calculation of River Ganga using Decision Tree Algorithm
 
Recuriter Recommendation System
Recuriter Recommendation SystemRecuriter Recommendation System
Recuriter Recommendation System
 
Combinatorial optimization and deep reinforcement learning
Combinatorial optimization and deep reinforcement learningCombinatorial optimization and deep reinforcement learning
Combinatorial optimization and deep reinforcement learning
 
Automation and robotics ii
Automation and robotics iiAutomation and robotics ii
Automation and robotics ii
 
Performance Comparision of Machine Learning Algorithms
Performance Comparision of Machine Learning AlgorithmsPerformance Comparision of Machine Learning Algorithms
Performance Comparision of Machine Learning Algorithms
 
Testing Machine Learning-enabled Systems: A Personal Perspective
Testing Machine Learning-enabled Systems: A Personal PerspectiveTesting Machine Learning-enabled Systems: A Personal Perspective
Testing Machine Learning-enabled Systems: A Personal Perspective
 

Recently uploaded

Odoo 17 Events - Attendees List Scanning
Odoo 17 Events - Attendees List ScanningOdoo 17 Events - Attendees List Scanning
Odoo 17 Events - Attendees List Scanning
Celine George
 
New Features in Odoo 17 Sign - Odoo 17 Slides
New Features in Odoo 17 Sign - Odoo 17 SlidesNew Features in Odoo 17 Sign - Odoo 17 Slides
New Features in Odoo 17 Sign - Odoo 17 Slides
Celine George
 
Lecture_Notes_Unit4_Chapter_8_9_10_RDBMS for the students affiliated by alaga...
Lecture_Notes_Unit4_Chapter_8_9_10_RDBMS for the students affiliated by alaga...Lecture_Notes_Unit4_Chapter_8_9_10_RDBMS for the students affiliated by alaga...
Lecture_Notes_Unit4_Chapter_8_9_10_RDBMS for the students affiliated by alaga...
Murugan Solaiyappan
 
Benchmarking Sustainability: Neurosciences and AI Tech Research in Macau - Ke...
Benchmarking Sustainability: Neurosciences and AI Tech Research in Macau - Ke...Benchmarking Sustainability: Neurosciences and AI Tech Research in Macau - Ke...
Benchmarking Sustainability: Neurosciences and AI Tech Research in Macau - Ke...
Alvaro Barbosa
 
How to Add a Filter in the Odoo 17 - Odoo 17 Slides
How to Add a Filter in the Odoo 17 - Odoo 17 SlidesHow to Add a Filter in the Odoo 17 - Odoo 17 Slides
How to Add a Filter in the Odoo 17 - Odoo 17 Slides
Celine George
 
modul ajar kelas x bahasa inggris 24/254
modul ajar kelas x bahasa inggris 24/254modul ajar kelas x bahasa inggris 24/254
modul ajar kelas x bahasa inggris 24/254
NurFitriah45
 
CTD Punjab Police Past Papers MCQs PPSC PDF
CTD Punjab Police Past Papers MCQs PPSC PDFCTD Punjab Police Past Papers MCQs PPSC PDF
CTD Punjab Police Past Papers MCQs PPSC PDF
hammadmughal76316
 
RDBMS Lecture Notes Unit4 chapter12 VIEW
RDBMS Lecture Notes Unit4 chapter12 VIEWRDBMS Lecture Notes Unit4 chapter12 VIEW
RDBMS Lecture Notes Unit4 chapter12 VIEW
Murugan Solaiyappan
 
1. Importance_of_reducing_postharvest_loss.pptx
1. Importance_of_reducing_postharvest_loss.pptx1. Importance_of_reducing_postharvest_loss.pptx
1. Importance_of_reducing_postharvest_loss.pptx
UmeshTimilsina1
 
FINAL MATATAG Kindergarten CG 2023 pdf
FINAL MATATAG Kindergarten CG 2023   pdfFINAL MATATAG Kindergarten CG 2023   pdf
FINAL MATATAG Kindergarten CG 2023 pdf
maricelrudela
 
Mail Server Configuration Using App passwords in Odoo 17
Mail Server Configuration Using App passwords in Odoo 17Mail Server Configuration Using App passwords in Odoo 17
Mail Server Configuration Using App passwords in Odoo 17
Celine George
 
E-learning Odoo 17 New features - Odoo 17 Slides
E-learning Odoo 17  New features - Odoo 17 SlidesE-learning Odoo 17  New features - Odoo 17 Slides
E-learning Odoo 17 New features - Odoo 17 Slides
Celine George
 
A beginner’s guide to project reviews - everything you wanted to know but wer...
A beginner’s guide to project reviews - everything you wanted to know but wer...A beginner’s guide to project reviews - everything you wanted to know but wer...
A beginner’s guide to project reviews - everything you wanted to know but wer...
Association for Project Management
 
Allopathic M1 Srudent Orientation Powerpoint
Allopathic M1 Srudent Orientation PowerpointAllopathic M1 Srudent Orientation Powerpoint
Allopathic M1 Srudent Orientation Powerpoint
Julie Sarpy
 
ASP.NET Core Interview Questions PDF By ScholarHat.pdf
ASP.NET Core Interview Questions PDF By ScholarHat.pdfASP.NET Core Interview Questions PDF By ScholarHat.pdf
ASP.NET Core Interview Questions PDF By ScholarHat.pdf
Scholarhat
 
How to Create a New Article in Knowledge App in Odoo 17
How to Create a New Article in Knowledge App in Odoo 17How to Create a New Article in Knowledge App in Odoo 17
How to Create a New Article in Knowledge App in Odoo 17
Celine George
 
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ..."DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
thanhluan21
 
How to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POSHow to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POS
Celine George
 
Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.
DrRavindrakshirsagar1
 
SEQUNCES Lecture_Notes_Unit4_chapter11_sequence
SEQUNCES  Lecture_Notes_Unit4_chapter11_sequenceSEQUNCES  Lecture_Notes_Unit4_chapter11_sequence
SEQUNCES Lecture_Notes_Unit4_chapter11_sequence
Murugan Solaiyappan
 

Recently uploaded (20)

Odoo 17 Events - Attendees List Scanning
Odoo 17 Events - Attendees List ScanningOdoo 17 Events - Attendees List Scanning
Odoo 17 Events - Attendees List Scanning
 
New Features in Odoo 17 Sign - Odoo 17 Slides
New Features in Odoo 17 Sign - Odoo 17 SlidesNew Features in Odoo 17 Sign - Odoo 17 Slides
New Features in Odoo 17 Sign - Odoo 17 Slides
 
Lecture_Notes_Unit4_Chapter_8_9_10_RDBMS for the students affiliated by alaga...
Lecture_Notes_Unit4_Chapter_8_9_10_RDBMS for the students affiliated by alaga...Lecture_Notes_Unit4_Chapter_8_9_10_RDBMS for the students affiliated by alaga...
Lecture_Notes_Unit4_Chapter_8_9_10_RDBMS for the students affiliated by alaga...
 
Benchmarking Sustainability: Neurosciences and AI Tech Research in Macau - Ke...
Benchmarking Sustainability: Neurosciences and AI Tech Research in Macau - Ke...Benchmarking Sustainability: Neurosciences and AI Tech Research in Macau - Ke...
Benchmarking Sustainability: Neurosciences and AI Tech Research in Macau - Ke...
 
How to Add a Filter in the Odoo 17 - Odoo 17 Slides
How to Add a Filter in the Odoo 17 - Odoo 17 SlidesHow to Add a Filter in the Odoo 17 - Odoo 17 Slides
How to Add a Filter in the Odoo 17 - Odoo 17 Slides
 
modul ajar kelas x bahasa inggris 24/254
modul ajar kelas x bahasa inggris 24/254modul ajar kelas x bahasa inggris 24/254
modul ajar kelas x bahasa inggris 24/254
 
CTD Punjab Police Past Papers MCQs PPSC PDF
CTD Punjab Police Past Papers MCQs PPSC PDFCTD Punjab Police Past Papers MCQs PPSC PDF
CTD Punjab Police Past Papers MCQs PPSC PDF
 
RDBMS Lecture Notes Unit4 chapter12 VIEW
RDBMS Lecture Notes Unit4 chapter12 VIEWRDBMS Lecture Notes Unit4 chapter12 VIEW
RDBMS Lecture Notes Unit4 chapter12 VIEW
 
1. Importance_of_reducing_postharvest_loss.pptx
1. Importance_of_reducing_postharvest_loss.pptx1. Importance_of_reducing_postharvest_loss.pptx
1. Importance_of_reducing_postharvest_loss.pptx
 
FINAL MATATAG Kindergarten CG 2023 pdf
FINAL MATATAG Kindergarten CG 2023   pdfFINAL MATATAG Kindergarten CG 2023   pdf
FINAL MATATAG Kindergarten CG 2023 pdf
 
Mail Server Configuration Using App passwords in Odoo 17
Mail Server Configuration Using App passwords in Odoo 17Mail Server Configuration Using App passwords in Odoo 17
Mail Server Configuration Using App passwords in Odoo 17
 
E-learning Odoo 17 New features - Odoo 17 Slides
E-learning Odoo 17  New features - Odoo 17 SlidesE-learning Odoo 17  New features - Odoo 17 Slides
E-learning Odoo 17 New features - Odoo 17 Slides
 
A beginner’s guide to project reviews - everything you wanted to know but wer...
A beginner’s guide to project reviews - everything you wanted to know but wer...A beginner’s guide to project reviews - everything you wanted to know but wer...
A beginner’s guide to project reviews - everything you wanted to know but wer...
 
Allopathic M1 Srudent Orientation Powerpoint
Allopathic M1 Srudent Orientation PowerpointAllopathic M1 Srudent Orientation Powerpoint
Allopathic M1 Srudent Orientation Powerpoint
 
ASP.NET Core Interview Questions PDF By ScholarHat.pdf
ASP.NET Core Interview Questions PDF By ScholarHat.pdfASP.NET Core Interview Questions PDF By ScholarHat.pdf
ASP.NET Core Interview Questions PDF By ScholarHat.pdf
 
How to Create a New Article in Knowledge App in Odoo 17
How to Create a New Article in Knowledge App in Odoo 17How to Create a New Article in Knowledge App in Odoo 17
How to Create a New Article in Knowledge App in Odoo 17
 
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ..."DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
 
How to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POSHow to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POS
 
Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.
 
SEQUNCES Lecture_Notes_Unit4_chapter11_sequence
SEQUNCES  Lecture_Notes_Unit4_chapter11_sequenceSEQUNCES  Lecture_Notes_Unit4_chapter11_sequence
SEQUNCES Lecture_Notes_Unit4_chapter11_sequence
 

Analysis of Educational Robotics activities using a machine learning approach

  • 1. Analysis of Educational Robotics activities using a machine learning approach L. Cesaretti*,**, L. Screpanti*, D. Scaradozzi*,***, E. Mangina**** * Department of Information Engineering, Università Politecnica delle Marche, Ancona, Italy ** Talent srl, Osimo, Italy *** LSIS- umr CNRS 6168, Laboratoire des Sciences de l'Information et des Systèmes Equipe I&M (ESIL) **** School of Computer Science, University College Dublin, Dublin, Ireland
  • 2. Why this research project? In the Educational Robotics (ER) field researchers have identified lack of quantitative analysis on how robotics can improve skills and increase learning achievements in students (Benitti 2012; Alimisis, 2013). Alimisis, D. (2013). Educational robotics: Open questions and new challenges. Themes in Science and Technology Education, 6(1), 63-71. Benitti, F. B. V. (2012). Exploring the educational potential of robotics in schools: A systematic review. Computers & Education, 58(3), 978-988.
  • 3. Machine learning and Data mining are everywhere! Made for me That's a prediction “of what Netflix thinks you may enjoy watching, based on your own unique tastes”.
  • 4. Real time monitoring of students’ activity (Registered Patent: n. 102018000009636, «APPARATO DI MONITORAGGIO DI OPERAZIONI DI ASSEMBLAGGIO E PROGRAMMAZIONE»)
  • 5. Case Study: Research Questions 1. identification of different patterns in the students’ problem-solving trajectories; 2. accurate prediction of students’ team final performance; and 3. correlation of the discovered patterns of students’ problem-solving with the evaluation given by the educators Applying data mining and machine learning methods to data collected from the educational environments can allow to predict and classify students’ behaviours and discover latent structural regularities to large educational dataset. Berland, M., Baker, R. S., & Blikstein, P. (2014). Educational data mining and learning analytics: Applications to constructionist research. Technology, Knowledge and Learning, 19(1-2), 205-220.
  • 6. Case Study: Participants and Procedure Students from seven Italian lower and higher secondary schools, located in the Emilia Romagna and Marche regions. The total number of students involved in this study is 197. The experimentation was carried out from March 2018 to March 2019.
  • 7. STUDENT’S ACTIVITY (Think – Make – Improve approach)
  • 8. Program the robot so that it covers a given distance (1 m), trying to be as precise as possible. Students’ teams involved in the research project had to take into account some constraints: • the amount of time within they had to design and test their solution (15 - 20 minutes); • the teams could test the programming sequence as many times as they wanted; • they were allowed to use measuring instruments only to measure some robot’s parameters (for example the radius of the wheel). Case Study: the Introductory Exercise EVALUATION • if the error was < 4 cm, the educator considered the challenge completed; • if the error was >= 4 cm the educator considered the challenge not completed.
  • 9. Case Study: how to represent the participants' activities in the robot programming activity Students’ teams designed 1113 programming sequences to solve the introductory Exercise. Each programming test realised by the students’ team can be represented as a vector composed by these 12 elements: ● Motors: the n° of Motor blocks in the sequence ● Loops: the n° of Loop blocks in the sequence. ● Conditionals: the n° of Conditional and Sensors blocks in the sequence. ● Others: the n° of blocks in the sequence belonging to different categories than Motors, Loops and Conditionals. ● Added: the n° of blocks added, compared to the previous sequence; ● Deleted: the n° of blocks deleted, compared to the previous sequence; ● Changed: the n° of blocks changed, compared to the previous sequence; ● Equal: the n° of the same blocks, compared to the previous sequence; ● Delta Motors: amount of change in Motor blocks parameters (first, second or third parameter), compared to the previous sequence (calculated only for blocks of the “Changed” category); ● Delta Loops: amount of change in Loop blocks parameters, compared to the previous sequence; ● Delta Conditionals: amount of change in Conditional blocks parameters, compared to the previous sequence; ● Delta Others: amount of change in Other blocks parameters, compared to the previous sequence.
  • 10. SUPERVISED APPROACH The feature matrix was created by calculating the mean value and the standard deviation for each indicator presented in the previous section, taking into account all the trials performed by a students’ team to solve an exercise; then, the authors compared the performances of four different machine learning algorithms: • Logistic Regression • Support Vector Machine • k-nearest neighbors • Random Forest classifier in the prediction of the students’ teams final result. MIXED APPROACH Characterized by the benefits of both supervised and unsupervised methods: a k-means algorithm was applied to calculate in which clusters the programming sequences could be divided. Following the clustering, the percentage of each cluster in the programming activity of the students’ groups was calculated and these new features were then used to create a feature matrix as an input for the previously cited four supervised algorithms. Case Study: Results
  • 11. Case Study: Results − Accuracy − Mean Precision (calculated considering the average value between precision in the prediction of students’ positive performance and negative performance) − Mean Recall (calculated considering the average value between the recall in the prediction of students’ positive performance and negative performance) − Mean F1 – Score (calculated considering the average value between the F1-score in the prediction of students’ positive performance and negative performance) To obtain these parameters a repeated 10-fold cross validation was performed, so that the average value and standard deviation of the previous four parameters repeating the 10-fold validation multiple times were calculated.
  • 13. Case Study: Problem solving patterns 0 5 10 15 20 25 30 Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6 Cluster 7 Cluster 8 Trials #ofprogrammingsequences Different teams' behaviours Mathematical / Planning Tinkering (with prevalent refining behaviuor) Tinkering (with significant high changes) Pearson correlation coefficient (between the features extracted with the k-means algorithms and the final results obtained by students’ teams) Two features show a statistically significant negative correlation: • number of trials (PCC = -0.48, p- value < 0.001) • percentage of sequences of the cluster named “HIGH MOTORS PARAMETERS CHANGE” (PCC = - 0.39, p-value < 0.01); High values of these features (typical for the “Tinkering with significantly high changes” teams) indicate higher probability of a negative performance.
  • 14. Case Study: Mathematical behaviour FINAL ERROR = 1 CM
  • 15. Case Study: Tinkering behaviour (with refining) FINAL ERROR = 1,5 CM
  • 16. 0 5 10 15 20 25 30 35 40 45 50 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 DeltaMotors Test number Case Study: Tinkering behaviour (with large changes) FINAL ERROR = 50 CM
  • 17. Conclusions • Compare the performance of the machine learning techniques already applied (SVM, Logistic Regression, KNN, Random Forest) with MLP Neural Network → Larger Dataset • Another improvement for future development of this study will be time tracking in the log files generated by the system. • Authors intend also to utilise recurrent neural network, in particular the long short-term memory autoencoders (a structure specifically designed to support sequences of input data), in order to translate the programming sequences created by students into fixed-length vectors (compress representation of the input data), maintaining high level of information content. • Another planned development is the update of the current system design with a personalised e- learning system: an educational recommender system could give real-time feedback to teachers and students involved in Educational Robotics activities, or propose personalised learning path to learners.
  • 18. Thank you! Questions? Contacts: Lorenzo Cesaretti l.cesaretti@pm.univpm.it David Scaradozzi d.scaradozzi@univpm.it Laura Screpanti l.screpanti@pm.univpm.it