This work presents an approach to assist teachers, tutors and students from online learning environments. It is a four-steps process called Pedagogical Recommendation Process that uses the coordinated efforts of human actors (pedagogical and technological specialists) and artificial actors (computational artifacts). The process' objective is to find relevant information in educational data to help creating personalized recommendations. Using the process it was possible to detect issues within a learning environment (UFAL Línguas), and discovered why some students were facing difficulties, and what other students were doing in order to succeed in the course. This information was used to personalize pedagogical recommendations.

1. A Systematic Approach for Providing
Personalized Pedagogical Recommendations
Based on Educational Data Mining
International Conference on Intelligent Tutoring System
ITS2014 – Honolulu, Hawaii
Ranilson Oscar Araújo Paiva
Ig Ibert Bittencourt
Alan Pedro da Silva
Seiji Isotani
Patrícia Jaques
ranilson@copin.ufcg.edu.br
ig.ibert@ic.ufal.br
alanpedro@ic.ufal.br
sisotani@icmc.usp.br
pjaques@unisinos.br

2. 2
Agenda
 Contextualization
 Problem
 Proposal
 Case Study
 Conclusion
 References
Agenda
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

3. 3
Grand Challenge 8: Learning for Life
 […] re-engage computer scientists to drive progress towards
computing machinery and devices, networks, software that
are actually up to the job so that every learner who wants to
learn can learn what they want, in the way they want,
whoever they are, whenever they are in the world, and
whatever their age or literacy level; and every person who
wants to teach can teach whoever wants to be taught in the
way they want to teach, whoever they are and wherever
they are in the world.
[UK COMPUTING RESEARCH COMMITTEE, 2008]
Contextualization
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

4. 4
Learning in the Future
 AAA Learning [BITTENCOURT, 2009]
 Learning available…
 To anyone
 From anywhere
 At anytime
 Relies on Information and Communication
Technologies
 Increasing number of online courses [CHRYSAFIADI,
2013]
 Online courses are based on Online Learning Environments
Contextualization
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

5. 5
Some Issues Found
Online environments almost inevitably produce huge
volume of data[…]
[FOLEY, 2006]
Web-based learning environments are able to record
most learning behaviors of the students, and are
hence able to provide a huge amount of learning
profile.
[ROMERO, 2007]
Problem
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

6. 6
Some Issues Found
80% of teachers surveyed felt uncomfortable using
computers, and reported difficulty in
understanding how to use technology to support
an engaging and meaningful learning environment.
[DUHANEY, 2000]
Problem
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

7. 7
Some Issues Found
Among the reasons for the lack of deeper technology knowledge by
teachers, the greatest is that it takes many hours of specific
training.
Teachers’ technology knowledge is limited, consequently applying it
to a meaningful learning context needs direction and support.
This support must come from a collaborative effort using the
combined knowledge base of teachers, technology facilitators and
other educational stakeholders.
[MOREHEAD, 2005]
Problem
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

8. 8
Some Questions Raised
1. How can we use educational data in order to
discover relevant educational information?
2. What can we do with the resulting information
in order to help teachers/tutors assisting their
students/groups?
3. How can we organize the responsibilities,
collaborations and tasks of those involved in the
process?
Problem
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

9. 9
Possible Answers
How can we use educational data in order to
discover relevant information/patterns?
The primary goal of Educational Data Mining
is to use large-scale educational datasets to better
understand learning and to provide information
about the learning process.
[ROMERO, 2011]
Problem
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

10. 10
Possible Answers
What can we do with the resulting information in
order to help teachers/tutors assisting their
students/groups?
Recommender Systems are software tools and
techniques providing suggestions for items to be of
use to a user. The suggestions provided are aimed at
supporting their users in various decision-making
processes.
[RICCI, 2011]
Problem
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

11. 11
Possible Answers
How can we organize the responsibilities, collaborations
and tasks of those involved in the process?
A Systematic Approach is a process used to determine
the viability of a project or procedure based on the
experiential application of clearly defined and repeatable
steps and an evaluation of the outcomes. Its goal is to
identify the most efficient means to generate consistent,
optimum results.
[INVESTORWORDS]
Problem
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

12. 12
Possible Answers
How can we organize the responsibilities, collaborations and
tasks of those involved in the process?
The Pedagogical Recommendation Process relies on the
coordinated efforts of specialists in the pedagogical and
technological domains and computational artifacts, using the
students’ interactional data as input, in order to personalize
pedagogical recommendations based on the detection and
understanding of pedagogical difficulties. Finally, the students'
performance is evaluated to check for progress.
[PAIVA, 2013]
Problem
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

13. Pedagogical Recommendation Process
13Proposal
Educational Data Mining
+
Recommender Systems
+
Systematic Approach
=
Pedagogical Recommendation
Process
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

14. Pedagogical Recommendation Process
14Proposal
STEP 1 – Detect Practices
• Practices are pedagogical
situations of interest
• Pedagogical Actor (PA)
• Defines alert values for important
educational data
• Technological Actor (TA)
• Maps the data defined by the PA
to the data in the learning
environment
• Computational Artifact (CA)
• Detects specific practices and
sends the alerts to the next step
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

15. Pedagogical Recommendation Process
15Proposal
STEP 2 – Discover Patterns
• Pedagogical Actor (PA)
• Defines the criteria to
confirm/reject that a particular
practice is happening
• Technological Actor (TA)
• Defines ways to pre-process, mine
and post-process educational data
to reach criteria defined by the PA
• Computational Artifact (CA)
• Operationalizes the data mining
process, based on the TA’s
definitions
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

16. Pedagogical Recommendation Process
16Proposal
STEP 3 – Recommend
• Pedagogical Recommendations are
reactive or preventive actions, to address a
particular pedagogical issues.
• Pedagogical Actor (PA)
• Creates recommendations to address
specific pedagogical issues discovered
• Technological Actor (TA)
• Creates ways to personalize and offer
teachers the recommendations, based
on the educational resources available
• Computational Artifact (CA)
• Operationalizes the TA’s creations,
returning the most relevant
recommendations for the issue found
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

17. Pedagogical Recommendation Process
17Proposal
STEP 4 – Monitor and Evaluate
• Pedagogical Actor (PA)
• Defines the process’ success
criteria
• Technological Actor (TA)
• Creates ways to monitor and
evaluate the students’ and
groups’ performance, checking
if the PA’s criteria were reached
• Computational Artifact (CA)
• Operationalizes the TA’s
creations
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

18. Case Study
18Case Study
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining
• Online environment for language learning
– Developed by the Federal University of Alagoas – Brazil
– Offered free online courses in a Gamified environment
• 2075 registration forms
– 200 submissions were accepted
• 100 Students from the University (UFAL)
• 100 Students from public schools (High School)
• 5 monthes course (October 2012 to February 2013)
– 3 Modules (With 2 Units Each)
– All classes were online
– Some exams were held in classroom
– 1 Teacher + 8 Tutors
• 780 megabytes of interactional data
– 1,200,000 RDF Triples

19. 19
Case Study – Detect Practices
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining
Case Study

20. 20
Case Study – Detect Practices
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining
Case Study

21. 21
Case Study – Detect Practices
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining
• Low Performance Interactions
– 0 to 1499 points
– 120 Students
– All failed
• Medium Performance Interactions
– 1500 to 3499 points
– 63 Students
– 40 failed / 23 succeeded
• High Performance Interactions
– 3500 to 5000 points
– 14 Students
– All succeeded
Case Study

22. 22
Case Study – Discover Patterns
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining
Case Study

23. 23
Case Study – Discover Patterns
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining
Case Study

24. 24
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining
Case Study – Discover Patterns
Case Study

26. 26
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining
Case Study – Recommend
Case Study

27. 27
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining
Case Study – Recommend
Case Study

28. 28
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining
Case Study – Recommend
Case Study

29. 29
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining
Case Study – Monitor and Evaluate
Case Study
• Pedagogical Actor (PA): Defines the process’ success criteria
• For example: a 20% improvement in a group’s performance
• Technological Actor (TA): Creates ways to monitor and
evaluate the students’ and groups’ performance, checking if the
PA’s criteria were reached
• For example: create computational ways to compare a group’s
performance before the recommendation, to its performance after the
recommendation and flag if the success threshold was achieved
• Computational Artifact (CA): Operationalizes the TA’s
creations
• For example: execute the TA’s program, signaling if a new process
iteration will be necessary (the issue is not solved)

30. 30
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining
Conclusion
Conclusion
The use of the Pedagogical Recommendation Process…
• Integrated and coordinated the efforts of education and
technology actors
• Enabled the discovery of situations where students were
facing difficulties
• Prepared the learning environment to identify and react to
some pedagogical situations.
• Can also be done in other courses based on the same
learning environment

31. References
1. BAYER, Jaroslav; BUDZOVSKA, Hana; GERYK, Jan; OBSIVAC, Tomas; POPELINSKY, Lubomir. Predicting drop-out from social behaviour
of students. Educational Data Mining Conference, 2012.
2. BITTENCOURT, Ig Ibert; COSTA, Evandro de Barros ; Marlos Silva ; SOARES, Elvys . A Computational Model for Developing Semantic
Web-based Educational Systems. Knowledge-Based Systems, 2009, vol. 22, pp. 302-315.
3. CHRYSAFIADI, Konstantina; VIRVOU, Maria. Student modeling approaches: A literature review for the last decade. Expert Syst. Appl.
40(11): 4715-4729 (2013).
4. GIBERT, Karina; IZQUIERDO, Joaqun; HOLMES, Geo; ATHANASIADIS, Ioannis; COMAS, Joaquim; SNCHEZ-MARR, Miquel. On the role
of pre and postprocessing in environmental data mining. International Congress on Environmental Modeling and Software, 2008.
5. KAVANAGH, J., AND HALL, W. Grand challenges in computing research 2008. In Grand challenges in computing research 2008 (United
Kingdom, 2008), GCCR'08, UK Computer Research Committee.
6. MORAN, Jose Manoel. O que aprendi sobre avaliação em cursos semipresenciais. In: SILVA, Marco; SANTOS, Edmea (Orgs). Avaliação
da Aprendizagem em Educação Online. S~ao Paulo: Loyola, 2006. Available at: http://www.eca.usp.br/prof/moran/aprendi.html.
Accessed in: 03/07/2013.
7. PAIVA, Ranilson O. Araújo; BITTENCOURT, Ig Ibert; PACHECO, Henrique; SILVA, Alan Pedro da; JAQUES, Patrcia; ISOTANI, Seiji.
Mineração de Dados e a Gestão Inteligente da Aprendizagem: Desaos e Direcionamentos in XXXII Congresso da Sociedade Brasileira de
Computação, 2012.
8. PARK, Ji-Hye; CHOI, Hee Jun. Factors Inuencing Adult Learners' Decision to Drop Out or Persist in Online Learning. Educational
Technology & Society, 2009, vol. 12, pp 207-217.
9. WITTEN, Ian; FRANK, Eibe; HALL, Mark. Data Mining: Practical Machine Learning Tools and Techniques. Elsevier, 2011. ed. 3.
Massachusetts.
References
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining

32. Thank you!
Obrigado!
ITS2014 | Honolulu, HAWAII - A Systematic Approach for Providing Personalized Pedagogical Recommendations Based on Educational Data Mining