2013 07 05 (uc3m) lasi emadrid rcrespo igutierrez uc3m analitica aprendizaje ayuda just in-time teaching

1. Learning Analytics support for
Just-in-time Teaching
Raquel M. Crespo
Universidad Carlos III de Madrid (Spain)
<rcrespo@it.uc3m.es>
#lasi
#emadridnet
5 July 2013 1LASI-Madrid

2. Agenda
• Introduction
• Learning analytics for just-in-time teaching
• The classON system
– Functionality
– Implementation
– Experimental results
• Conclusions and future work
5 July 2013 LASI-Madrid 2

3. Learning Analytics
“The measurement, collection, analysis
and reporting of data about learners and
their contexts, for purposes of
understanding and optimizing learning and
the environments in which it occurs.”
SoLAR
5 July 2013 LASI-Madrid 3

4. Applications
• Predicting learner performance
• Learner modelling
• Recommendation of learning resources
• Enhance social learning environment
• Detect undesired learner behaviours
• Detect affects of the learner
• Increase learner awareness and reflection
5 July 2013 LASI-Madrid 4

5. Awareness Applications
• Awareness about student accesses to
learning resources (Mazza & Milani, 2005)
• Visualization techniques to make students
activities in the course and inform the self-
reflection (Govaerts, Verbert, Klerkx, &
Duval, 2010)
• Visualization-based awareness: GLASS
(Leony, Pardo, De la Fuente Valentín, De
Castro, & Delgado Kloos, 2012)
5 July 2013 LASI-Madrid 5
Mostly in
eLearning

6. Awareness
5 July 2013 LASI-Madrid 6
• Knowing who is around, what they
do in terms of activities and
interactions - CSCW
(Dourish & Bly, 1992)
• Lamps as distributed awareness
mechanisms in recitations
sections
(Alavi, Dillenbourg, & Kaplan, 2009)
• System that minimizes learning
progress differences in software
teaching classes
(Dong & Hwang, 2012)

7. Just-in-time teaching
Teaching and learning strategy
(Novak, Patterson, Gavrin, & Enger, 1998)
Adaptive Hypermedia & ITS
(Brusilovsky & Peylo, 2003)
Personalization,
active learning,
constructivism
5 July 2013 LASI-Madrid 7

8. Context
• Active learning activities
• Lab sessions using computers (f2f)
• Intensive tutoring support
• Engineering Degrees
• Overcrowded classes
photo: berkeleylab at flickr
5 July 2013 LASI-Madrid 8

9. The traditional approach
Students work at lab supported by the teacher’s
personalized/general explanations
• Face-to-face
• Direct interaction
• Progress monitoring?
• Questions solving?
• Particularly in overcrowded classes.
photo: vickicaruana.blogspot.com.es/2011/01/are-you-afraid-to-raise-your-hand.html
5 July 2013 LASI-Madrid 9

10. Problems
Collected from literature, experts and experiments
• Teacher
– … overwhelmed with questions
– … not efficiently answering them
– … not aware of students learning situation
• Students
– … monitor teacher to get their questions answered
– … not meeting the objectives of the session
photo: vickicaruana.blogspot.com.es/2011/01/are-you-afraid-to-raise-your-hand.html
5 July 2013 LASI-Madrid 10

11. Requirements
• Personalized monitoring
• Teacher awareness to
decide interventions
• Improve teacher efficiency
• Make students confident
that will be helped so they
make well use of the
session time
• Formative assessment
• Storage of learning traces
for summative assessment
5 July 2013 LASI-Madrid 11
Image source:
http://vizconsult.wordpress.com/2011/03/09/requirements-definition/

12. classON
in-Class Live Analytics for Assessment and Orchestration
• Just-in-time teaching support
• Supporting technologies
– Mobile devices
– Visualization techniques
– Learning analytics
• Web-based solution
– Student interface
– Teacher interface
5 July 2013 LASI-Madrid 12
www.class-on.org

13. 5 July 2013 LASI-Madrid 13
classON architecture

14. Student UI
5 July 2013 LASI-Madrid 14

15. Student UI
5 July 2013 LASI-Madrid 15
Navigation
Helper
Assignment

16. Student UI
5 July 2013 LASI-Madrid 16
Questions
& Answers

17. Teacher UI
5 July 2013 LASI-Madrid 17

18. Teacher UI
5 July 2013 LASI-Madrid 18

19. Teacher UI
5 July 2013 LASI-Madrid 19
Question
Tutoring time
Progress
Students

20. 5 July 2013 LASI-Madrid 20
classON in action

21. Analytics
about questions
5 July 2013 LASI-Madrid 21

22. Analytics about learning traces
5 July 2013 LASI-Madrid 22

23. Experimental results
Multimedia
Applications
Systems
architecture
Multimedia
Applications
Date 2012 2012 2013
Teachers 4 3 1
Students 70 121 30
Group settings Pairs Pairs Pairs
• Data sources
– Interviews with teachers
– Students surveys
– Traces collected by the system
5 July 2013 LASI-Madrid 23

24. Experimental results
0%
10%
20%
30%
40%
50%
60%
70%
80%
Solved
quickly
Solved but
waiting
Unresolved
Help requests
classON No classON Other
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
classON No classON Other
Fairness in doubt solving
Yes No (time) No (order) No (both)
5 July 2013 LASI-Madrid 24

25. Lessons learned
5 July 2013 LASI-Madrid 25

26. Conclusions
• Positive evaluation
& valuable suggestions by
– Students
– Teachers
– TEL experts discussion groups
• Q&A by students expected to
reduce help requests
– Reduce waiting time
– Reduce unresolved doubts
5 July 2013 LASI-Madrid 26
Collected results validate that classON fulfills the expected
requirements, and thus helps teachers and students to make the
most of the learning sessions with scarce resources.

27. Future Work
• New metrics
– Students comparison with the mean of the class
• Gamification
– of the activities in the assignment, Q&A
• Application to other contexts
– Project-based learning
– Extension to massive face-to-face environemnts
5 July 2013 LASI-Madrid 27

28. Related publications
• Gutiérrez Rojas, I., & Crespo García, R. M. (2012). Towards efficient
provision of feedback in lab sessions. International Conference on
Advanced Learning Technologies (ICALT) (Vol. 1). Rome.
• Gutiérrez Rojas, I., Crespo García, R. M., & Delgado Kloos, C. (2011).
Orchestration and feedback in lab sessions: improvements in quick
feedback provision. Towards Ubiquitous Learning. 6th European
Conference of Technology Enhanced Learning, EC-TEL 2011
Palermo, Italy, September 20-23, 2011 Proceedings. LNCS, 6964.
doi:10.1007/978-3-642-23985-4
• Gutiérrez Rojas, I., Crespo García, R. M., & Delgado Kloos, C. (2012).
Enhancing orchestration of lab sessions by means of awareness
mechanisms. EC-TEL 2012. Saarbrücken.
5 July 2013 LASI-Madrid 28

29. 5 July 2013 LASI-Madrid 29
Thanks for your
attention!
Questions?

30. Learning Analytics support for
Just-in-time Teaching
Raquel M. Crespo
Universidad Carlos III de Madrid (Spain)
<rcrespo@it.uc3m.es>
#lasi
#emadridnet
5 July 2013 30LASI-Madrid