The document discusses using learning analytics to detect student emotions and provide affective recommendations of learning resources. It describes collecting event data from virtual learning environments, using methods like hidden Markov models to classify emotions, and modifying collaborative filtering for recommendation based on detected emotions. The goal is to complete the learning analytics cycle by collecting data, analyzing it to detect affective states, and intervening with personalized recommendations. Future work includes integrating sensors to detect emotions and experiments to evaluate the approach.

1. Pushing the awareness
envelope: from interactions
to emotions
Israel Gutiérrez
Derick Leony
University Carlos III of Madrid

2. Awareness across spaces
with learning analytics
Data Process / Inter-
collection Analysis ventions
• Subject: Teachers and students
• Tool: Learning analytics with traces across spaces
• Objective:
• Recommend: classON and affective recommender

3. Guess what?
Number of months to reach 1 million users
Company 1
Company 2
0 2,5 5 7,5 10

4. Guess what?
Number of months to reach 1 million users
Company 1
Company 2
0 2,5 5 7,5 10

5. Guess what?
Number of months to reach 1 million users
Company 1
Company 2
0 2,5 5 7,5 10

6. Online learning and...
flipping the classroom
• Lectures at home
• Homework in class
• Let’s support this new
types of classes!!!

7. Flipped session
= computer lab session
• Observations in lab sessions
• Inefficient interactions
• Problems orchestrating the session
• Definition of metrics (ICALT’12)
• Waiting time
• Order

8. Proposed solution
Data Process / Inter-
collection Analysis ventions
Attach info
Events from Recommend
to augmented
web sensors interactions
physical space

9. http://www.class-on.org
• Awareness tool supporting teachers
• Context
• Lab sessions (f2f) = flipped session
• Regulate interactions
• Empower teacher for improving
orchestration

10. BART SIMPSON
LISA SIMPSON
Student interface

11. BUSY
2 3
5 2
Teacher interface

12. BUSY
BART SIMPSON LISA SIMPSON
Teacher interface

13. Orchestration aspects
that could be improved
• Based on the 5+3 aspects framework
(ECTEL’12)
• Management
• Intervention
• Assessment
• Design

14. classON Evaluation
• Quantitative data: not enough improvement
• Teachers perception
• info in a glance
• fair distribution of feedback
• Students perception
• more fair (time/order)
• trust the help is coming

15. classON Further Steps
• Suggest peer support interactions
• New strategies for feedback dispatching
• Annotation system
• Exploitation of qualitative data in questions &
answers
• New experiment(authoring tool): Superpowers to
teachers (workshop at Madrid Science Week)
• Visualization + gamification (explored in my stay at
KUL, but in the context of Research 2.0)

16. Ambient displays
• Not the main focus of • Design factors
user attention
• Appeal (usability +
• Used for aesthetics)
• Awareness • Learnability:
match user’s
• Persuade expectations
• Contexts
• Awareness: match
user’s interests
• Health / good habits • Distraction: low

17. Gamification
• Applying game design and game mechanics
to non-game contexts
• Engage users in a new system
• Engage users keep using the system
• Metaphors for ambient displays using game
concepts

18. GambientFication
Ambient displays + Gamification
(Powered by Analytics)
• KU Leuven group
• TiNYARM
• Use iPads as ambient displays
• Use during working hours
• Use a holder as a photo frame
• 10 last days of the stay

21. Evaluation
• Awareness of research activity in the short/
medium/long term
• Appeal (usability + aesthetics)
• Distraction
• User perception
• Awareness/reflection/sense-making/
behavior change

22. And now, let’s jump to
emotions...

23. Context
● Life-cycle of Learning Analytics process
Data Process / Inter-
Collection Analysis ventions
Events from
Virtual URLs
Machine Bash commands
Files
Compilations
13/09/2012 EEE Meeting 2012 1

24. Data collection
● Collect events from:
– Web browser
– Text editor
– Programming
tools
– Session
● More information:
– Partial grades
– Forums
13/09/2012 EEE Meeting 2012 2

25. Problem statement
● Detecting emotions in educational settings
– Affective computing (Picard et al., 1997)
– Use of physical sensors
– Applied mainly in ITS
● Recommender systems in TEL
– SoA Review (Draschler et al., 2011)
– There are no implementations that
consider the affective state of the learner
13/09/2012 EEE Meeting 2012 3

26. Objectives
● Detect learner emotions from the events
generated within a learning environment
● Recommend learning resources based on
the detected emotions (and other variables)
● Reflecting affective states into the virtual
environment
● Providing awareness of affective states
13/09/2012 EEE Meeting 2012 4

27. Proposed solution
● Complete Learning Analytics cycle
Data Process / Inter-
Collection Analysis ventions
Events from Detecting Recommend
Virtual affective Learning
Machine state Resources
13/09/2012 EEE Meeting 2012 5

28. Classification of emotions
● Two prevalent ways:
– Two-dimensional features (valence, arousal)
– Discrete sets (e.g. Ekman's basic emotions:
anger, disgust, fear, joy, sadness, surprise)
● Learning-domain set proposed by D'Mello et al.
(2007): Boredom, confusion, frustration, eureka,
motivation, neutral
13/09/2012 EEE Meeting 2012 6

29. Processing events
● Input: Events collected during a working session
● Methods: Hidden Markov Models, GSP
13/09/2012 EEE Meeting 2012 7

30. Intervention: Recommend
resources
● Resource recommendation
● User-based (vs item) collaborative filtering
● Modified process:
– Define preference model
– Calculate user similarity including emotions
– Select neighbors of learner
– Obtain list of recommended items
– Filter recommendations according to current
emotion
● Implementation based on Apache Mahout
13/09/2012 EEE Meeting 2012 8

31. Evaluation
● Affective state detection
– Learner feedback
– Instructor feedback
– External sensors:
● Galvanic skin response
● Facial gestures
● Resource recommendation
– Information system metrics:
● Precision and recall
13/09/2012 EEE Meeting 2012 9

32. First Prototype
● Widget: Learning
Resource Affective
Recommender
● Among the winners of
the 3rd ROLE Widget
Enchantment
● Paper accepted in
workshop MATEL 2012
● Demo available!
13/09/2012 EEE Meeting 2012 10

33. Future work
● Implementing two sensors for emotion: Galvanic skin response
(Arduino-based), facial gestures (OpenCV-based)
● Analyzing the inclusion of other variables:
– Grades, session duration, amount of events
● Experiments planned:
– Detection: Madrid's Science Week 2012
– Recommendation: C programming class
● Deployment on Amazon EC2
● Integration with LearnGLASS
13/09/2012 EEE Meeting 2012 11

34. Thank you!
Questions?
13/09/2012 EEE Meeting 2012 12