Presentation given at the EEE meeting in Barcelona 13/09/2012 By Derick Leony and Israel Gutiérrez

1. Pushing the awareness
envelope: from interactions
to emotions
Israel Gutiérrez
Derick Leony
University Carlos III of Madrid

2. Awareness across spaces
with learning analytics
Data Process / Inter-
collection Analysis ventions
• Subject: Teachers and students
• Tool: Learning analytics with traces across spaces
• Objective:
• Recommend: classON and affective recommender

3. Guess what?
Number of months to reach 1 million users
Company 1
Company 2
0 2,5 5 7,5 10

4. Guess what?
Number of months to reach 1 million users
Company 1
Company 2
0 2,5 5 7,5 10

5. Guess what?
Number of months to reach 1 million users
Company 1
Company 2
0 2,5 5 7,5 10

6. Online learning and...
flipping the classroom
• Lectures at home
• Homework in class
• Let’s support this new
types of classes!!!

7. Flipped session
= computer lab session
• Observations in lab sessions
• Inefficient interactions
• Problems orchestrating the session
• Definition of metrics (ICALT’12)
• Waiting time
• Order

8. Proposed solution
Data Process / Inter-
collection Analysis ventions
Attach info
Events from Recommend
to augmented
web sensors interactions
physical space

9. http://www.class-on.org
• Awareness tool supporting teachers
• Context
• Lab sessions (f2f) = flipped session
• Regulate interactions
• Empower teacher for improving
orchestration

10. BART SIMPSON
LISA SIMPSON
Student interface

11. BUSY
2 3
5 2
Teacher interface

12. BUSY
BART SIMPSON LISA SIMPSON
Teacher interface

13. Orchestration aspects
that could be improved
• Based on the 5+3 aspects framework
(ECTEL’12)
• Management
• Intervention
• Assessment
• Design

14. classON Evaluation
• Quantitative data: not enough improvement
• Teachers perception
• info in a glance
• fair distribution of feedback
• Students perception
• more fair (time/order)
• trust the help is coming

15. classON Further Steps
• Suggest peer support interactions
• New strategies for feedback dispatching
• Annotation system
• Exploitation of qualitative data in questions &
answers
• New experiment(authoring tool): Superpowers to
teachers (workshop at Madrid Science Week)
• Visualization + gamification (explored in my stay at
KUL, but in the context of Research 2.0)

16. Ambient displays
• Not the main focus of • Design factors
user attention
• Appeal (usability +
• Used for aesthetics)
• Awareness • Learnability:
match user’s
• Persuade expectations
• Contexts
• Awareness: match
user’s interests
• Health / good habits • Distraction: low

17. Gamification
• Applying game design and game mechanics
to non-game contexts
• Engage users in a new system
• Engage users keep using the system
• Metaphors for ambient displays using game
concepts

18. GambientFication
Ambient displays + Gamification
(Powered by Analytics)
• KU Leuven group
• TiNYARM
• Use iPads as ambient displays
• Use during working hours
• Use a holder as a photo frame
• 10 last days of the stay

21. Evaluation
• Awareness of research activity in the short/
medium/long term
• Appeal (usability + aesthetics)
• Distraction
• User perception
• Awareness/reflection/sense-making/
behavior change

22. And now, let’s jump to
emotions...

23. Context
● Life-cycle of Learning Analytics process
Data Process / Inter-
Collection Analysis ventions
Events from
Virtual URLs
Machine Bash commands
Files
Compilations
13/09/2012 EEE Meeting 2012 1

24. Data collection
● Collect events from:
– Web browser
– Text editor
– Programming
tools
– Session
● More information:
– Partial grades
– Forums
13/09/2012 EEE Meeting 2012 2

25. Problem statement
● Detecting emotions in educational settings
– Affective computing (Picard et al., 1997)
– Use of physical sensors
– Applied mainly in ITS
● Recommender systems in TEL
– SoA Review (Draschler et al., 2011)
– There are no implementations that
consider the affective state of the learner
13/09/2012 EEE Meeting 2012 3

26. Objectives
● Detect learner emotions from the events
generated within a learning environment
● Recommend learning resources based on
the detected emotions (and other variables)
● Reflecting affective states into the virtual
environment
● Providing awareness of affective states
13/09/2012 EEE Meeting 2012 4

27. Proposed solution
● Complete Learning Analytics cycle
Data Process / Inter-
Collection Analysis ventions
Events from Detecting Recommend
Virtual affective Learning
Machine state Resources
13/09/2012 EEE Meeting 2012 5

28. Classification of emotions
● Two prevalent ways:
– Two-dimensional features (valence, arousal)
– Discrete sets (e.g. Ekman's basic emotions:
anger, disgust, fear, joy, sadness, surprise)
● Learning-domain set proposed by D'Mello et al.
(2007): Boredom, confusion, frustration, eureka,
motivation, neutral
13/09/2012 EEE Meeting 2012 6

29. Processing events
● Input: Events collected during a working session
● Methods: Hidden Markov Models, GSP
13/09/2012 EEE Meeting 2012 7

30. Intervention: Recommend
resources
● Resource recommendation
● User-based (vs item) collaborative filtering
● Modified process:
– Define preference model
– Calculate user similarity including emotions
– Select neighbors of learner
– Obtain list of recommended items
– Filter recommendations according to current
emotion
● Implementation based on Apache Mahout
13/09/2012 EEE Meeting 2012 8

31. Evaluation
● Affective state detection
– Learner feedback
– Instructor feedback
– External sensors:
● Galvanic skin response
● Facial gestures
● Resource recommendation
– Information system metrics:
● Precision and recall
13/09/2012 EEE Meeting 2012 9

32. First Prototype
● Widget: Learning
Resource Affective
Recommender
● Among the winners of
the 3rd ROLE Widget
Enchantment
● Paper accepted in
workshop MATEL 2012
● Demo available!
13/09/2012 EEE Meeting 2012 10

33. Future work
● Implementing two sensors for emotion: Galvanic skin response
(Arduino-based), facial gestures (OpenCV-based)
● Analyzing the inclusion of other variables:
– Grades, session duration, amount of events
● Experiments planned:
– Detection: Madrid's Science Week 2012
– Recommendation: C programming class
● Deployment on Amazon EC2
● Integration with LearnGLASS
13/09/2012 EEE Meeting 2012 11

34. Thank you!
Questions?
13/09/2012 EEE Meeting 2012 12