Overview of the current research around emotionsense.org

1. how can sensor data be used to
infer how people feel?
@neal_lathia
university of cambridge

2. how many smartphones do
you have?

3. how often do you carry your
phone with you (%) ?

4. how often do you carry your
phone with you (%) ?
research: 90% of the time it is
in the same room as you
making it a powerful, personal
device that can sense,
monitor, and interact with you

5. sense (infer): activities, social interactions,
mobility, ambient sound
in the moment

6. monitor: green transportation habits and modes,
fitness levels
individuals over time

7. interact: social network broadcasts, ubiquitous
technology evaluation, persuasive interfaces
understand & persuade

8. now: a tool for psychology research
“ubiquitous, intimate, sensor-rich, computationally
powerful, remotely accessible”

9. sense
infer
mental
processes
interact

11. infers
sensed “in the reported
context moment” mood
(responds)

12. inference →
challenges
system →
challenges
user →
challenges

13. system →
challenges
(kiran rachuri)

14. battery
smartphone
processing sensors
system → weren't built
challenges memory for continuous
monitoring
network

15. system → challenges
:: adaptive sensor sampling
sense sleep
co-location -2% accuracy - 40% accuracy
-45% battery -5 % battery
continuous adaptive interval (50%)
accelerometer -1% accuracy - 34% accuracy
-44% battery -8 % battery

16. inference →
challenges
(ubicomp 2010)

17. inference → challenges
:: machine learning {sensed context} → emotion
5 broad emotions
microphone emotional
sample prosody-based
model
MAP likelihood
+ 71% accuracy
daily diary inferred emotion

18. user →
challenges

19. user → challenges
:: protocols for experience sampling
experience sampling
signal respond
compliance, bias, delay

20. inference →
challenges
system → putting it
challenges together
user →
challenges

21. emotion
conflict? inference
learning
adaptive putting it conflict?
sampling together?
conflict? experience
sampling

22. {sensed context} → {current emotion}
in the moment

23. {sensed context} → { }
{sensed context} →
{current emotion}
{ } → {current emotion}

24. {sensed context} →
design for sensor data
{current emotion}
design for participant
compliance

25. design for sensor data
{sensed context} →
just finishing: a/b {current emotion}
test with a variety of
triggers. results?
design for participant
compliance

26. so, to conclude..

27. you
who can
misinterpret
your emotions?
a psychologist your social circles

28. you your phone
who can
misinterpret
your emotions?
a psychologist your social circles

29. infers
sensed “in the reported
context moment” mood
responds

30. emotionsense.org
@neal_lathia
university of cambridge