Contextual Dissonance: Design Bias in Sensor-Based Experience Sampling Methods

918 views

Published on

Paper presented at #ubicomp13. Full paper pdf here: http://www.cl.cam.ac.uk/~nkl25/publications/papers/lathia_ubicomp13.pdf

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
918
On SlideShare
0
From Embeds
0
Number of Embeds
22
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Contextual Dissonance: Design Bias in Sensor-Based Experience Sampling Methods

  1. 1. Contextual Dissonance: Design Bias in Sensor-Enhanced Experience Sampling Methods @neal_lathia, k. rachuri, c. mascolo (@cecim), j. rentfrow computer laboratory, university of cambridge #ubicomp13
  2. 2. Contextual Dissonance?
  3. 3. Contextual Dissonance: Design Bias in Sensor-Enhanced Experience Sampling Methods
  4. 4. You are tasked with researching X (e.g., X = emotions) in daily life using ubiquitous tech; so you decide to build a system that will: ● Ask participants for assessments of the X they experience ● Collect sensor data to 'objectively' measure participants' contexts and quantify their behaviour Research Scenario
  5. 5. why would you do this? ● … to explore whether machine learning approaches could infer people's subjective responses/complex behaviours ● … to understand the extent that the broad set of sensor data reflects self- reported behaviour
  6. 6. “...automated tracing is widely used to provide insight into what and when; however, it does not provide the why...” Froehlich et al.
  7. 7. Contextual Dissonance: Design Bias in Sensor-Enhanced Experience Sampling Methods
  8. 8. Contextual Dissonance: Design Bias in Sensor-Enhanced Experience Sampling Methods
  9. 9. “...researchers are faced with concrete decisions regarding design [...] studies have often been classifed into the three categories of interval-, signal-, and event-contingent protocols...” Bolger et. al ESM design: how should I ask questions?
  10. 10. “...sampling to capture data from the sensors of the phone cannot be performed continuously, as this will drain the battery rapidly. However, conservative sampling leads to the loss of valuable behavioural data...” K. Rachuri sensor design: how should I sample from sensors?
  11. 11. Both of these design protocols will affect the quantity and quality of data that you receive from participants.
  12. 12. ● Shouldn't sense everything all the time: triggers a survey based on a particular sensor ● Ask for subjective responses and, while doing so, sample data from other sensors to gather behavioural signals Research Scenario
  13. 13. We built a system like this. It includes: sensor data collection, ESM interfaces, etc., and remote reconfguration.
  14. 14. Open Source Smartphone Libraries for Computational Social Science N. Lathia, K. Rachuri, C. Mascolo, G. Roussos. 2nd ACM Workshop on Mobile Systems for Computational Social Science. as an aside...
  15. 15. 22 users; 1-month; questions about mood & current context (location, sociability); background sensing from many sensors; triggers remotely reconfgured weekly.
  16. 16. Contextual Dissonance: Design Bias in Sensor-Enhanced Experience Sampling Methods … by asking the “what if” question:
  17. 17. Your ESM protocol is driven by the accelerometer's state: questionnaires will be triggered based on when the participant is moving. Example Research Scenario
  18. 18. ...skews sampling towards the later hours of the day
  19. 19. But all sensors have their own distribution... … so how have I skewed my other results?
  20. 20. P( state(sensor1) = b | state(accelerometer) = a) ~ P( state(sensor1)) = b
  21. 21. Data that I would have received by continuous sampling
  22. 22. Data received by triggering on one sensor's state.
  23. 23. Bias?
  24. 24. Accelerometer ~ Non-Stationary 10.61% of the data is non-stationary. When it is, participants are: 95.23% non-silent; 39.24% at home; 14.43% communicating with others.
  25. 25. Full Sample vs. Accelerometer Trigger Non-silent? 37.78% | 95.23% Communicating with others? 4.60% | 14.43%
  26. 26. More Examples? Microphone ~ Silent/Non-Silent Accelerometer ~ Moving/Not-Moving Location ~ Home/Away Screen ~ Using the device SMS/Calls ~ Communicating with others Proximity ~ Near the phone
  27. 27. Microphone ~ Non-Silent 37.78% of the data is non-silent. When it is, participants are: 26.75% non-stationary; 47.12% at home; 9.48% communicating with others.
  28. 28. Full Sample vs. Microphone Trigger Moving? 10.61% | 26.75% Communicating with others? 4.60% | 9.48%
  29. 29. Dissonance; a tension or clash resulting from the combination of two disharmonious elements
  30. 30. Dissonance; between using sensor states to trigger ESM surveys while using sensor data to quantify context and behaviour.
  31. 31. Ok; so replace the accelerometer trigger with sampling uniformly across time. Example Research Scenario
  32. 32. temporal sampling is more likely to fnd your participant in “dominant” contexts, e.g., at home.
  33. 33. But the response data I get back from participants will not be affected by the choices that I make... right? Research Scenario
  34. 34. 1-month; 4 groups with random weekly trigger orders: (a) screen, (b) communication events, (c) immediately during non-silence, (d) some time after non-silence
  35. 35. “4 of the 6 tests found that the negative affect ratings (and 2 out of 6 for the positive ratings) were signifcantly different from one another with at least 90% confdence.”
  36. 36. who are you with? alone 33.33% of the time (screen trigger) to 60.77% of the time (microphone trigger)
  37. 37. Contextual Dissonance: Design Bias in Sensor-Enhanced Experience Sampling Methods
  38. 38. Where do we go from here? Opportunities for more research...
  39. 39. Generalise sensor- enhanced experience sampling tool. Currently in alpha testing.
  40. 40. Working with Android sensors? Try out library! One of the goals is to enable easy and quick access to sensor data in 2 lines of code. https://github.com/nlathia/SensorManager
  41. 41. Contextual Dissonance: Design Bias in Sensor-Enhanced Experience Sampling Methods @neal_lathia, k. rachuri, @cecim, j. rentfrow ACM Ubicomp 2013
  42. 42. References ● Smyth and Stone. “Ecological Momentary Assessment Research in Behavioral Medicine.” Journal of Happiness Studies 2003. ● Froehlich et al. “MyExperience: A System for In Situ Tracing and Capturing User Feedback on Mobile Phones.” ACM MobiSys 2007. ● Froehlich et al. “UbiGreen: Investigating a Mobile Tool for Tracking and Supporting Green Transportation Habits” ACM CHI 2009. ● Rachuri. “Smartphones Based Social Sensing: Adaptive Sampling, Sensing and Computation Offloading.” PhD Thesis 2013. ● Bolger et. al. “Diary Methods: Capturing Life as it is Lived” Ann. Rev. Psychology 2003.

×