Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

iSenseStress: Assessing Stress Through Human-Smartphone Interaction Analysis

696 views

Published on

Thank You for referencing this work, if you find it useful!

Reference/Citation: Matteo Ciman, Katarzyna Wac, Ombretta Gaggi, iSenseStress: Assessing Stress Through Human-Smartphone Interaction Analysis, 9th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth), Istanbul, Turkey, May 2015.

Published in: Health & Medicine
  • Be the first to comment

iSenseStress: Assessing Stress Through Human-Smartphone Interaction Analysis

  1. 1. iSenseStress: Assessing Stress Through Human-Smartphone Interaction Analysis Matteo Ciman1, Katarzyna Wac2 and Ombretta Gaggi1 1 University of Padua Padua, Italy 2 University of Geneva and University of Copenhagen Geneva, Switzerland and Copenhagen, Denmark
  2. 2. PervasiveHealth 2015, Istanbul, Turkey /26 Stress Experience • Stress is mental condition experienced every day • Long exposure can lead to anxiety, depression etc. => increase of healthcare costs • In 2013 American teens reported stress experienced at unhealthy levels (and at increasing lower ages) [http://www.apa.org/news/press/releases/stress/2013/teenstress.aspx] 2 • Early assessment of stress condition can help to provide feedback to improve health state of individuals
  3. 3. PervasiveHealth 2015, Istanbul, Turkey /26 Stress Assessment - State of the Art • Stress assessment using wearable and ubiquitous devices can increase individuals’s acceptance without interfering with their life • MouStress: project for stress assessment considering computer mouse movements or keyboard [1] • Usage smartphone sensors (WiFi, GPS, Bluetooth, calls, SMS) [2] 3 [1] D. Sun, P. Paredes, and J. Canny, “Moustress: Detecting stress from mouse motion,” in SIGCHI Conference on Human Factors in Computing Systems, 2014, pp. 61–70. [2] G. Bauer and P. Lukowicz, “Can smartphones detect stress-related changes in the behaviour of individuals?” in PERCOM Workshops, 2012.
  4. 4. PervasiveHealth 2015, Istanbul, Turkey /26 The idea 4
  5. 5. PervasiveHealth 2015, Istanbul, Turkey /26 Our Approach • No external devices used, just smartphone (less expensive, more usable) • No privacy-related information (i.e., calls, messages, location etc.) • Possible to run a phone background service all the day long • Based on human-smartphone interaction analysis • Limitation: an interaction with the smartphone is required to make an assessment 5
  6. 6. PervasiveHealth 2015, Istanbul, Turkey /26 Human-Smartphone Interaction 6 Tap Scroll Swipe Text Writing Double Tap Rotate Zoom Pinch Long press
  7. 7. PervasiveHealth 2015, Istanbul, Turkey /26 Tasks Definition • Search Task: • Scroll, swipe and tap • Write Task: • Tap, Text Writing 7 Tap Scroll Swipe Tap Text Writing
  8. 8. PervasiveHealth 2015, Istanbul, Turkey /26 Search Task • Find inside a 21x15 grid the right icon (s) • Scroll and Swipe to inspect all the icons • Tap to select the right icon 8
  9. 9. PervasiveHealth 2015, Istanbul, Turkey /26 Search Task Features • Tap {min, max, average} pressure / length / size • Scroll, Swipe • {min, max, average} speed / time length / acceleration / pixels length / pressure • Linearity • D(interaction, center), D(interaction, top_left_screen) 9
  10. 10. PervasiveHealth 2015, Istanbul, Turkey /26 Write Task • Paragraph writing as text message • Keyboard without autocorrection or word suggestion • English as text language 10
  11. 11. PervasiveHealth 2015, Istanbul, Turkey /26 Write Task Features • Tap {min, max, average} pressure, length, size • Tap movement and duration • Writing: • Speed • # errors • Back digits 11
  12. 12. PervasiveHealth 2015, Istanbul, Turkey /26 Protocol 12 Initial Relax (5’) Relaxed Tasks (~30’) Stressor (5’-10’) Stressed Tasks (~10’) Self Assessment Negative Valence Low energy Not Stressed Positive Valence High Energy Stressed 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 ESM 1 ESM 2 ESM 3 ESM 4 ESM 5
  13. 13. PervasiveHealth 2015, Istanbul, Turkey /26 Protocol (II) 13 Initial Relax (5’) Relaxed Tasks (~30’) Stressor (5’-10’) Stressed Tasks (~10’)
  14. 14. PervasiveHealth 2015, Istanbul, Turkey /26 How to Stress People • Most common used stressors for tests • Mathematical problems • Timing pressure • Social evaluation • Repetition • Uncontrollability 14 We used these
  15. 15. PervasiveHealth 2015, Istanbul, Turkey /26 Stressor Task: Math 15 Large prime number
  16. 16. PervasiveHealth 2015, Istanbul, Turkey /26 Stressor Task: Math (II) 16 Progress bar + tic-tac sound Random decrease Digits back to 0 every time Each wrong answer annoying sound and going back
  17. 17. PervasiveHealth 2015, Istanbul, Turkey /26 Search Task - Stressed 17 Progress bar + tic-tac sound Sound + vibration
  18. 18. PervasiveHealth 2015, Istanbul, Turkey /26 Write Task - Stressed 18 Progress bar + tic-tac sound
  19. 19. PervasiveHealth 2015, Istanbul, Turkey /26 User Study • 13 Participants (7M, 6F), average age 26,38 (± 2,53) • Own phone, no constraints for the type to use (Android OS) • Different English literacy level • Average protocol duration: 1 hour • Cover story: New Google interface analysis 19
  20. 20. PervasiveHealth 2015, Istanbul, Turkey /26 Stress Induction Analysis 20 Initial Relax Relaxed Tasks Stressor Stressed Tasks ESM 1 ESM 2 ESM 3 ESM 4 ESM 5 TEST t(13) p-value ESM 3 VS ESM 4 1.99 0,007 * ESM 3 VS ESM 5 -2.84 0,009 * ESM 4 VS ESM 5 -2.74 0.5 Participants were stressed Different stress level at the end of tasks Kept stressed during stress tasks
  21. 21. PervasiveHealth 2015, Istanbul, Turkey /26 Features Evaluation • Statistical analysis for significance evaluation • Stress prediction model using Decision Tree (DT), k- Nearest Neighbourhood (kNN), Bayes Network (BN), Support Vector Machine (SVM) and Neural Networks (NN) • User and global model (evaluated using 10-Fold cross validation and leave-one-out) 21
  22. 22. PervasiveHealth 2015, Istanbul, Turkey /26 Search Task - Statistical Correlation • Only weak correlation between our features • Global Model • Average swipe pressure (p-value = 0,09) • Scroll distance from center (p-value = 0,065) • Scroll distance from top left (p-value = 0,07) 22 • User model • Scroll interaction length (strong correlation for 61% of users) • Scroll delta (strong correlation for 40% of users) • Scroll linearity (strong correlation for 45% of users)
  23. 23. PervasiveHealth 2015, Istanbul, Turkey /26 Search Task - Prediction Model 23 F-measure for Scroll interaction models MODEL DT KNN SVM NN BN USER (AVERAGE) 0.79 0.80 0.81 0.80 0.77 GLOBAL (AVERAGE) 0.73 0.71 0.78 0.74 0.67 F-measure for Swipe interaction models MODEL DT KNN SVM NN BN USER (AVERAGE) 0.86 0.86 0.79 0.87 0.85 GLOBAL (AVERAGE) 0.92 0.75 0.81 0.82 0.77
  24. 24. PervasiveHealth 2015, Istanbul, Turkey /26 Write Task - Statistical Correlation • User Model • Digits size (64% of users with strong correlation) • Pressure/Size ratio (55% of users with strong correlation) • Global Model • Wrong Words / Total words ratio (p-value = 0,028) • Digits time distance (p-value = 0,012) • Digit duration (p-value = 0,08) 24
  25. 25. PervasiveHealth 2015, Istanbul, Turkey /26 Conclusions • Stress assessment using data from non-intrusive devices can increase people’ acceptance • Human-smartphone interaction analysis can be leveraged to assess stress state in users • Scroll and Swipe: F-measure of stress prediction between 79% and 85% for user models, and between 70% and 80% for global model. • Text writing: several features showed strong correlation 25
  26. 26. PervasiveHealth 2015, Istanbul, Turkey /26 Future works • Real-time background service for stress assessment • Behaviour suggestion implementation • Stress assessment in the wild (ongoing study, 29 participants) 26 • My PhD Thesis :)
  27. 27. iSenseStress: Assessing Stress Through Human-Smartphone Interaction Analysis Matteo Ciman, Katarzyna Wac and Ombretta Gaggi {mciman,gaggi}@math.unipd.it katarzyna.wac@unige.ch wac@di.ku.dk

×