Your SlideShare is downloading. ×
Sensing Opportunities and Zero Effort Applications for Mobile Health Persuasion
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Sensing Opportunities and Zero Effort Applications for Mobile Health Persuasion

4,825
views

Published on

This is my Mobile Health 2010 (#mh2010) talk that I gave on May 24th for the session: "The Sweet Spot of Behavior Change via Mobile Devices." …

This is my Mobile Health 2010 (#mh2010) talk that I gave on May 24th for the session: "The Sweet Spot of Behavior Change via Mobile Devices."

I use lots of animations, so I strongly encourage you to download the PowerPoint pptx here:
http://www.cs.washington.edu/homes/jfroehli/talks.html

Published in: Technology, Business

2 Comments
6 Likes
Statistics
Notes
No Downloads
Views
Total Views
4,825
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
82
Comments
2
Likes
6
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Maybe add in social persuasion here?Maybe write on arrow instead?
  • So, from a technology perspective, there’s two things going on here: (1) we have a sensing system that can track location and infer particular decision moments and (2) we have a navigation application that uses persuasion tactics to motivate behavior.
  • Thus, for these applications to work, there needs to be some _sensing_ and some _feedback_
  • one potential opportunity here is to use sensing systems that passively sense and infer human activities---http://www.adajournal.org/article/S0002-8223(02)90316-0/abstractEnergy Intake and Energy Expenditure: A Controlled Study Comparing Dietitians and Non-dietitiansCATHERINE M. CHAMPAGNE, PhD, RD, FADAa, GEORGE A. BRAY, MDa, APRIL A. KURTZ, MS, RDa, JOSEFINA BRESSAN RESENDE MONTEIRO, PhDb,ELIZABETH TUCKER, JULIA VOLAUFOVA, PhDa, JAMES P. DELANY, PhDahttp://content.nejm.org/cgi/content/abstract/327/27/1893Discrepancy between self-reported and actual caloric intake and exercise in obese subjectsSW Lichtman, K Pisarska, ER Berman, M Pestone, H Dowling, E Offenbacher, H Weisel, S Heshka, DE Matthews, and SB HeymsfieldInt J ObesRelatMetabDisord. 1999 Aug;23(8):881-8.Dietary underreporting is prevalent in middle-aged British women and is not related to adiposity (percentage body fat).Samaras K, Kelly PJ, Campbell LV.Twin Research and Genetic Epidemiology Unit, St Thomas' Hospital, London, UKhttp://www.ncbi.nlm.nih.gov/pubmed/10490791?ordinalpos=6&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_DefaultReportPanel.Pubmed_RVDocSum
  • In the last 20 years, sensing and activity inference has come a very long way—originally much work was looking at instrumenting the environment with sensors like cameras to infer what people were doing in that space; with the costs and sizes of hardware shrinking, the early 2000s were marked by instrumenting the person or clothing with wearable sensorsThese wearable sensors are now making their ways into our cell phones in the form of GPS, accelerometers, proximity sensors and cameras. The mobile phone is, of course the ideal platform because it’s nearly always on and always with us.
  • It’s a new source of measurement information for behavior change.
  • The sounds are acquired with a microphone located insidethe ear canal. This is an unobtrusive location widely accepted inother applications (hearing aids, headsets). To validate our method we present experimental results containing 3500 seconds of chewing datafrom four subjects on four different food types typically found in a meal.Up to 99% accuracy is achieved on eating recognition and between 80%to 100% on food type classification.
  • Sensing system need not be on phone; can also be communicated via the cloud.System used optical, ion selective electrical pH, and conductivity sensors in order to sense and classify liquid in a cup in a practical way. Feedback system can still be phone.Drinks make up a surprisingly large portion of daily caloric intake with some research suggesting that 21% of a person’s daily caloric intake comes from beverages (458 calories) [5]. These tend to be ‘optional’ calories, not consumed exclusively to satiate hunger, and thus potentially easier to eliminate or replace with healthier alternatives Automatic Classification of Daily Fluid Intake by Jonathan Lester, Desney Tan, Shwetak Patel, and A.J. Brush22 March 2010; IEEE Pervasive Health 2010 Proceedings
  • Coughing is the most frequently cited symptom when people seek medical advice in the united states…
  • In our lab, we’ve been exploring the use of commodity mobile phones to automatically detect and classify when a person coughs.Sensing system doesn’t have to be complicated; it’s the software algorithms that are smart.
  • Ratio of about 4 to 1 annotation hours to recording hours
  • The participants were informed to monitor….
  • similar findings for people estimating their caloric intake and the amount of exercise they get per week.If you can’t measure it, you can’t change it.http://www.adajournal.org/article/S0002-8223(02)90316-0/abstracthttp://content.nejm.org/cgi/content/abstract/327/27/1893http://www.ncbi.nlm.nih.gov/pubmed/10490791?ordinalpos=6&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_DefaultReportPanel.Pubmed_RVDocSum
  • One of the best established findings in psychology is the role of feedback on performance.Think about how ingrained feedback is in our environment. For example, driving:this type of feedback is always available, passively given allowing us to sort of pre-attentively recognize it
  • We want to create these sorts of passive awareness systems.
  • Passive sensing enables a new type of application space I call “zero-effort applications”Avoid the challenging problem of determining when is the right time to prompt people with information; instead surround the user with persuasive information in a very accessible manner.
  • The goal is for the primary interactions to be completely passive; for both ubifit and ubigreen
  • As Jon mentioned, we had real time access to participant’s phone data so we could monitor the state of their phones in real timeThis screen shot was taken on a Monday, towards the beginning of the week, and shows the initial stages of their transit behavior. [next slide] Here, the shot was taken towards the end of the week; you can see how participants’ icons progressed throughout the week. At the bottom left you can see that one participant reached the final state of the progression and saw the Northern Lights.
  • [The glanceable display] was a constant reminder…whereas if you didn’t have a screen [glanceable display], you probably—I wouldn’t think about it [physical activity] as much, you know, I think about it maybe subconsciously every time I look at my phone. {S5}
  • Abstractions allowed for playfulnessProgress was visibleAchievements were rewardedCould build collection of achievements
  • I don’t have time to go through all of the persuasive/influence tactics that we thought about… but it’s worth mentioning that we specifically didn’t use loss aversion as a motivation technique here…We wanted people to enjoy using and feel good about using the applications.
  • I would like to see some graph or raw data. - Participant 13
  • Consolvo conducted a 3-month study with a control and experimental group and found that those with the ambient display outperformed those without.
  • This talk was not meant to be prescriptive but rather to inspire thought and creativity around the ideas of passive sensing for human activities. And secondly, to think creatively about how information can be passively displayed to the user.In the next 3 – 5 years, I believe activity inference will be the key technological innovation in mobile phone technology.* New opportunities in passively sensing activitiesOne of the biggest problems in hospitals is hygiene and hand washing; we
  • The sounds are acquired with a microphone located insidethe ear canal. This is an unobtrusive location widely accepted inother applications (hearing aids, headsets). To validate our method we present experimental results containing 3500 seconds of chewing datafrom four subjects on four different food types typically found in a meal.Up to 99% accuracy is achieved on eating recognition and between 80%to 100% on food type classification.
  • ubigreen context-triggered survey
  • with] a web site, it’s so easy, ‘Oh, I didn’t do anything, I'm not going to click on it.’ It’s so easy to ignore it. But on the phone, you can’t really ignore it as easily…otherwise, it’s just…out of sight, out of mind. {S9}
  • Transcript

    • 1. sensing opportunities
      for mobile health persuasion
      jonfroehlich@uw.edu
      phd candidate in computer science
      university of washington
      mobile health conference
      stanford university, 05.24.2010
      design:
      use:
      build:
      ubicomp lab
      sustainability research
      university of washington
      university of washington
    • 2. sensing opportunities
      for mobile health persuasion
    • 3. sensing opportunities
      for mobile health persuasion
      Twin Falls is only 1.4 mi away and you’ll burn an estimated 170 calories round trip.
      Twin Falls is only 1.4 mi away.
      Twin Falls!
      Burn 170
      cal & reach your walking goal for the week!
    • 4. sensing opportunities
      for mobile health persuasion
      Kairostechnology that suggests a behavior at the most opportune moment
      -fogg, 2003
      Twin Falls is only 1.4 mi away and you’ll burn an estimated 170 calories round trip.
      Twin Falls is only 1.4 mi away.
      Twin Falls!
      Burn 170
      cal & reach your walking goal for the week!
    • 5. two things you need
      a method to passivelymonitor human activity
      a method to provide feedback about behavior
    • 6. self-report
      useful for measuring
      beliefs, feelings, goals
      simple
      low cost
      • burdensome
      • 7. people are not good at monitoring their own behaviors:
      • 8. eating [champagne, 2002]
      • 9. exercise [lichtman, 1992]
      • 10. routine activities [klasnja, 2008]
      • 11. coughing [liu et al., in submission]
    • activity inference
      a very brief history
      miluzzo et al., sensys
      lester et al., ijcai
      bao et al., pervasive
      want, pers com
      gavrila et al, c. vision
      2010
      2000
      1990
      instrumenting the environment
      instrumenting the person/clothing
      instrumenting the cell phone
    • 12. not just sensor hardware progressions
      • also, advances in machine learning
      • 13. ability to store lots of information
      • 14. constant connectivity / the cloud
    • just as location aware computing has ushered in a new era of mobile phone software
      so to will activity inference for future mobile phone generations
    • 15. running
      lester et al., ijcai 2005
      choudury et al., ieee pervasive 2008
    • 16. walking
      lester et al., ijcai 2005
      choudury et al., ieee pervasive 2008
    • 17. sitting
      lester et al., ijcai 2005
      choudury et al., ieee pervasive 2008
    • 18. transit modes
      patterson et al., ubicomp 2003
      zheng et al., ubicomp 2008
      reddy et al., sensor networks2009
    • 19. eating
      jawbone microphone
      eating
      microphone in ear detects when a person is eating with 99% accuracy
      amft et al., ubicomp 2007
      cheng et al., pervasive 2010
    • 20. identifying fluids
      instrumented cup
      79% classification accuracy
      68 different fluids including sodas, juices, beers, wines
      lester et al., pervasive health 2010
    • 21. coughing
      liu et al., in submission
    • 22. automatically detecting coughs with a commodity mobile phone
      microphone
      liu et al., in submission
    • 23. collecting and analyzing the cough dataset
      17 participants
      72 hours of naturalistic audio recording
      6 graduate students annotated recordings
      2542 coughs labeled by annotators
      84.4% of coughs were correctly classified
      0.7% false positive rate (3.3/hr)
      liu et al., in submission
    • 24. 19
      number of coughs
      measured vs. self-report
    • 25. inaccuracy of self-report
      20
      diff: mean (22.8/hr), std (33/hr)
      number of coughs
      measured vs. self-report
      diff: mean (22.8/hr), std (33/hr)
    • 26. two things you need
      a method to passivelymonitor human activity
      a method to provide feedback about behavior
    • 27. speedometer
      gas gauge
    • 28.
    • 29. zero effort applications
      for behavior change
      goal: minimize interaction costs
      approach: passive sensing + passive display
      basically, do the activities that you normally do and the mobile phone will automatically respond
    • 30. two examples:
      ubifit
      ubigreen
      encouraging fitness behaviors through passive sensing and feedback
      consolvo et al., chi 2008
      consolvo et al., ubicomp2008
      encouraging proenvironmental behaviors through passive sensing and feedback
      froehlich et al., chi 2009
    • 31. ubisystem components
      collects data about physical activities
      activity recognition device
      glanceable display
      phone wallpaper!
      +
      communicates data about physical activities
    • 32. ubisystem components
      towards zero effort applications
      collects data about physical activities
      activity recognition device
      interactive application
      glanceable display
      +
      +
      communicates data about physical activities
    • 33. ubifit
      personal ambient display
      walk
      cardio
      strength
      flexibility
      primary goal met
      alternate goal met
      recent goal met
    • 34. ubigreentracked 6 transit activities
      walk
      bike
      drive alone
      train
      carpool
      bus
      “green”
      “not-green”
      minimum activity duration: 7 minutes
      29
    • 35. ubigreen
      personal ambient display
      phone
      background
      (wallpaper)
      current activity
      evolving image
      value icon bar
    • 36. sense of anticipation for how story would unfold
    • 37. ubigreen
      personal ambient display
      tree
      design:
      03/30/08
      03/31/08
      03/31/08
      03/31/08
      03/31/08
      04/01/08
      04/01/08
      04/01/08
      04/01/08
      04/02/08
      04/02/08
      04/03/08
      04/03/08
      04/03/08
      04/03/08
      04/03/08
      04/03/08
      04/03/08
      04/04/08
      04/04/08
      04/04/08
      04/04/08
      04/05/08
      04/05/08
      04/06/08
      7:27 PM
      7:45 AMT-Mobile
      8:05 AMT-Mobile
      5:15 PMT-Mobile
      5:30 PMT-Mobile
      7:23 AMT-Mobile
      8:05 AMT-Mobile
      5:38 PMT-Mobile
      6:11 PMT-Mobile
      7:41 AMT-Mobile
      5:22 PMT-Mobile
      6:55 AMT-Mobile
      7:15 AMT-Mobile
      4:48 PMT-Mobile
      5:56 PMT-Mobile
      6:09 PMT-Mobile
      9:45 PMT-Mobile
      10:07 PMT-Mobile
      07:28 AMT-Mobile
      05:41 PMT-Mobile
      07:35 PMT-Mobile
      11:01 PMT-Mobile
      08:31 AMT-Mobile
      10:19 AMT-Mobile
      12:00 AMT-Mobile
      everything resets on sunday
    • 38. ubigreen
      personal ambient display
      polar bear
      design:
      03/30/08
      7:27 PM
      03/31/08
      03/31/08
      03/31/08
      03/31/08
      04/01/08
      04/01/08
      04/01/08
      04/01/08
      04/02/08
      04/02/08
      04/03/08
      04/03/08
      04/03/08
      04/03/08
      04/03/08
      04/03/08
      04/03/08
      04/04/08
      04/04/08
      04/04/08
      04/04/08
      7:45 AM
      8:05 AM
      5:15 PM
      5:30 PM
      7:23 AM
      8:05 AM
      5:38 PM
      6:11 PM
      7:41 AM
      5:22 PM
      6:55 AM
      7:15 AM
      4:48 PM
      5:56 PM
      6:09 PM
      9:45 PM
      10:07 PM
      07:28 AM
      05:41 PM
      07:35 PM
      11:01 PM
    • 39. Monday
      Saturday
    • 40. personal ambient display
      impressions of ubifit
      If you didn’t have a screen [display], I wouldn’t think about it [physical activity] as much… I think about it maybe subconsciously every time I look at my phone.
      - P5UF
      With a website, it’s so easy to ignore… it’s just out of sight, out of mind. But on the phone, you can’t really ignore it…
      - P9UF
    • 41.
    • 42. game mechanics
      measured progress
      playful
      collections
      virtual achievements
    • 43. loss aversion
    • 44. need for quantitative data
      I would like to see some graph or raw data.
      - P13UG
      quantitative data
      • builds trust system is working
      • 45. allows for self-comparison
      • 46. some people like it better
    • effectiveness of the ubifit glanceable display
      Best Fit
      Linear Trend
      Best Fit
      Linear Trend
      no glanceable display
      glanceable display
      study occurred over thanksgiving, christmas, and new years.
      40
    • 47. in conclusion
      imagine that you can sense….
    • 48. running
      lester et al., ijcai 2005
      choudury et al., ieee pervasive 2008
    • 49. walking
      lester et al., ijcai 2005
      choudury et al., ieee pervasive 2008
    • 50. sitting
      lester et al., ijcai 2005
      choudury et al., ieee pervasive 2008
    • 51. eating
      jawbone microphone
      eating
      amft et al., ubicomp 2007
      cheng et al., pervasive 2010
    • 52. lakefront property
      phone home screen:
      most valuable real estate in all of technology
    • 53. lakefront property
      phone home/lock screen:
      most valuable real estate in all of technology
    • 54. thanks to:
      sunny consolvo
      pedjaklasnja
      jameslanday
      ericlarson
      seanliu
      shwetakpatel
      thank you
      @jonfroehlich
      design:
      use:
      build:
      ubicomp lab
      university of washington
      university of washington
    • 55. extra slides
    • 56. sink usage
      froehlich et al., ubicomp 2009
      larson et al., pervasive & mobile computing 2010
    • 57. ubigreen sensing transit
      1
      3
      2
      wearable activity recognition device
      cell towers
      user
      Walk
      Bike
      Drive Alone
      Train
      Carpool
      Bus
      minimum activity duration: 7 minutes
      51
    • 58. precursor to ubifit
      pedometer cell phone fitness study
      consolvo, et al., chi 2006
    • 59. ubigreen context-triggered survey
      using the myexperience toolkit
      froehlich et al., mobisys 2007
      http://myexperience.sourceforget.net
    • 60. limitations
      of sensing
      can‘t infer thoughts, feelings, intentions
      can be expensive
      sensing may not yet exist for behavior
      froehlich et al., mobisys 2007
      http://myexperience.sourceforget.net
    • 61. intelmsp
      lester et al., ijcai 2005
    • 62. personal ambient display
      impressions of ubigreen
      It’s omnipresent
      - P9UG
      It definitely keeps you more aware of it [personal transportation]. You use your phone every single day so you know.
      - P6UG