Sensing Opportunities and Zero Effort Applications for Mobile Health Persuasion

5,363
-1

Published on

This is my Mobile Health 2010 (#mh2010) talk that I gave on May 24th for the session: "The Sweet Spot of Behavior Change via Mobile Devices."

I use lots of animations, so I strongly encourage you to download the PowerPoint pptx here:
http://www.cs.washington.edu/homes/jfroehli/talks.html

Published in: Technology, Business
2 Comments
7 Likes
Statistics
Notes
No Downloads
Views
Total Views
5,363
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
86
Comments
2
Likes
7
Embeds 0
No embeds

No notes for slide
  • Maybe add in social persuasion here?Maybe write on arrow instead?
  • So, from a technology perspective, there’s two things going on here: (1) we have a sensing system that can track location and infer particular decision moments and (2) we have a navigation application that uses persuasion tactics to motivate behavior.
  • Thus, for these applications to work, there needs to be some _sensing_ and some _feedback_
  • one potential opportunity here is to use sensing systems that passively sense and infer human activities---http://www.adajournal.org/article/S0002-8223(02)90316-0/abstractEnergy Intake and Energy Expenditure: A Controlled Study Comparing Dietitians and Non-dietitiansCATHERINE M. CHAMPAGNE, PhD, RD, FADAa, GEORGE A. BRAY, MDa, APRIL A. KURTZ, MS, RDa, JOSEFINA BRESSAN RESENDE MONTEIRO, PhDb,ELIZABETH TUCKER, JULIA VOLAUFOVA, PhDa, JAMES P. DELANY, PhDahttp://content.nejm.org/cgi/content/abstract/327/27/1893Discrepancy between self-reported and actual caloric intake and exercise in obese subjectsSW Lichtman, K Pisarska, ER Berman, M Pestone, H Dowling, E Offenbacher, H Weisel, S Heshka, DE Matthews, and SB HeymsfieldInt J ObesRelatMetabDisord. 1999 Aug;23(8):881-8.Dietary underreporting is prevalent in middle-aged British women and is not related to adiposity (percentage body fat).Samaras K, Kelly PJ, Campbell LV.Twin Research and Genetic Epidemiology Unit, St Thomas' Hospital, London, UKhttp://www.ncbi.nlm.nih.gov/pubmed/10490791?ordinalpos=6&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_DefaultReportPanel.Pubmed_RVDocSum
  • In the last 20 years, sensing and activity inference has come a very long way—originally much work was looking at instrumenting the environment with sensors like cameras to infer what people were doing in that space; with the costs and sizes of hardware shrinking, the early 2000s were marked by instrumenting the person or clothing with wearable sensorsThese wearable sensors are now making their ways into our cell phones in the form of GPS, accelerometers, proximity sensors and cameras. The mobile phone is, of course the ideal platform because it’s nearly always on and always with us.
  • It’s a new source of measurement information for behavior change.
  • The sounds are acquired with a microphone located insidethe ear canal. This is an unobtrusive location widely accepted inother applications (hearing aids, headsets). To validate our method we present experimental results containing 3500 seconds of chewing datafrom four subjects on four different food types typically found in a meal.Up to 99% accuracy is achieved on eating recognition and between 80%to 100% on food type classification.
  • Sensing system need not be on phone; can also be communicated via the cloud.System used optical, ion selective electrical pH, and conductivity sensors in order to sense and classify liquid in a cup in a practical way. Feedback system can still be phone.Drinks make up a surprisingly large portion of daily caloric intake with some research suggesting that 21% of a person’s daily caloric intake comes from beverages (458 calories) [5]. These tend to be ‘optional’ calories, not consumed exclusively to satiate hunger, and thus potentially easier to eliminate or replace with healthier alternatives Automatic Classification of Daily Fluid Intake by Jonathan Lester, Desney Tan, Shwetak Patel, and A.J. Brush22 March 2010; IEEE Pervasive Health 2010 Proceedings
  • Coughing is the most frequently cited symptom when people seek medical advice in the united states…
  • In our lab, we’ve been exploring the use of commodity mobile phones to automatically detect and classify when a person coughs.Sensing system doesn’t have to be complicated; it’s the software algorithms that are smart.
  • Ratio of about 4 to 1 annotation hours to recording hours
  • The participants were informed to monitor….
  • similar findings for people estimating their caloric intake and the amount of exercise they get per week.If you can’t measure it, you can’t change it.http://www.adajournal.org/article/S0002-8223(02)90316-0/abstracthttp://content.nejm.org/cgi/content/abstract/327/27/1893http://www.ncbi.nlm.nih.gov/pubmed/10490791?ordinalpos=6&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_DefaultReportPanel.Pubmed_RVDocSum
  • One of the best established findings in psychology is the role of feedback on performance.Think about how ingrained feedback is in our environment. For example, driving:this type of feedback is always available, passively given allowing us to sort of pre-attentively recognize it
  • We want to create these sorts of passive awareness systems.
  • Passive sensing enables a new type of application space I call “zero-effort applications”Avoid the challenging problem of determining when is the right time to prompt people with information; instead surround the user with persuasive information in a very accessible manner.
  • The goal is for the primary interactions to be completely passive; for both ubifit and ubigreen
  • As Jon mentioned, we had real time access to participant’s phone data so we could monitor the state of their phones in real timeThis screen shot was taken on a Monday, towards the beginning of the week, and shows the initial stages of their transit behavior. [next slide] Here, the shot was taken towards the end of the week; you can see how participants’ icons progressed throughout the week. At the bottom left you can see that one participant reached the final state of the progression and saw the Northern Lights.
  • [The glanceable display] was a constant reminder…whereas if you didn’t have a screen [glanceable display], you probably—I wouldn’t think about it [physical activity] as much, you know, I think about it maybe subconsciously every time I look at my phone. {S5}
  • Abstractions allowed for playfulnessProgress was visibleAchievements were rewardedCould build collection of achievements
  • I don’t have time to go through all of the persuasive/influence tactics that we thought about… but it’s worth mentioning that we specifically didn’t use loss aversion as a motivation technique here…We wanted people to enjoy using and feel good about using the applications.
  • I would like to see some graph or raw data. - Participant 13
  • Consolvo conducted a 3-month study with a control and experimental group and found that those with the ambient display outperformed those without.
  • This talk was not meant to be prescriptive but rather to inspire thought and creativity around the ideas of passive sensing for human activities. And secondly, to think creatively about how information can be passively displayed to the user.In the next 3 – 5 years, I believe activity inference will be the key technological innovation in mobile phone technology.* New opportunities in passively sensing activitiesOne of the biggest problems in hospitals is hygiene and hand washing; we
  • The sounds are acquired with a microphone located insidethe ear canal. This is an unobtrusive location widely accepted inother applications (hearing aids, headsets). To validate our method we present experimental results containing 3500 seconds of chewing datafrom four subjects on four different food types typically found in a meal.Up to 99% accuracy is achieved on eating recognition and between 80%to 100% on food type classification.
  • ubigreen context-triggered survey
  • with] a web site, it’s so easy, ‘Oh, I didn’t do anything, I'm not going to click on it.’ It’s so easy to ignore it. But on the phone, you can’t really ignore it as easily…otherwise, it’s just…out of sight, out of mind. {S9}
  • Sensing Opportunities and Zero Effort Applications for Mobile Health Persuasion

    1. sensing opportunities<br />for mobile health persuasion<br />jonfroehlich@uw.edu<br />phd candidate in computer science<br />university of washington<br />mobile health conference<br />stanford university, 05.24.2010<br />design:<br />use:<br />build:<br />ubicomp lab<br />sustainability research<br />university of washington<br />university of washington<br />
    2. sensing opportunities<br />for mobile health persuasion<br />
    3. sensing opportunities<br />for mobile health persuasion<br />Twin Falls is only 1.4 mi away and you’ll burn an estimated 170 calories round trip.<br />Twin Falls is only 1.4 mi away.<br />Twin Falls!<br />Burn 170 <br />cal & reach your walking goal for the week!<br />
    4. sensing opportunities<br />for mobile health persuasion<br />Kairostechnology that suggests a behavior at the most opportune moment<br />-fogg, 2003<br />Twin Falls is only 1.4 mi away and you’ll burn an estimated 170 calories round trip.<br />Twin Falls is only 1.4 mi away.<br />Twin Falls!<br />Burn 170 <br />cal & reach your walking goal for the week!<br />
    5. two things you need<br />a method to passivelymonitor human activity<br />a method to provide feedback about behavior<br />
    6. self-report<br />useful for measuring<br />beliefs, feelings, goals<br />simple<br />low cost<br /><ul><li>burdensome
    7. people are not good at monitoring their own behaviors:
    8. eating [champagne, 2002]
    9. exercise [lichtman, 1992]
    10. routine activities [klasnja, 2008]
    11. coughing [liu et al., in submission]</li></li></ul><li>activity inference<br />a very brief history<br />miluzzo et al., sensys<br />lester et al., ijcai<br />bao et al., pervasive<br />want, pers com<br />gavrila et al, c. vision<br />2010<br />2000<br />1990<br />instrumenting the environment<br />instrumenting the person/clothing<br />instrumenting the cell phone<br />
    12. not just sensor hardware progressions<br /><ul><li>also, advances in machine learning
    13. ability to store lots of information
    14. constant connectivity / the cloud</li></li></ul><li>just as location aware computing has ushered in a new era of mobile phone software<br />so to will activity inference for future mobile phone generations<br />
    15. running<br />lester et al., ijcai 2005<br />choudury et al., ieee pervasive 2008<br />
    16. walking<br />lester et al., ijcai 2005<br />choudury et al., ieee pervasive 2008<br />
    17. sitting<br />lester et al., ijcai 2005<br />choudury et al., ieee pervasive 2008<br />
    18. transit modes<br />patterson et al., ubicomp 2003<br />zheng et al., ubicomp 2008<br />reddy et al., sensor networks2009<br />
    19. eating<br />jawbone microphone<br />eating<br />microphone in ear detects when a person is eating with 99% accuracy<br />amft et al., ubicomp 2007<br />cheng et al., pervasive 2010<br />
    20. identifying fluids<br />instrumented cup<br />79% classification accuracy <br />68 different fluids including sodas, juices, beers, wines<br />lester et al., pervasive health 2010<br />
    21. coughing<br />liu et al., in submission<br />
    22. automatically detecting coughs with a commodity mobile phone<br />microphone<br />liu et al., in submission<br />
    23. collecting and analyzing the cough dataset<br />17 participants<br />72 hours of naturalistic audio recording<br />6 graduate students annotated recordings<br />2542 coughs labeled by annotators<br />84.4% of coughs were correctly classified<br />0.7% false positive rate (3.3/hr)<br />liu et al., in submission<br />
    24. 19<br />number of coughs<br />measured vs. self-report<br />
    25. inaccuracy of self-report<br />20<br />diff: mean (22.8/hr), std (33/hr)<br />number of coughs<br />measured vs. self-report<br />diff: mean (22.8/hr), std (33/hr)<br />
    26. two things you need<br />a method to passivelymonitor human activity<br />a method to provide feedback about behavior<br />
    27. speedometer<br />gas gauge<br />
    28. zero effort applications<br />for behavior change<br />goal: minimize interaction costs<br />approach: passive sensing + passive display <br />basically, do the activities that you normally do and the mobile phone will automatically respond<br />
    29. two examples:<br />ubifit<br />ubigreen<br />encouraging fitness behaviors through passive sensing and feedback<br />consolvo et al., chi 2008<br />consolvo et al., ubicomp2008<br />encouraging proenvironmental behaviors through passive sensing and feedback<br />froehlich et al., chi 2009<br />
    30. ubisystem components<br />collects data about physical activities<br />activity recognition device<br />glanceable display<br />phone wallpaper!<br />+<br />communicates data about physical activities<br />
    31. ubisystem components<br />towards zero effort applications<br />collects data about physical activities<br />activity recognition device<br />interactive application<br />glanceable display<br />+<br />+<br />communicates data about physical activities<br />
    32. ubifit<br />personal ambient display<br />walk<br />cardio<br />strength<br />flexibility<br />primary goal met<br />alternate goal met<br />recent goal met<br />
    33. ubigreentracked 6 transit activities<br />walk<br />bike<br />drive alone<br />train<br />carpool<br />bus<br />“green”<br />“not-green”<br />minimum activity duration: 7 minutes<br />29<br />
    34. ubigreen<br />personal ambient display<br />phone<br />background<br />(wallpaper)<br />current activity<br />evolving image<br />value icon bar<br />
    35. sense of anticipation for how story would unfold<br />
    36. ubigreen<br />personal ambient display<br />tree<br />design:<br />03/30/08<br />03/31/08<br />03/31/08<br />03/31/08<br />03/31/08<br />04/01/08<br />04/01/08<br />04/01/08<br />04/01/08<br />04/02/08<br />04/02/08<br />04/03/08<br />04/03/08<br />04/03/08<br />04/03/08<br />04/03/08<br />04/03/08<br />04/03/08<br />04/04/08<br />04/04/08<br />04/04/08<br />04/04/08<br />04/05/08<br />04/05/08<br />04/06/08<br />7:27 PM<br />7:45 AMT-Mobile<br />8:05 AMT-Mobile<br />5:15 PMT-Mobile<br />5:30 PMT-Mobile<br />7:23 AMT-Mobile<br />8:05 AMT-Mobile<br />5:38 PMT-Mobile<br />6:11 PMT-Mobile<br />7:41 AMT-Mobile<br />5:22 PMT-Mobile<br />6:55 AMT-Mobile<br />7:15 AMT-Mobile<br />4:48 PMT-Mobile<br />5:56 PMT-Mobile<br />6:09 PMT-Mobile<br />9:45 PMT-Mobile<br />10:07 PMT-Mobile<br />07:28 AMT-Mobile<br />05:41 PMT-Mobile<br />07:35 PMT-Mobile<br />11:01 PMT-Mobile<br />08:31 AMT-Mobile<br />10:19 AMT-Mobile<br />12:00 AMT-Mobile<br />everything resets on sunday<br />
    37. ubigreen<br />personal ambient display<br />polar bear<br />design:<br />03/30/08<br />7:27 PM<br />03/31/08<br />03/31/08<br />03/31/08<br />03/31/08<br />04/01/08<br />04/01/08<br />04/01/08<br />04/01/08<br />04/02/08<br />04/02/08<br />04/03/08<br />04/03/08<br />04/03/08<br />04/03/08<br />04/03/08<br />04/03/08<br />04/03/08<br />04/04/08<br />04/04/08<br />04/04/08<br />04/04/08<br />7:45 AM<br />8:05 AM<br />5:15 PM<br />5:30 PM<br />7:23 AM<br />8:05 AM<br />5:38 PM<br />6:11 PM<br />7:41 AM<br />5:22 PM<br />6:55 AM<br />7:15 AM<br />4:48 PM<br />5:56 PM<br />6:09 PM<br />9:45 PM<br />10:07 PM<br />07:28 AM<br />05:41 PM<br />07:35 PM<br />11:01 PM<br />
    38. Monday<br />Saturday<br />
    39. personal ambient display<br />impressions of ubifit<br />If you didn’t have a screen [display], I wouldn’t think about it [physical activity] as much… I think about it maybe subconsciously every time I look at my phone. <br /> - P5UF<br />With a website, it’s so easy to ignore… it’s just out of sight, out of mind. But on the phone, you can’t really ignore it…<br />- P9UF<br />
    40. game mechanics<br />measured progress<br />playful<br />collections<br />virtual achievements<br />
    41. loss aversion<br />
    42. need for quantitative data<br />I would like to see some graph or raw data.<br /> - P13UG<br />quantitative data<br /><ul><li>builds trust system is working
    43. allows for self-comparison
    44. some people like it better</li></li></ul><li>effectiveness of the ubifit glanceable display<br />Best Fit<br />Linear Trend<br />Best Fit<br />Linear Trend<br />no glanceable display<br />glanceable display<br />study occurred over thanksgiving, christmas, and new years. <br />40<br />
    45. in conclusion<br />imagine that you can sense….<br />
    46. running<br />lester et al., ijcai 2005<br />choudury et al., ieee pervasive 2008<br />
    47. walking<br />lester et al., ijcai 2005<br />choudury et al., ieee pervasive 2008<br />
    48. sitting<br />lester et al., ijcai 2005<br />choudury et al., ieee pervasive 2008<br />
    49. eating<br />jawbone microphone<br />eating<br />amft et al., ubicomp 2007<br />cheng et al., pervasive 2010<br />
    50. lakefront property<br />phone home screen:<br />most valuable real estate in all of technology<br />
    51. lakefront property<br />phone home/lock screen:<br />most valuable real estate in all of technology<br />
    52. thanks to:<br />sunny consolvo<br />pedjaklasnja<br />jameslanday<br />ericlarson<br />seanliu<br />shwetakpatel<br />thank you<br />@jonfroehlich<br />design:<br />use:<br />build:<br />ubicomp lab<br />university of washington<br />university of washington<br />
    53. extra slides<br />
    54. sink usage<br />froehlich et al., ubicomp 2009<br />larson et al., pervasive & mobile computing 2010<br />
    55. ubigreen sensing transit<br />1<br />3<br />2<br />wearable activity recognition device<br />cell towers<br />user<br />Walk<br />Bike<br />Drive Alone<br />Train<br />Carpool<br />Bus<br />minimum activity duration: 7 minutes<br />51<br />
    56. precursor to ubifit<br />pedometer cell phone fitness study<br />consolvo, et al., chi 2006<br />
    57. ubigreen context-triggered survey<br />using the myexperience toolkit<br />froehlich et al., mobisys 2007<br />http://myexperience.sourceforget.net<br />
    58. limitations<br />of sensing<br />can‘t infer thoughts, feelings, intentions<br />can be expensive<br />sensing may not yet exist for behavior<br />froehlich et al., mobisys 2007<br />http://myexperience.sourceforget.net<br />
    59. intelmsp<br />lester et al., ijcai 2005<br />
    60. personal ambient display<br />impressions of ubigreen<br />It’s omnipresent<br />- P9UG<br />It definitely keeps you more aware of it [personal transportation]. You use your phone every single day so you know. <br />- P6UG<br />

    ×