Sensing Opportunities and Zero Effort Applications for Mobile Health Persuasion
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Sensing Opportunities and Zero Effort Applications for Mobile Health Persuasion

on

  • 5,246 views

This is my Mobile Health 2010 (#mh2010) talk that I gave on May 24th for the session: "The Sweet Spot of Behavior Change via Mobile Devices." ...

This is my Mobile Health 2010 (#mh2010) talk that I gave on May 24th for the session: "The Sweet Spot of Behavior Change via Mobile Devices."

I use lots of animations, so I strongly encourage you to download the PowerPoint pptx here:
http://www.cs.washington.edu/homes/jfroehli/talks.html

Statistics

Views

Total Views
5,246
Views on SlideShare
4,914
Embed Views
332

Actions

Likes
6
Downloads
80
Comments
2

9 Embeds 332

http://www.slideshare.net 266
http://mobilehealthsystem.com 47
http://www.linkedin.com 7
http://kristinasaric.com 4
http://www.lmodules.com 3
http://cmuedu643.wikispaces.com 2
http://192.168.0.2:3000 1
http://localhost:3000 1
http://facebook.slideshare.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • I've now graduated from UW so the above link is no longer valid. The original pptx can still be downloaded here (along with all my other talks)

    http://www.cs.umd.edu/~jonf/talks.html
    Are you sure you want to
    Your message goes here
    Processing…
  • I use lots of animations in my talks to demonstrate ideas and demo applications. In this talk, for example, there are animations of UbiFit and UbiGreen. Unfortunately, SlideShare doesn't support animations so I strongly encourage you to download the PowerPoint pptx here:

    http://www.cs.washington.edu/homes/jfroehli/talks.html
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Maybe add in social persuasion here?Maybe write on arrow instead?
  • So, from a technology perspective, there’s two things going on here: (1) we have a sensing system that can track location and infer particular decision moments and (2) we have a navigation application that uses persuasion tactics to motivate behavior.
  • Thus, for these applications to work, there needs to be some _sensing_ and some _feedback_
  • one potential opportunity here is to use sensing systems that passively sense and infer human activities---http://www.adajournal.org/article/S0002-8223(02)90316-0/abstractEnergy Intake and Energy Expenditure: A Controlled Study Comparing Dietitians and Non-dietitiansCATHERINE M. CHAMPAGNE, PhD, RD, FADAa, GEORGE A. BRAY, MDa, APRIL A. KURTZ, MS, RDa, JOSEFINA BRESSAN RESENDE MONTEIRO, PhDb,ELIZABETH TUCKER, JULIA VOLAUFOVA, PhDa, JAMES P. DELANY, PhDahttp://content.nejm.org/cgi/content/abstract/327/27/1893Discrepancy between self-reported and actual caloric intake and exercise in obese subjectsSW Lichtman, K Pisarska, ER Berman, M Pestone, H Dowling, E Offenbacher, H Weisel, S Heshka, DE Matthews, and SB HeymsfieldInt J ObesRelatMetabDisord. 1999 Aug;23(8):881-8.Dietary underreporting is prevalent in middle-aged British women and is not related to adiposity (percentage body fat).Samaras K, Kelly PJ, Campbell LV.Twin Research and Genetic Epidemiology Unit, St Thomas' Hospital, London, UKhttp://www.ncbi.nlm.nih.gov/pubmed/10490791?ordinalpos=6&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_DefaultReportPanel.Pubmed_RVDocSum
  • In the last 20 years, sensing and activity inference has come a very long way—originally much work was looking at instrumenting the environment with sensors like cameras to infer what people were doing in that space; with the costs and sizes of hardware shrinking, the early 2000s were marked by instrumenting the person or clothing with wearable sensorsThese wearable sensors are now making their ways into our cell phones in the form of GPS, accelerometers, proximity sensors and cameras. The mobile phone is, of course the ideal platform because it’s nearly always on and always with us.
  • It’s a new source of measurement information for behavior change.
  • The sounds are acquired with a microphone located insidethe ear canal. This is an unobtrusive location widely accepted inother applications (hearing aids, headsets). To validate our method we present experimental results containing 3500 seconds of chewing datafrom four subjects on four different food types typically found in a meal.Up to 99% accuracy is achieved on eating recognition and between 80%to 100% on food type classification.
  • Sensing system need not be on phone; can also be communicated via the cloud.System used optical, ion selective electrical pH, and conductivity sensors in order to sense and classify liquid in a cup in a practical way. Feedback system can still be phone.Drinks make up a surprisingly large portion of daily caloric intake with some research suggesting that 21% of a person’s daily caloric intake comes from beverages (458 calories) [5]. These tend to be ‘optional’ calories, not consumed exclusively to satiate hunger, and thus potentially easier to eliminate or replace with healthier alternatives Automatic Classification of Daily Fluid Intake by Jonathan Lester, Desney Tan, Shwetak Patel, and A.J. Brush22 March 2010; IEEE Pervasive Health 2010 Proceedings
  • Coughing is the most frequently cited symptom when people seek medical advice in the united states…
  • In our lab, we’ve been exploring the use of commodity mobile phones to automatically detect and classify when a person coughs.Sensing system doesn’t have to be complicated; it’s the software algorithms that are smart.
  • Ratio of about 4 to 1 annotation hours to recording hours
  • The participants were informed to monitor….
  • similar findings for people estimating their caloric intake and the amount of exercise they get per week.If you can’t measure it, you can’t change it.http://www.adajournal.org/article/S0002-8223(02)90316-0/abstracthttp://content.nejm.org/cgi/content/abstract/327/27/1893http://www.ncbi.nlm.nih.gov/pubmed/10490791?ordinalpos=6&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_DefaultReportPanel.Pubmed_RVDocSum
  • One of the best established findings in psychology is the role of feedback on performance.Think about how ingrained feedback is in our environment. For example, driving:this type of feedback is always available, passively given allowing us to sort of pre-attentively recognize it
  • We want to create these sorts of passive awareness systems.
  • Passive sensing enables a new type of application space I call “zero-effort applications”Avoid the challenging problem of determining when is the right time to prompt people with information; instead surround the user with persuasive information in a very accessible manner.
  • The goal is for the primary interactions to be completely passive; for both ubifit and ubigreen
  • As Jon mentioned, we had real time access to participant’s phone data so we could monitor the state of their phones in real timeThis screen shot was taken on a Monday, towards the beginning of the week, and shows the initial stages of their transit behavior. [next slide] Here, the shot was taken towards the end of the week; you can see how participants’ icons progressed throughout the week. At the bottom left you can see that one participant reached the final state of the progression and saw the Northern Lights.
  • [The glanceable display] was a constant reminder…whereas if you didn’t have a screen [glanceable display], you probably—I wouldn’t think about it [physical activity] as much, you know, I think about it maybe subconsciously every time I look at my phone. {S5}
  • Abstractions allowed for playfulnessProgress was visibleAchievements were rewardedCould build collection of achievements
  • I don’t have time to go through all of the persuasive/influence tactics that we thought about… but it’s worth mentioning that we specifically didn’t use loss aversion as a motivation technique here…We wanted people to enjoy using and feel good about using the applications.
  • I would like to see some graph or raw data. - Participant 13
  • Consolvo conducted a 3-month study with a control and experimental group and found that those with the ambient display outperformed those without.
  • This talk was not meant to be prescriptive but rather to inspire thought and creativity around the ideas of passive sensing for human activities. And secondly, to think creatively about how information can be passively displayed to the user.In the next 3 – 5 years, I believe activity inference will be the key technological innovation in mobile phone technology.* New opportunities in passively sensing activitiesOne of the biggest problems in hospitals is hygiene and hand washing; we
  • The sounds are acquired with a microphone located insidethe ear canal. This is an unobtrusive location widely accepted inother applications (hearing aids, headsets). To validate our method we present experimental results containing 3500 seconds of chewing datafrom four subjects on four different food types typically found in a meal.Up to 99% accuracy is achieved on eating recognition and between 80%to 100% on food type classification.
  • ubigreen context-triggered survey
  • with] a web site, it’s so easy, ‘Oh, I didn’t do anything, I'm not going to click on it.’ It’s so easy to ignore it. But on the phone, you can’t really ignore it as easily…otherwise, it’s just…out of sight, out of mind. {S9}

Sensing Opportunities and Zero Effort Applications for Mobile Health Persuasion Presentation Transcript

  • 1. sensing opportunities
    for mobile health persuasion
    jonfroehlich@uw.edu
    phd candidate in computer science
    university of washington
    mobile health conference
    stanford university, 05.24.2010
    design:
    use:
    build:
    ubicomp lab
    sustainability research
    university of washington
    university of washington
  • 2. sensing opportunities
    for mobile health persuasion
  • 3. sensing opportunities
    for mobile health persuasion
    Twin Falls is only 1.4 mi away and you’ll burn an estimated 170 calories round trip.
    Twin Falls is only 1.4 mi away.
    Twin Falls!
    Burn 170
    cal & reach your walking goal for the week!
  • 4. sensing opportunities
    for mobile health persuasion
    Kairostechnology that suggests a behavior at the most opportune moment
    -fogg, 2003
    Twin Falls is only 1.4 mi away and you’ll burn an estimated 170 calories round trip.
    Twin Falls is only 1.4 mi away.
    Twin Falls!
    Burn 170
    cal & reach your walking goal for the week!
  • 5. two things you need
    a method to passivelymonitor human activity
    a method to provide feedback about behavior
  • 6. self-report
    useful for measuring
    beliefs, feelings, goals
    simple
    low cost
    • burdensome
    • 7. people are not good at monitoring their own behaviors:
    • 8. eating [champagne, 2002]
    • 9. exercise [lichtman, 1992]
    • 10. routine activities [klasnja, 2008]
    • 11. coughing [liu et al., in submission]
  • activity inference
    a very brief history
    miluzzo et al., sensys
    lester et al., ijcai
    bao et al., pervasive
    want, pers com
    gavrila et al, c. vision
    2010
    2000
    1990
    instrumenting the environment
    instrumenting the person/clothing
    instrumenting the cell phone
  • 12. not just sensor hardware progressions
    • also, advances in machine learning
    • 13. ability to store lots of information
    • 14. constant connectivity / the cloud
  • just as location aware computing has ushered in a new era of mobile phone software
    so to will activity inference for future mobile phone generations
  • 15. running
    lester et al., ijcai 2005
    choudury et al., ieee pervasive 2008
  • 16. walking
    lester et al., ijcai 2005
    choudury et al., ieee pervasive 2008
  • 17. sitting
    lester et al., ijcai 2005
    choudury et al., ieee pervasive 2008
  • 18. transit modes
    patterson et al., ubicomp 2003
    zheng et al., ubicomp 2008
    reddy et al., sensor networks2009
  • 19. eating
    jawbone microphone
    eating
    microphone in ear detects when a person is eating with 99% accuracy
    amft et al., ubicomp 2007
    cheng et al., pervasive 2010
  • 20. identifying fluids
    instrumented cup
    79% classification accuracy
    68 different fluids including sodas, juices, beers, wines
    lester et al., pervasive health 2010
  • 21. coughing
    liu et al., in submission
  • 22. automatically detecting coughs with a commodity mobile phone
    microphone
    liu et al., in submission
  • 23. collecting and analyzing the cough dataset
    17 participants
    72 hours of naturalistic audio recording
    6 graduate students annotated recordings
    2542 coughs labeled by annotators
    84.4% of coughs were correctly classified
    0.7% false positive rate (3.3/hr)
    liu et al., in submission
  • 24. 19
    number of coughs
    measured vs. self-report
  • 25. inaccuracy of self-report
    20
    diff: mean (22.8/hr), std (33/hr)
    number of coughs
    measured vs. self-report
    diff: mean (22.8/hr), std (33/hr)
  • 26. two things you need
    a method to passivelymonitor human activity
    a method to provide feedback about behavior
  • 27. speedometer
    gas gauge
  • 28.
  • 29. zero effort applications
    for behavior change
    goal: minimize interaction costs
    approach: passive sensing + passive display
    basically, do the activities that you normally do and the mobile phone will automatically respond
  • 30. two examples:
    ubifit
    ubigreen
    encouraging fitness behaviors through passive sensing and feedback
    consolvo et al., chi 2008
    consolvo et al., ubicomp2008
    encouraging proenvironmental behaviors through passive sensing and feedback
    froehlich et al., chi 2009
  • 31. ubisystem components
    collects data about physical activities
    activity recognition device
    glanceable display
    phone wallpaper!
    +
    communicates data about physical activities
  • 32. ubisystem components
    towards zero effort applications
    collects data about physical activities
    activity recognition device
    interactive application
    glanceable display
    +
    +
    communicates data about physical activities
  • 33. ubifit
    personal ambient display
    walk
    cardio
    strength
    flexibility
    primary goal met
    alternate goal met
    recent goal met
  • 34. ubigreentracked 6 transit activities
    walk
    bike
    drive alone
    train
    carpool
    bus
    “green”
    “not-green”
    minimum activity duration: 7 minutes
    29
  • 35. ubigreen
    personal ambient display
    phone
    background
    (wallpaper)
    current activity
    evolving image
    value icon bar
  • 36. sense of anticipation for how story would unfold
  • 37. ubigreen
    personal ambient display
    tree
    design:
    03/30/08
    03/31/08
    03/31/08
    03/31/08
    03/31/08
    04/01/08
    04/01/08
    04/01/08
    04/01/08
    04/02/08
    04/02/08
    04/03/08
    04/03/08
    04/03/08
    04/03/08
    04/03/08
    04/03/08
    04/03/08
    04/04/08
    04/04/08
    04/04/08
    04/04/08
    04/05/08
    04/05/08
    04/06/08
    7:27 PM
    7:45 AMT-Mobile
    8:05 AMT-Mobile
    5:15 PMT-Mobile
    5:30 PMT-Mobile
    7:23 AMT-Mobile
    8:05 AMT-Mobile
    5:38 PMT-Mobile
    6:11 PMT-Mobile
    7:41 AMT-Mobile
    5:22 PMT-Mobile
    6:55 AMT-Mobile
    7:15 AMT-Mobile
    4:48 PMT-Mobile
    5:56 PMT-Mobile
    6:09 PMT-Mobile
    9:45 PMT-Mobile
    10:07 PMT-Mobile
    07:28 AMT-Mobile
    05:41 PMT-Mobile
    07:35 PMT-Mobile
    11:01 PMT-Mobile
    08:31 AMT-Mobile
    10:19 AMT-Mobile
    12:00 AMT-Mobile
    everything resets on sunday
  • 38. ubigreen
    personal ambient display
    polar bear
    design:
    03/30/08
    7:27 PM
    03/31/08
    03/31/08
    03/31/08
    03/31/08
    04/01/08
    04/01/08
    04/01/08
    04/01/08
    04/02/08
    04/02/08
    04/03/08
    04/03/08
    04/03/08
    04/03/08
    04/03/08
    04/03/08
    04/03/08
    04/04/08
    04/04/08
    04/04/08
    04/04/08
    7:45 AM
    8:05 AM
    5:15 PM
    5:30 PM
    7:23 AM
    8:05 AM
    5:38 PM
    6:11 PM
    7:41 AM
    5:22 PM
    6:55 AM
    7:15 AM
    4:48 PM
    5:56 PM
    6:09 PM
    9:45 PM
    10:07 PM
    07:28 AM
    05:41 PM
    07:35 PM
    11:01 PM
  • 39. Monday
    Saturday
  • 40. personal ambient display
    impressions of ubifit
    If you didn’t have a screen [display], I wouldn’t think about it [physical activity] as much… I think about it maybe subconsciously every time I look at my phone.
    - P5UF
    With a website, it’s so easy to ignore… it’s just out of sight, out of mind. But on the phone, you can’t really ignore it…
    - P9UF
  • 41.
  • 42. game mechanics
    measured progress
    playful
    collections
    virtual achievements
  • 43. loss aversion
  • 44. need for quantitative data
    I would like to see some graph or raw data.
    - P13UG
    quantitative data
    • builds trust system is working
    • 45. allows for self-comparison
    • 46. some people like it better
  • effectiveness of the ubifit glanceable display
    Best Fit
    Linear Trend
    Best Fit
    Linear Trend
    no glanceable display
    glanceable display
    study occurred over thanksgiving, christmas, and new years.
    40
  • 47. in conclusion
    imagine that you can sense….
  • 48. running
    lester et al., ijcai 2005
    choudury et al., ieee pervasive 2008
  • 49. walking
    lester et al., ijcai 2005
    choudury et al., ieee pervasive 2008
  • 50. sitting
    lester et al., ijcai 2005
    choudury et al., ieee pervasive 2008
  • 51. eating
    jawbone microphone
    eating
    amft et al., ubicomp 2007
    cheng et al., pervasive 2010
  • 52. lakefront property
    phone home screen:
    most valuable real estate in all of technology
  • 53. lakefront property
    phone home/lock screen:
    most valuable real estate in all of technology
  • 54. thanks to:
    sunny consolvo
    pedjaklasnja
    jameslanday
    ericlarson
    seanliu
    shwetakpatel
    thank you
    @jonfroehlich
    design:
    use:
    build:
    ubicomp lab
    university of washington
    university of washington
  • 55. extra slides
  • 56. sink usage
    froehlich et al., ubicomp 2009
    larson et al., pervasive & mobile computing 2010
  • 57. ubigreen sensing transit
    1
    3
    2
    wearable activity recognition device
    cell towers
    user
    Walk
    Bike
    Drive Alone
    Train
    Carpool
    Bus
    minimum activity duration: 7 minutes
    51
  • 58. precursor to ubifit
    pedometer cell phone fitness study
    consolvo, et al., chi 2006
  • 59. ubigreen context-triggered survey
    using the myexperience toolkit
    froehlich et al., mobisys 2007
    http://myexperience.sourceforget.net
  • 60. limitations
    of sensing
    can‘t infer thoughts, feelings, intentions
    can be expensive
    sensing may not yet exist for behavior
    froehlich et al., mobisys 2007
    http://myexperience.sourceforget.net
  • 61. intelmsp
    lester et al., ijcai 2005
  • 62. personal ambient display
    impressions of ubigreen
    It’s omnipresent
    - P9UG
    It definitely keeps you more aware of it [personal transportation]. You use your phone every single day so you know.
    - P6UG