Your SlideShare is downloading. ×

AAAI Beth Logan


Published on

Keynote for AAAI Fall Symposium, AI in Eldercare track.

Keynote for AAAI Fall Symposium, AI in Eldercare track.

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Transcript

    • 1. Do try this at home! Living with In-Home Health Monitoring Systems Beth Logan Intel Digital Health Product Research and Incubation With collaborators from Intel Research Seattle, MIT and University of Washington
    • 2. Where would you rather live as you age?
      • Or in your home?
        • Likely cheaper with more privacy
      • In a nursing home?
        • More companionship
        • Less privacy
        • Far away?
        • More expensive
        • Easy monitoring
    • 3. Barriers to Staying Home Alone
      • Disabilities
        • Unable to perform ADLs
        • Cognition problems
      • Fear of worst case scenarios
        • Falls, fires, heat/air-conditioning failures, break-ins
      • Burden on caregivers too great
        • Constant visits or phone calls needed
        • Can sensor-based activity monitoring increase quality of life for elders home alone and reduce the effort of their caregivers?
    • 4. ADL Monitoring Wish List
      • Detect basic ADLs with good accuracy
        • Eating, movement, bathroom use, etc
        • Possibly detect emergencies
      • Minimal invasion of privacy
      • Easy installation
      • Sensors are unobtrusive, work flawlessly and need little maintenance
      • Wearables are a pleasure to wear and easy to maintain
      • User understands enough about the system to fix small problems
      • Remote maintenance for upgrades and major problems
    • 5. Baby Steps
      • Monitor people’s activities in a sensorized home
      • Questions
        • Can activities be detected with useful accuracy?
        • Do people like/mind living in a sensorized home?
    • 6. MIT House_n PlaceLab
        • Condo near MIT
        • Hundreds of sensors
        • Video for annotation
        • Subjects live undisturbed for weeks or months
          • Data uploaded weekly
          • Residents have ‘veto rights’
    • 7. Built-In Sensors
      • 206 inputs total
        • 101 wired reed switches
        • 14 water flow
        • 37 current
        • 36 temperature
        • 1 pressure
        • 10 humidity
        • 1 gas
        • 6 light
      Barometric pressure Humidity Temperature IR video camera Color video camera Top-down camera Light sensor Microphone Switches to detect open/close Temperature Water flow Current
    • 8. RFID Tags and Bracelet
      • 435 tags installed throughout condo
      RFID Bracelet RFID Tags
    • 9. Motion Based Sensors
      • 281 inputs total
        • 265 on-object accelerometers
        • 2x3axis on-body accelerometers
        • 10 infra-red motion detectors
    • 10. Typical study Joint with Intel 2007
      • Studied a young married couple living for 10 weeks in the PlaceLab
      • 104 hours of data annotated
        • Covers 15 days from middle of experiment
        • Chose periods when male subject was home
      • Independent annotator
        • Annotated male’s activities from the video
        • Used a 98 activity ontology
        • Limited/no video in bathrooms and bedroom; still had audio
    • 11. Living in a sensorized home
      • Participants maintained as normal routine as possible
      • Brought many possessions into the apartment to feel “at home”
      • Wore some sensors
        • Both subjects wore accelerometers on wrist, hips and thigh
        • Male subject wore a bulky RFID bracelet
      • Installed some sensors
    • 12. Participants’ reactions
      • “ We weren’t as conscious as I thought we would be, it was actually kind of natural being here…I didn’t notice some things as much as I thought I would, like the cameras.” Male subject
      • “ I wasn’t bothered by it, really at all, I thought I might get weirded out every once in a while, but there were very few times where I was totally tired of being in the project, and I felt pretty comfortable here” Female subject
    • 13. What do people do at home?
      • Most and least observed activities in 104h annotated set, male subject only
      Activity Total Observed Time Using a computer 1866 min Background listening to music or radio 813 min Actively watching tv or movies 732 min Sleeping deeply 728 min Reading book/paper/magazine 359 min Preparing a snack 44 s Leaving the home 42 s Making the bed 34 s Washing hands 24 s Drying dishes 6 s
    • 14. Activity Recognition Results Excellent  Poor  Sensors
    • 15. Take-aways (more detail in the paper)
      • People behave unpredictably “in the wild”
        • Revealed by our long-term study with unaffiliated subjects and annotator
        • Interrupted activities, multi-tasking, activities in multiple and unexpected locations
      • Many activities are short and infrequent
        • Collecting sufficient “real life” data very challenging
      • Location “good enough” for recognizing many simple activities
        • Likely not good enough for fine-grained activities
      • Even with 100s of sensors, our sensor density was sometimes insufficient
        • Opportunity for new sensors
    • 16. Next Steps
      • Move to in-home installed systems
        • Address lack of data by studying ways users can give feedback to an adaptive system
          • Ongoing work with MIT
        • Shift focus to older subjects
          • Collaboration with Intel Research Seattle
          • (rest of this talk)
      • FYI PlaceLab annotated dataset available online
        • Sensor data, activity ontology, annotations
    • 17. Technologies for Long-Term Care (TLC) Joint with Intel Research Seattle
      • Sensors in elders’ homes track activities and generate a summary display for caregivers
      RFID bracelet and RFID tags “ shake” sensors Display
    • 18. Display Detail 5 day trend Eating and ambulation timeline Today’s grooming and medication status Customizable photo background
    • 19. How to have an experiment in your home…
      • My official roles
        • Inference Lead responsible for algorithms to detect activities
        • Middleware co-designer and contributor
      • My unofficial roles
        • Chief QA engineer
        • Chief guinea pig
          • Only way to get the data I needed to design the inference!
    • 20.
      • 12 shake sensors
      • 10 RFID tags and RFID bracelet
      • 1 aggregator (laptop) with attached 802.15.4 antenna
      • Wore a phone camera around neck for 1 week to get ground truth
      • Kept display at work to easily show demos
      Home Deployment Sep 2007 – Oct 2008
    • 21. Technical Experience
      • Ease of install and setup 9/10
        • Some issues with sensors falling off at first
        • Good debugging beforehand paid off
        • Some tweaking of inference
      • Maintenance 6/10
        • Constantly changing shake sensor batteries – typically one per week
        • Bracelet needed as much attention as a cell phone
        • Some work to keep the Ceiva display up and running
        • Aggregator had 0 downtime due to bugs or failures!
      • Ease of uninstall 10/10
        • no marks anywhere!
    • 22. User Experience
      • Aesthetics 8/10
        • Over time found a way to hide most sensors but hated having the laptop and the antenna in the dining room
      • Ease of use
        • Barely noticed shake sensors 9/10
        • Hated the bracelet after about a month 1/10
          • Fragile, uncomfortable, needed constant attention, too ugly to wear around guests
      • Friends and family 10/10 (non-bracelet)
        • Many people didn’t even notice the experiment until I pointed it out!
          • I have plenty of computers
          • Only wore the bracelet around my boyfriend
      • Privacy
        • No issues with the main experiment 9/10
        • Hated wearing camera phone for ground truth 0/10
          • bulky and felt very invasive
      Antenna and laptop
    • 23. My Negative Bias
      • I was developer, maintainer and user
        • Typically if there was a problem, I owned it
        • Felt like I was bringing work home
      • I had the display screen at work, not home
        • Much more public place
      • I don’t (yet!) need TLC
        • A burden, especially maintaining the bracelet
    • 24. Findings from TLC Elder Study 2008
      • Interviews from 4 elders, 2 formal, 2 family caregivers in Seattle area
      • Roughly 10,000 hrs of data
      • 3.5 days of camera ground truth
      • Experience from 9 installs
      • Still finishing the study
        • 14 elders total but don’t expect major changes to findings
    • 25. Finding: TLC is Useful!
      • “ I think it’s a marvelous thing… my sister will call me up and say “I don’t have any apples, why haven’t you eaten ?” Elder
      • “ He brushes his teeth more often now because I can keep track. That wasn't something I really did before.” Caregiver
      • “ [T]here's footprints all over it-- so she wasn't sleeping last night … So, I won't bother her for a while.” Caregiver
    • 26. Finding: TLC is Easy to Use
      • Elder’s ability to tolerate and manage sensor bracelet was a concern
      • Feedback from elders is reassuring:
        • “ Ain't nothing to it. Just takes 30 seconds. I take it off when I leave the house, but sometimes I won't take it off for a week…”
      • TLC elders wear bracelets on over 95% of days.
    • 27. Finding: TLC is Delightful
      • Attractive design and appealing backgrounds are a key part of the displays
      • “ I love it. It's interesting… I wish I could get the cat to come back more. It comes only every week and a half.” Elder
    • 28. Finding: TLC is Well Behaved
      • False-positive rates for activity recognition are under 10%
        • Very simple inference (Naïve Bayes + rules)
        • Acceptable for human-processed output
      • Installation time is under 2 hours per user
      • Failures are as expected from pilots
        • Picture frame backup
        • Battery drain
        • Bracelet antenna damage
    • 29. Are we there yet?
      • We’re making good progress!
      • Heartening to see our system was well received and even liked
      • Debugging at home
          • Great way to increase robustness and get useful feedback instead of focusing on technical problems
          • Sometimes painful but a good way to show faith in a system
          • Definitely not 100% predictive of real users
    • 30. Where to next?
      • We are not far away from useful systems!
        • Already some products on the market
      • Biggest challenges
        • Installation and maintenance
        • Lack of real-world data
          • Work on adaptive systems
          • Make anonymized data public?
          • CHI2009 workshop "Developing Shared Home Behavior Datasets”
        • Move beyond sensing to actuation
    • 31. Collaborators
      • Intel
        • Matthai Philipose, TLC project lead
        • Jennifer Healey, Brad Needham, Ken Lafond, Adam Rea, Polly Powledge, Beverly Harrison, Jean-Manuel Van Thong, Scott Blackwell.
      • MIT House_n
        • Stephen Intille, Director of Technology
        • Emmanuel Munguia-Tapia, Jennifer Beudin
      • University of Washington School of Public Health
        • Sheri Reder, Clinical Instructor
        • Susan Hedrick, Professor
    • 32. Backup