AAAI Beth Logan


Published on

Keynote for AAAI Fall Symposium, AI in Eldercare track.

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • AAAI Beth Logan

    1. 1. Do try this at home! Living with In-Home Health Monitoring Systems Beth Logan Intel Digital Health Product Research and Incubation With collaborators from Intel Research Seattle, MIT and University of Washington
    2. 2. Where would you rather live as you age? <ul><li>Or in your home? </li></ul><ul><ul><li>Likely cheaper with more privacy </li></ul></ul><ul><li>In a nursing home? </li></ul><ul><ul><li>More companionship </li></ul></ul><ul><ul><li>Less privacy </li></ul></ul><ul><ul><li>Far away? </li></ul></ul><ul><ul><li>More expensive </li></ul></ul><ul><ul><li>Easy monitoring </li></ul></ul>
    3. 3. Barriers to Staying Home Alone <ul><li>Disabilities </li></ul><ul><ul><li>Unable to perform ADLs </li></ul></ul><ul><ul><li>Cognition problems </li></ul></ul><ul><li>Fear of worst case scenarios </li></ul><ul><ul><li>Falls, fires, heat/air-conditioning failures, break-ins </li></ul></ul><ul><li>Burden on caregivers too great </li></ul><ul><ul><li>Constant visits or phone calls needed </li></ul></ul><ul><ul><li>Can sensor-based activity monitoring increase quality of life for elders home alone and reduce the effort of their caregivers? </li></ul></ul>
    4. 4. ADL Monitoring Wish List <ul><li>Detect basic ADLs with good accuracy </li></ul><ul><ul><li>Eating, movement, bathroom use, etc </li></ul></ul><ul><ul><li>Possibly detect emergencies </li></ul></ul><ul><li>Minimal invasion of privacy </li></ul><ul><li>Easy installation </li></ul><ul><li>Sensors are unobtrusive, work flawlessly and need little maintenance </li></ul><ul><li>Wearables are a pleasure to wear and easy to maintain </li></ul><ul><li>User understands enough about the system to fix small problems </li></ul><ul><li>Remote maintenance for upgrades and major problems </li></ul>
    5. 5. Baby Steps <ul><li>Monitor people’s activities in a sensorized home </li></ul><ul><li>Questions </li></ul><ul><ul><li>Can activities be detected with useful accuracy? </li></ul></ul><ul><ul><li>Do people like/mind living in a sensorized home? </li></ul></ul>
    6. 6. MIT House_n PlaceLab <ul><ul><li>Condo near MIT </li></ul></ul><ul><ul><li>Hundreds of sensors </li></ul></ul><ul><ul><li>Video for annotation </li></ul></ul><ul><ul><li>Subjects live undisturbed for weeks or months </li></ul></ul><ul><ul><ul><li>Data uploaded weekly </li></ul></ul></ul><ul><ul><ul><li>Residents have ‘veto rights’ </li></ul></ul></ul>
    7. 7. Built-In Sensors <ul><li>206 inputs total </li></ul><ul><ul><li>101 wired reed switches </li></ul></ul><ul><ul><li>14 water flow </li></ul></ul><ul><ul><li>37 current </li></ul></ul><ul><ul><li>36 temperature </li></ul></ul><ul><ul><li>1 pressure </li></ul></ul><ul><ul><li>10 humidity </li></ul></ul><ul><ul><li>1 gas </li></ul></ul><ul><ul><li>6 light </li></ul></ul>Barometric pressure Humidity Temperature IR video camera Color video camera Top-down camera Light sensor Microphone Switches to detect open/close Temperature Water flow Current
    8. 8. RFID Tags and Bracelet <ul><li>435 tags installed throughout condo </li></ul>RFID Bracelet RFID Tags
    9. 9. Motion Based Sensors <ul><li>281 inputs total </li></ul><ul><ul><li>265 on-object accelerometers </li></ul></ul><ul><ul><li>2x3axis on-body accelerometers </li></ul></ul><ul><ul><li>10 infra-red motion detectors </li></ul></ul>
    10. 10. Typical study Joint with Intel 2007 <ul><li>Studied a young married couple living for 10 weeks in the PlaceLab </li></ul><ul><li>104 hours of data annotated </li></ul><ul><ul><li>Covers 15 days from middle of experiment </li></ul></ul><ul><ul><li>Chose periods when male subject was home </li></ul></ul><ul><li>Independent annotator </li></ul><ul><ul><li>Annotated male’s activities from the video </li></ul></ul><ul><ul><li>Used a 98 activity ontology </li></ul></ul><ul><ul><li>Limited/no video in bathrooms and bedroom; still had audio </li></ul></ul>
    11. 11. Living in a sensorized home <ul><li>Participants maintained as normal routine as possible </li></ul><ul><li>Brought many possessions into the apartment to feel “at home” </li></ul><ul><li>Wore some sensors </li></ul><ul><ul><li>Both subjects wore accelerometers on wrist, hips and thigh </li></ul></ul><ul><ul><li>Male subject wore a bulky RFID bracelet </li></ul></ul><ul><li>Installed some sensors </li></ul>
    12. 12. Participants’ reactions <ul><li>“ We weren’t as conscious as I thought we would be, it was actually kind of natural being here…I didn’t notice some things as much as I thought I would, like the cameras.” Male subject </li></ul><ul><li>“ I wasn’t bothered by it, really at all, I thought I might get weirded out every once in a while, but there were very few times where I was totally tired of being in the project, and I felt pretty comfortable here” Female subject </li></ul>
    13. 13. What do people do at home? <ul><li>Most and least observed activities in 104h annotated set, male subject only </li></ul>Activity Total Observed Time Using a computer 1866 min Background listening to music or radio 813 min Actively watching tv or movies 732 min Sleeping deeply 728 min Reading book/paper/magazine 359 min Preparing a snack 44 s Leaving the home 42 s Making the bed 34 s Washing hands 24 s Drying dishes 6 s
    14. 14. Activity Recognition Results Excellent  Poor  Sensors
    15. 15. Take-aways (more detail in the paper) <ul><li>People behave unpredictably “in the wild” </li></ul><ul><ul><li>Revealed by our long-term study with unaffiliated subjects and annotator </li></ul></ul><ul><ul><li>Interrupted activities, multi-tasking, activities in multiple and unexpected locations </li></ul></ul><ul><li>Many activities are short and infrequent </li></ul><ul><ul><li>Collecting sufficient “real life” data very challenging </li></ul></ul><ul><li>Location “good enough” for recognizing many simple activities </li></ul><ul><ul><li>Likely not good enough for fine-grained activities </li></ul></ul><ul><li>Even with 100s of sensors, our sensor density was sometimes insufficient </li></ul><ul><ul><li>Opportunity for new sensors </li></ul></ul>
    16. 16. Next Steps <ul><li>Move to in-home installed systems </li></ul><ul><ul><li>Address lack of data by studying ways users can give feedback to an adaptive system </li></ul></ul><ul><ul><ul><li>Ongoing work with MIT </li></ul></ul></ul><ul><ul><li>Shift focus to older subjects </li></ul></ul><ul><ul><ul><li>Collaboration with Intel Research Seattle </li></ul></ul></ul><ul><ul><ul><li>(rest of this talk) </li></ul></ul></ul><ul><li>FYI PlaceLab annotated dataset available online </li></ul><ul><ul><li>Sensor data, activity ontology, annotations </li></ul></ul><ul><ul><li> </li></ul></ul>
    17. 17. Technologies for Long-Term Care (TLC) Joint with Intel Research Seattle <ul><li>Sensors in elders’ homes track activities and generate a summary display for caregivers </li></ul>RFID bracelet and RFID tags “ shake” sensors Display
    18. 18. Display Detail 5 day trend Eating and ambulation timeline Today’s grooming and medication status Customizable photo background
    19. 19. How to have an experiment in your home… <ul><li>My official roles </li></ul><ul><ul><li>Inference Lead responsible for algorithms to detect activities </li></ul></ul><ul><ul><li>Middleware co-designer and contributor </li></ul></ul><ul><li>My unofficial roles </li></ul><ul><ul><li>Chief QA engineer </li></ul></ul><ul><ul><li>Chief guinea pig </li></ul></ul><ul><ul><ul><li>Only way to get the data I needed to design the inference! </li></ul></ul></ul>
    20. 20. <ul><li>12 shake sensors </li></ul><ul><li>10 RFID tags and RFID bracelet </li></ul><ul><li>1 aggregator (laptop) with attached 802.15.4 antenna </li></ul><ul><li>Wore a phone camera around neck for 1 week to get ground truth </li></ul><ul><li>Kept display at work to easily show demos </li></ul>Home Deployment Sep 2007 – Oct 2008
    21. 21. Technical Experience <ul><li>Ease of install and setup 9/10 </li></ul><ul><ul><li>Some issues with sensors falling off at first </li></ul></ul><ul><ul><li>Good debugging beforehand paid off </li></ul></ul><ul><ul><li>Some tweaking of inference </li></ul></ul><ul><li>Maintenance 6/10 </li></ul><ul><ul><li>Constantly changing shake sensor batteries – typically one per week </li></ul></ul><ul><ul><li>Bracelet needed as much attention as a cell phone </li></ul></ul><ul><ul><li>Some work to keep the Ceiva display up and running </li></ul></ul><ul><ul><li>Aggregator had 0 downtime due to bugs or failures! </li></ul></ul><ul><li>Ease of uninstall 10/10 </li></ul><ul><ul><li>no marks anywhere! </li></ul></ul>
    22. 22. User Experience <ul><li>Aesthetics 8/10 </li></ul><ul><ul><li>Over time found a way to hide most sensors but hated having the laptop and the antenna in the dining room </li></ul></ul><ul><li>Ease of use </li></ul><ul><ul><li>Barely noticed shake sensors 9/10 </li></ul></ul><ul><ul><li>Hated the bracelet after about a month 1/10 </li></ul></ul><ul><ul><ul><li>Fragile, uncomfortable, needed constant attention, too ugly to wear around guests </li></ul></ul></ul><ul><li>Friends and family 10/10 (non-bracelet) </li></ul><ul><ul><li>Many people didn’t even notice the experiment until I pointed it out! </li></ul></ul><ul><ul><ul><li>I have plenty of computers </li></ul></ul></ul><ul><ul><ul><li>Only wore the bracelet around my boyfriend </li></ul></ul></ul><ul><li>Privacy </li></ul><ul><ul><li>No issues with the main experiment 9/10 </li></ul></ul><ul><ul><li>Hated wearing camera phone for ground truth 0/10 </li></ul></ul><ul><ul><ul><li>bulky and felt very invasive </li></ul></ul></ul>Antenna and laptop
    23. 23. My Negative Bias <ul><li>I was developer, maintainer and user </li></ul><ul><ul><li>Typically if there was a problem, I owned it </li></ul></ul><ul><ul><li>Felt like I was bringing work home </li></ul></ul><ul><li>I had the display screen at work, not home </li></ul><ul><ul><li>Much more public place </li></ul></ul><ul><li>I don’t (yet!) need TLC </li></ul><ul><ul><li>A burden, especially maintaining the bracelet </li></ul></ul>
    24. 24. Findings from TLC Elder Study 2008 <ul><li>Interviews from 4 elders, 2 formal, 2 family caregivers in Seattle area </li></ul><ul><li>Roughly 10,000 hrs of data </li></ul><ul><li>3.5 days of camera ground truth </li></ul><ul><li>Experience from 9 installs </li></ul><ul><li>Still finishing the study </li></ul><ul><ul><li>14 elders total but don’t expect major changes to findings </li></ul></ul>
    25. 25. Finding: TLC is Useful! <ul><li>“ I think it’s a marvelous thing… my sister will call me up and say “I don’t have any apples, why haven’t you eaten ?” Elder </li></ul><ul><li>“ He brushes his teeth more often now because I can keep track. That wasn't something I really did before.” Caregiver </li></ul><ul><li>“ [T]here's footprints all over it-- so she wasn't sleeping last night … So, I won't bother her for a while.” Caregiver </li></ul>
    26. 26. Finding: TLC is Easy to Use <ul><li>Elder’s ability to tolerate and manage sensor bracelet was a concern </li></ul><ul><li>Feedback from elders is reassuring: </li></ul><ul><ul><li>“ Ain't nothing to it. Just takes 30 seconds. I take it off when I leave the house, but sometimes I won't take it off for a week…” </li></ul></ul><ul><li>TLC elders wear bracelets on over 95% of days. </li></ul>
    27. 27. Finding: TLC is Delightful <ul><li>Attractive design and appealing backgrounds are a key part of the displays </li></ul><ul><li>“ I love it. It's interesting… I wish I could get the cat to come back more. It comes only every week and a half.” Elder </li></ul>
    28. 28. Finding: TLC is Well Behaved <ul><li>False-positive rates for activity recognition are under 10% </li></ul><ul><ul><li>Very simple inference (Naïve Bayes + rules) </li></ul></ul><ul><ul><li>Acceptable for human-processed output </li></ul></ul><ul><li>Installation time is under 2 hours per user </li></ul><ul><li>Failures are as expected from pilots </li></ul><ul><ul><li>Picture frame backup </li></ul></ul><ul><ul><li>Battery drain </li></ul></ul><ul><ul><li>Bracelet antenna damage </li></ul></ul>
    29. 29. Are we there yet? <ul><li>We’re making good progress! </li></ul><ul><li>Heartening to see our system was well received and even liked </li></ul><ul><li>Debugging at home </li></ul><ul><ul><ul><li>Great way to increase robustness and get useful feedback instead of focusing on technical problems </li></ul></ul></ul><ul><ul><ul><li>Sometimes painful but a good way to show faith in a system </li></ul></ul></ul><ul><ul><ul><li>Definitely not 100% predictive of real users </li></ul></ul></ul>
    30. 30. Where to next? <ul><li>We are not far away from useful systems! </li></ul><ul><ul><li>Already some products on the market </li></ul></ul><ul><li>Biggest challenges </li></ul><ul><ul><li>Installation and maintenance </li></ul></ul><ul><ul><li>Lack of real-world data </li></ul></ul><ul><ul><ul><li>Work on adaptive systems </li></ul></ul></ul><ul><ul><ul><li>Make anonymized data public? </li></ul></ul></ul><ul><ul><ul><li>CHI2009 workshop &quot;Developing Shared Home Behavior Datasets” </li></ul></ul></ul><ul><ul><li>Move beyond sensing to actuation </li></ul></ul>
    31. 31. Collaborators <ul><li>Intel </li></ul><ul><ul><li>Matthai Philipose, TLC project lead </li></ul></ul><ul><ul><li>Jennifer Healey, Brad Needham, Ken Lafond, Adam Rea, Polly Powledge, Beverly Harrison, Jean-Manuel Van Thong, Scott Blackwell. </li></ul></ul><ul><li>MIT House_n </li></ul><ul><ul><li>Stephen Intille, Director of Technology </li></ul></ul><ul><ul><li>Emmanuel Munguia-Tapia, Jennifer Beudin </li></ul></ul><ul><li>University of Washington School of Public Health </li></ul><ul><ul><li>Sheri Reder, Clinical Instructor </li></ul></ul><ul><ul><li>Susan Hedrick, Professor </li></ul></ul>
    32. 32. Backup