The document discusses living with in-home health monitoring systems to allow elderly individuals to live independently at home. It describes a study where participants lived in an apartment equipped with hundreds of sensors to detect daily activities. The study found that participants were comfortable with the sensors and that basic activities could be detected with useful accuracy. The document advocates moving such systems into real homes to better understand challenges and gather more real-world data.

1. Do try this at home! Living with In-Home Health Monitoring Systems Beth Logan Intel Digital Health Product Research and Incubation With collaborators from Intel Research Seattle, MIT and University of Washington

2. Where would you rather live as you age? Or in your home? Likely cheaper with more privacy In a nursing home? More companionship Less privacy Far away? More expensive Easy monitoring

3. Barriers to Staying Home Alone Disabilities Unable to perform ADLs Cognition problems Fear of worst case scenarios Falls, fires, heat/air-conditioning failures, break-ins Burden on caregivers too great Constant visits or phone calls needed Can sensor-based activity monitoring increase quality of life for elders home alone and reduce the effort of their caregivers?

4. ADL Monitoring Wish List Detect basic ADLs with good accuracy Eating, movement, bathroom use, etc Possibly detect emergencies Minimal invasion of privacy Easy installation Sensors are unobtrusive, work flawlessly and need little maintenance Wearables are a pleasure to wear and easy to maintain User understands enough about the system to fix small problems Remote maintenance for upgrades and major problems

5. Baby Steps Monitor people’s activities in a sensorized home Questions Can activities be detected with useful accuracy? Do people like/mind living in a sensorized home?

6. MIT House_n PlaceLab Condo near MIT Hundreds of sensors Video for annotation Subjects live undisturbed for weeks or months Data uploaded weekly Residents have ‘veto rights’

7. Built-In Sensors 206 inputs total 101 wired reed switches 14 water flow 37 current 36 temperature 1 pressure 10 humidity 1 gas 6 light Barometric pressure Humidity Temperature IR video camera Color video camera Top-down camera Light sensor Microphone Switches to detect open/close Temperature Water flow Current

8. RFID Tags and Bracelet 435 tags installed throughout condo RFID Bracelet RFID Tags

9. Motion Based Sensors 281 inputs total 265 on-object accelerometers 2x3axis on-body accelerometers 10 infra-red motion detectors

10. Typical study Joint with Intel 2007 Studied a young married couple living for 10 weeks in the PlaceLab 104 hours of data annotated Covers 15 days from middle of experiment Chose periods when male subject was home Independent annotator Annotated male’s activities from the video Used a 98 activity ontology Limited/no video in bathrooms and bedroom; still had audio

11. Living in a sensorized home Participants maintained as normal routine as possible Brought many possessions into the apartment to feel “at home” Wore some sensors Both subjects wore accelerometers on wrist, hips and thigh Male subject wore a bulky RFID bracelet Installed some sensors

12. Participants’ reactions “ We weren’t as conscious as I thought we would be, it was actually kind of natural being here…I didn’t notice some things as much as I thought I would, like the cameras.” Male subject “ I wasn’t bothered by it, really at all, I thought I might get weirded out every once in a while, but there were very few times where I was totally tired of being in the project, and I felt pretty comfortable here” Female subject

13. What do people do at home? Most and least observed activities in 104h annotated set, male subject only Activity Total Observed Time Using a computer 1866 min Background listening to music or radio 813 min Actively watching tv or movies 732 min Sleeping deeply 728 min Reading book/paper/magazine 359 min Preparing a snack 44 s Leaving the home 42 s Making the bed 34 s Washing hands 24 s Drying dishes 6 s

14. Activity Recognition Results Excellent  Poor  Sensors

15. Take-aways (more detail in the paper) People behave unpredictably “in the wild” Revealed by our long-term study with unaffiliated subjects and annotator Interrupted activities, multi-tasking, activities in multiple and unexpected locations Many activities are short and infrequent Collecting sufficient “real life” data very challenging Location “good enough” for recognizing many simple activities Likely not good enough for fine-grained activities Even with 100s of sensors, our sensor density was sometimes insufficient Opportunity for new sensors

16. Next Steps Move to in-home installed systems Address lack of data by studying ways users can give feedback to an adaptive system Ongoing work with MIT Shift focus to older subjects Collaboration with Intel Research Seattle (rest of this talk) FYI PlaceLab annotated dataset available online Sensor data, activity ontology, annotations http://architecture.mit.edu/house_n/data/PlaceLab/PLCouple1.htm

17. Technologies for Long-Term Care (TLC) Joint with Intel Research Seattle Sensors in elders’ homes track activities and generate a summary display for caregivers RFID bracelet and RFID tags “ shake” sensors Display

18. Display Detail 5 day trend Eating and ambulation timeline Today’s grooming and medication status Customizable photo background

19. How to have an experiment in your home… My official roles Inference Lead responsible for algorithms to detect activities Middleware co-designer and contributor My unofficial roles Chief QA engineer Chief guinea pig Only way to get the data I needed to design the inference!

20. 12 shake sensors 10 RFID tags and RFID bracelet 1 aggregator (laptop) with attached 802.15.4 antenna Wore a phone camera around neck for 1 week to get ground truth Kept display at work to easily show demos Home Deployment Sep 2007 – Oct 2008

21. Technical Experience Ease of install and setup 9/10 Some issues with sensors falling off at first Good debugging beforehand paid off Some tweaking of inference Maintenance 6/10 Constantly changing shake sensor batteries – typically one per week Bracelet needed as much attention as a cell phone Some work to keep the Ceiva display up and running Aggregator had 0 downtime due to bugs or failures! Ease of uninstall 10/10 no marks anywhere!

22. User Experience Aesthetics 8/10 Over time found a way to hide most sensors but hated having the laptop and the antenna in the dining room Ease of use Barely noticed shake sensors 9/10 Hated the bracelet after about a month 1/10 Fragile, uncomfortable, needed constant attention, too ugly to wear around guests Friends and family 10/10 (non-bracelet) Many people didn’t even notice the experiment until I pointed it out! I have plenty of computers Only wore the bracelet around my boyfriend Privacy No issues with the main experiment 9/10 Hated wearing camera phone for ground truth 0/10 bulky and felt very invasive Antenna and laptop

23. My Negative Bias I was developer, maintainer and user Typically if there was a problem, I owned it Felt like I was bringing work home I had the display screen at work, not home Much more public place I don’t (yet!) need TLC A burden, especially maintaining the bracelet

24. Findings from TLC Elder Study 2008 Interviews from 4 elders, 2 formal, 2 family caregivers in Seattle area Roughly 10,000 hrs of data 3.5 days of camera ground truth Experience from 9 installs Still finishing the study 14 elders total but don’t expect major changes to findings

25. Finding: TLC is Useful! “ I think it’s a marvelous thing… my sister will call me up and say “I don’t have any apples, why haven’t you eaten ?” Elder “ He brushes his teeth more often now because I can keep track. That wasn't something I really did before.” Caregiver “ [T]here's footprints all over it-- so she wasn't sleeping last night … So, I won't bother her for a while.” Caregiver

26. Finding: TLC is Easy to Use Elder’s ability to tolerate and manage sensor bracelet was a concern Feedback from elders is reassuring: “ Ain't nothing to it. Just takes 30 seconds. I take it off when I leave the house, but sometimes I won't take it off for a week…” TLC elders wear bracelets on over 95% of days.

27. Finding: TLC is Delightful Attractive design and appealing backgrounds are a key part of the displays “ I love it. It's interesting… I wish I could get the cat to come back more. It comes only every week and a half.” Elder

28. Finding: TLC is Well Behaved False-positive rates for activity recognition are under 10% Very simple inference (Naïve Bayes + rules) Acceptable for human-processed output Installation time is under 2 hours per user Failures are as expected from pilots Picture frame backup Battery drain Bracelet antenna damage

29. Are we there yet? We’re making good progress! Heartening to see our system was well received and even liked Debugging at home Great way to increase robustness and get useful feedback instead of focusing on technical problems Sometimes painful but a good way to show faith in a system Definitely not 100% predictive of real users

30. Where to next? We are not far away from useful systems! Already some products on the market Biggest challenges Installation and maintenance Lack of real-world data Work on adaptive systems Make anonymized data public? CHI2009 workshop "Developing Shared Home Behavior Datasets” Move beyond sensing to actuation

31. Collaborators Intel Matthai Philipose, TLC project lead Jennifer Healey, Brad Needham, Ken Lafond, Adam Rea, Polly Powledge, Beverly Harrison, Jean-Manuel Van Thong, Scott Blackwell. MIT House_n Stephen Intille, Director of Technology Emmanuel Munguia-Tapia, Jennifer Beudin University of Washington School of Public Health Sheri Reder, Clinical Instructor Susan Hedrick, Professor

32. Backup