Utilising the Ubiquity of the Cell    Phone to Record Physiological               Activities                           Aid...
Overview• BACKGROUND  • Cell Phone Ubiquity  • Lifelogging• CELL PHONE DATA LOGGER  • A Cell Phone Data Logging Framework•...
CLARITYPrincipal InvestigatorsPrincipal Investigators  Prof. Barry Smyth             - Personalization, recommender system...
CLARITYCLARITY What ? “The Sensor Web” – Increasing availability of cheap, robust, and deployable   sensor technologies us...
Cell phone ubiquity-4 billion Cell Phones inWorld  -Approx 1bn PCs- Almost 70% of new cellphone subscriptionscome from dev...
RPF on Cellphone as Platform for Healthcare• 14 universities supported• Cell phones can provide people with access to tech...
How often do you visit your Doctor?                                                                                       ...
LifeloggingLifelogging is about digitally recording your daily lifeSometimes its for a reason Work           e.g. security...
Lifelogging DevicesMuch past research focus on miniaturising hardware and increasing battery‐life + storage e.g. visual li...
Aims of this project• Utilise cell phone ubiquity  •   Logging platform on Windows Mobile devices  •   Framework allows ea...
Overview• BACKGROUND  • Cell Phone Ubiquity  • Lifelogging• CELL PHONE DATA LOGGER  • A Cell Phone Data Logging Framework•...
SmartLogger Overview                                                                    Cellular Heart Rate               ...
Easily include new sensors  Heart Rate                   XML                                                              ...
Data Logger Summary• .NET Windows Mobile• Easy to incorporate additional Bluetooth  sensors• Has to deal with incomplete a...
Overview• BACKGROUND  • Cell Phone Ubiquity  • Lifelogging• CELL PHONE DATA LOGGER  • A Cell Phone Data Logging Framework•...
How to review lots of data?Physiological   data:  Little emphasis on   visualisation                 IEEE TRANSACTIONS ON ...
Simple view of Human Memory•SENSORY•SHORT – TERM•LONG – TERM  –PROCEDURAL  –DECLARATIVE     • Semantic     • EPISODIC/ AUT...
Cued Recall & Visual Encoding •“Cued Recall” better than “Free Recall” (Purdy, ’01) •Memories can be temporally encoded (L...
Our Take…To effectively help people understand their physiological data:• Passively logged cell phone data gives potential...
My Physiological Diary      QUERY                                                                    CONTEXT      PANEL   ...
My Physiological Diary:Location Context         UNIVERSITY COLLEGE DUBLIN      DUBLIN CITY UNIVERSITY      TYNDALL NATIONA...
My Physiological Diary:Image Context         UNIVERSITY COLLEGE DUBLIN      DUBLIN CITY UNIVERSITY      TYNDALL NATIONAL I...
My Physiological Diary:Compare across days/months/years        UNIVERSITY COLLEGE DUBLIN      DUBLIN CITY UNIVERSITY      ...
My Physiological Diary:Display normalised values         UNIVERSITY COLLEGE DUBLIN      DUBLIN CITY UNIVERSITY      TYNDAL...
My Physiological Diary:Delve deeper into data         UNIVERSITY COLLEGE DUBLIN      DUBLIN CITY UNIVERSITY      TYNDALL N...
My Physiological Diary:Adaptively query based on time         UNIVERSITY COLLEGE DUBLIN      DUBLIN CITY UNIVERSITY      T...
Result – OpenSource• Allows sensor device researchers concentrate on the   their hardware/chemistry/physics strengths• Wil...
Overview• BACKGROUND  • Cell Phone Ubiquity  • Lifelogging• CELL PHONE DATA LOGGER  • A Cell Phone Data Logging Framework•...
Use Case – Sleep Apnea• 12 million people in USA have   sleep apnea• Process of diagnosis can involve   going to “sleep la...
Future challenge – Security                                                                    Cellular Heart Rate        ...
Future – Health Vault                                                                    Cellular Heart Rate              ...
Future – Symptom detection                                                                           Cellular Heart Rate  ...
Dublin SenseCam WorkActivityRecognition27 “concepts”Outputs manually judgedon ~95k images (5 users)                UNIVERS...
Comparison of Lifestyle Within                                                           Social Groups                    ...
Future – Zigbee                                                                    Cellular Heart Rate                    ...
Conclusions•Utilising cell phone ubiquity  • Logging platform on Windows Mobile devices  • Framework allows easy integrati...
Thank YouSpecial thanks to Kristin Tolle, Tim Chou, Mike Sinclair, Kristin Lauter, Eric                  Horvitz, Roger Ba...
Upcoming SlideShare
Loading in …5
×

Cellphone as a platform for health care (given to Microsoft Research Redmond)

2,073 views

Published on

Presentation given at Microsoft Research on a framework which leverages the ubiquity of the cell phone, to aggregate multiple wearable biological sensors.

Seminar at the Health & Wellbeing External Research Group, 23 April 2009, Microsoft Research, Redmond, WA, USA.

Abstract:
Lifelogging is a term used to describe recording different aspects of your daily life, in digital form, for your own exclusive personal use. It is a form of reverse surveillance, sometimes termed sousveillance, referring to us - the subjects - doing the watching, of ourselves. In today's healthcare world where we react to conditions that have already developed, lifelogging technologies may offer a glimpse into a future world of proactive healthcare where symptoms of conditions are detected at much earlier stages.

At the end of last year it was estimated that there were 4 billion cell phones in the world, in comparison to just over 1 billon PCs. In this presentation I will discuss a framework, which leverages the ubiquity of the cell phone, to aggregate multiple wearable biological sensors. This physiological lifelogged data can then be easily queried via an interface which utilises contextual memory retrieval cues to assist people remember what type of activity they were doing at a particular time. This may be helpful for the diagnosis of potential medical conditions e.g. to explain that my heart rate was very high because I was at the gym, or that I had a disturbed night's sleep because I was in an unfamiliar hotel room.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,073
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Cellphone as a platform for health care (given to Microsoft Research Redmond)

  1. 1. Utilising the Ubiquity of the Cell  Phone to Record Physiological  Activities Aiden Doherty Mentor: Kristin Tolle External Research, Microsoft Research &CLARITY: Centre for Sensor Web Technologies, Dublin, Ireland UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  2. 2. Overview• BACKGROUND • Cell Phone Ubiquity • Lifelogging• CELL PHONE DATA LOGGER • A Cell Phone Data Logging Framework• MY PHYSIOLOGICAL DIARY • Reviewing Physiological Data Using Contextual Information• ONGOING WORK UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  3. 3. CLARITYPrincipal InvestigatorsPrincipal Investigators Prof. Barry Smyth - Personalization, recommender systems, mobile computing Prof. Alan Smeaton - Content-based information retrieval Prof. Dermot Diamond - Materials research, wearable sensors Prof. Noel O’Connor - Audio-visual analysis, multi-modal information processing Mr. Gregory O’Hare - Ubiquitous computing, multi-agent systemsAssociate PIs Prof. Paddy Nixon - Pervasive computing, middleware, security, trust, privacy Prof. Niall Moyna - Sports Science, wearable sensing Dr. Simon Dobson - Middleware, pervasive computing Dr. Cian O’Mathuna - Sensor devices, energy-aware hardware Dr. Brian Caulfield - Physiotherapy, therapeutic gaming, wearable sensorsQuick Stats•$21.4M over 5 years ($15.4M from Irish government, $6M from industry)•84 researchers (28 academics, 31 post-docs, 25 PhD students)•12 support staff UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  4. 4. CLARITYCLARITY What ? “The Sensor Web” – Increasing availability of cheap, robust, and deployable  sensor technologies ushering in a wave of new information  sources; – Ubiquitous, dynamic, noisy, reactive and yielding  unstructured data‐streams == sensor web – Realizing the sensor web demands a large‐scale, multi‐ disciplinary research effort == CLARITY – Moving beyond our research silos to novel research  interactions; – Demonstrator projects in: Personal health and wellness; Environmental monitoring; UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  5. 5. Cell phone ubiquity-4 billion Cell Phones inWorld -Approx 1bn PCs- Almost 70% of new cellphone subscriptionscome from developingnations (Source: International Telecommunications Union)- Bluetooth is nowstandard on most cell Source: http://www.dialaphone.co.uk/blog/?p=1485phones UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  6. 6. RPF on Cellphone as Platform for Healthcare• 14 universities supported• Cell phones can provide people with access to technology‐based healthcare solutions •Who otherwise would have no such opportunities UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  7. 7. How often do you visit your Doctor? Visits to Dr. My Life UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  8. 8. LifeloggingLifelogging is about digitally recording your daily lifeSometimes its for a reason Work e.g. security personnel, medical staff, etc.Personal e.g. diaries, etc.Sometimes its for posterityRecording vacations, family gatherings, social occasionsSometimes its because we canAnd we’re not yet sure what we’ll do with it e.g. MyLifeBits UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE  8
  9. 9. Lifelogging DevicesMuch past research focus on miniaturising hardware and increasing battery‐life + storage e.g. visual lifelogging domain Steve Mann. Wearable computing: a first step toward personal imaging. Computer, 30:25–32, Feb 1997. TIMELINE Tano et. al. University of Electro-Communications, Tokyo, Japan Microsoft Research SenseCam UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE  9
  10. 10. Aims of this project• Utilise cell phone ubiquity • Logging platform on Windows Mobile devices • Framework allows easy integration of new BT sensors• Reviewing physiological values • Interface to monitor, analyse & browse through huge  volumes of sensor data • “Individualise” medical baselines UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  11. 11. Overview• BACKGROUND • Cell Phone Ubiquity • Lifelogging• CELL PHONE DATA LOGGER • A Cell Phone Data Logging Framework• MY PHYSIOLOGICAL DIARY • Reviewing Physiological Data Using Contextual Information• ONGOING WORK UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  12. 12. SmartLogger Overview Cellular Heart Rate Wi‐Fi XML LAN Data  ActiveSync Store … Location Images Body Temperature UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  13. 13. Easily include new sensors Heart Rate XML Sleep apnea neck cuff with oximeter, accelerometer, microphone, & gsr sensors Location Zephyr HxM: An example of a modern biometric sensor sending out BT readings Images Body Temperature UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  14. 14. Data Logger Summary• .NET Windows Mobile• Easy to incorporate additional Bluetooth sensors• Has to deal with incomplete and heterogeneous data UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  15. 15. Overview• BACKGROUND • Cell Phone Ubiquity • Lifelogging• CELL PHONE DATA LOGGER • A Cell Phone Data Logging Framework• MY PHYSIOLOGICAL DIARY • Reviewing Physiological Data Using Contextual  Information• ONGOING WORK UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  16. 16. How to review lots of data?Physiological  data: Little emphasis on  visualisation IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 8, NO. 4, DECEMBER 2004 439 A Wireless PDA-Based Physiological Monitoring System for Patient Transport Yuan-Hsiang Lin, I-Chien Jan, Patrick Chow-In Ko, Yen-Yu Chen, Jau-Min Wong, and Gwo-Jen Jan UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  17. 17. Simple view of Human Memory•SENSORY•SHORT – TERM•LONG – TERM –PROCEDURAL –DECLARATIVE • Semantic • EPISODIC/ AUTOBIOGRAPHICAL UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  18. 18. Cued Recall & Visual Encoding •“Cued Recall” better than “Free Recall” (Purdy, ’01) •Memories can be temporally encoded (Larsen, ’96) •Distinct memories are more strongly encoded (Purdy, ’01) •Memories stored by association (Baddeley, ’04) UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  19. 19. Our Take…To effectively help people understand their physiological data:• Passively logged cell phone data gives potential “cues”• Query data on “temporal” axes (calendar constraints)• Highlight more “distinctive” events (charts)• “Associate” related events (location + images) UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  20. 20. My Physiological Diary QUERY CONTEXT PANEL PANEL CHART PANEL UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  21. 21. My Physiological Diary:Location Context UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  22. 22. My Physiological Diary:Image Context UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  23. 23. My Physiological Diary:Compare across days/months/years UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  24. 24. My Physiological Diary:Display normalised values UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  25. 25. My Physiological Diary:Delve deeper into data UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  26. 26. My Physiological Diary:Adaptively query based on time UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  27. 27. Result – OpenSource• Allows sensor device researchers concentrate on the  their hardware/chemistry/physics strengths• Will allow machine learning researchers easily  aggregate data to apply their techniques• Will allow health conscious individuals more easily  make sense of the data they’ve been collecting UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  28. 28. Overview• BACKGROUND • Cell Phone Ubiquity • Lifelogging• CELL PHONE DATA LOGGER • A Cell Phone Data Logging Framework• MY PHYSIOLOGICAL DIARY • Reviewing Physiological Data Using Contextual Information• ONGOING WORK UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  29. 29. Use Case – Sleep Apnea• 12 million people in USA have  sleep apnea• Process of diagnosis can involve  going to “sleep lab” Sleep apnea neck cuff with oximeter, accelerometer, microphone, & gsr sensors• In preliminary discussions with  Sleep Disorders Center in UW  Medical School UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  30. 30. Future challenge – Security Cellular Heart Rate Wi‐Fi XML LAN Data  ActiveSync Store … Location Images Body Temperature UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  31. 31. Future – Health Vault Cellular Heart Rate Wi‐Fi XML LAN Data  ActiveSync Store … Location Images Body Temperature UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  32. 32. Future – Symptom detection Cellular Heart Rate Wi‐Fi XML LAN Data  ActiveSync Store … You appear to be showing  Hey it looks like you’re  Only kidding !!! some symptoms of sleep  Location writing a letter!!! aponea Images Body Temperature UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  33. 33. Dublin SenseCam WorkActivityRecognition27 “concepts”Outputs manually judgedon ~95k images (5 users) UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  34. 34. Comparison of Lifestyle Within  Social Groups steeringWheel 3.5 eating insideVehicle vehiclesExternal readingstandard deviations away from sample mean 2.5 holdingPhone steeringWheel vehiclesExternal holdingPhone reading 1.5 eating insideVehicle 0.5 -0.5 user 1 user 2 user 3 user 4 user 5 -1.5 UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  35. 35. Future – Zigbee Cellular Heart Rate Wi‐Fi XML LAN Data  ActiveSync Store … Location Images Body Temperature UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  36. 36. Conclusions•Utilising cell phone ubiquity • Logging platform on Windows Mobile devices • Framework allows easy integration of new BT sensors•Reviewing physiological values • Interface to monitor, analyse & browse through huge volumes of sensor data • “Individualise” medical baselines•Lot’s of exciting future directions!!! UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 
  37. 37. Thank YouSpecial thanks to Kristin Tolle, Tim Chou, Mike Sinclair, Kristin Lauter, Eric  Horvitz, Roger Barga, and Jim Gemmell. further information: http://www.cdvp.dcu.ie/SenseCam UNIVERSITY COLLEGE DUBLIN    DUBLIN CITY UNIVERSITY    TYNDALL NATIONAL INSTITUTE 

×