• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Phone As A Sensor Technology: mHealth and Chronic Disease
 

Phone As A Sensor Technology: mHealth and Chronic Disease

on

  • 138 views

The mHealth “revolution” has promised to deliver in-home healthcare that parallels the care we might receive in a physician’s office. However, the panacea of digital health has proven to be more ...

The mHealth “revolution” has promised to deliver in-home healthcare that parallels the care we might receive in a physician’s office. However, the panacea of digital health has proven to be more problematic and messy than its vision, especially for collecting and interpreting medical quantities from the home. In this talk I will discuss several successful projects for sensing medical quantities from a mobile phone using the embedded sensors (i.e., camera, microphone, accelerometer) and how these projects can increase compliance as well as enhance doctor patient relationships. I will focus on the reliability and calibration of the sensing and the role of computer scientists and engineers in the future of mHealth.

Statistics

Views

Total Views
138
Views on SlideShare
138
Embed Views
0

Actions

Likes
0
Downloads
3
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Phone As A Sensor Technology: mHealth and Chronic Disease Phone As A Sensor Technology: mHealth and Chronic Disease Presentation Transcript

    • phone-as-a-sensor technology: mhealth and chronic disease > slide to unlock eric c. larson | eclarson.com Assistant Professor Computer Science and Engineering
    • iPhone 9 M Now with iColon. > slide to unlock
    • Comp Sci. & Engr.
    • arch algorithms OS databases & data mining software Comp Sci. & Engr. languages AI & robotics networking symbolic computing
    • arch algorithms OS databases & data mining software mobile Comp Sci. & Engr. languages AI & robotics networking symbolic computing
    • arch algorithms OS databases & data mining software mobile Comp Sci. & Engr. mhealth languages AI & robotics networking symbolic computing
    • arch algorithms OS databases & data mining software Comp Sci. & Engr. mhealth languages AI & robotics networking symbolic computing mobile
    • what is mhealth?
    • the promise of mhealth:
    • the promise of mhealth: revolutionize medicine
    • the promise of mhealth: revolutionize medicine eliminate doctor visits
    • the promise of mhealth: revolutionize medicine eliminate doctor visits remote / automatic diagnosis
    • the promise of mhealth: revolutionize medicine eliminate doctor visits remote / automatic diagnosis equalize developing countries
    • the promise of mhealth: revolutionize medicine eliminate doctor visits remote / automatic diagnosis equalize developing countries
    • the promise of mhealth: revolutionize medicine eliminate doctor visits remote / automatic diagnosis equalize developing countries
    • heart rate zombie run fitness trainer stress check current mhealth glucose buddy
    • heart rate zombie run stress check current mhealth 43,000 apps for health on the app store fitness trainer glucose buddy
    • heart rate zombie run stress check current mhealth 43,000 apps for health on the app store 96% are for calorie counting & exercise fitness trainer glucose buddy
    • heart rate zombie run stress check current mhealth 43,000 apps for health on the app store 96% are for calorie counting & exercise 4% are remote monitoring fitness trainer glucose buddy
    • consider physician’s needs
    • consider physician’s needs connecting with patient
    • consider physician’s needs connecting with patient tracking baselines
    • consider physician’s needs connecting with patient tracking baselines personalized trending data
    • consider physician’s needs connecting with patient tracking baselines personalized trending data managing chronic disease
    • consider physician’s needs connecting with patient tracking baselines personalized trending data managing chronic disease 30% of all US healthcare spending is on chronic disease
    • mhealth and chronic disease management
    • mhealth and chronic disease management compliance? cost? doctor patient? data reliability?
    • compliance
    • compliance
    • baseline sensor quantity
    • phone as a sensor baseline sensor quantity
    • phone as a sensor baseline embedded sensors sensor quantity
    • phone as a sensor baseline embedded sensors sensor estimated quantity processing
    • accelerometer gyroscope magnetometer /compass dual camera / flash 1+ microphones proximity sensor capacitive sensor gps motorized actuator wireless antenna (s)
    • compliance++; cost--; dr_pat *= 10; accelerometer gyroscope magnetometer /compass dual camera / flash 1+ microphones proximity sensor capacitive sensor gps motorized actuator wireless antenna (s)
    • accelerometer gyroscope magnetometer /compass dual camera / flash 1+ microphones compliance++; proximity sensor cost--; capacitive sensor dr_pat *= 10; gps motorized actuator data reliability? wireless antenna (s)
    • what can the mobile phone sense with clinical accuracy?
    • lung function jaundice future research
    • lung function jaundice future research
    • spirometer ? lung function?
    • lung function evaluates pulmonary impairments asthma COPD cystic fibrosis
    • spirometer device that measures amount of air inhaled and exhaled.
    • volume flow using a spirometer time volume
    • volume flow using a spirometer time volume
    • volume flow using a spirometer time volume
    • volume volume-time graph time
    • volume volume-time graph time
    • volume-time graph FVC volume FEV1 time FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
    • volume-time graph FVC volume FEV1 1 sec. time FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
    • volume-time graph FEV1% = FEV1/FVC FVC volume FEV1 1 sec. time FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
    • FEV1% = FEV1/FVC FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
    • FEV1% = FEV1/FVC > 80% healthy 60 - 79% mild 40 - 59% moderate < 40% severe FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
    • flow flow-volume graph volume
    • flow flow-volume graph volume
    • flow-volume graph PEF flow 1 sec. FEV1 FVC volume PEF: Peak Expiratory Flow FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
    • flow-volume graph flow normal volume
    • flow-volume graph normal flow obstructive volume
    • obstructive diseases ! resistance in air path leads to reduced air flow
    • obstructive diseases ! resistance in air path leads to reduced air flow
    • restrictive diseases ! lungs are unable to pump enough air and pressure
    • restrictive diseases ! lungs are unable to pump enough air and pressure
    • flow-volume graph normal Flow obstructive Volume
    • flow-volume graph normal Flow obstructive restrictive Volume
    • clinical spirometry
    • home spirometry
    • home spirometry ! faster detection rapid recovery trending
    • challenges with home spirometry high cost barrier patient compliance less coaching limited integration
    • SpiroSmart availability cost portability more effective coaching interface integrated uploading
    • Using SpiroSmart
    • Using SpiroSmart
    • Using SpiroSmart ]
    • Using SpiroSmart ]
    • Using SpiroSmart ]
    • Using SpiroSmart
    • Using SpiroSmart
    • flow rate volume airflow sensor lung function
    • flow rate volume airflow sensor sound pressure microphone lung function
    • flow rate volume airflow sensor estimated lung function sound pressure microphone processing
    • audio
    • audio flow features
    • audio flow features measures regression FEV1 FVC PEF
    • audio flow features measures regression curve regression Flow(L/s) 10 10 5 5 0 0 1 0 40 Volume(L) Volume(L) FEV1 FVC PEF Flow(L/s) 15 15 2 Volume(L) 4 2 Volume(L) 1 3 3 3 4 2 1 3 0 20 1 2 4 6 time(s) 8 10 4
    • audio flow features measures regression curve regression Flow(L/s) lung function 10 10 5 5 0 0 1 0 40 Volume(L) Volume(L) FEV1 FVC PEF Flow(L/s) 15 15 2 Volume(L) 4 2 Volume(L) 1 3 3 3 4 2 1 3 0 20 1 2 4 6 time(s) 8 10 4
    • flow features estimation amplitude 1 0.5 0 −0.5 −1 0 1 2 3 4 time(s) 5 6 7
    • flow features estimation 0.5 0 −0.5 −1 0 1 2 3 4 5 6 time(s) source vocal tract 2500 frequency(Hz) amplitude 1 2000 1500 1000 500 0 output 7
    • flow features estimation lpc8raw amplitude 1 0.5 0.5 0 −0.5 0 flow estimation features amplitude −0.5 1 2 3 4 5 6 7 time(s) 0.5 0 −0.5 resonance tracking lpc8raw 1 2 3 4 5 6 time(s) source vocal tract 2500 2000 1500 1000 500 0 −1 1 0 7 amplitude −1 0 envelope detection −1 10 frequency(Hz) amplitude 1 output 1 2 3 4 5 6 7 time(s) 0.5 0 −0.5 −1 0 auto-regressive estimate 1 2 3 4 time(s) 5 6 7
    • measures regression 8 flow(L/s) 6 4 2 ground truth 0 −1 0 1 2 3 4 5 time(s) feature value 0.4 0.3 0.2 0.1 0 feature 1 0 1 2 4 5 time(s) 0.4 feature value 3 0.3 0.2 0.1 feature 2 0 −1 0 1 2 time(s) 3 4 5
    • 8 6 flow(L/s) measures regression 7.1 4 2 PEF features ground truth 0 −1 0 1 2 3 4 5 time(s) feature value 0.4 0.3 0.2 0.1 0 feature 1 0 1 2 4 5 time(s) 0.4 feature value 3 0.3 0.2 0.1 feature 2 0 −1 0 1 2 time(s) 3 4 5
    • 8 6 flow(L/s) measures regression 7.1 4 2 PEF features ground truth 0 −1 0 1 2 3 4 5 time(s) feature value 0.4 0.3 0.2 0.1 0 feature 1 0 1 2 4 5 time(s) 0.4 feature value 3 0.3 0.2 0.1 feature 2 0 −1 0 1 2 time(s) 3 4 5 0.35 0.33
    • 8 6 flow(L/s) measures regression 7.1 3.2 4 2 PEF features ground truth 0 −1 0 1 2 3 4 5 time(s) feature value 0.4 0.3 0.2 0.1 0 FEV1 features feature 1 0 1 2 3 4 5 time(s) 0.4 feature value 0.35 0.33 0.3 0.2 0.1 feature 2 0 −1 0 1 2 time(s) 3 4 5
    • 8 6 flow(L/s) measures regression 7.1 3.2 4 2 PEF features ground truth 0 −1 0 1 2 3 4 5 time(s) feature value 0.4 0.3 0.2 0.1 0 FEV1 features feature 1 0 1 2 3 4 0.12 0.17 5 time(s) 0.4 feature value 0.35 0.33 0.3 0.2 0.1 feature 2 0 −1 0 1 2 time(s) 3 4 5
    • measures regression FEV1 features PEF features
    • measures regression FEV1 features PEF features bagged decision tree bagged decision tree
    • measures regression FEV1 features PEF features bagged decision tree bagged decision tree output output
    • curve regression feature 1 0.3 0.2 0.1 0 1 3 4 5 time(s) 0.4 feature value 2 feature N 0.3 0.2 0.1 0 ... 0 −1 ... windowed machine learning regression feature value 0.4 0 1 2 time(s) 3 4 5
    • curve regression feature 1 0.3 0.2 0.1 0 1 3 4 5 time(s) 0.4 feature value 2 ... 0 −1 ... feature N 0.3 0.2 0.1 0 8 6 flow(L/s) windowed machine learning regression feature value 0.4 4 2 0 0 1 2 3 4 5 time(s) curve output
    • curve regression 0.4 feature value feature value 0.4 0.3 0.2 0.1 0 −1 0 1 2 3 time(s) bagged decision tree 4 0.3 0.2 0.1 0 5 0 1 2 time(s) 3 4 5
    • curve regression 0.4 feature value feature value 0.4 0.3 0.2 0.1 0 −1 0 1 2 3 time(s) bagged decision tree 4 0.3 0.2 0.1 0 5 0 1 2 3 4 time(s) CRF 5
    • curve regression 0.4 feature value feature value 0.4 0.3 0.2 0.1 0 −1 0 1 2 3 4 0.3 0.2 0.1 0 5 0 1 2 bagged decision tree ! 6 flow(L/s) 4 CRF 8 4 2 0 −1 3 time(s) time(s) 0 1 2 time(s) 3 4 5 5
    • study design x3 x3
    • study enrollment
    • study enrollment study a participants 52 18-75 years old, mostly healthy
    • study enrollment study a participants 52 18-75 years old, mostly healthy study b participants 10 12-17 years old, mixed healthy/abnormal
    • study enrollment study a participants 52 18-75 years old, mostly healthy study b 10 12-17 years old, mixed healthy/abnormal study c participants 56 10-69 years old, mostly abnormal enrolled by pulmonologists participants
    • measures regression results
    • measures regression results
    • curves regression results 15 6 flow(L/s) flow(L/s) 8 4 2 0 −1 0 1 2 volume(L) 3 4 10 5 0 −2 0 2 volume(L) 4 6
    • curves regression results 15 6 flow(L/s) flow(L/s) 8 4 2 0 −1 0 1 2 volume(L) 3 4 10 5 0 −2 0 2 volume(L) 4 6
    • curves regression results 15 flow(L/s) 6 4 2 0 −1 0 1 2 3 10 5 0 −2 4 0 volume(L) 2 volume(L) 15 flow(L/s) flow(L/s) 8 10 5 0 −2 0 2 volume(L) 4 6 4 6
    • curves regression results 15 flow(L/s) 6 4 2 0 −1 0 1 2 3 10 5 0 −2 4 0 volume(L) 2 volume(L) 15 flow(L/s) flow(L/s) 8 10 5 0 −2 0 2 volume(L) 4 6 4 6
    • can SpiroSmart curves be used for diagnosis?
    • survey
    • survey • normal/abnormal subjects curves
    • survey • normal/abnormal subjects curves • 5 pulmonologists
    • survey • normal/abnormal subjects curves • 5 pulmonologists • unaware if from SpiroSmart / spirometer
    • results restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive
    • results restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive identical 64%
    • results restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive one off 10% identical 64%
    • results restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive one off 10% identical 64%
    • results restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive false positive 14% one off 10% identical 64%
    • results false negative 4% restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive false positive 14% one off 10% identical 64%
    • results error 8% false negative 4% restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive false positive 14% one off 10% identical 64%
    • results error 8% false negative 4% restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive false positive 14% ! ! abnormal vs normal one off 96% identical 10% 64%
    • appropriate for trending and screening
    • global health non-profit
    • global health non-profit patient and doctors
    • global health non-profit patient and doctors pharmaceutical drug trials
    • lung function jaundice future research
    • lung function jaundice future research
    • neonatal jaundice in the US kernicterus: 21 hazardous jaundice: 1158 extreme jaundice: 2,317 severe jaundice: 35,000 phototherapy: 290,000 visible jaundice: 3.5 million births/year: 4.1 million
    • Method Accuracy Disadvantages TSB Gold standard (r=1.0) Painful, costly, inconvenient, delayed Accurate (r= 0.75 -0.93) Meter = $7000 tips $5 unavailable in most physician offices TcB Visual assessment (provider or parent) not accurate (r= 0.36 - 0.7), underestimates severity No standardization lighting, pigmentation
    • bilirubin level in blood blood draw jaundice level
    • bilirubin level in blood blood draw yellowness camera jaundice level
    • bilirubin level in blood blood draw estimated jaundice level yellowness camera processing
    • bilicam
    • study A participants 48 newborns 0-4 days old, collected in nursery 3 hospitals in Washington & Philadelphia
    • study A participants 48 newborns 0-4 days old, collected in nursery 3 hospitals in Washington & Philadelphia
    • bilicam Color Linearization • Camera Settings Adjustment • Light Source Estimation Image Segmentation • Quality Control for Distance, Lighting, and Shadow • Sternum, Forehead, Card Segmented Color Calibration • Dynamic Least Squares Regression • Automatic Feature Selection Neonatal Skin Response to Bilirubin • Skin Independent Color Transformations Applied • Multivariate Machine Learning Regression
    • bilicam initial results bilirubin level 20 15 10 5 0 r=0.91 mg/dl 20 5 10 15 bilicam estimation
    • bilicam initial results bilirubin level 20 15 10 5 0 r=0.91 non white mg/dl white 20 5 10 15 bilicam estimation
    • bilicam initial results bilirubin level 20 TcB = 0.85 15 10 5 0 r=0.91 non white mg/dl white 20 5 10 15 bilicam estimation
    • bilicam initial results bilirubin level 20 TcB = 0.85 BiliCam = 0.84 15 10 5 0 r=0.91 non white mg/dl white 20 5 10 15 bilicam estimation
    • bilicam future work
    • bilicam future work • near term: screening
    • bilicam future work • near term: screening • medium term: more data
    • bilicam future work • near term: screening • medium term: more data • long term: developing world
    • bilicam future work • near term: screening • medium term: more data • long term: developing world “in many resource poor nations, hyperbilirubinemia is the second or third leading cause of infant mortality and disability”
    • lung function jaundice future research
    • lung function jaundice future research
    • future research oxygen volume, VO2
    • cardiac output and blood pressure
    • intra ocular pressure
    • intra ocular pressure PressCam
    • eclarson.com eclarson@lyle.smu.edu > slide to unlock @ec_larson Thank You!
    • phone-as-a-sensor technology: mhealth and chronic disease eclarson.com eclarson@lyle.smu.edu > slide to unlock @ec_larson eric c. larson | eclarson.com Assistant Professor Computer Science and Engineering collaborators: ! Joseph Camp Shwetak Patel Jim Stout, MD Jim Taylor, MD Margaret Rosenfeld, MD Gaetano Boriello Mayank Goel Lilian DeGreef
    • phone-as-a-sensor technology: mhealth and chronic disease eclarson.com eclarson@lyle.smu.edu @ec_larson eric c. larson | eclarson.com Assistant Professor Computer Science and Engineering collaborators: ! Joseph Camp Shwetak Patel Jim Stout, MD Jim Taylor, MD Margaret Rosenfeld, MD Gaetano Boriello Mayank Goel Lilian DeGreef