phone-as-a-sensor technology:
mhealth and chronic disease

> slide to unlock

eric c. larson | eclarson.com
Assistant Prof...
iPhone 9

M

Now with iColon.

> slide to unlock
Comp
Sci. & Engr.
arch

algorithms
OS

databases
& data
mining

software

Comp
Sci. & Engr.

languages

AI &
robotics
networking

symbolic
c...
arch

algorithms
OS

databases
& data
mining

software

mobile

Comp
Sci. & Engr.

languages

AI &
robotics
networking

sy...
arch

algorithms
OS

databases
& data
mining

software

mobile

Comp
Sci. & Engr.

mhealth

languages

AI &
robotics
netwo...
arch

algorithms
OS

databases
& data
mining

software

Comp
Sci. & Engr.

mhealth

languages

AI &
robotics
networking

s...
what is mhealth?
the promise of mhealth:
the promise of mhealth:
revolutionize medicine
the promise of mhealth:
revolutionize medicine
eliminate doctor visits
the promise of mhealth:
revolutionize medicine
eliminate doctor visits
remote / automatic diagnosis
the promise of mhealth:
revolutionize medicine
eliminate doctor visits
remote / automatic diagnosis
equalize developing co...
the promise of mhealth:
revolutionize medicine
eliminate doctor visits
remote / automatic diagnosis
equalize developing co...
the promise of mhealth:
revolutionize medicine
eliminate doctor visits
remote / automatic diagnosis
equalize developing co...
heart rate
zombie run

fitness trainer

stress check

current mhealth

glucose buddy
heart rate
zombie run

stress check

current mhealth

43,000 apps for health on the app store

fitness trainer

glucose bud...
heart rate
zombie run

stress check

current mhealth

43,000 apps for health on the app store
96% are for calorie counting...
heart rate
zombie run

stress check

current mhealth

43,000 apps for health on the app store
96% are for calorie counting...
consider physician’s needs
consider physician’s needs
connecting with patient
consider physician’s needs
connecting with patient
tracking baselines
consider physician’s needs
connecting with patient
tracking baselines
personalized trending data
consider physician’s needs
connecting with patient
tracking baselines
personalized trending data
managing chronic disease
consider physician’s needs
connecting with patient
tracking baselines
personalized trending data
managing chronic disease
...
mhealth and chronic disease
management
mhealth and chronic disease
management
compliance?
cost?
doctor patient?
data reliability?
compliance
compliance
baseline

sensor

quantity
phone as a sensor
baseline

sensor

quantity
phone as a sensor
baseline

embedded
sensors

sensor

quantity
phone as a sensor
baseline

embedded
sensors

sensor

estimated
quantity

processing
accelerometer
gyroscope
magnetometer /compass
dual camera / flash
1+ microphones
proximity sensor
capacitive sensor
gps
mot...
compliance++;
cost--;
dr_pat *= 10;

accelerometer
gyroscope
magnetometer /compass
dual camera / flash
1+ microphones
proxi...
accelerometer
gyroscope
magnetometer /compass
dual camera / flash
1+ microphones
compliance++;
proximity sensor
cost--;
cap...
what can the mobile phone
sense with clinical accuracy?
lung
function

jaundice

future
research
lung
function

jaundice

future
research
spirometer ?

lung function?
lung function
evaluates pulmonary
impairments
asthma
COPD
cystic fibrosis
spirometer

device that measures
amount of air inhaled and
exhaled.
volume

flow

using a spirometer

time

volume
volume

flow

using a spirometer

time

volume
volume

flow

using a spirometer

time

volume
volume

volume-time graph

time
volume

volume-time graph

time
volume-time graph
FVC

volume

FEV1

time

FEV1: Forced Expiratory Volume in 1 second
FVC: Forced Vital Capacity
volume-time graph
FVC

volume

FEV1

1 sec.

time

FEV1: Forced Expiratory Volume in 1 second
FVC: Forced Vital Capacity
volume-time graph
FEV1% = FEV1/FVC

FVC

volume

FEV1

1 sec.

time

FEV1: Forced Expiratory Volume in 1 second
FVC: Force...
FEV1% = FEV1/FVC

FEV1: Forced Expiratory Volume in 1 second
FVC: Forced Vital Capacity
FEV1% = FEV1/FVC

> 80%

healthy

60 - 79%

mild

40 - 59%

moderate

< 40%

severe

FEV1: Forced Expiratory Volume in 1 s...
flow

flow-volume graph

volume
flow

flow-volume graph

volume
flow-volume graph
PEF

flow

1 sec.

FEV1

FVC
volume

PEF: Peak Expiratory Flow
FEV1: Forced Expiratory Volume in 1 second
...
flow-volume graph

flow

normal

volume
flow-volume graph
normal
flow

obstructive

volume
obstructive diseases

!

resistance in air path leads to reduced air flow
obstructive diseases

!

resistance in air path leads to reduced air flow
restrictive diseases

!

lungs are unable to pump enough air and pressure
restrictive diseases

!

lungs are unable to pump enough air and pressure
flow-volume graph
normal
Flow

obstructive

Volume
flow-volume graph
normal
Flow

obstructive
restrictive
Volume
clinical spirometry
home spirometry
home spirometry

!

faster detection
rapid recovery
trending
challenges with

home spirometry

high cost barrier
patient compliance
less coaching
limited integration
SpiroSmart
availability
cost
portability
more effective coaching interface
integrated uploading
Using SpiroSmart
Using SpiroSmart
Using SpiroSmart

]
Using SpiroSmart

]
Using SpiroSmart

]
Using SpiroSmart
Using SpiroSmart
flow rate
volume

airflow
sensor

lung function
flow rate
volume

airflow
sensor

sound
pressure

microphone

lung function
flow rate
volume

airflow
sensor

estimated
lung function

sound
pressure

microphone

processing
audio
audio
flow features
audio
flow features
measures
regression
FEV1
FVC
PEF
audio
flow features
measures
regression

curve
regression
Flow(L/s)

10

10

5

5
0

0

1

0
40

Volume(L) Volume(L)

FEV1
...
audio
flow features
measures
regression

curve
regression
Flow(L/s)

lung function

10

10

5

5
0

0

1

0
40

Volume(L) V...
flow features estimation
amplitude

1
0.5
0
−0.5
−1
0

1

2

3

4

time(s)

5

6

7
flow features estimation
0.5
0
−0.5
−1
0

1

2

3

4

5

6

time(s)

source

vocal tract
2500

frequency(Hz)

amplitude

1
...
flow features estimation
lpc8raw

amplitude

1

0.5

0.5
0
−0.5

0

flow estimation features

amplitude

−0.5

1

2

3

4

5...
measures
regression

8

flow(L/s)

6
4
2

ground truth

0
−1

0

1

2

3

4

5

time(s)

feature value

0.4
0.3
0.2
0.1
0
...
8
6

flow(L/s)

measures
regression

7.1

4
2

PEF features

ground truth

0
−1

0

1

2

3

4

5

time(s)

feature value
...
8
6

flow(L/s)

measures
regression

7.1

4
2

PEF features

ground truth

0
−1

0

1

2

3

4

5

time(s)

feature value
...
8
6

flow(L/s)

measures
regression

7.1
3.2

4
2

PEF features

ground truth

0
−1

0

1

2

3

4

5

time(s)

feature va...
8
6

flow(L/s)

measures
regression

7.1
3.2

4
2

PEF features

ground truth

0
−1

0

1

2

3

4

5

time(s)

feature va...
measures regression
FEV1 features

PEF features
measures regression
FEV1 features

PEF features

bagged
decision tree

bagged
decision tree
measures regression
FEV1 features

PEF features

bagged
decision tree

bagged
decision tree

output

output
curve regression
feature 1

0.3
0.2
0.1

0

1

3

4

5

time(s)

0.4

feature value

2

feature N

0.3
0.2
0.1
0

...

0
−...
curve regression
feature 1

0.3
0.2
0.1

0

1

3

4

5

time(s)

0.4

feature value

2

...

0
−1

...

feature N

0.3
0.2...
curve regression
0.4

feature value

feature value

0.4
0.3
0.2
0.1
0
−1

0

1

2

3

time(s)

bagged
decision tree

4

0....
curve regression
0.4

feature value

feature value

0.4
0.3
0.2
0.1
0
−1

0

1

2

3

time(s)

bagged
decision tree

4

0....
curve regression
0.4

feature value

feature value

0.4
0.3
0.2
0.1
0
−1

0

1

2

3

4

0.3
0.2
0.1
0
5

0

1

2

bagged
...
study design

x3
x3
study enrollment
study enrollment
study a
participants

52

18-75 years old, mostly healthy
study enrollment
study a
participants

52

18-75 years old, mostly healthy
study b
participants

10

12-17 years old, mixe...
study enrollment
study a
participants

52

18-75 years old, mostly healthy
study b
10

12-17 years old, mixed healthy/abno...
measures regression results
measures regression results
curves regression results
15

6

flow(L/s)

flow(L/s)

8

4
2
0
−1

0

1

2

volume(L)

3

4

10

5

0
−2

0

2

volume(L)...
curves regression results
15

6

flow(L/s)

flow(L/s)

8

4
2
0
−1

0

1

2

volume(L)

3

4

10

5

0
−2

0

2

volume(L)...
curves regression results
15

flow(L/s)

6
4
2
0
−1

0

1

2

3

10

5

0
−2

4

0

volume(L)

2

volume(L)

15

flow(L/s)...
curves regression results
15

flow(L/s)

6
4
2
0
−1

0

1

2

3

10

5

0
−2

4

0

volume(L)

2

volume(L)

15

flow(L/s)...
can SpiroSmart curves be
used for diagnosis?
survey
survey
• normal/abnormal subjects curves
survey
• normal/abnormal subjects curves
• 5 pulmonologists
survey
• normal/abnormal subjects curves
• 5 pulmonologists
• unaware if from SpiroSmart / spirometer
results
restrictive
inadequate
normal
minimal obstructive
mild obstructive
moderate obstructive
severe obstructive
results
restrictive
inadequate
normal
minimal obstructive
mild obstructive
moderate obstructive
severe obstructive

identi...
results
restrictive
inadequate
normal
minimal obstructive
mild obstructive
moderate obstructive
severe obstructive

one of...
results
restrictive
inadequate
normal
minimal obstructive
mild obstructive
moderate obstructive
severe obstructive

one of...
results
restrictive
inadequate
normal
minimal obstructive
mild obstructive
moderate obstructive
severe obstructive

false ...
results
false negative
4%
restrictive
inadequate
normal
minimal obstructive
mild obstructive
moderate obstructive
severe o...
results
error
8%

false negative
4%
restrictive
inadequate
normal
minimal obstructive
mild obstructive
moderate obstructiv...
results
error
8%

false negative
4%
restrictive
inadequate
normal
minimal obstructive
mild obstructive
moderate obstructiv...
appropriate for
trending and screening
global health non-profit
global health non-profit

patient and doctors
global health non-profit

patient and doctors

pharmaceutical drug trials
lung
function

jaundice

future
research
lung
function

jaundice

future
research
neonatal jaundice in the US
kernicterus:
21

hazardous
jaundice: 1158
extreme jaundice:
2,317
severe jaundice: 35,000

pho...
Method

Accuracy

Disadvantages

TSB

Gold standard
(r=1.0)

Painful, costly,
inconvenient, delayed

Accurate
(r= 0.75 -0....
bilirubin
level in blood

blood
draw

jaundice level
bilirubin
level in blood

blood
draw

yellowness

camera

jaundice level
bilirubin
level in blood

blood
draw

estimated
jaundice level

yellowness

camera

processing
bilicam
study A
participants

48 newborns

0-4 days old, collected in nursery
3 hospitals in Washington & Philadelphia
study A
participants

48 newborns

0-4 days old, collected in nursery
3 hospitals in Washington & Philadelphia
bilicam
Color
Linearization
• Camera Settings
Adjustment
• Light Source Estimation

Image
Segmentation
• Quality Control f...
bilicam initial results
bilirubin level

20
15
10
5
0

r=0.91
mg/dl

20
5
10
15
bilicam estimation
bilicam initial results
bilirubin level

20
15
10
5
0

r=0.91
non
white
mg/dl
white

20
5
10
15
bilicam estimation
bilicam initial results
bilirubin level

20

TcB = 0.85

15
10
5
0

r=0.91
non
white
mg/dl
white

20
5
10
15
bilicam estim...
bilicam initial results
bilirubin level

20

TcB = 0.85
BiliCam = 0.84

15
10
5
0

r=0.91
non
white
mg/dl
white

20
5
10
1...
bilicam future work
bilicam future work
• near term: screening
bilicam future work
• near term: screening
• medium term: more data
bilicam future work
• near term: screening
• medium term: more data
• long term: developing world
bilicam future work
• near term: screening
• medium term: more data
• long term: developing world
“in many resource poor n...
lung
function

jaundice

future
research
lung
function

jaundice

future
research
future research

oxygen volume, VO2
cardiac output and blood pressure
intra ocular pressure
intra ocular pressure

PressCam
eclarson.com
eclarson@lyle.smu.edu
> slide to unlock
@ec_larson

Thank You!
phone-as-a-sensor technology:
mhealth and chronic disease

eclarson.com
eclarson@lyle.smu.edu
> slide to unlock
@ec_larson...
phone-as-a-sensor technology:
mhealth and chronic disease

eclarson.com
eclarson@lyle.smu.edu
@ec_larson

eric c. larson |...
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Phone As A Sensor Technology: mHealth and Chronic Disease
Upcoming SlideShare
Loading in …5
×

Phone As A Sensor Technology: mHealth and Chronic Disease

195
-1

Published on

The mHealth “revolution” has promised to deliver in-home healthcare that parallels the care we might receive in a physician’s office. However, the panacea of digital health has proven to be more problematic and messy than its vision, especially for collecting and interpreting medical quantities from the home. In this talk I will discuss several successful projects for sensing medical quantities from a mobile phone using the embedded sensors (i.e., camera, microphone, accelerometer) and how these projects can increase compliance as well as enhance doctor patient relationships. I will focus on the reliability and calibration of the sensing and the role of computer scientists and engineers in the future of mHealth.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
195
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Phone As A Sensor Technology: mHealth and Chronic Disease

  1. 1. phone-as-a-sensor technology: mhealth and chronic disease > slide to unlock eric c. larson | eclarson.com Assistant Professor Computer Science and Engineering
  2. 2. iPhone 9 M Now with iColon. > slide to unlock
  3. 3. Comp Sci. & Engr.
  4. 4. arch algorithms OS databases & data mining software Comp Sci. & Engr. languages AI & robotics networking symbolic computing
  5. 5. arch algorithms OS databases & data mining software mobile Comp Sci. & Engr. languages AI & robotics networking symbolic computing
  6. 6. arch algorithms OS databases & data mining software mobile Comp Sci. & Engr. mhealth languages AI & robotics networking symbolic computing
  7. 7. arch algorithms OS databases & data mining software Comp Sci. & Engr. mhealth languages AI & robotics networking symbolic computing mobile
  8. 8. what is mhealth?
  9. 9. the promise of mhealth:
  10. 10. the promise of mhealth: revolutionize medicine
  11. 11. the promise of mhealth: revolutionize medicine eliminate doctor visits
  12. 12. the promise of mhealth: revolutionize medicine eliminate doctor visits remote / automatic diagnosis
  13. 13. the promise of mhealth: revolutionize medicine eliminate doctor visits remote / automatic diagnosis equalize developing countries
  14. 14. the promise of mhealth: revolutionize medicine eliminate doctor visits remote / automatic diagnosis equalize developing countries
  15. 15. the promise of mhealth: revolutionize medicine eliminate doctor visits remote / automatic diagnosis equalize developing countries
  16. 16. heart rate zombie run fitness trainer stress check current mhealth glucose buddy
  17. 17. heart rate zombie run stress check current mhealth 43,000 apps for health on the app store fitness trainer glucose buddy
  18. 18. heart rate zombie run stress check current mhealth 43,000 apps for health on the app store 96% are for calorie counting & exercise fitness trainer glucose buddy
  19. 19. heart rate zombie run stress check current mhealth 43,000 apps for health on the app store 96% are for calorie counting & exercise 4% are remote monitoring fitness trainer glucose buddy
  20. 20. consider physician’s needs
  21. 21. consider physician’s needs connecting with patient
  22. 22. consider physician’s needs connecting with patient tracking baselines
  23. 23. consider physician’s needs connecting with patient tracking baselines personalized trending data
  24. 24. consider physician’s needs connecting with patient tracking baselines personalized trending data managing chronic disease
  25. 25. consider physician’s needs connecting with patient tracking baselines personalized trending data managing chronic disease 30% of all US healthcare spending is on chronic disease
  26. 26. mhealth and chronic disease management
  27. 27. mhealth and chronic disease management compliance? cost? doctor patient? data reliability?
  28. 28. compliance
  29. 29. compliance
  30. 30. baseline sensor quantity
  31. 31. phone as a sensor baseline sensor quantity
  32. 32. phone as a sensor baseline embedded sensors sensor quantity
  33. 33. phone as a sensor baseline embedded sensors sensor estimated quantity processing
  34. 34. accelerometer gyroscope magnetometer /compass dual camera / flash 1+ microphones proximity sensor capacitive sensor gps motorized actuator wireless antenna (s)
  35. 35. compliance++; cost--; dr_pat *= 10; accelerometer gyroscope magnetometer /compass dual camera / flash 1+ microphones proximity sensor capacitive sensor gps motorized actuator wireless antenna (s)
  36. 36. accelerometer gyroscope magnetometer /compass dual camera / flash 1+ microphones compliance++; proximity sensor cost--; capacitive sensor dr_pat *= 10; gps motorized actuator data reliability? wireless antenna (s)
  37. 37. what can the mobile phone sense with clinical accuracy?
  38. 38. lung function jaundice future research
  39. 39. lung function jaundice future research
  40. 40. spirometer ? lung function?
  41. 41. lung function evaluates pulmonary impairments asthma COPD cystic fibrosis
  42. 42. spirometer device that measures amount of air inhaled and exhaled.
  43. 43. volume flow using a spirometer time volume
  44. 44. volume flow using a spirometer time volume
  45. 45. volume flow using a spirometer time volume
  46. 46. volume volume-time graph time
  47. 47. volume volume-time graph time
  48. 48. volume-time graph FVC volume FEV1 time FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
  49. 49. volume-time graph FVC volume FEV1 1 sec. time FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
  50. 50. volume-time graph FEV1% = FEV1/FVC FVC volume FEV1 1 sec. time FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
  51. 51. FEV1% = FEV1/FVC FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
  52. 52. FEV1% = FEV1/FVC > 80% healthy 60 - 79% mild 40 - 59% moderate < 40% severe FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
  53. 53. flow flow-volume graph volume
  54. 54. flow flow-volume graph volume
  55. 55. flow-volume graph PEF flow 1 sec. FEV1 FVC volume PEF: Peak Expiratory Flow FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
  56. 56. flow-volume graph flow normal volume
  57. 57. flow-volume graph normal flow obstructive volume
  58. 58. obstructive diseases ! resistance in air path leads to reduced air flow
  59. 59. obstructive diseases ! resistance in air path leads to reduced air flow
  60. 60. restrictive diseases ! lungs are unable to pump enough air and pressure
  61. 61. restrictive diseases ! lungs are unable to pump enough air and pressure
  62. 62. flow-volume graph normal Flow obstructive Volume
  63. 63. flow-volume graph normal Flow obstructive restrictive Volume
  64. 64. clinical spirometry
  65. 65. home spirometry
  66. 66. home spirometry ! faster detection rapid recovery trending
  67. 67. challenges with home spirometry high cost barrier patient compliance less coaching limited integration
  68. 68. SpiroSmart availability cost portability more effective coaching interface integrated uploading
  69. 69. Using SpiroSmart
  70. 70. Using SpiroSmart
  71. 71. Using SpiroSmart ]
  72. 72. Using SpiroSmart ]
  73. 73. Using SpiroSmart ]
  74. 74. Using SpiroSmart
  75. 75. Using SpiroSmart
  76. 76. flow rate volume airflow sensor lung function
  77. 77. flow rate volume airflow sensor sound pressure microphone lung function
  78. 78. flow rate volume airflow sensor estimated lung function sound pressure microphone processing
  79. 79. audio
  80. 80. audio flow features
  81. 81. audio flow features measures regression FEV1 FVC PEF
  82. 82. audio flow features measures regression curve regression Flow(L/s) 10 10 5 5 0 0 1 0 40 Volume(L) Volume(L) FEV1 FVC PEF Flow(L/s) 15 15 2 Volume(L) 4 2 Volume(L) 1 3 3 3 4 2 1 3 0 20 1 2 4 6 time(s) 8 10 4
  83. 83. audio flow features measures regression curve regression Flow(L/s) lung function 10 10 5 5 0 0 1 0 40 Volume(L) Volume(L) FEV1 FVC PEF Flow(L/s) 15 15 2 Volume(L) 4 2 Volume(L) 1 3 3 3 4 2 1 3 0 20 1 2 4 6 time(s) 8 10 4
  84. 84. flow features estimation amplitude 1 0.5 0 −0.5 −1 0 1 2 3 4 time(s) 5 6 7
  85. 85. flow features estimation 0.5 0 −0.5 −1 0 1 2 3 4 5 6 time(s) source vocal tract 2500 frequency(Hz) amplitude 1 2000 1500 1000 500 0 output 7
  86. 86. flow features estimation lpc8raw amplitude 1 0.5 0.5 0 −0.5 0 flow estimation features amplitude −0.5 1 2 3 4 5 6 7 time(s) 0.5 0 −0.5 resonance tracking lpc8raw 1 2 3 4 5 6 time(s) source vocal tract 2500 2000 1500 1000 500 0 −1 1 0 7 amplitude −1 0 envelope detection −1 10 frequency(Hz) amplitude 1 output 1 2 3 4 5 6 7 time(s) 0.5 0 −0.5 −1 0 auto-regressive estimate 1 2 3 4 time(s) 5 6 7
  87. 87. measures regression 8 flow(L/s) 6 4 2 ground truth 0 −1 0 1 2 3 4 5 time(s) feature value 0.4 0.3 0.2 0.1 0 feature 1 0 1 2 4 5 time(s) 0.4 feature value 3 0.3 0.2 0.1 feature 2 0 −1 0 1 2 time(s) 3 4 5
  88. 88. 8 6 flow(L/s) measures regression 7.1 4 2 PEF features ground truth 0 −1 0 1 2 3 4 5 time(s) feature value 0.4 0.3 0.2 0.1 0 feature 1 0 1 2 4 5 time(s) 0.4 feature value 3 0.3 0.2 0.1 feature 2 0 −1 0 1 2 time(s) 3 4 5
  89. 89. 8 6 flow(L/s) measures regression 7.1 4 2 PEF features ground truth 0 −1 0 1 2 3 4 5 time(s) feature value 0.4 0.3 0.2 0.1 0 feature 1 0 1 2 4 5 time(s) 0.4 feature value 3 0.3 0.2 0.1 feature 2 0 −1 0 1 2 time(s) 3 4 5 0.35 0.33
  90. 90. 8 6 flow(L/s) measures regression 7.1 3.2 4 2 PEF features ground truth 0 −1 0 1 2 3 4 5 time(s) feature value 0.4 0.3 0.2 0.1 0 FEV1 features feature 1 0 1 2 3 4 5 time(s) 0.4 feature value 0.35 0.33 0.3 0.2 0.1 feature 2 0 −1 0 1 2 time(s) 3 4 5
  91. 91. 8 6 flow(L/s) measures regression 7.1 3.2 4 2 PEF features ground truth 0 −1 0 1 2 3 4 5 time(s) feature value 0.4 0.3 0.2 0.1 0 FEV1 features feature 1 0 1 2 3 4 0.12 0.17 5 time(s) 0.4 feature value 0.35 0.33 0.3 0.2 0.1 feature 2 0 −1 0 1 2 time(s) 3 4 5
  92. 92. measures regression FEV1 features PEF features
  93. 93. measures regression FEV1 features PEF features bagged decision tree bagged decision tree
  94. 94. measures regression FEV1 features PEF features bagged decision tree bagged decision tree output output
  95. 95. curve regression feature 1 0.3 0.2 0.1 0 1 3 4 5 time(s) 0.4 feature value 2 feature N 0.3 0.2 0.1 0 ... 0 −1 ... windowed machine learning regression feature value 0.4 0 1 2 time(s) 3 4 5
  96. 96. curve regression feature 1 0.3 0.2 0.1 0 1 3 4 5 time(s) 0.4 feature value 2 ... 0 −1 ... feature N 0.3 0.2 0.1 0 8 6 flow(L/s) windowed machine learning regression feature value 0.4 4 2 0 0 1 2 3 4 5 time(s) curve output
  97. 97. curve regression 0.4 feature value feature value 0.4 0.3 0.2 0.1 0 −1 0 1 2 3 time(s) bagged decision tree 4 0.3 0.2 0.1 0 5 0 1 2 time(s) 3 4 5
  98. 98. curve regression 0.4 feature value feature value 0.4 0.3 0.2 0.1 0 −1 0 1 2 3 time(s) bagged decision tree 4 0.3 0.2 0.1 0 5 0 1 2 3 4 time(s) CRF 5
  99. 99. curve regression 0.4 feature value feature value 0.4 0.3 0.2 0.1 0 −1 0 1 2 3 4 0.3 0.2 0.1 0 5 0 1 2 bagged decision tree ! 6 flow(L/s) 4 CRF 8 4 2 0 −1 3 time(s) time(s) 0 1 2 time(s) 3 4 5 5
  100. 100. study design x3 x3
  101. 101. study enrollment
  102. 102. study enrollment study a participants 52 18-75 years old, mostly healthy
  103. 103. study enrollment study a participants 52 18-75 years old, mostly healthy study b participants 10 12-17 years old, mixed healthy/abnormal
  104. 104. study enrollment study a participants 52 18-75 years old, mostly healthy study b 10 12-17 years old, mixed healthy/abnormal study c participants 56 10-69 years old, mostly abnormal enrolled by pulmonologists participants
  105. 105. measures regression results
  106. 106. measures regression results
  107. 107. curves regression results 15 6 flow(L/s) flow(L/s) 8 4 2 0 −1 0 1 2 volume(L) 3 4 10 5 0 −2 0 2 volume(L) 4 6
  108. 108. curves regression results 15 6 flow(L/s) flow(L/s) 8 4 2 0 −1 0 1 2 volume(L) 3 4 10 5 0 −2 0 2 volume(L) 4 6
  109. 109. curves regression results 15 flow(L/s) 6 4 2 0 −1 0 1 2 3 10 5 0 −2 4 0 volume(L) 2 volume(L) 15 flow(L/s) flow(L/s) 8 10 5 0 −2 0 2 volume(L) 4 6 4 6
  110. 110. curves regression results 15 flow(L/s) 6 4 2 0 −1 0 1 2 3 10 5 0 −2 4 0 volume(L) 2 volume(L) 15 flow(L/s) flow(L/s) 8 10 5 0 −2 0 2 volume(L) 4 6 4 6
  111. 111. can SpiroSmart curves be used for diagnosis?
  112. 112. survey
  113. 113. survey • normal/abnormal subjects curves
  114. 114. survey • normal/abnormal subjects curves • 5 pulmonologists
  115. 115. survey • normal/abnormal subjects curves • 5 pulmonologists • unaware if from SpiroSmart / spirometer
  116. 116. results restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive
  117. 117. results restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive identical 64%
  118. 118. results restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive one off 10% identical 64%
  119. 119. results restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive one off 10% identical 64%
  120. 120. results restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive false positive 14% one off 10% identical 64%
  121. 121. results false negative 4% restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive false positive 14% one off 10% identical 64%
  122. 122. results error 8% false negative 4% restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive false positive 14% one off 10% identical 64%
  123. 123. results error 8% false negative 4% restrictive inadequate normal minimal obstructive mild obstructive moderate obstructive severe obstructive false positive 14% ! ! abnormal vs normal one off 96% identical 10% 64%
  124. 124. appropriate for trending and screening
  125. 125. global health non-profit
  126. 126. global health non-profit patient and doctors
  127. 127. global health non-profit patient and doctors pharmaceutical drug trials
  128. 128. lung function jaundice future research
  129. 129. lung function jaundice future research
  130. 130. neonatal jaundice in the US kernicterus: 21 hazardous jaundice: 1158 extreme jaundice: 2,317 severe jaundice: 35,000 phototherapy: 290,000 visible jaundice: 3.5 million births/year: 4.1 million
  131. 131. Method Accuracy Disadvantages TSB Gold standard (r=1.0) Painful, costly, inconvenient, delayed Accurate (r= 0.75 -0.93) Meter = $7000 tips $5 unavailable in most physician offices TcB Visual assessment (provider or parent) not accurate (r= 0.36 - 0.7), underestimates severity No standardization lighting, pigmentation
  132. 132. bilirubin level in blood blood draw jaundice level
  133. 133. bilirubin level in blood blood draw yellowness camera jaundice level
  134. 134. bilirubin level in blood blood draw estimated jaundice level yellowness camera processing
  135. 135. bilicam
  136. 136. study A participants 48 newborns 0-4 days old, collected in nursery 3 hospitals in Washington & Philadelphia
  137. 137. study A participants 48 newborns 0-4 days old, collected in nursery 3 hospitals in Washington & Philadelphia
  138. 138. bilicam Color Linearization • Camera Settings Adjustment • Light Source Estimation Image Segmentation • Quality Control for Distance, Lighting, and Shadow • Sternum, Forehead, Card Segmented Color Calibration • Dynamic Least Squares Regression • Automatic Feature Selection Neonatal Skin Response to Bilirubin • Skin Independent Color Transformations Applied • Multivariate Machine Learning Regression
  139. 139. bilicam initial results bilirubin level 20 15 10 5 0 r=0.91 mg/dl 20 5 10 15 bilicam estimation
  140. 140. bilicam initial results bilirubin level 20 15 10 5 0 r=0.91 non white mg/dl white 20 5 10 15 bilicam estimation
  141. 141. bilicam initial results bilirubin level 20 TcB = 0.85 15 10 5 0 r=0.91 non white mg/dl white 20 5 10 15 bilicam estimation
  142. 142. bilicam initial results bilirubin level 20 TcB = 0.85 BiliCam = 0.84 15 10 5 0 r=0.91 non white mg/dl white 20 5 10 15 bilicam estimation
  143. 143. bilicam future work
  144. 144. bilicam future work • near term: screening
  145. 145. bilicam future work • near term: screening • medium term: more data
  146. 146. bilicam future work • near term: screening • medium term: more data • long term: developing world
  147. 147. bilicam future work • near term: screening • medium term: more data • long term: developing world “in many resource poor nations, hyperbilirubinemia is the second or third leading cause of infant mortality and disability”
  148. 148. lung function jaundice future research
  149. 149. lung function jaundice future research
  150. 150. future research oxygen volume, VO2
  151. 151. cardiac output and blood pressure
  152. 152. intra ocular pressure
  153. 153. intra ocular pressure PressCam
  154. 154. eclarson.com eclarson@lyle.smu.edu > slide to unlock @ec_larson Thank You!
  155. 155. phone-as-a-sensor technology: mhealth and chronic disease eclarson.com eclarson@lyle.smu.edu > slide to unlock @ec_larson eric c. larson | eclarson.com Assistant Professor Computer Science and Engineering collaborators: ! Joseph Camp Shwetak Patel Jim Stout, MD Jim Taylor, MD Margaret Rosenfeld, MD Gaetano Boriello Mayank Goel Lilian DeGreef
  156. 156. phone-as-a-sensor technology: mhealth and chronic disease eclarson.com eclarson@lyle.smu.edu @ec_larson eric c. larson | eclarson.com Assistant Professor Computer Science and Engineering collaborators: ! Joseph Camp Shwetak Patel Jim Stout, MD Jim Taylor, MD Margaret Rosenfeld, MD Gaetano Boriello Mayank Goel Lilian DeGreef
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×