Asthma is a high-burden chronic inflammatory disease with prevalence in children with twice the rate compared to adults. It can be improved by continuously monitoring patients and their environment using the Internet of Things (IoT) based devices. These sensor data streams so obtained are essential to comprehend multiple factors triggering asthma symptoms. In order to support physicians in exploring causal associations and finding actionable insights, a visualization system with a scalable cloud infrastructure that can process multimodal sensor data and Patient Generated Health Data (PGHD) is necessary. In this thesis, we describe a cloud-based asthma management and visualization platform that integrates personalized PGHD from kHealth1 kit and outdoor environmental observations from web services2. When applied to data from an individual, the tool assists in analyzing and explaining symptoms using ”personalized” causes, monitor disease progression, and improve asthma management. The front-end visualization was built with Bootstrap Framework and Highcharts. Elasticsearch was used as back-end storage to aggregate data from various sources. Further, Node.js and Express Framework were used to develop several Representational State Transfer services useful for the visualization.

1. Sensor Data Streams Correlation Platform for
Asthma Management
Vaikunth Sridharan
Master’s Thesis Defense
April 11, 2018
Master’s Thesis Committee
Dr. Amit Sheth (Advisor)
Dr. Krishnaprasad Thirunarayan
Dr. Valerie Shalin
Dr. Maninder Kalra

2. Internet of Things (IoT)
2
Image Source: Where the IoT will be used in 2025 (August 31, 2017). IoT [JPG]. Retrieved Mar 20th, 2017, from
https://hbr.org/2014/10

3. IoT devices used in Healthcare Sector
● Fall detection
TN Gia et al., NORCAS (2016)
S Greene et al., iNIS (2016)
● Identifying anomalies in heart functioning
C Puri et al., IoT of Health (2016)
A Ukil et al., AINA (2016)
● Monitoring sleep
Fitbit Sleep Study., (2018)
3

4. Asthma
A chronic disease characterized by airway inflammation
and bronchoconstriction.
- CDC reports, ~27,739 asthmatic children
hospitalized
- Poorly controlled and managed disease (Masoli., et al)
- Multifactorial disease
Image Source: Ingelheim, [Boehringer] (Jan 3rd, 2009). Asthma [PNG]. Retrieved
November 6th, 2017, from https://www.pinterest.com/pin/2955555988759704/
4

5. Asthma Triggers
- Each patient reacts differently
Some patients might be sensitive
to poor air quality
And
Others might be sensitive to
pollen
5
- Triggered by, World Health Organization, Asthma Fact Sheet, CDC
(Retrieved 2017)
- Environmental Variations
- Example: change in temperature, increase
in humidity, etc.
- Pollutants
- Example: Dust particles, etc.
- Genetic Factors
- Difficult to diagnose with extant methods
Vulnerability
Severity

6. Patient
Examination
Review Patient History
Estimate Patient
Health Status
Check
Disease Progression
Check Current
Symptoms
Devise Appropriate
Treatment Plans
Traditional Healthcare Scenario
Lippa, K. D., & Shalin, V. L. (2016). Creating a common trajectory: Shared decision making and distributed cognition in medical
consultations. Patient Experience Journal, 3(2), 73.
Drawbacks
6
- Episodic clinical visits
- Patients’ filtered memory
- Data parameters not available
- Patient’s Environment
- Medication Usage

7. Augmented Personalized Health (APH)
Physical
Examination
Review Patient History
Estimate Patient
Health Status
Check
Disease Progression
Check Current
Symptoms
Device and Create
Management Plans
Traditional
Healthcare
Model
Health
Strategies
Self
monitoring
Self appraisal
Self management
Intervention
Disease
Progression
& Tracking
Patients
Doctors
7
A. Sheth, U. Jaimini, H. Yip, How Will the Internet of Things Enable Augmented Personalized Health? IEEE Intelligent
Systems, Jan/Feb 2018.

8. Temperature
Humidity
Air Quality
Pollen
Parameters to be monitored
Long-term Meds
Short-term Meds
Other Meds
Symptoms
8
What contributed to the patient’s symptoms?
Why did the patient take medication?

9. kHealth for Asthma
Sheth, A., Anantharam, P., & Thirunarayan, K. (2014). kHealth: Proactive Personalized Actionable Information for Better
Healthcare. In Workshop on Personal Data Analytics in the Internet of Things (PDA@ IOT 2014), collocated at VLDB.
1
http://wiki.knoesis.org/index.php/Asthma9
~2.5 million data
points for 50
completed patients
each one-month
period
IRB*

10. - Diverse Parameters
- The kHealth kit collects 29* parameters per patient due to the multifactorial nature of asthma
- Example: Symptoms, medication usage, ozone, pollen, etc
- Higher Sampling Rates of Sensors
- Indoor and outdoor environmental sensor data are captured at much higher rate compared patient recorded readings
using the kHealth kit
- Difficult to analyze manually
Challenges
10 * Calculated up till April 11, 2018

11. 11
Related Studies
11
No
No
No
No
No
Patient Trial
Or
Evaluation
No
No
●
● Spirometer
● Electronic Stethoscope
● Sensordrone , Oximeter and
Smartphone
● Activity--Biking, Hiking,
Walking
● Sleep duration
● Outdoor Environmental
data
Sensors/Parameters
No
No
● Patient’s medication usage,
● Peak Flow Meter readings
● Symptoms reported
● Serves as an electronic-diary
4 LinkMedica.,
(Retrieved 2018)
No● Inhaler Sensor
Tracks medication usage:
● Time
● Location
●
5 Propeller
Health Platform.,
(Retrieved 2018)
Main Objectives
● Sensor based monitoring
● Collection and detection of wheeze sounds
Ho-Kyeong Ra et
al., (2016)
● Visualization with Health coaches
● Aims to reduces the information-seeking between
patient-clinician interaction (5 participants)
●
2 Ryokai et al.
(2015)
● Ubiquitous Warning System
● Patient’s location
3 Chu et al.
(2006)
Studies
1

12. Summary
- Asthma is a multifactorial disease
- IoT-devices could enable continuous monitoring and data collection
- Challenges:
- Diverse Parameters
- Higher sampling rates of sensors compared to patient readings
12

13. Thesis Statement
Multimodal sensor data about activity, sleep, indoor and outdoor environmental
conditions, along with patient-reported symptoms and medication usage, can be
collected, analyzed, visualized and summarized so as to enable correlating
triggers with associated symptoms, to obtain actionable insights.
13

14. 14
Outline
Data sources
kHealth kit (4 devices)
Web Services
Data Sources Aggregation
Queries and Processing
Visualization Platform
Overall System
System Evaluation
Conclusion and Future work

15. 15
Outline
Data sources
kHealth kit (4 devices)
Web Services
Data Sources Aggregation
Queries and Processing
Visualization Platform
Overall System
System Evaluation
Conclusion and Future work

16. What symptoms are youhaving now?
Cough
Wheeze
Chest Tightness
Nose Opens Wide
Can’t Talk in full sentences
kHealth kit: kHealth-Asthma Android App Patient Questionnaire
Symptoms and its types
Long acting medication usage
Short-acting medication usage
16 parameters per day per patient
What symptoms are you
having now?
Cough
Wheeze
Chest Tightness
Nose Opens Wide
Can’t Talk in full sentences
16

17. Parameters*
kHealth kit: Digital Peak Flow Meter
Peak Expiratory Flow (PEF)
Forced Expiratory Volume
in one sec (FEV)
Patients
breathe
Period - Twice daily (4 data points per day per patient)
17
*Sawyer, G., Miles, J., Lewis, S., Fitzharris, P., Pearce, N., & Beasley, R. (1998). Classification of asthma severity: should
the international guidelines be changed?. Clinical and Experimental Allergy, 28(12), 1565-1570.
Image inspired from: https://www.microlife.com/consumer-products/respiratory-care/asthma-monitor/pf-100

18. Parameters*
Activity (Steps, Distance, Active, Sedentary & Light)
Sleep (Duration, REM, Light, Deep)
Heart rate (BPM)
kHealth kit: Fitbit Charge 2™
wearable
8 parameters tracked per day per patient
18
Bian, J., Guo, Y., Xie, M., Parish, A. E., Wardlaw, I., Brown, R., ... & Perry, T. T. (2017). Exploring the Association Between Self-Reported Asthma
Impact and Fitbit-Derived Sleep Quality and Physical Activity Measures in Adolescents. JMIR mHealth and uHealth, 5(7).
Image source: http://fitbit.com

19. Parameters*
Volatile Compounds (ppb)
Indoor Temperature (o
F)
Indoor Humidity (%)
Particulate Matter 2.5 (µg/m³)
Carbon Dioxide (ppm)
Global Pollution Index (no unit)
kHealth kit: Indoor Air Quality Sensor
Data is collected every 5 minutes,
288 per day for each of the 6 parameters
(288x6 = 1728 data points per day per patient)
19
Good Air Quality Poor Air Quality
*Infante-Rivard, C. (1993). Childhood asthma and indoor environmental risk factors. American Journal of Epidemiology, 137(8), 834-844.
Jaimini, U., Banerjee, T., Romine, W., Thirunarayan, K., Sheth, A., & Kalra, M. (2017). Investigation of an indoor air quality sensor for asthma management in children. IEEE sensors letters, 1(2), 1-4.
Image inspired from: http://foobot.io
Foobot

20. Estimated
Pollutants
Observations by
Monitoring Stations
Periodically
Monitoring Stations
Outdoor Parameters
Ozone (constructed unit)
Particulate Matter (constructed unit)
Humidity (%)
Temperature (o
F)
Pollen Count (PI)
Web Services for Outdoor Environment
Pollen - Every 12 hours daily
Other parameters - Every hour of the day
108 data points per patient per locationIQVIA™

21. 21
Data Collected by kHealth kit per day per patient
Active sensing (4 parameters)
Tablet
Symptom - 6
Short acting med - 1
Long acting med - 1
Total - 8 x 2 (twice a day) = 16
Peak Flow meter = 2 (twice a day)
Total = 16+2 = 18 data points/day
Passive sensing (19 parameters)
Foobot
CO2
, VOC, Humidity,
Temperature, PM2.5,
Global Pollution Index
Fitbit
Sleep - 4 (REM,Light sleep,Deep sleep, # minutes active)
Activity - 4 (minutes active, sedentary minutes, minutes lightly active, #
steps)
Subtotal - 8
Outdoor Parameters
Ozone, PM2.5, Temperature, Humidity = 24 x 4 = 96
Pollen = 2
Subtotal = 108
Total = 1834 data points / day
288 (every five minutes) x 6 = 1728
Total number of data points per patient per day = 18 + 1844 = 1852 data points/ day

22. 22
30 days x 4 params. = 120
30 days x 4 params. = 120
30 days x 6 params.x 288 (every 5 min)
= 51840
30 days x 6 params. x 2(twice a day) = 360
30 days x 2 params x 2(twice a day). = 120
Consider Patient-A, deployed for one-month
For 30 days = 55, 500 data points
30 days x 98 params. = 2940

23. 23
- We need a cloud infrastructure to collect and
process data
- Appropriate visualization and summarization
Infrastructure

24. 24
Outline
Data sources
kHealth kit (4 devices)
Web Services
Data Sources Aggregation
Queries and Processing
Visualization Platform
Overall System
System Evaluation
Conclusion and Future work

25. Data Source Aggregation
Fitbit via Authentication
Enabled API
Mapping
inventory
Access,
Refresh
Tokens
Crawling
Service
Authentication based
Fitbit Cloud
25
Environmental Web
Services
Monitoring stations
Crawling
Service
Web API Endpoints
IQVIA™
Defining Schema
Patient recorded data
via Android device
Firebase
Active Sensing
Firebase
Administrator
Storing
Index
2Index
1
Elasticsearch
Cloud
Storage
Access Token based
Foobot via Web API
Crawling
Service
Foobot Cloud
Mapping
inventory

26. Storage
26
Built-in RESTful APIs
Aggregations
Time filtering
Geo-Distance SortingQueries
Elasticsearch
Cloud Storage
Index 1
Index 2
- Apache Lucene
- Dynamic Mapping
- NoSQL Database
Basic
Statistical Functions
Missing document and minimum count
RESTful
Services

27. 27
Solution
1. Diversity (29 parameters)
a. Indexed to individual indices (like SQL tables) with appropriate schema
b. Schema includes numeric-, geo-, or date-
2. Highly varying sampling rates compared to patient recorded readings
a. Performing aggregations (observations per day or an hour or a 12 hour period)
b. Applying basic statistical functions such as minimum, maximum, or average

28. 28
Outline
Data sources
kHealth kit (4 devices)
Web Services
Data Sources Aggregation
Queries and Processing
Visualization Platform
Overall System
System Evaluation
Conclusion and Future work

29. 29
Querying
Activity, Indoor and Outdoor Observations Patient recorded Observations
1. Deployment period based
filtering (temporal-filtering)
2. Geo-distance querying (sorting
- minimum)
3. Filtering symptoms based
questions answered
4. Performing aggregations,
statistical functions, etc.

30. 30
Outline
Data sources
kHealth kit (4 devices)
Web Services
Data Sources Aggregation
Queries and Processing
Visualization Platform
Overall System
System Evaluation
Conclusion and Future work

31. Visualization Platform - kHealthDash
31

32. 32
Overall System

33. Clinical Decision Making
kHealthDash
Firebase
Data Aggregation
Temporal Querying
REST
APIS
Geo-Distance Sorting
Cloud Storage
Elasticsearch
Cluster
Index
Services
kHealth Cloud
33
Real-time
kHealthkit for Asthma
Foobot
[ Indoor Air Quality ]
Peak Flow Meter
[Lung Functioning]
Asthma Patient
Fitbit Charge 2
[ Activity + Sleep ]
kHealth
Mobile App
[Contextual Questions ]
Outdoor Environmental Data
Air Quality Temperature
HumidityPollen
Periodically
Third-party
APIs
Fitbit
Foobot
TM

34. 34
Real Patient Scenarios

35. Records readings using kHealth kit
Humidity
Temperature
Y-axisissplit
35
Ozone
Pollen Index
Particulate Matter 2.5
What contributed to the patient’s symptoms?Why did the patient take any short-acting
medication?
Asthma symptoms Medication usage
PATIENT-12
PATIENT-A

36. 36
Outdoor Environmental Observations versus Asthmatic Symptoms PATIENT-B

37. 37
Indoor Environmental Observations, Activity, Sleep versus Asthmatic Symptoms PATIENT-B

38. 38
Ozone and PM2.5 Pollen Index TemperatureHumidity
Source: EPA’s AirNow Source: Pollen.com Medical Knowledge from Clinician Medical Knowledge from Clinician

39. 39
Outline
Data sources
kHealth kit (4 devices)
Web Services
Data Sources Aggregation
Queries and Processing
Visualization Platform
Overall System
System Evaluation
Conclusion and Future work
-

40. 40
System Evaluation
- We included 5 researchers and 5 Healthcare providers
- Questionnaire survey containing 2 asthma patients data
was designed with Qualtrics
- Participants were asked to respond to questions relevant to
asthma using tabular data, followed by using kHealthDash
- Measures:
- Usefulness
- Overall Usability (SUS)
With kHealthDash
System Usability Scale
Questionnaire
Without kHealthDash
Questions (Q)
Questions (Q)

41. 41
Asthma-relevant Questionnaire*
Question Choices (likert scale)
How likely were you able to identify symptoms for
Patient-A? 0 to 10
How likely were you able to find the outdoor
parameters contributing for Patient-A ?
(5 sub-questions)
0 to 10
How likely were you able to find correlation
between short-acting medication and symptoms
for Patient-A
0 to 10
* Reviewed by clinical collaborator

42. 42
Results: With vs Without Platform
Without kHealthDash
With kHealthDash
Number of responses = 10
Asthma relevant Questionnaire
Responses provided by clinical professionals for domain related questionnaire in a
scale of 0 - 10, 0 - Least Likely and 10 - Most Likely
LikertScale
Least Likely
Most Likely
P-value using t-test, 0.0001*
(<0.005)

43. 43
System Usability Scale, John Brooke., (1986)
What do users think of the overall usability of
the application?
SUS score (0-100) and average is 68 (from 500
studies)
Studies also recommends minimum sample size
can be 5, source
Provided with predefined set of 10 questions to
measure usability of user interfaces
5 Healthcare providers 5 Researchers
68
SUS (Mean)
SUSscore

44. 44
Outline
Data sources
kHealth kit (4 devices)
Web Services
Data Sources Aggregation
Queries and Processing
Visualization Platform
Overall System
System Evaluation
Conclusion and Future work

45. Conclusion
- Asthma being multifactorial and a challenging problem
- Multimodal sources has to be explored
- A broad and integrated system is necessary
- A scalable cloud infrastructure capable of integrating multimodal data and
represented better to:
- Summarize trigger information
- Allow clinicians to explore for correlating triggers with asthma outcomes
45

46. Future work
Develop heuristics based rules from the
clinician verified evidence and to use them in
predicting the occurrence of symptoms.
Choosing patient reported symptoms and
relevant observations as instances and train a
machine learning model for predicting asthmatic
symptoms.
46

47. Questions?
47

48. This research is supported by National Institutes of Health under the
Grant Number: 1 R01HD087132. Any opinions, findings, and conclusions or
recommendations expressed in this material are those of the author(s) and do not
necessarily reflect the views of the National Institute of Health.
48

49. 49
Acknowledgements
Dr. Amit Sheth
(Advisor)
Dr. Krishnaprasad
Thirunarayanan
Dr. Maninder
Kalra
Dr. Valerie Shalin
Dr. Pramod
Anantharam
(Mentor)
Dr. Tanvi Banerjee

50. 50
Acknowledgements
Dipesh
Kadariya
Collaborator
Revathy
Venkataramanan
Collaborator
Alan Smith
Technical Mentor
Jeremy Brunn
Technical Mentor

51. kHealth Team
Revathy VenkataramananUtkarshini Jaimini Hong Yung Yip Quintin Oliver
Clinical Collaborator
Dr. Maninder Kalra
Co-investigator
(Pulmonologist at Dayton Children’s Hospital)
Graduate & Undergraduate Students
Dipesh Kadaria
Dr. Tanvi Banerjee
(Co-investigator)
Faculty
Dr. Amit Sheth
Principal
Investigator
Dr. Krishnaprasad
Thirunarayan
Co-investigator
51

52. 52
Kno.e.sis

53. 53

54. References
54