The document proposes a software architecture for interoperable ambient monitoring applications to enable predictive and personalized medicine. The architecture uses a publish-subscribe model with loosely coupled components that exchange data through a common communication bus. Sensors and other data sources act as publishers that provide ambient medical and behavioral data. Machine learning and complex event processing components act as subscribers and transformers to analyze the data and detect events or conditions. Semantic standards are used to ensure semantic interoperability between the components. The goal is to scale up ambient monitoring from clinical trials to sustainable home monitoring services tailored to individual patients.

1. Personalizing
MedicalTreatments
based on Ambient Information
Towards Interoperable
Monitoring Applications
Rémi Bastide
ISIS – IRIT, France
Remi.Bastide@irit.fr
http://www.irit.fr/~Remi.Bastide

2. Big Data for Predictive and
Personalized Medicine
• Data mining : finding useful information from
vast data repositories
– Combination of statistical and computational
approaches
– Finding unexpected correlations from seemingly
unrelated data
• Correlation is not causation !
2

3. Sources of Medical Information
• X-omics
• Electronic Health Records
• Medical Reimbursement History
• Social Media
Sensors and bio-Sensors
3

4. Outline of the talk
• Introduction (done)
• State of the art in ambient monitoring
– Monitoring bio-signals
– Monitoring activities of daily life
• Problems
• Technical Proposal
– Software architecture
– Semantic Interoperability
4

5. Ambient Data for Predictive and
Personalized Medicine
• Ambient Data is collected continuously,
unobtrusively, without direct action from the
user who continues performing his daily life
activities as usual
– Ambient biomedical data
– Ambient behavioral data
5

6. Capturing biomedical data
6

7. Connected Health Devices
7

8. Connected Health Devices
• Monitor activity,
calories burnt, heart
rate, sleeping… 8

9. Continous Sensing of bio-signals
9

10. Smart clothing
10

11. SmartToilets
11

12. Implanted or Ingestible Sensors
12
Fraunhofer Intravascular Monitoring System : placed in the femoral artery,
measures blood pressure 30 times /s

13. Monitoring medication adherence
13
Feasibility of an Ingestible Sensor-Based System for Monitoring Adherence toTuberculosis Therapy,
Belknap et al. 2012

14. Lab-on-a-Chip
14Nano-Tera project, EPFL, Switzerland

15. Ambient sensors in smart housing
15

16. Motion Sensing
• Computer vision (e.g. kinect, LeapMotion…)
• “X-ray” vision using wireless (wifi) signals
– Monitoring Breathing via Signal Strength inWireless
Networks (Patwari et al. 2011)
– Wisee system
• Indoor location systems, RFID tags, sensors in
soles, accelerometer and gyroscope…
16

17. Smart Meters
17

18. LifeLogging
• The technical ability to
record and store every event
and information about one’s life
18

19. From sensors to long-term
monitoring
19

20. Techniques for inferring ADLs
from sensed data
• Machine-learning techniques
– Pre-training a computer system with benchmark samples of the
activity to be recognized
• Model-based techniques (e.g. Complex Event Processing)
– Pre-defining a computer model of the sequence of events that
characterize the activity to be detected
• The old fashioned way : clinical interviews and questionnaires
– “Human as sensor”
20

21. From clinical studies to
personalized home-care
• Many of the tools and techniques
presented above are currently
experimented in clinical trials
– Controlled cohorts and
experimental setup
– Ad-hoc software architecture
– Usually targeted at a single
pathology
 Challenges in scaling up these
results to the general population
• Monitoring services for the
elderly
– Proportion of old people rising in
the population
– Developing chronic diseases,
multi-pathology
– Desire for home-care
 Developing sustainable
monitoring services, that can be
tailored to the specific case of the
patient
21
2003 HeatWave :
15 000 over-mortality
in France, about 70 000
in Europe

22. Software engineering principles
• Weak coupling
– Construct software
applications as
assemblies of
components that are as
independent as possible
to each other
• Syntactic and Semantic
Interoperability
– Syntactic : all software
components speak the
same language
– Semantic : the meaning
of exchanged
information is preserved
22

23. Weak coupling : publish /
subscribe architecture
• Components do not know each
other, nor speak directly to each
other
• Instead components « publish »
information about a designated
« topic », or manifest their
interest in a topic by
« subscribing » to it
– « Software bus »
23
Publisher
Subscriber Subscriber
« Provider », « Consumer » and
«Transformer » components

24. • Provide data to the communication bus
• Sensor components
– Act as proxies for hardware sensors
• Motion sensors
• Intelligent pillow
• Inertial navigation sensors carried on
by the patient
• Medical equipment
• …
– Translation from proprietary
language to bus-compliant data
Providers
Sensor Component
Hardware
Sensors
Data Communication Bus
Proprietary
Language

25. Providers
– Scheduler
• Simulate the activity of the user and feed simulated
data to the bus
• Useful for “benchmarking” and validating detection
algorithms or systems
– Based on simulation
– Based on real-time captured data logged during previous
experiments
25
DataCommunicationBus
XML
Emulation scenario
Scheduler
Component
data

26. Consumers
• Consumers are components that are only using the data transmitted on
the communication bus
– Logger: Store the data exchanged on the communication
– 3DVisualization Component
26
DataCommunicationBus
XML
Emulation
scenario
Logger Component
data
Database

27. Transformers
• Transformers act both as
consumers and producers
– Based on Machine Learning or
Complex Event Processing
– Simple transformers
• only use data produced by regular
producers
– Advanced transformers
• use data produced by producers
and/or by other transformers
• Simple transformers
– Fall detection (e.g. from skin’s electrical
resistance and heart rate [Noury 2013])
– Sleeping monitors
– Activity monitor (e.g. smart meters +
location sensors detects the act of
preparing breakfast)
• Advanced transformers
– Denutrition detector : variations in the
rate of preparing food + readings from
a wireless scale
27

28. Semantic Interoperability :
Semantic Sensor Networks
28
• Using and extending the Semantic Sensor
Network ontology developed by theW3C
– Data exchanged between producers and
consumers is expressed in terms of this ontology
(« observation » concept)

29. Towards Big-Data-Driven
Predictive Medicine
– Technology Providers What is possible ?
• or will become possible in the next few years thanks to
Moore’s law
– Medicine Practitioners  What is useful ?
• Sustainability, cost / benefit ratio for the Health
system
– Society at large What is ethical ?
• Issues about data security, privacy, screening…
29

30. Personalizing
MedicalTreatments
based on Ambient Information
Towards Interoperable
Monitoring Applications
Rémi Bastide
ISIS – IRIT, France
Remi.Bastide@irit.fr
http://www.irit.fr/~Remi.Bastide

31. 31