Presentation CBMS 2014, Mount Sinai, NY

  • 64 views
Uploaded on

The Medical Cyber-Physical Systems Activity at EIT: A Look under the Hood …

The Medical Cyber-Physical Systems Activity at EIT: A Look under the Hood

In the future, clinical environments will develop into medical cyber-physical systems of their own. Patients will get direct treatment according to a direct data acquisition and interpretation workflow. In the end, the doctor’s decision support will be provided according to the data the cyber- physical system (CPS) collects from the individual patients. This approach will be scalable, extend patient monitoring to data collections at home by using portable sensors providing information about a patient’s recovery status, and influence healthcare of the future.

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
64
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
2
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The Medical Cyber-Physical Systems Activity at EIT: A Look under the Hood CBMS 2014 NEW YORK 29/5/2014 Daniel Sonntag(DFKI), Sonja Zillner (Siemens), Samarjit Chakraborty (TUM), Andras Lorincz (ELTE), Esko Strommer (VTT), and Luciano Serafini (FBK) Wednesday, May 28, 14
  • 2. Vision • In the future, clinical environments will develop into medical cyber-physical systems of their own. Patients will get direct treatment according to a direct data acquisition and interpretation workflow. In the end, the doctor’s decision support will be provided according to the data the cyber- physical system (CPS) collects from the individual patients. This approach will be scalable, extend patient monitoring to data collections at home by using portable sensors providing information about a patient’s recovery status, and influence healthcare of the future. 2 Wednesday, May 28, 14
  • 3. UPEN’S DEFINITION Medical Cyber-Physical Systems (MCPS) are complex, safety-critical, intelligent systems of interconnected medical devices. An MCPS brings together monitoring devices, such as heart-rate monitors, and delivery devices, such as medication infusion pumps. 3 Wednesday, May 28, 14
  • 4. 4 Networked Closed-loop Medical Cyber-Physical Systems Closed-Loop Decision Support? Wednesday, May 28, 14
  • 5. MEDICAL CPS ENVIRONMENTS Objective, Workscope, Outcomes Wednesday, May 28, 14
  • 6. • EIT ICT Labs is the Knowledge and Innovation Com- munity of EIT with a focus on future Information and Communication Society. In this framework, we have a specific innovation objective within cyber- physical systems, namely to combine active and passive user input modes in clinical environments for knowledge acquisition and decision support. 6 Wednesday, May 28, 14
  • 7. The EIT ICT Labs' investment model, the so-called "catalyst-carrier model" 7 http://www.dfki.de/ MedicalCPS/ New co-operation of European institutions in a future European healthcare sector. Rich capture of clinical data, including symptoms and signs rather than just a single diagnosis, are among the objectives of parallel European efforts (e.g., http://www.transformproject.eu/). Wednesday, May 28, 14
  • 8. EIT Activity: Specific Objectives • We will combine active and passive user input modes in clinical environments for knowledge acquisition and decision support. • Active input modes include digital pens, smartphones, and automatic handwriting recognition for a direct digitalisation of patient data. • Passive input modes include sensors of the clinical environment and/or mobile smartphones. • This combination for knowledge acquisition and decision support (while using machine learning techniques) has not yet been explored in clinical environments and is of specific interest because it combines previously unconnected information sources. • The innovative aspect is a holistic view on individual patients whereby individual active and passive patient data can be taken into account for clinical decision support. Wednesday, May 28, 14
  • 9. 9 In our first running testbed system, sensor data aggregate clinical databases automatically whereupon scalable patient data can be recorded on a daily basis in a clinical database. Difference to IBM WATSON: Realtime aspect, direct digitisation, holistic view Wednesday, May 28, 14
  • 10. 10 • We hope that the reference architecture will provide new insights into the interplay of active and passive sensor modes in the medical domain which we will additionally exploit in the technology transfer into the European healthcare sector. • three main steps: (1) digitisation of patient data, (2) usage of en- vironmental (medical) sensor technology, (3) distributed data access and real-time clinical decision support. Wednesday, May 28, 14
  • 11. 11 Milestones • knowledge acquisition by intelligent user interfaces (active modes); • networked embedded systems development and sensors development in presence, posture, and activity monitoring of humans by non- intrusive sensors, thereby including a functional sensor architecture (passive modes); • data mining: segmentation of behavioural patterns, finding analogies between behaviours, and predicting using these analogies; • the knowledge integration (including a semantic model for clinical information) according to the data intelligence models in combination with clinical patient data towards clinical decision support. Wednesday, May 28, 14
  • 12. Overall rationale • The overall rationale is to address the knowledge acquisition bottleneck: future clinical care relies on digitised patient information; this information can be collected manually (active mode) by a doctor or a patient, or automatically (passive mode) by using suitable environmental sensors, e.g., of a portable device the patient is carrying. • Suitable data mining models should enable us to combine those independent information sources for a combined decision support model. For example, a patient’s recovery from a back injury can be monitored by the movement data while climbing steps and combined with the digitised patient disease record at the hospital’s side. 12 Wednesday, May 28, 14
  • 13. Wednesday, May 28, 14
  • 14. COMPONENTS Wednesday, May 28, 14
  • 15. 15 Input/Output Real-Time Dialogue Server Wednesday, May 28, 14
  • 16. Real-time communication • The communication between activity sensor server and system server is realized using XML-RPC. 16 Wednesday, May 28, 14
  • 17. DFKI Active User Input Wednesday, May 28, 14
  • 18. 18 Wednesday, May 28, 14
  • 19. 19Technische Universität München Philipp Kindt <kindt@rcs.ei.tum.de> 5/ TopicSuggestion:ECG+activitymonitoringat home ● Heart data is collected and surveilled continuously ● Compressed form of data is sent to hospital through a wireless link to the smartphone or a base station ● Seamless integration into hospital's information processing system ● Long-time data will become available to doctor ● Motion artifacts should be filtered by accellerometer data Wednesday, May 28, 14
  • 20. Mobile Sensor Architecture / Data Flow 20 1) The activity sensor measures acceleration data in three dimensions and transmits them via a Bluetooth Low Energy (BLE)- link to the activity sensor server. 2) The activity sensor server receives the streaming data. It converts the raw acceleration signals from the sensors to physical units. The signal is then processed, transformed to the frequency domain and activity detection performed. Wednesday, May 28, 14
  • 21. 21 Wednesday, May 28, 14
  • 22. EöFacultyofInformaticsNeuralInfProcessingGroup ? ? Hint… Research: Pattern completion  spatio-temporal Sparse methods  dictionary learning  structure discovery  NLP Applications: Face  action-units Educational games  recommender systems CPS  pattern discovery and forecasting ELTE Wednesday, May 28, 14
  • 23. 02/19/13 23 Another pilot use case: a sofa/bed with capacitive sensors 11 9 8 7 5 1 3 0 4 Applications: • Sleep disorder monitoring • Supporting independent living of elderly Wednesday, May 28, 14
  • 24. 05/28/14 24 Pilot sensor installation in a bed bottom plate Unfolded bed bottom plate Bed bottom plate folded for transportation Leg end instrumentation of the bed bottom plate: • four sensor elements, • a sensor board, • a communication board • USB connection Size: • unfolded: 80 cm x 200 cm • folded: 80 cm x 50 cm Material: foam PVC Wednesday, May 28, 14
  • 25. 05/28/14 25 User interface User is laying twisted User is laying flat User is sitting High activity Medium activity Low activity Wednesday, May 28, 14
  • 26. 05/28/14 26 Functionality of the bed installation Self calibration • Sensor platform powers up and calibrates itself when the USB connection is established • Persons are not allowed on the bed at this time, but all the bed clothes (mattress, bed- sheet, etc.) must be on place • After a few seconds the sensor system is calibrated and operational Measurement • Sensor Board A measures the capacitance of 5 sensor electrodes, and communicates with the Communication Board by using SPI bus • Sensor Board B operates the same but with 4 sensor electrodes • Communication Board combines the results from both of the Sensor boards and sends the measurement results to the PC by using USB connection Data processing • The data is processed on a PC or in a server • The system detects if the bed is empty or if there is a person on the bed • It also detects the posture of the person: sitting on the side of the bed, laying flat on one’s back, laying twisted in fetal position • Also the activity of the person is detected: whether the person has low, medium, high or shocking activity Wednesday, May 28, 14
  • 27. • The motivation for additional environmental sensors lies in tracking the behaviour of patients or care home residents and detecting abnormal living patterns. • In addition, the medical CPS perspective includes the monitoring of clinical care of bedridden patients. • We take an approach for such an (eHealth) monitoring detector by implementing an intelligent furniture network (here, a bedplate). • Human behaviour in the form of postures and activity levels is monitored by using a set of very low cost low-intrusive capacitive proximity sensors. 27 Wednesday, May 28, 14
  • 28. SEMANTIC MODEL FOR CLINCAL INFORATION 28 Wednesday, May 28, 14
  • 29. LOGO 29 Data Integration Steps Knowledge Lifecyle in Multimodal Medical CPS “Retrieve patient record” “Add captured input data” Wednesday, May 28, 14
  • 30. • We developed a generic clinical data access strategy within this medical CPS activity that allows us to seamlessly and flexibly align patient data from various data sources, including sensor data. • First, an integrated clinical information model that establishes the foundation to integrate and structure clinical data by providing concepts covering meta-information and interpretations of clinical patient data; • Second, an natural language processing (NLP) information extraction pipeline that provides the basis algorithms needed to extract the relevant information from unstructured clinical textual documents. • Goal: include sensory data towards real-time clinical decision support • (Currently, we are in the process of aligning the semantically described clinical data with the patient sensor data.) • Increased data availability should make individualised treatments possible; the problem we, however, face is that these data are not semantically integrated. 30 Wednesday, May 28, 14
  • 31. • Standardised representation requires the use of established ontologies, vocabularies or coding systems like, e.g., the ICD10, LOINC1 or SNOMED CT2. • We propose a semantic Model for Clinical Information (MCI) based on the Ontology of General Medical Science (OGMS). • The resulting semantic model allows us to monitor for example the sleeping status which can be associated with health information; or the duration a patient is bedridden can be associated with administrative data. 31 Wednesday, May 28, 14
  • 32. Model for Clincial Information 32 Link to terminologies Storage of application data SNOMED CT—Systematized Nomenclature of Medicine Clinical Terms; ICD—International Classification of Diseases; LOINC—Logical Observation Identifier Names and Codes: ATC—Anatomical Therapeutic Chemical Classification System; RadLex—Radiological Lexicon; DOID— Disease Ontology; FMA—Foundational Model of Anatomy; OPS— Operationen- und Prozedurenschluessel (German coding system for pro- cedures); SYMP—Symptom Ontology; LinkedCT—Linked Clinical Trials; DrugBank—Open Drug Data; SIDER—Side Effect Resource. The current version of MCI has links to ICD-10, OPS10, ATC11, LOINC, the FMA, and RadLex. By using federated SPARQL queries, it becomes possible t combine data of MCI with knowledge contained in other ontologies RDF RDF RDF OWL Wednesday, May 28, 14
  • 33. Integrating processes and ontologies 33 Roles / Organization Documents Semantic Annotations Actions Wednesday, May 28, 14
  • 34. 34 In Medical CPS, we modelled the ACR guidelines: http://www.acr.org/Quality-Safety/Standards-Guidelines/ Practice-Guidelines-by-Modality/Breast-Imaging. (1) support for the construction of integrated domain and process models; (2) easy editing of a wiki page by means of forms accessible to the medical experts; (3) automatic import and export in OWL and BPMN; easy import of lists of elements organised according to predefined semantic structures (taxonomy or partonomy); (4) term extraction functionalities; (5) graphical browsing/editing of the domain and process models; and (6) fully-integrated model evaluation functionalities (model checklist and quality indicators) Wednesday, May 28, 14
  • 35. Forthcoming • http://www.kuenstliche-intelligenz.de/ index.php?id=7801 • EIT action line health and wellbeing • “Clinical Data Intelligence” (BMWi) • “Kognit” (BMBF) • Augmented Reality 35 Wednesday, May 28, 14
  • 36. CONCLUSIONS • We described how we combined active and passive user input modes in clinical environments for knowledge discovery and knowledge acquisition towards real-time decision support in clinical environments. • The main contributions are, first, technical components for aggregating digitised patient information, combining manual data acquisition and sensory data together with a dialogue server, and second, the integration of data acquisition technologies into a clinical testbed environment based on semantic models for clinical information. • The expected outcomes within EIT include enabling technologies for clinical decision support; and a medical CPS reference architecture that can contribute to the definition of a European Healthcare Infrastructure in Horizon 2020, see http://ec.europa.eu/ programmes/horizon2020. • In the next project phase (2015), we will focus on CPS related programming questions, i.e., physiological close loop control with human in the loop extension (proactively request users for real- time expert input). 36 Wednesday, May 28, 14
  • 37. Thank you! 37 Wednesday, May 28, 14
  • 38. 38 Wednesday, May 28, 14
  • 39. NEW DIRECTIONS DFKI / Siemens / ELTE Wednesday, May 28, 14
  • 40. 40 Augmented Reality in Medicine Video: https://dl.dropboxusercontent.com/u/48051165/ERmed_CHI2013video.mp4 Wednesday, May 28, 14
  • 41. 41 Wednesday, May 28, 14
  • 42. CPS PROGRAMMING OBJECTIVES 2014-2015 DFKI / Siemens / KTH / ELTE Wednesday, May 28, 14
  • 43. 43 Decision support systems: professionals and patients, as well as patient guidance services, which build on multimodal data fusion, data and pattern analysis, and modeling and predictive algorithms of patient health status Wednesday, May 28, 14
  • 44. CPS Action CPS Action CPS Action Wednesday, May 28, 14
  • 45. 45 Server/Client infrastructure security & resilience (DFKI) “the capacity to recover quickly from difficulties” Quality assurance of Medical CPS programs (ELTE) Clinical workflow standardisation (SIEMENS) CPS Action CPS Action CPS Action Task Structure Wednesday, May 28, 14
  • 46. Valorisation potential and expected revenues to be generated • Our experts predict a change of the medical healthcare sector in Europe (according to the US developments) with three main steps: • (1) digitalisation of patient data, • (2) usage of environmental (medical) sensor technology, • (3) distributed data access and real-time clinical decision support. This activity addresses all three steps in very specific but related tasks. • The European healthcare market profits directly by the provided reference architecture, and indirectly by a new co-operation of European institutions in a future European healthcare sector. Wednesday, May 28, 14