Situational Awareness for Smart Health ApplicationsABSTRACTWith sensors and sensing applications proliferating in the mode...
awareness to the operator. With experience, the pattern-recognition/action-selectionsequence can become automated and redu...
Figure 2. A screenshot from Google Health ( for a fictional user, “Jane.”     By aggregating information...
address the various data types and characteristics of information collected, as well as    possible schema evolution, the ...
Figure 2. In this conceptual representation of a data-centric model, applications read and       write data without requir...
SA Systems Often Need to Analyze Data in Real TimeTo provide situational awareness, systems need to aggregate, correlate, ...
Figure 3. Snapshot from a research application that integrates information from pathology,     doctor and ER notes to prov...
and middleware will be better able to control and monitor the flow of information, makingit easier to design compliance to...
respiration monitor {DEVICE_ID, TIME, RESPIRATION_RATE_PER_MINUTE}. By using DDS,and by having these sensor feeds updating...
enough to adapt to new events and new sensors. Effective systems are never built bymerely aggregating compelling features,...
Upcoming SlideShare
Loading in …5

Situational Awareness for Smart Health


Published on

Published in: Health & Medicine
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Situational Awareness for Smart Health

  1. 1. Situational Awareness for Smart Health ApplicationsABSTRACTWith sensors and sensing applications proliferating in the modern world, traditional datasilos becoming available through service-oriented architecture (SOA), and new social databecoming accessible through apps, we now have the ability to consolidate large volumes ofdata to better comprehend and analyze the environment around us. By consolidating andintegrating this data in real time, we have opportunities to develop new suites of smartapplications that can change the way we manage our health, drive our cars, track inventory—the possibilities are endless.These “smart” applications will need to be more aware of their environment; they will needattributes of what is commonly called situational awareness (SA). This article begins bydescribing some common attributes of situational awareness. Next, we will introduce theconcept of a data-centric architecture and discuss why it is essential for building SAapplications. The article will discuss some complimentary technologies required to buildsuch a system. To narrate the concepts, we will use examples from how new applications inhealth care (Smart Health) plan to change the way we care for chronic diseases and manageour health.What is Situational Awareness?Situational awareness (SA) refers to a system being aware of its surroundings, its users andtheir working context, with the ability to show relevant information that will assist users indecision making. SA creates a model that captures the system state and provides anunderstanding of how events affect that state. A good SA model integrates relevantinformation from multiple sources, determines the relative importance of different events,and projects the state of the system based on events. To build a system that is situationallyaware, the model must be accurate and must update quickly to reflect current events.SA systems are not the same as systems that do multi-sensor data fusion. Multi-sensor datafusion techniques combine data from multiple sensors, providing more accuratemeasurements of the environment. However, a multi-sensor system does not understandthe context of the user or the state of the system, and has little intelligence to process thedata. Consider for example a device that measures body temperature from multiple placeson the body. The device may use sensor fusion techniques to eliminate faulty readings andprovide the most accurate body temperature reading based on its sensor-fusionalgorithms. A SA system, by realizing that one particular sensor always returns an outlyingreading, may recommend the user to check if it is working or properly connected.An SA system by itself does not provide value. It is entirely possible for an operator to havean excellent SA system and still make an incorrect decision. This could be due to poorstrategies, poor training, or poor interpretation, among other reasons. Where possible, anSA system needs to take the next step: it needs to recognize patterns and either takeautonomous action or proactively direct operator attention. Pattern-matching technologyor machine-learning techniques can recognize correlated events and assist with delivering
  2. 2. awareness to the operator. With experience, the pattern-recognition/action-selectionsequence can become automated and reduce demands on the operator.As an example, in health care, practitioners of bio-informatics have recognized the value ofproviding situational awareness in their sensing applications. By correlating – in real time--sensor data for ECG, blood oxygen, blood pressure, respiration or pulse, and applyingpatterns to monitor events of interest, we can build systems that can be used to managepatients with chronic conditions (such as heart diseases or diabetes) and alert the patientor the medical provider for anomalous events. Figure 1. This proposed iPad® Smart Health application integrates sensor information with operator input and knowledge-base data to provide SA for tracking blood sugar levels (see specific techniques used by each application to provide SA will vary. However, todeliver such awareness of the environment to real-time systems, application architects andsystem developers must follow the guidelines described below.True SA Requires Integrating and Interpreting InformationWith the informational deluge, there is a gap between the large volume of sensing dataproduced and a human’s ability to process the information. Ironically, overwhelmed orunder-trained operators may be even less informed with numerous, highly capable sensingdevices than with fewer, simpler ones. For the information to be processed correctly, itmust be integrated and interpreted correctly. For example, if a home-based monitoringsystem were to be developed for managing cardiac patients, it will not be possible to becontinually watching for ECG, pulse, or blood pressure reading and trying to detect eventsof interest – the system will not be usable or valuable. What is required is a system thatintegrates and correlates data from different medical devices in real time. 2
  3. 3. Figure 2. A screenshot from Google Health ( for a fictional user, “Jane.” By aggregating information for diverse sources, Jane can track how exercise and medication help improve her cholesterol and blood pressure. Notice that the information required to derive SA comes from multiple sources (pathology tests, pedometer sensors, and user input).However, integrating information from distributed sensor systems such as these medicaldevices is more complicated than integrating data in traditional enterprise systems.• Integration of heterogeneous architectures: Unlike traditional enterprise systems, embedded and RTOS markets for operating systems are heavily fragmented; typical sensing systems use a range of operating systems (INTEGRITY, VxWorks, LynxOS, TinyOS, …), devices and network protocols (UDP, TCP, Bluetooth, Infiniband, wireless, radio, …) and middleware protocols (JMS, HTTP, DDS, ...). Often, no single sensing system can provide a comprehensive event-detection or monitoring system. Instead, a combination of best-of-breed components—each designed for a specific purpose, operating system, and network protocol—must together provide a comprehensive solution. Data from multiple sources must be organized and prioritized to support distributed, cooperative decision-making.• Dynamic, evolvable, and type-safe data representation and encapsulation: These SA models must allow for the collection of a variety of data types from sensor probes. To 3
  4. 4. address the various data types and characteristics of information collected, as well as possible schema evolution, the SA model must provide an approach for having self- describing data or a similar mechanism that allows clients to discover and process the schema dynamically. What this means is that a SA system cannot define a unified and complete data structure upfront, which all the medical devices use. What is required is a methodology where different devices/sensors can still use different data types, and can still be integrated without complex code.• Event correlation and aggregation: SA is about inferring activity of interest—events— either by monitoring for known abnormalities or intelligently adapting to the environment to infer abnormal events. To do so, events from different sensors must be correlated and aggregated. For example, some events are immediately recognizable (such as systolic blood pressure > 200). Other events are characterized by intermittent activity spanning a much longer timeframe (hours, days, or even weeks) and may not even be identified as an event until a vast collection of records are considered in aggregate (for example, systolic blood pressure = 160 but has been steadily rising for the last week). Thus, while every event has a definitive beginning, this starting point is not always discernible at the time of occurrence. Neither is the time it takes for such an event to unfold or its ultimate duration predictable. Event-detection tools are needed to normalize events from different sources and correlate them by time or distance to identify possible information of interest.A Data-Centric Architecture is Key to Building an SA SystemThe architecture for connecting sensors and distributing their data can either follow amessage-centric or data-centric design pattern. In a message-centric model, theinfrastructure does not understand your data. The infrastructure carries “opaque” contentthat varies in structure from message to message. Because the messages have no identity,they are indistinguishable to the infrastructure. They also lack lifecycle management. Oftenused for Enterprise Service Bus (ESB) messaging, the Java Message Service (JMS) API andAdvanced Message Queuing Protocol (AMQP) are examples of such message-centrictechnologies. For these technologies, there is no need for a semantic understanding of thedata.A data-centric architecture uses the principles of a global data-space. It resembles a virtual,centralized database. From the operator’s viewpoint, the data collected from differentsources appears as if it is from a single source. The operator does not have to worry aboutaccessing the data source from each sensor, normalizing the data, etc. All sensorscontribute their data to the global data-space. Applications access this data, similar to adatabase, with no concern for the distributed nature of the system. 4
  5. 5. Figure 2. In this conceptual representation of a data-centric model, applications read and write data without requiring any information on the location of the data, transport protocol, operating system, and so on.With a data-centric design (see Figure 2), developers specify only the data requirements—inputs and outputs—of each subsystem. Applications focus only on the data they intend toconsume or produce and leave the mechanism to procure, normalize, filter, and enrich thedata to a data bus. What this implies is that while developing applications for providingsituation awareness, say tracking cardiac health, we do not need to worry about how toconnect to the medical device, how to do endian conversion, how to transform data types,how to poll for the next sample, how to demarshall messages on the socket to a datastructure – the data centric middleware should be able to manage all these operations.In a data-centric model, the infrastructure does understand your data. In particular, itunderstands: • What data schemas will be used • Which data items are distinct from which others • The lifecycles for the data items • How to attach behavior (e.g., filters and Quality of Service) to individual data itemsA data-centric architecture removes the tight coupling between the data producer andconsumer, thus making the design scalable and evolvable. Examples of data-centric designtechnologies include the Data Distribution Service (DDS) for Real-Time Systems standardand the Real-Time Publish-Subscribe (RTPS) wire protocol, both from the ObjectManagement Group (OMG).This requirement is critical for building a situational aware system, which becomes moreaware as it senses more sources and analyzes them in real time. With a data-centricarchitecture, the middleware understands the data; only the relevant data is put on thewire, avoiding performance bottlenecks. As an example, while the sensor may be makingtemperature readings at 5 Hz, we can use the middleware to only send the information at 1Hz or when the temperature exceeds 99 F. This capability is not conveniently possiblewithout having a data-centric architecture. 5
  6. 6. SA Systems Often Need to Analyze Data in Real TimeTo provide situational awareness, systems need to aggregate, correlate, cleanse, andprocess sensor data in real time.New technologies such as Complex Event Processing (CEP) allow users to performtraditional database and data-mining tasks like data validation, cleaning, enrichment, andanalysis without first persisting the data. By using CEP, application developers can query,filter, and transform data from multiple sensors for event detection in real time. With theability to automate pattern monitoring in real time through CEP, operators can developautonomic event-response mechanisms and get critical information for isolating events.For example, sticking with the cardiac tracking application, we can have a system thatintegrates data from medical devices measuring ECG, temperature, pulse. By using CEP, wecan define watching for events with patterns of interest: temperature < 97 AND pulse > 110SA systems often need to add new queries in a monitoring system without recompilingcode or restarting the system. With CEP, operators can develop new pattern filters withoutrecompiling the code. This feature becomes very critical in use-cases where the situation isdynamic, for example, in agent monitoring or the clinician adding new patterns formonitoring.Visualization Frameworks Need to Adapt for SAWith the avalanche of sensing information, to have as complete an SA model as possible, itis imperative for the operator to address risks with the highest importance and have accessto all the data. This system must evaluate, prioritize, and present time-sensitive data tousers in an understandable format.The visualization technology should provide a layered view of the data so the operator hasa comprehensive perspective of the environment, with the option to observe events ofinterest in greater detail (i.e., zoom in, zoom out). A layered view will also allow data to beaggregated across many dimensions: time, network components, hosts, and applications.The ease of browsing such state information can be provided through heat maps that canimmediately convey the visual message. The information processed by the heat maps mustbe measurable and segmented so it can provide different status levels.Visualization is a very powerful way to provide a sensing system’s status to an operator.However, a visualization framework should be extendable to include new types of sensorsand it must provide interoperability. A visualization framework needs to use a layered viewof the data so operators can get high-level information about the system as well as detailedinformation. 6
  7. 7. Figure 3. Snapshot from a research application that integrates information from pathology, doctor and ER notes to provide situational awareness to improve patient care in an intensive care unit (see visualization framework for an SA system should have the ability to provide negativereasoning for diagnosis. In this approach, the operator seeks evidence that will disprove ahypothesis. That is, unlike monitoring data for events of interest, the system monitors datato perform negative reasoning. This kind of diagnosis becomes very important in use casessuch as medical pathology. In addition, an SA system should understand the context of theuser and the state of the system so it can prioritize information and help the operator focuson information and events of interest.SA Systems Should Address Privacy and Security ConcernsCorrelating information from multiple sensors and historical data implies that the thornyissues of access control and privacy have been addressed. Privacy defines policies andprocedures for sharing information correctly. Access control is the mechanism toimplement privacy.Often requiring a cultural and legal context, privacy is about controlling how informationflows. For example, while there have been many implementations of access-controlmechanisms for enterprise applications, there is no compelling implementation thataddresses compliance for the Health Insurance Portability and Accountability Act (HIPAA)privacy law. Privacy implications for sharing information will obviously vary amongindustries and use cases, but an SA system that is built on a content-aware infrastructure 7
  8. 8. and middleware will be better able to control and monitor the flow of information, makingit easier to design compliance to privacy rules for the application. Figure 4. HIPAA Compliance Checker, a project by Professor John Mitchell, provides HIPAA privacy compliance to messaging by understanding user and message contexts (see systems that collect data from multiple sensors are inherently distributed. Each “remotesensor node” represents a potential point of attack from intruders. The network may bedispersed over a large area, further exposing it to attackers who may capture andreprogram individual sensor nodes. Since it may not be possible to guarantee security forall remote sensor sources, the SA system must be vigilant in monitoring for intrusions.Putting it TogetherConsider, for the sake of illustrating the principles, an in-home cardiac monitoring devicethat consists of (minimally obtrusive) sensors that measure blood oxygen, pulse,respiration rate, and body temperature. The sensor readings for a healthy individualusually show 97%-99% Oxygen saturation, 14-20 breaths per minute, a pulse rate of 60-80per minute, and an oral body temperature of 98.6 F.There are considerable challenges in providing effective monitoring if all these devicesprovide readings to the caregiver or the patient in isolation. Instead of documenting all thelimitations, let us discuss how a situational-aware system built on data-centric principlesprovided by Data Distribution Service (DDS) can enable new use cases.By using a data-centric middleware, all the sensing devices are updating the information, inreal time, to the common global data space. To better understand, consider the Oxygenmeter updating data instances with attributes {DEVICE_ID, TIME, PERCENTAGE_OXYGEN}.Similarly, the pulse sensor is publishing {DEVICE_ID, TIME, PULSE_PER_MINUTE}, and the 8
  9. 9. respiration monitor {DEVICE_ID, TIME, RESPIRATION_RATE_PER_MINUTE}. By using DDS,and by having these sensor feeds updating information in real-time, we have a virtualdatabase that can be analyzed very efficiently.For this simple use case, the monitoring application does not need to integrate and parsemessages from each sensor to update and correlate the application data structures. Butthere are many other advantages to using a data-centric, publish-subscribe middleware: bydefining the quality of service (QoS) contracts for each publishing sensor, we can make themonitoring device more efficient. For example, while we may be interested in getting pulsemeasurements every second, getting temperature readings at five-minute intervals may bemore relevant, saving traffic on the wire and reducing load on the system. Such contractscan be easily defined without updating the sensors or the application code. The ObjectManagement Group (OMG) DDS specification provides a rich library of real-timenetworking behavior that can be availed simply by turning on an XML parameter— withoutwriting complex code.Once the data is acquired, we can use a Complex Event Processing (CEP) engine to performreal-time analysis and set alerts. For example, by using the CCL scripting languagedescribed in previous sections, we can develop alerts such as: “when WITHIN five minutesthe BASELINE (average) respiration increases by 10% and BASELINE Oxygen falls by 3%.”Keep in mind that the CEP engine would not process sensor readings in real-time if thestream had not first been normalized into data structures, which a data-centric middlewarelike DDS so efficiently enables.More interestingly, we can analyze real-time trends with CEP and provide more instructivemonitoring and care. With CEP’s real-time data-mining capabilities, for example, we coulddetect that a patient’s respiration rate significantly increases a few minutes after atemperature spike, risking the heart. In that case, care may be directed (depending on themedical specifics of the case, of course) towards managing the fever rather than medicatingto calm the heart.In addition, with an intelligently built visualization interface, the system can continuallyreceive patient feedback to establish baseline trends and detect correlations in anomalies(example: patient reports pain each time the sensor is reading an increase in pulse, butonly between 2-4 am).Obviously, there are many challenges in building such a system. Perhaps the firstgeneration of such technologies would only provide information for the eventual doctorvisit, where the patient (traditionally) could not either recall the specific anomalies or didnot have all the data. By providing such richness in information, enabled by a newgeneration of sensing devices, real-time networking technologies such as DDS, and real-time stream processing, we could change the way we provide care.ConclusionA system with SA capabilities can only provide true situational awareness if the operatorsare trained correctly, the relevant data sources are integrated, and the system is flexible 9
  10. 10. enough to adapt to new events and new sensors. Effective systems are never built bymerely aggregating compelling features, as that technique does not deliver value to theoperator. By focusing on the operators and their need to comprehend and act on the data,we can deliver true SA applications using multi-sensor systems.We are on the cusp of another information revolution, where real-world data will help usbetter understand and react to the world around us. This revolution will come fromintegrating, in real time, information that traditionally resides in application silos and byapplying multi-sensor data fusion to provide true situational awareness to the applicationuser. By converting the data deluge into actionable intelligence, we can change how wedeliver patient care, drive our automobiles, manage our assets, and much more. 10