3.3 - A Methodology for Developing Quality of Information Metrics for Bod…


Published on

Tuesday, October 23, 2012
Technical Session #3

Italo Armenti (University of Virginia, US), Philip Asare (University of Virginia, US), Juliana Su (University of Virginia, US), John Lach (University of Virginia, US)

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Next slide!
  • In wireless health research engineers have to face many optimization issuesTrade-offs are technical (power, delay, cost…) and health metrics (health decisions, diagnosis)Engineers are at good dealing with technical trade-offs but are not good with the health metrics, we focus on QoI in this workAll optimizations have to take QoI into account
  • Goal: provide best possible data as we can… but other metrics suffer from our designTo save power, we may do lossy compression on data… could help wearability less power means smaller battery, QoI is less but still goodIf we try another design: less sensors, wearability, cost and power are better… but QoI is impactedWe can’t ignore QoI (emphasis)
  • A QoI measure assesses how different design points of a system affect medical decisionsIt is a measure that has to be determined in collaboration with doctors. We can evaluate design points without involving a doctor anytime we come up with a new one. Developing the measure we also get a threshold of the minimum QoI that doctors can tolerate. This way we engineers can proceed normally with our DSE.
  • The first and third point in this slide are written in the previous slide’s notes
  • 1) Authors observed that for an ECG there is no need to transmit the whole signal, features of it are sufficient. More in general, if waveform on top left (input in our system) can be turned in waveform on top right, we preserve the features of interest: QoI is high and we can save energy for transmission.2) The new measure proposed in the WDD paper correlated better with health decisions, this measure was indeed based on 18 features extracted from each PQRST complex.3) Our goal: generalize to assess QoI on any signal and application (not only ECG and compression)
  • In the design space there are many knobs we can turn. These are going to affect all the metrics of interest e.g. if we compress data in lossy manner we save energy but QoI is affectedSampling -> if lower -> less energy but QoI may be affectedProcessing -> digital filterMany other knobs that impact metrics (power, wearability, QoI…)The framework we propose helps explore the impact of tuning the knobs on QoI
  • In QoI eval. we use an Ideal System (highest QoI, doesn’t distort the data) as reference. It’s a design impractical for development as a wireless health system.Then we compare each design point to the ideal sys. The QoI value is indeed a score we assign design points with respect to the ideal sys.
  • 1) Every d.p. is evaluated feeding it data from all subjects available. Working with clinicians we also obtain a threshold of tolerability with which we can determine d.p. that provide poor QoI – significantly impact (in a negative way) medical decisions2) Once we evaluated all possible d.p. we can select the most promising ones.
  • IMUs on wrist, sacrum, left ankleInteresting case bc not based on sensing modality formally established in health care  determining QoI becomes particularly challenging3) Read the rest of the slide,ppl away could not be able to readAbnormal cadence is an indicator of low mobility which is an indicator of several diseases (related to parkinsonian gait)
  • In our case study we evaluated 7 different design points and obtained the following results…D2 node senses data and streams it as is to the base station where classification is done.D5 classifies on node and compresses (RateRes) only walking segments (QoI here is close to D2 but requires less energy… this seems a good design trade-off!!!)  we couldn’t have got to this tradeoff without the framework!!!- With these result we can then use in conjuction with information on resource consumption or cost of the design point to do our trade-offs(the focus of this work, was more on techniques for providing these kinds of quality of information results and not the trade-offs. We will explore those in future work)
  • Application with large design space may take a long time to be evaluated, we need an estimator to save on DSE timeMore applications: traditional sensing modalities and other emerging ones
  • When optimizing to design a wireless health system we cannot ignore QoIWe developed a framework that helps engineers optimize also for QoI
  • Going more into details of the methodology we propose… it has two main stepsBlock evaluation. Applications can be divided into processing blocks each with a function (filtering, compression, processing). In this step we evaluate how a single block performs QoI-wise with respect to the next one.Good to rule out bad d.p. in the beginning2) Then we perform the QoI evaluation (as already showed): in collaboration with doctors we come up with a QoI function to evaluate different d.p.
  • 1) Problem: devices are resource constrained so information presented by a BSN to a doctor my mislead his/her decisions2) Define bad information
  • We assume practitioners have an objective analysis to diagnose a condition. Let’s assume we have the objective analysis: the diagnostic index is the output of the objective analysis of the practitioner: given a design point there is a threshold that tells whether the condition is present or absent. I0 is the output of the objective analysis on an ideal system (with unlimited resources)Every application has different specific metrics, the diagnostic index could change even for the same sensing modality.2) this shows also that we need application-specific metrics bc this index would change depending on the condition, even for the same sensing modality
  • Raw signal or featuresDiagnosis to perform on the dataWe have two design solutions with different configurations (compression algorithms)
  • Engineers have to face many trade-offs (battery lifetime, small cost, wearability, reliability…)Problem is that the BSN may distort the signal (incomprehensible or misleading for the doctor). Cannot let this happen, patients’ health is at stake.The research question we want to answer with this paper is:…
  • In our vision of the BSN, not only we see the BSN as a data collector, but also as a tool whose functions can be modified by the doctor who observes the patient’s signal
  • With respect to impact on medical diagnosisThey used ECG data and tried different compression algorithms, then they assessed how these affected practitioners’ diagnosis.Drawbacks: only one physiologic signal, only effect of compression. We hope BSN will do more than that. Too generalPicture of features
  • we used two lossy compression algorithms
  • BSNs allow for long-term, continuous, remote monitoring of vital signs outside clinical settingsAllow for research on medical devices that will make life betterHow do we realize this?3) Notes on bullets - Long term monitoring is just examples of general areas it can help - Medical research: Longitudinal students that couldn’t be done without BSN We have data on how people’s bodies behave in normal everyday life as opposed to in the hospital which is where most medical data today comes from.
  • In our lab, the INERTIA lab we strive for these goalsWe developed a 6DoF Motion sensor in the form of a watch (used and will be used in many projects)An example is longitudinal assessment of Ankle-Foot Orthosis prescribed to children affected by cerebral palsy
  • Engineers always try to design their system as much close as an ideal oneWhile working towards their goal, there are designs of the system that may give different performance resultsIn wireless health, when designing our system we don’t want to ignore quality of informationSuggestion: a big X on the last design to indicate that even with good delay, cost and power, we cannot ignore quality?
  • Traditional metrics not always are proper, because they are information agnostic and are useful when the physiologic signal in input to the BSN has to be accurately reproduced at the outputBut doctors don’t always look at the whole signal, rather they look at parts of it or features of it (arrythmya -> R peak)
  • We want to know how design points of a BSN impact practitioner’s decision. ECG, EEG, gait analisysWe consider system where there are multiple blocks affecting information
  • First: we don’t want all configurations, we wanna eliminate soon bad design points (e.g. FFT)How well does particular solution preserve information relevant to practitionerProcessing block pic, arrow with ?Block evaluation is relevant for preliminary design choicesMost importantly, for helping develop predictor.
  • Easily applicable on existing sensing modalities bc literature already tells us what are the features of interestIn emerging modalities features of interest are not know yet bc no one has done long-term continuous sensing, the QoI methodology could help find those features.
  • 3.3 - A Methodology for Developing Quality of Information Metrics for Bod…

    1. 1. A Methodology for DevelopingQuality of Information Metrics for Body Sensor Design Italo Armenti1,2, Philip Asare1,2, Juliana Su1,2, John Lach1 1Charles L. Brown Department of Electrical and Computer Engineering 2Department of Computer Science UVA Center for Wireless Health University of Virginia Wireless Health 2012 October 22nd -25th San Diego, CA
    2. 2. Trade-offs• Engineering is all about managing trade-offs• Traditional technical metrics – Power, cost, delay, throughput, …• Emerging Wireless Health metrics – Wearability, safety, quality of information (QoI), …• Optimizations must simultaneously consider all metrics 2
    3. 3. System Optimization Power Power Power Cost CostSystem0 Cost Highest Wearability Wearability Wearability QoI QoI QoI QoI ??? Power Power PowerSystem1 Cost Cost Cost Compress Wearability Wearability Data Wearability QoI QoI ??? QoI Power PowerSystem2 Power Cost Cost Fewer Cost Wearability Sensors Wearability Wearability QoI QoI ??? 3
    4. 4. QoI: Quality of Information System0 Health Highest Decision QoI System1 Health QoI Compress Decision Assessment Data System2 Health Fewer Decision Sensors 4
    5. 5. Project Overview• QoI  potential for leading to correct health decision• Quality metrics are application-specific• Requires input from health practitioners• Goal: Provide framework for QoI-based Wireless Health system optimization 5
    6. 6. Promising Example: Weighted Diagnostic Distortion (Zigel, et al. 2000)• Assessed impact of ECG compression on medical decisions• New measure correlated better with physician assessment• Our goal: generalize for Wireless Health systems & applications 6
    7. 7. Optimization Knobs Processing Communication Sampling Knc-link Kcd-link knode,1 … knode,n kcell,1 … kcell,n kdb,1… kdb,nOriginal End Signal Signal node cellphone database• Other knobs include number/location of sensors, sensor networking, etc.• Framework models all design knobs – Affect power, delay, throughput, etc. – Also affect QoI! 7
    8. 8. QoI Evaluation decisiondata1 Ideal … (I0) decisiondatan [0,1] | I – Di | 0 QoIDi : f(|I0 – Di |) decisiondata1 Designs … (Di) decisiondatan Function (f) determined in collaboration with dataset health practitionerCollected with “Ideal” system(e.g., in-patient 12-lead ECG) 8
    9. 9. Expected End Results • Evaluation of design points on all data points • Assignment of a QoI score: – Relative to ideal system Determined incollaboration with health practitioner 9
    10. 10. Case Study• Assisted living residents monitored with IMUs for ~3 hours• Emerging sensing modality – Particularly challenging QoI assessment• Practitioners interested in: – Data of walking segments – Cadence• System setup: – Sacrum IMU  walking detection – Left Ankle IMU  cadence computation• Bad QoI: – Misclassified walking segments – Distortion in cadence 10
    11. 11. Case Study ResultsD2 D5-No compression -On-node classification-Workstation classification -Walking segment compression-Workstation cadence calculation -Workstation cadence calculation Power(D5) << Power(D2) QoI(D5) QoI(D2) Good tradeoff! 11
    12. 12. Future Work• Develop a QoI predictor – For complex systems and applications with large design space• Expand to other Wireless Health systems and applications – Including other emerging sensing modalities not currently part of standard healthcare 12
    13. 13. Summary• Need measure for quality of information (QoI) in Wireless Health system optimization Power Cost System Wearability QoI !!!• Developed general framework for – Application-specific QoI assessment – QoI-based system optimization 13
    14. 14. Thank YouQuestions?Italo Armenti – ia3he@virginia.edu 14