Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

On Temporal Logic and Signal Processing

2,115 views

Published on

Published in: Technology, Health & Medicine
  • Be the first to comment

  • Be the first to like this

On Temporal Logic and Signal Processing

  1. 1. On Temporal Logic and Signal Processing Ezio Bartocci Vienna University of Technology in collaboration with: Alexandre Donzé, Oded Maler, Dejan Nickovic, Radu Grosu, Scott A. SmolkaUC Berkley, Verimag, IST Austria, TU Wien, SUNY Stony Brook
  2. 2. Outline•  Motivation•  Temporal Logic for Signals•  Temporal Frequency Logic•  Case Study•  Future Works
  3. 3. Motivation
  4. 4. Signals come from everywhere !!From the Earth From the Climate Changes From the Economy Seismometer El Niño/La Niña-Southern Oscillation Stock Market From the Heart From Circuits From Music ECG Amplifier Music Sheet
  5. 5. Signals from the Heart The signal pattern can change in timeECG R Normal ECG J Normal Heart Rhythm Ventricular Ventricular P T Tachycardia Fibrillation Q S QRS interval P-Q interval Q-T interval J R Brugada ECG Simulation P Experiment Q T S
  6. 6. Signals from the Heart The signal pattern can change in spaceECG Normal Heart Rhythm Ventricular Ventricular Tachycardia Fibrillation Simulation Experiment Grosu, et al., Bartocci, Learning and detecting emergent behavior in networks of cardiac myocytes, Comnunications of the ACM, Vol. 52, (3)
  7. 7. Signals from the Heart Heart Rate Dynamics in Health and Disease http://physionet.org/tour/ECG Disease Normal Heart Rhythm Ventricular Ventricular Tachycardia Fibrillation Healthy Disease Simulation Disease Experiment The signal pattern can change in frequency
  8. 8. Signals from the HeartECG Normal Heart Rhythm Ventricular Ventricular Tachycardia Fibrillation Simulation Experiment The signal pattern can change in time, space and frequency
  9. 9. Research Challenges (1) 1.  Can we formally specify the emergent behavior of a disease in a logical language ? What are the ingredients ? (Specification) •  Time (A priori of our knowledge for Kant, •  Space Critique of Pure Reason) •  Frequency (The cells are oscillators, but social and physical phenomena are in general coupled periodic or quasi-periodic oscillators)Example:“Eventually an atrial fibrillation that last for at least 10 seconds will occur.” Holter Recording
  10. 10. Research Challenges (2) 2.  Can we learn a logic formula that satisfies the emergent behavior of a disease and does not satisfy an healthy behavior (Robustness) ? 3.  Can we use the logic formula to explore the regions of the parameter space that will make the model satisfying (or not) the formula (Diagnosis) ?(A) Wave forms on a1000x1000 network ofcardiac cells for varyingparameter values.(B) The associated tipmovement and regions offibrillation.
  11. 11. Research Challenges (3)4.  Can we control the input of a plant model, using the logic formula as our reference input ? (Control)5.  Can we minimally modify the model, such that the logic formula may be satisfied ? (Model Repair)6.  Can we monitor real-time signals using the logic formula as a signal pattern specification ? (Real-time Verification)
  12. 12. Temporal Logic for Signals
  13. 13. Families of Temporal Logics Linear Temporal Logic (LTL) Discrete-time semantics A. Pnueli, 1977Syntax:Syntactic sugar for Eventually and Globally:Semantics: Kripke Structure
  14. 14. Families of Temporal Logics Linear Temporal Logic (LTL) A. Pnueli, 1977 Metric Interval Time Propositional Temporal Logic (MITL) Temporal Logic (TPTL)R. Alur, T. Feder, T. A. Henzinger, 1996 R. Alur, T. A. Henzinger, 1994 They extend LTL with continuous-time semantics
  15. 15. Families of Temporal Logics Linear Temporal Logic (LTL) A. Pnueli, 1977 Metric Interval Time Propositional Temporal Logic (MITL) Temporal Logic (TPTL) R. Alur, T. Feder, T. A. Henzinger, 1991 R. Alur, T. A. Henzinger, 1989Syntax: Boolean Signals:Syntactic sugar for Eventually and Globally:Semantics: Minkowski sum and difference:
  16. 16. Families of Temporal Logics Linear Temporal Logic (LTL) A. Pnueli, 1977 Metric Interval Time Propositional Temporal Logic (MITL) Temporal Logic (TPTL)R. Alur, T. Feder, T. A. Henzinger, 1991 R. Alur, T. A. Henzinger, 1989 Signal Temporal Logic (STL) They extend MITL with Real-Valued Signals O. Maler, D. , Nickovic, 2004
  17. 17. Signal Temporal LogicProperty to verify:
  18. 18. Families of Temporal Logics Linear Temporal Logic (LTL) A. Pnueli, 1977 Metric Interval Time Propositional Temporal Logic (MITL) Temporal Logic (TPTL)R. Alur, T. Feder, T. A. Henzinger, 1991 R. Alur, T. A. Henzinger, 1989 Signal Temporal Logic (STL) O. Maler, D. , Nickovic, 2004 Temporal Frequency We extend the Signal Temporal Logic Logic (STL) to deal with time-dependent frequency Our attempt, 2012 amplitudes
  19. 19. Temporal Frequency Logic
  20. 20. Frequency Analysis in a Nutshell Fourier TransformFourier series: 1T0 = ω0 a0 +∞x[t] = + ∑ ak cos ( 2πω k t ) + bk sin ( 2πω k t ) 2 k=0with ω k = kω 0Eulers formula: eix = cos(x) + i sin(x) +∞x[t] = ∑ce k 2iπ kω 0t k=−∞Continuos Fourier Transform: +∞cω = x (ω ) = ∫ ˆ x[t]e−2iπω t dt −∞Inverse Continuos Fourier Transform: +∞x[t] = ∫ cω e2iπω t dt −∞
  21. 21. t we recorded. Indeed, our melody does not have has other notes that sound cool with an E line). Combining Time and Frequencyr (but definitely bluesy) formula which looks for a The Short-Time Fourier Transform and the so-called “turn-around” (the sequence B Continuous-time STFT: 1 ∞ STFT {x(t)}(τ , ω ) ≡ X(τ , ω ) = ∫ x ( t )ω (t − τ )e − jω t dt 0b] µA ^ ⌃[8b,9b] (µB ^ ⌃[b,2b] µA ^ ⌃[2b,3b] µE ) −∞ Discrete-time STFT: ⌥1 (10) 0 1 2 3 4 5 ∞Figuren ]5. ω ) ≡ X(τ ,ω ) = ∑ x [ n ]ω [n − m]e STFT {x [ }(m, − jω n 50 n=−∞ 45 Fixed Resoultion: 40 35mented function pitch in Matlab and defined STL 30ool [5]. All examples in this paper are also avail- 25 20g.imag.fr/~window size It depends on the donze/breach_music_example.html 15STL syntax (Boolean and temporal operators) on 10 5form predicate for the logic: New 0 0 1 2 3 4 5 0 0.5 1 1.5 µ = f (x, p) > ✓ 0.2 0.1 0nal operator and p is one or more parameters. The 0 1 2 3 4 5
  22. 22. Combining Time and Frequency The Wavelet Transform Mexican Hat Morlet HaarFourier Transform Short Time Fourier Transform Wavelet Transform
  23. 23. Case Study: Music
  24. 24. meaning that pitch! (x)[t] is the amplitude of ! in x around time t. To be useful, this operator must satisfy two conflicting constraints: to be able to tolerate small pitch variations (for instruments which are not perfectly Case Study: Music tuned) and, on the other hand, to discriminate a note from its two closest distinct 1 1 neighboring notes. These notes have frequencies !1 = 2 12 ! and !2 = 2 12 !. Chromatic Scale: This makes the closest frequency to be discriminated at a distance d! = ! !1 Note Rest Value of !. Thus in our implementation, pitch is defined as the STFT pitch! (x) = Whole note xL (!), where the size L of the window function is automatically chosen (based on ˆ frequency-temporal resolution G4 G4# A4 A# STFT described in the Half note C4 C4# D4 D4# E4 F4 F4# constraints of B4 C5 Appendix) to achieve the required resolution. Once pitch! is defined we can define the pitch-261.63 Hz 523.25 Hz detection predicates. For instance, a predicate detecting the note A Quarter note 1 1 with pitch −ω C 4 = 2= ω C 4 # ω Dwill2be C 4 # !A 12 440Hz 4 = 12 ω written as µA = pitch!A (x) > ✓. The only parameter which eighth note remains to be fixed is the threshold ✓. This will determine the robustness of the predicate to variations in volume, pitch or duration. Octave ω C for 2ω C 4 Note that the Breach tool, used 5 = monitoring all examples in this paper, implements the quantitative semantics of [7]. Breach can thus measured in Tempo return a satis- Note Rest Value faction signal equal to pitch! (x) ✓ such that the signal’s sign beats for minute bpm = and magnitude 4 sec determine Boolean satisfaction and robustness, resp. Robustness is propagated to STL formulas using the min-max equivalent of the Boolean 40-60temporal 60 bpm means that: 2 sec Largo and bpm operators as defined in (2). In Fig. 3, we show the result Adagio 66-76 bpm of applying the pitch Moderato 108-120 bpm function to the detection of an F. sec 1 Allegro 120-168 bpm Presto 168-200 bpm 0.5 sec
  25. 25. Single note detection %&%%-" !" %&%%-+" B8" !#" %&%%-" %&%%," $" %&%%,+" %&%%+" %&%%," %&%%*" %&%%++" !" %&%%)" %&%%+" %&%%(" %&%%*+" %&%%" =8" %&%%*" $" !" !#" %" %&" %&(" %&)" %&*" %&+" %&," %&-" %&." %&/" :A3&" ))%"" )*%"" )+%"" ),%"" ?@" )-%""" :1;<=>" 38"%" 01234567!8"9"24" 01234567!#8"9"24" 01234567$8"9"24" %" %&%+" %&" %&+" %&(" %&(+" %&)" %&)+" %&*" %&*+" :A3&"
  26. 26. Diatonic Scale with rest 675">?@" !" #" $" %" &" )*+," " (" 123420" 5" 5" -)./0" 5" 6" 7" 8" 9" :" ;" +*<=" 6" 123420" 5" 6" -)./0" 5" 5" 6" 7" 8" 9" :" ;" +*<=" 6" 123420" 5" 6" -)./0" 5" 5" 6" 7" 8" 9" :" ;" +*<=" 6" 123420" 5" 6" -)./0" 5" 5" 6" 7" 8" 9" :" ;" +*<=" 6" 123420" 5" 6" -)./0" 5" 5" 6" 7" 8" 9" :" ;" +*<="
  27. 27. authors, was indeed a Blues melody. To do this, we built aon the fact that standard blues is characterized by a 12-bar Recognizing a piece of bluesr being basically four beats). In the key of E, it is as follows: E E | B A E E. Note that a bar in E does not mean that weof E notes in a row. There can be di↵erent notes, but the overall d like a melody in the key of E (music is a mix of strict and very 1 that is likely to both fascinate and exasperate the most rigorous 0.5 ists). If we assume though that in0 a bar of E there should be ate played, and similarly for A and B, it is easy to write a formula ⌃0.5anslates the above structure. For ⌃1 example, if b is the duration of 0 5 10 15 20 25 30 ⌃3 x 10^⌃[b,2b] µE ^ ⌃[2b,3b] µE ^ ⌃[3b,4b] µE4^⌃[4b,5b] µA ^ ⌃[5b,6b] µA ^ ⌃[6b,7b] µE ^ ⌃[7b,8b] µE 2 by one of the authors, was indeed a Blues melody. To do this, we built a^⌃[8b,9b] µB ^ ⌃[9b,10b] µAfact [10b,11b] µE ^ ⌃[11b,12b] µE ) formula based on the ^ ⌃ that standard blues is characterized by a 12-bar 0 structure (a bar beingbar): Blues pattern (12 basically four beats). In the key of E, it is as follows: E E E E | A A E E | B A E E. Note that a bar in E does not mean that we 0 5 10 15 20 25still, however, too E notes in a row. There canabove blues pattern, the overall play four beats of strict in transcribing the be di↵erent notes, but x 10 ⌃3blues should sound like a melody in the key of E (music is a not have bar line that we recorded. Indeed, our melody does mix of strict and very 2 rth1bar (but beats is likely to that sound cool with an E line). subjective rules that bar = 4 has other notes both fascinate and exasperate the most rigorous 1 computer scientists). If we assume though that in a bar of E there should be at fied a one E note played, and similarly for A and whichislooks to write a formula least simpler (but definitely bluesy) formula B, it easy for a 0A that directly translates so-called “turn-around”example, if b is the duration of in bars 5-6, and the the above ⌃1 structure. For (the sequence B11: bar: one Turn around 0 5 10 15 20 25= µE ^ ⌃[5b,6b] µ^⌃[b,2b] µE ^ ⌃B ^ ⌃[b,2b] µA ^ ⌃µE µE ) µE A ^ ⌃[8b,9b] (µ [2b,3b] µE ^ ⌃[3b,4b] [2b,3b] (10)
  28. 28. Future Works•  Test our tool with ECG patterns and circuits signals•  Introduce quantitative operator for robustness•  Add a space operator for detecting frequency- spatio-temporal patterns•  Sensitivity Analysis guided by a logic formula•  Automatic learning of logic formulae for specifying observed anomalous signal patterns (disease)•  Synthesis of online monitors

×