08 probabilistic inference over time

771 views

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
771
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
48
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

08 probabilistic inference over time

  1. 1. Bayesian Networks Unit 8 Probabilistic Inference over Time Wang, Yuan-Kai, 王元凱 ykwang@mails.fju.edu.tw http://www.ykwang.tw Department of Electrical Engineering, Fu Jen Univ. 輔仁大學電機工程系 2006~2011 Reference this document as: Wang, Yuan-Kai, “Probabilistic Inference over Time," Lecture Notes of Wang, Yuan-Kai, Fu Jen University, Taiwan, 2011.Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  2. 2. Bayesian Networks Unit - Probabilistic Inference over Time p. 2 Goal of This Unit • Know the uncertainty concept in temporal models • Learn four inference types in temporal models – Filtering, Prediction, Smoothing, Most Likely Explanation • See some temporal models – HMM, Kalman/Particle filtering – Dynamic Bayesian networks Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  3. 3. Bayesian Networks Unit - Probabilistic Inference over Time p. 3 Related Units • Background – Probabilistic graphical model – Exact inference in BN – Approximate inference in BN • Next units – HMM – Kalman filter – Particle filter – DBN Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  4. 4. Bayesian Networks Unit - Probabilistic Inference over Time p. 4 Self-Study Reference • Chapter 15, Sections 15.1-15.2, Artificial Intelligence - a modern approach, 2nd, by S. Russel & P. Norvig, Prentice Hall, 2003. Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  5. 5. Bayesian Networks Unit - Probabilistic Inference over Time p. 5 Structure of Related Lecture Notes Problem Structure Data Learning PGM B E Representation Learning A Unit 5 : BN Units 16~ : MLE, EM Unit 9 : Hybrid BN J M Units 10~15: Naïve Bayes, MRF, HMM, DBN, Kalman filter P(B) Parameter P(E) Learning P(A|B,E) P(J|A) Query Inference P(M|A) Unit 6: Exact inference Unit 7: Approximate inference Unit 8: Temporal inference Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  6. 6. Bayesian Networks Unit - Probabilistic Inference over Time p. 6 Contents 1. Time and Uncertainty …………………...... 7 2. Inference in Temporal Models ……...……. 46 3. Various Models .…….................................... 90 4. References …………………………………. 96 Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  7. 7. Bayesian Networks Unit - Probabilistic Inference over Time p. 7 1. Time and Uncertainty • What is probabilistic reasoning over time – There are a lot of time-series data • Ex: Stock data, weather data, radar signal, ... – We want to • Predict its next data • Recover correct values of its current data • Recover correct values of its previous data Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  8. 8. Bayesian Networks Unit - Probabilistic Inference over Time p. 8 Example – Stock Data Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  9. 9. Bayesian Networks Unit - Probabilistic Inference over Time p. 9 Example 2 - Visual Tracking • What is visual tracking – Continuously detect objects in video – Time series data • What kind of objects – Face, – Facial features (eye, eyebrow, ...) – Human body – Hand – ... Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  10. 10. Bayesian Networks Unit - Probabilistic Inference over Time p. 10 Why Visual Tracking (1/2) • A simple idea to detect objects in all frames of a video – "Detect object at every frame with the same detection method • Disadvantage – A detection of a frame may be slow – Detections at all frames become very slow • So, if you have a very quick detection method, the simple method is OK? Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  11. 11. Bayesian Networks Unit - Probabilistic Inference over Time p. 11 Why Visual Tracking (2/2) • A better approach to detect objects in all frames of a video – Detect objects at the first frame – Find objects at succeeding frames with a quick method  tracking • Goal of visual tracking – Fast and accurate detection of objects Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  12. 12. Bayesian Networks Unit - Probabilistic Inference over Time p. 12 Front-View Face Tracking Single frame detector Temporal detector Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  13. 13. Bayesian Networks Unit - Probabilistic Inference over Time p. 13 Side-View Face Tracking without temporal continuity without temporal continuity Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  14. 14. Bayesian Networks Unit - Probabilistic Inference over Time p. 14 Two Kinds of Approaches • Neighborhood-based – Search the neighborhood of the objects location in previous frame • Prediction-based – Search the neighborhood of the predicted location in current frame Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  15. 15. Bayesian Networks Unit - Probabilistic Inference over Time p. 15 Basic Algorithm • Basic idea of both the two approaches 1.Read first frame 2.Detect moving object O Obtain Region of Interest (ROI), usually rectangle or ellipse 3.Read next frame 4.For all possible ROI candidate Oc a)Compare the similarity between O and Oc b)If similarity is high, tracking successfully. Break Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  16. 16. Bayesian Networks Unit - Probabilistic Inference over Time p. 16 Neighborhood-search Tracking • Basic idea 1. Read first frame 2. Detect face O Obtain Region of Interest (ROI), usually rectangle or ellipse 3. Read next frame 4. For all possible ROI candidate Oc a) Compare the similarity between O and Oc b) If similarity is high, break Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  17. 17. Bayesian Networks Unit - Probabilistic Inference over Time p. 17 Basic Ideas Face Detection O First frame Face Tracking O Oc Search Region Next frame Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  18. 18. Bayesian Networks Unit - Probabilistic Inference over Time p. 18 Prediction-based Tracking • Three steps – Predict next position of moving objects with a probabilistic model (parameters) – Detect new position around the predicted position • Prediction error – Update • The correct position • The probabilistic model with the prediction error Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  19. 19. Bayesian Networks Unit - Probabilistic Inference over Time p. 19 Predict Next Position P ( zt | xt ) Current frame Previous frames Real position : xt Predicted position Detected position : zt x-t+1  P ( zt | xt ) P(x t 1 | xt ) Probabilistic P ( x t | x t 1 ) model Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  20. 20. Bayesian Networks Unit - Probabilistic Inference over Time p. 20 Detect New Position by LSE Predicted position Search region SE = 1032, 2560, LSE = 104 1968, 104, 2223, ... Detected position: zt+1 Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  21. 21. Bayesian Networks Unit - Probabilistic Inference over Time p. 21 Update x-t+1 Prediction Error zt+1 x-t+1-zt+1 Corrected position xt+1 Corrected P ( z t | x t ) Probabilistic model P ( x t | x t 1 ) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  22. 22. Bayesian Networks Unit - Probabilistic Inference over Time p. 22 Accurate Tracking = Smoothing Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  23. 23. Bayesian Networks Unit - Probabilistic Inference over Time p. 23 Example 3 - Robot Localization • Localization of AIBO robot in RoboCup • The robot has to – See landmark • Object detection & object recognition – Analyze the landmark • Calculate distance & angle between the robot and the landmark – Estimate its location Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  24. 24. Bayesian Networks Unit - Probabilistic Inference over Time p. 24 RoboCup Field (r , ) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  25. 25. Bayesian Networks Unit - Probabilistic Inference over Time p. 25 Tracking of Robot Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  26. 26. Bayesian Networks Unit - Probabilistic Inference over Time p. 26 Temporal Patterns • Deterministic patterns : – Traffic light – FSM (Finite State Machine) –… • Non-Deterministic patterns : – Weather – Speech – Tracking – … Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  27. 27. Bayesian Networks Unit - Probabilistic Inference over Time p. 27 How to Do It? • What we want? – Prediction: Predict its next data – Filtering: Recover correct values of its current data – Smoothing: Recover correct values of its previous data • How to achieve it? – Statistically model the data Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  28. 28. Bayesian Networks Unit - Probabilistic Inference over Time p. 28 Statistically Modeling y x Predict A set of time- y = 1.3x + 96 : Model Filter related data Smooth Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  29. 29. Bayesian Networks Unit - Probabilistic Inference over Time p. 29 State • There is a set of time-related data Time t = 0 1 2 3 ... ︵ ︵ ︵ ︵ 50, 100 49, 98 50, 96 48, 94 State s = 50, 180 50, 178 50, 176 47, 173 50, 160 49, 158 50, 156 48, 154 ︶ ︶ ︶ ︶ • We call each data – A state of the system, or – A state of the object Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  30. 30. Bayesian Networks Unit - Probabilistic Inference over Time p. 30 Observable v.s. Unobservable States • Observable state – Measurable values • Sensor values, feature values – Ex : Localization/Visual Tracking • Measured position, Measured speed – Ex : Facial Expression Recognition • Eyebrow up, eyebrow down, ... • Unobservable state – Real state of the system/object – Ex : Localization/Visual Tracking • Real position, real speed – Ex : Facial Expression Recognition • Smile, Cry, Anger, ... Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  31. 31. Bayesian Networks Unit - Probabilistic Inference over Time p. 31 Observable v.s. Unobservable States (Math) • Let – Xt = set of unobservable state variables at time t – Et = set of observable state variables at time t • Usually we observe – E0, E1, ...., Et : time-related data • But we want to derive – X0, X1, ..., Xt • Notation: Xa:b = Xa, Xa+1, ..., Xb Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  32. 32. Bayesian Networks Unit - Probabilistic Inference over Time p. 32 Markov Chain • Markov chain is an assumption – A state is dependent on previous state – Xt depends on X0:t-1 – Xt+1 will not influence Xt • Markov process – If we assume that a set of data obeys Markov assumption, – We say the data perform Markov process Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  33. 33. Bayesian Networks Unit - Probabilistic Inference over Time p. 33 Markov Process • First-order Markov process – P(Xt |X0:t-1)=P(Xt | Xt-1 ) • Second-order Markov process – P(Xt |X0:t-1)=P(Xt | Xt-2 , Xt-1 ) • Higher order Markov process ... – Complicate, seldom used Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  34. 34. Bayesian Networks Unit - Probabilistic Inference over Time p. 34 Transition Model & Sensor Model • Transition model – P(Xt | Xt-1 ) – P(Xt | Xt-2 , Xt-1 ) • Sensor model – We usually assume the evidence variables (sensor values) at time t, Et, depend only on the current state Xt – P(Et|X0:t, E0:t-1) = P(Et|Xt) – It is also called observation model Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  35. 35. Bayesian Networks Unit - Probabilistic Inference over Time p. 35 Diagram of Transition & Sensor Models for 1st Order Markov • P(Xt | Xt-1 ) Transition of Xt-1 Xt unobservable states Causal relationship • P(Et|Xt) Xt Et between observable & unobservable states Xt-1 Xt Xt+1 Xt+2 Et Et+1 Et+2 A special Bayesian network Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  36. 36. Bayesian Networks Unit - Probabilistic Inference over Time p. 36 An "Umbrella World" Example (1/2) • A security guard is always at a secret underground room, without going out • He wants to know if it is raining today • But he can not observe the outside world • He can only see each morning the director coming in with, or without, an umbrella • Rain is the unobservable state • Umbrella is the observable state (sensors values) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  37. 37. Bayesian Networks Unit - Probabilistic Inference over Time p. 37 An "Umbrella World" Example (2/2) • For each day t, the set Et contains a single evidence Ut (whether the umbrella appears) • The set Xt contains a single state variable Rt (whether it is raining) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  38. 38. Bayesian Networks Unit - Probabilistic Inference over Time p. 38 Stationary Process • The transition model P(Xt | Xt-1) and the sensor model P(Et | Xt) are all fixed for all time t • Stationary process assumption – Can reduce the complexity of the algorithm for inference Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  39. 39. Bayesian Networks Unit - Probabilistic Inference over Time p. 39 Inference for the Markov Process (1/2) X0 X1 X2 Xt E1 E2 Et • A Bayesian net with 2 random variables – X: X0, X1, ..., Xt – E: E1, ..., Et • We know that P(X0, X1, ..., Xt, E1, ..., Et), the FJD, can answer any query – But it can be reduced t P( X 0 , X 1 , X t , E1 , , Et )  P( X 0 ) P( X i | X i 1 ) P( Ei | X i ) i 1 Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  40. 40. Bayesian Networks Unit - Probabilistic Inference over Time p. 40 Inference for the Markov Process (2/2) • We need three PDFs – P(X0), P(Xi|Xi-1), P(Ei|Xi) • For discrete R.V., we need – 1 prior probability table P(X0) – 2 CPTs • CPT for transition model: P(Xi|Xi-1) • CPT for sensor model: P(Ei|Xi) • For continuous R.V., we need – Gaussian pdf, Gaussian Mixture, ... • Here we consider only discrete R.V. Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  41. 41. Bayesian Networks Unit - Probabilistic Inference over Time p. 41 Sequence Diagram (1/4) X P(X0) S0 0.2 X0 X1 X2 Xt S1 0.1 ... ... E1 E2 Et Si 0.3 P(X0) probability table ? • Suppose the unobservable variable X is a discrete R.V. •X = S1, S2, S3, ..., Si, ... • P(X0) is the probability of X=Si at t=0 Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  42. 42. Bayesian Networks Unit - Probabilistic Inference over Time p. 42 Sequence Diagram (2/4) X0 X1 X2 Xt E1 E2 Et P(Xt|Xt-1) conditional probability table ? Xt+1 S1 S2 ... Si Xt Transition S1 0.1 0.2 ... 0.05 probability S2 0.2 0.15 ... 0.18 ... ... ... ... ... Si 0.31 0.03 ... 0.22 Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  43. 43. Bayesian Networks Unit - Probabilistic Inference over Time p. 43 Sequence Diagram (3/4) X0 X1 X2 Xt E1 E2 Et P(Et|Xt) conditional probability table ? Et v1 v2 ... vj Xt Observation S1 0.1 0.2 ... 0.05 probability S2 0.2 0.15 ... 0.18 ... ... ... ... ... Si 0.31 0.03 ... 0.22 Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  44. 44. Bayesian Networks Unit - Probabilistic Inference over Time p. 44 Sequence Diagram (4/4) X1 X2 X3 X4 X5 X6 X7 S3 S3 S1 S1 S3 S2 S3 v2 v4 v1 v1 v2 v3 v4 E1 E2 E3 E4 E5 E6 E7 t =1 2 3 4 5 6 7 S1 S1 S1 S1 S1 S1 S1 S2 S2 S2 S2 S2 S2 S2 S3 S3 S3 S3 S3 S3 S3 Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  45. 45. Bayesian Networks Unit - Probabilistic Inference over Time p. 45 Short Summary X0 X1 X2 Xt E1 E2 Et • If we have the three PDFs/Tables – P(X0), P(Xi|Xi-1), P(Ei|Xi) • We can answer any query – P(X1, X3 | E2, E4), P(X1, E5 | X2, X4), ... • Do we need to ask many kinds of query? • Or we have some frequently asked queries? Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  46. 46. Bayesian Networks Unit - Probabilistic Inference over Time p. 46 2. Inference in Temporal Models • Four common query tasks in temporal inference/reasoning – Filtering: P(Xt | e1:t)= P(Xt | E1:t=e1:t) • Estimate correct current states – Prediction: P(Xt+k | e1:t) for k > 0 • Predict possible next states – Smoothing: P(Xk | e1:t) for 1  k < t • Better estimate of past states – Most likely explanation: arg maxX1:t P(X1:t | e1:t) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  47. 47. Bayesian Networks Unit - Probabilistic Inference over Time p. 47 Subsections • 2.1 Graphical models of the 4 inferences • 2.2 Mathematical formula of the 4 inferences Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  48. 48. Bayesian Networks Unit - Probabilistic Inference over Time p. 48 2.1 Graphical Models of the 4 Inferences • Use sequence diagram to illustrate what are – Filtering – Prediction – Smoothing – Most likely explanation Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  49. 49. Bayesian Networks Unit - Probabilistic Inference over Time p. 49 Graphical Models - Filtering • P(Xt | e1:t) X0 X1 X2 Xt E1 E2 Et A filtering example for 2D position of robot/WLAN card Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  50. 50. Bayesian Networks Unit - Probabilistic Inference over Time p. 50 Graphical Models - Prediction • P(Xt+k | e1:t) for k > 0 For k=1 X0 X1 X2 Xt Xt+1 E1 E2 Et Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  51. 51. Bayesian Networks Unit - Probabilistic Inference over Time p. 51 Graphical Models – Smoothing (1/3) • P(Xk | e1:t) for 1  k < t X0 X1 X2 Xk Xt E1 E2 Ek Et Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  52. 52. Bayesian Networks Unit - Probabilistic Inference over Time p. 52 Graphical Models – Smoothing (2/3) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  53. 53. Bayesian Networks Unit - Probabilistic Inference over Time p. 53 Graphical Models – Smoothing (3/3) Smoothing v.s. Filtering Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  54. 54. Bayesian Networks Unit - Probabilistic Inference over Time p. 54 Graphical Models - Most Likely Explanation (1/2) • arg maxX1:t P(X1:t | e1:t) X0 X1 X2 Xt E1 E2 Et Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  55. 55. Bayesian Networks Unit - Probabilistic Inference over Time p. 55 Graphical Models - Most Likely Explanation (2/2) t =1 2 3 4 5 6 7 E1=v2 E2=v4 E3=v1 E4=v1 E5=v2 E6=v3 E7=v4 S1 S1 S1 S1 S1 S1 S1 S2 S2 S2 S2 S2 S2 S2 S3 S3 S3 S3 S3 S3 S3 Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  56. 56. Bayesian Networks Unit - Probabilistic Inference over Time p. 56 2.2 Mathematical Formula of the 4 Inferences • Derive mathematical formula of – Prediction – Filtering – Smoothing – Most likely explanation Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  57. 57. Bayesian Networks Unit - Probabilistic Inference over Time p. 57 Prediction (1/3) • P(Xt+1 | e1:t ): one-step prediction as example X0 X1 X2 Xt Xt+1 E1 E2 Et P( X t 1 | e1:t )   X 0 , X t P( X 0 ) P ( X t 1 | X t )  P( X i | X i 1 ) P(ei | X i ) i 1t But more efficient formula can be derived Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  58. 58. Bayesian Networks Unit - Probabilistic Inference over Time p. 58 Prediction (2/3) • New formula for P(Xt+1 | e1:t ) – Xt+1 has no relationship to e1, e2, ..., et – But they both have relationship to xt – If X is a Boolean R.V., P(Xt+1|e1:t) =<P(xt+1=true|e1:t), P(xt+1=false|e1:t)> •  P(Xt+1 | e1:t ) – = xt P(Xt+1 | xt , e1:t )P(xt | e1:t ) transition model by – = xt P(Xt+1 | xt )P(xt | e1:t ) CPT of transition model Filtering Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  59. 59. Bayesian Networks Unit - Probabilistic Inference over Time p. 59 Prediction (3/3) P(Xt+1 | e1:t) = xt P(Xt+1| xt )P(xt | e1:t ) t=1 2 3 ... t t+1 Et=v2 v4 v1 ... v3  e1:t P(Xt+1=S2 | e1:t) S1 S1 S1 S1 P(S1|S2) ) S1 = xt P(Xt+1=S2| xt=Si) … P(S1|e1:t P(xt=Si | e1:t ) S2 P(S2|S2) S2 S2 S2 S2 = xt P(S2|Si)P(Si|e1:t) … P(S2|e1:t) = P(S2|S1)P(S1|e1:t) S3 S3 S3 S3 P(S3|S2) S3 + P(S2|S2)P(S2|e1:t) … P(S3|e1:t) … … … … … + P(S2|S3)P(S3|e1:t) P(SN|S2) + ... SN SN SN … SN P(SN|e1:t) SN Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  60. 60. Bayesian Networks Unit - Probabilistic Inference over Time p. 60 Filtering (1/3) • P(Xt+1 | e1:t+1) X X1 X2 Xt+1 0 ( or P(Xt | e1:t) ) E1 E2 Et+1 P( X t 1 | e1:t 1 )   X 0 , X t P( X 0 )  P ( X i | X i 1 ) P (ei | X i ) i 1t But more efficient formula can be derived Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  61. 61. Bayesian Networks Unit - Probabilistic Inference over Time p. 61 Filtering (2/3) • P(Xt+1 | e1:t+1) • P(Xt+1 | e1:t+1)= P(Xt+1 | e1:t, et+1) – = P(et+1 | Xt+1 , e1:t) P(Xt+1 | e1:t) by Bayes rule – = P(et+1 | Xt+1 ) P(Xt+1 | e1:t ) by sensor model CPT of sensor model Prediction by transition model – = P(et+1 | Xt+1 ) xt P(Xt+1 | xt , e1:t )P(xt | e1:t ) – = P(et+1 | Xt+1 ) xt P(Xt+1 | xt )P(xt | e1:t ) • We derive a recursive algorithm – P(Xt+1 | e1:t+1) can be calculated by P(Xt | e1:t) – There is a function f that P(Xt+1 | e1:t+1) = f(et+1, P(Xt | e1:t)) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  62. 62. Bayesian Networks Unit - Probabilistic Inference over Time p. 62 Filtering (3/3) P(Xt+1 | e1:t+1)= P(et+1 | Xt+1)P(Xt+1 | e1:t ) t=1 2 ... t t+1 Et=v2 v4 ... v3 v4 P(Xt+1=S2 | e1:t+1) P(S1|S2) S et+1 S1 S1 … S1 P(S |e ) 1 = P(et+1=v4|Xt+1=S2) 1 1:t P(et+1|Xt+1) P(Xt+1=S2 | e1:t ) S2 S2 … S2 P(S2|S2) S2 =P(v4|S2) P(S2|e1:t) P(Xt+1|e1:t) = P(v4 | S2) P(S3|S2) S =P(S2|e1:t) P(S2 | e1:t ) S3 S3 … S 3 P(S 3|e1:t) 3 … … … … P(SN|S2) SN SN … SN P(SN|e1:t) SN Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  63. 63. Bayesian Networks Unit - Probabilistic Inference over Time p. 63 Forward Variable • P(Xt+1 | e1:t ) = xt P(Xt+1 | xt )P(xt | e1:t ) • P(Xt+1 | e1:t+1) = P(et+1 | Xt+1) P(Xt+1 | e1:t ) • They are a kind of recursive function • Interesting points – Both the prediction & filtering of Xt+1 need P(Xt|e1:t) – We define the P(Xt|e1:t) as a forward variable f1:t – i.e., f1:t = P(Xt | e1:t ), f1:t(Si) = P(Xt=Si | e1:t ) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  64. 64. Bayesian Networks Unit - Probabilistic Inference over Time p. 64 Forward Procedure • P(Xt+1 | e1:t+1) = P(et+1 | Xt+1) P(Xt+1 | e1:t ) = P(et+1 | Xt+1) xt P(Xt+1 | xt )P(xt | e1:t ) • The filtering process is rewritten as f1:t+1 = Forward(f1:t , et+1) – A forward procedure (algorithm) : Forward(f1:t , et+1) = P(et+1 | Xt+1)xt P(Xt+1 | xt )P(xt | e1:t ) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  65. 65. Bayesian Networks Unit - Probabilistic Inference over Time p. 65 Filtering Example (1/4) • For the umbrella example P(Rt|Rt-1) P(Ut|Rt) P(ut|rt)=0.9 P(ut|rt)=0.1 P(ut|rt)=0.2 P(ut|rt)=0.8 P(Ut|rt)=<P(ut|rt),P(ut|rt)> = <0.9,0.1> P(Ut|rt)=<P(ut|rt),P(ut|rt)> = <0.2,0.8> P(ut|Rt)? Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  66. 66. Bayesian Networks Unit - Probabilistic Inference over Time p. 66 Filtering Example (2/4) • Assume the man believes that P(R0) = <0.5,0.5> = < P(r0), P(r0) > – The rain probability before the observation sequence begins • Now we has the observation sequence: umbrella1=true, umbrella2=true • We will use the P(R |U ) P(R2|U1,U2) 1 1 filtering process Rain1 Rain2 to find rain probability Umbrella1 Umbrella2 =true =true Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  67. 67. Bayesian Networks Unit - Probabilistic Inference over Time p. 67 Filtering Example (3/4) Rain0 Rt-1 P(Rt) Rain1 Rain2 Rt P(Ut) P(R1|u1) t 0.7 t 0.9 f 0.3= P(u1|R1)P(R1) f 0.2 Umbrella1 Umbrella2 P(R1) =true =true = r0P(R1|r0)P(r0) = <0.7,0.3>0.5 + <0.3,0.7>0.5 = <0.5,0.5> P(R1|u1) = P(u1|R1)P(R1) = <0.9,0.2><0.5,0,5> = <0.45,0.1>  <0.818, 0.182> Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  68. 68. Bayesian Networks Unit - Probabilistic Inference over Time p. 68 Filtering Example (4/4) Rain0 Rain1 Rain2 Rt-1 P(Rt) Rt P(Ut) t 0.7 t 0.9 f 0.3 f 0.2 P(R2|u1,u2) = P(u2|R2)P(R2|u1) Umbrella1 Umbrella2 =true =true P(R2|u1) = r1P(R2|r1)P(r1|u1) = <0.7,0.3>0.818 + <0.3,0.7>0.182 = <0.627,0.373> P(R2|u1,u2) = P(u2|R2)P(R2|u1) = <0.9,0.2><0.627,0,373> = <0.565,0.075>  <0.883, 0.117> Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  69. 69. Bayesian Networks Unit - Probabilistic Inference over Time p. 69 Smoothing (1/2) • P(Xk | e1:t) for 1  k < t – Divide e1:t into e1:k and ek+1:t – P(Xk | e1:t) = P(Xk | e1:k , ek+1:t) – =  P(Xk | e1:k)P(ek+1:t | Xk , e1:k) – =  P(Xk | e1:k)P(ek+1:t | Xk ) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  70. 70. Bayesian Networks Unit - Probabilistic Inference over Time p. 70 Smoothing (2/2) P(Xk=S2 | e1:t) t=1 k-1 k k+1 t Et= v2 ... v4 v3 v1 ... v4 S1 … S1 P(x |x ) S1 S1 … S1 1 2 S2 … S2 P(x2|x2) S2 S2 … S2 P(x3|x2) S3 … S3 S3 S3 … S3 … … … … … P(xN|x2) SN … SN SN SN … SN Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  71. 71. Bayesian Networks Unit - Probabilistic Inference over Time p. 71 Backward Variable • P(ek+1:t | Xk) – = xk+1P(ek+1:t | Xk, xk+1)P(xk+1 | Xk) – = xk+1P(ek+1:t | xk+1)P(xk+1 | Xk) – = xk+1P(ek+1 , ek+2:t | xk+1)P(xk+1 | Xk) – = xk+1P(ek+1 | xk+1)P(ek+2:t | xk+1)P(xk+1 | Xk) • This is also a recursive formula • We define a backward variable bk+1:t – bk+1:t = P(ek+1:t | Xk) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  72. 72. Bayesian Networks Unit - Probabilistic Inference over Time p. 72 Backward Procedure (1/2) • P(ek+1:t | Xk) = xk+1P(ek+1|xk+1)P(ek+2:t|xk+1)P(xk+1|Xk) • The formula is rewritten as bk+1:t = Backward(bk+2:t , ek+1) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  73. 73. Bayesian Networks Unit - Probabilistic Inference over Time p. 73 Backward Procedure (2/2) P(ek+1:t |xk) = xk+1P(ek+1|xk+1)P(ek+2:t|xk+1)P(xk+1|Xk) t=1 k k+1 t P(ek+1:t |xk=S2) Et= v2 ... v4 v1 ... v4 = xk+1P(v1|xk+1) S1 … S1 S1 … S1 P(ek+2:t|xk+1)P(xk+1|S2) = P(v |S )P(e |S )P(S |S ) S2 … S2 S2 … S2 1 1 k+2:t 1 1 2 + P(v1|S2)P(ek+2:t|S2)P(S2|S2) + ... S3 … S3 S3 … S3 … … … … + P(v1|SN)P(ek+2:t|SN)P(SN|S2) SN … SN SN … SN Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  74. 74. Bayesian Networks Unit - Probabilistic Inference over Time p. 74 The Smoothing Formula • P(Xk | e1:t) = P(Xk | e1:k , ek+1:t) – =  P(Xk | e1:k)P(ek+1:t | Xk , e1:k) – =  P(Xk | e1:k)P(ek+1:t | Xk ) – =  f1:kbk+1:t • Time complexity – Both the forward and backward recursions take a constant time per step – Complexity of smoothing P(Xk | e1:t) with e1:t is O(t) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  75. 75. Bayesian Networks Unit - Probabilistic Inference over Time p. 75 Smoothing Example (1/3) • For the umbrella example • P(R1 | u1, u2) – Computing the smoothed estimate for the probability of rain at t=1, – Given the umbrella observations on days 1 & 2 Rain0 Rain1 Rain2 Rt-1 P(Rt) Rt P(Ut) t 0.7 t 0.9 f 0.3 f 0.2 Umbrella1 Umbrella2 =true =true Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  76. 76. Bayesian Networks Unit - Probabilistic Inference over Time p. 76 Smoothing Example (2/3) • P(R1 | u1, u2) = P(R1|u1)P(u2|R1) – P(R1|u1) = <0.818, 0.182> – P(u2|R1) = r2P(u2|r2)P(|r2)P(r2|R1) = (0.91<0.7,0.3>) + (0.21<0.3,0.7>) = <0.69, 0.41> • P(R1 | u1, u2) = <0.818,0.182><0.69,0.41>  <0.883, 0.117> • Note: P(R1|u1) = <0.818, 0.182> • With more one observation u2, the probability of r1 increases  smoothing Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  77. 77. Bayesian Networks Unit - Probabilistic Inference over Time p. 77 Smoothing Example (3/3) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  78. 78. Bayesian Networks Unit - Probabilistic Inference over Time p. 78 Most Likely Explanation (1/2) • Smoothing P(Xk | e1:t) considers only one past state at time step k • Most likely explanation, arg maxX1:t P(X1:t | e1:t) – Considers all past states, and – Choose the best state sequence X0 X1 X2 Xt E1 E2 Et Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  79. 79. Bayesian Networks Unit - Probabilistic Inference over Time p. 79 Most Likely Explanation (2/2) • We will discuss 3 algorithms – Algorithm 1: • Very simple, directly using smoothing • Time complexity O(t2) – Algorithm 2(forward-backward algo.): • Improved usage of smoothing • Time complexity O(t) • But the result may not be the best state sequence – Algorithm 3(Viterbi algorithm): • Time complexity O(t) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  80. 80. Bayesian Networks Unit - Probabilistic Inference over Time p. 80 Algorithm 1 • The most simple idea for this problem – Call smoothing t times, smoothing one state each time – For (i=0; i<t; i++) P(Xi | e1:t) • Drawback – Time complexity of O(t2) : too slow • Improvement – Apply dynamic programming to reduce the complexity to O(t) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  81. 81. Bayesian Networks Unit - Probabilistic Inference over Time p. 81 Algorithm 2 (1/2) • Forward-backward algorithm – First, record the results of the forward filtering over the whole sequence from 1 to t – Then, run the backward recursion from t down to 1, and • Compute the smoothed estimate at each time step k, from the bk+1:t and the stored f1:k Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  82. 82. Bayesian Networks Unit - Probabilistic Inference over Time p. 82 Algorithm 2 (2/2) fv[i]= f1:t = P(Xt | e1:t ) forward procedure: f1:t+1 = Forward(f1:t , et+1) Smoothing: P(Xk | e1:t) =  f1:kbk+1:t backward procedure: bk+1:t = Backward(bk+2:t , ek+1) Fu Jen University in previous slidesElectrical Engineering Department of Wang, Yuan-Kai Copyright
  83. 83. Bayesian Networks Unit - Probabilistic Inference over Time p. 83 However (1/2) • For the umbrella example, suppose there is an observation sequence e1:t=[true, true, false, true, true] for umbrellas appearance • What is the weather sequence most likely to explain this? – Does the absence of the umbrella on day 3 mean that • Day 3 wasnt raining, or • The director forget to bring it? • If day 3 wasnt raining, day 4 may not be raining either, but the director brought the umbrella just in case Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  84. 84. Bayesian Networks Unit - Probabilistic Inference over Time p. 84 However (2/2) • The forward-backward algorithm uses smoothing for each single time step • But to find the most likely sequence, we must consider joint probabilities over all time steps • To consider joint probabilities of a sequence, we need to consider path Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  85. 85. Bayesian Networks Unit - Probabilistic Inference over Time p. 85 Path • A path is a possible sequence – There are 25 paths – Each path (sequence) has a probability – Only one path has the maximum probability Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  86. 86. Bayesian Networks Unit - Probabilistic Inference over Time p. 86 Probability of Path t P( X 1:t | e1:t )   P( X i 1 | X i ) P(ei | X i ) i 1 • arg maxX1:t P(X1:t | e1:t) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  87. 87. Bayesian Networks Unit - Probabilistic Inference over Time p. 87 Recursive View • An important idea for finding arg maxX1:t P(X1:t | e1:t) – A path in maxX1:t-1 P(X1:t-1 | e1:t-1) must be the path in maxX1:t P(X1:t | e1:t) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  88. 88. Bayesian Networks Unit - Probabilistic Inference over Time p. 88 The Viterbi Example Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  89. 89. Bayesian Networks Unit - Probabilistic Inference over Time p. 89 Algorithm 3 • Viterbi algorithm max P( x1 , , xt , X t 1 | e1:t 1 ) x1 xt  P (et 1 | X t 1 ) max P( X t 1 | xt ) max P ( x1 , , xt 1 , xt | e1:t )    xt  x1 xt 1  • It is similar to the filtering algorithm P(Xt+1 | e1:t+1) = P(et+1 | Xt+1) xt P(Xt+1 | xt )P(xt | e1:t ) Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  90. 90. Bayesian Networks Unit - Probabilistic Inference over Time p. 90 3. Various Models • Hidden Markov Models • Kalman Filter • Particle Filter • Dynamic Bayesian Networks Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  91. 91. Bayesian Networks Unit - Probabilistic Inference over Time p. 91 Hidden Markov Model (1/2) Hidden states eg. Real location X1 X2 X3 Observations eg. Detected Y1 Y2 Y3 location n P( x1 , x2 ,..., xn )   P( xi | pa( xi )) i 1 Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  92. 92. Bayesian Networks Unit - Probabilistic Inference over Time p. 92 Hidden Markov Model (2/2)  X1 A X2 X3 Parameter tyeing B Y1 Y3 Y2 Transition matrix Observation matrix Initial state distribution Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  93. 93. Bayesian Networks Unit - Probabilistic Inference over Time p. 93 Kalman Filtering X1 X2 X3 Y1 Y2 Y3 • The same graphical structure with HMM • But •In HMM, Xi and Yi are discrete (CPT) •In Kalman filter, Xi and Yi are continuous Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  94. 94. Bayesian Networks Unit - Probabilistic Inference over Time p. 94 Particle Filtering • TBU Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  95. 95. Bayesian Networks Unit - Probabilistic Inference over Time p. 95 Dynamic Bayesian Network (DBN) • TBU Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright
  96. 96. Bayesian Networks Unit - Probabilistic Inference over Time p. 96 4. References • Chapter 15, Sections 15.1-15.2, Artificial Intelligence - a modern approach, 2nd, by S. Russel & P. Norvig, Prentice Hall, 2003. Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai Copyright

×