This document provides an overview of Bayesian networks through a 3-day tutorial. Day 1 introduces Bayesian networks and provides a medical diagnosis example. It defines key concepts like Bayes' theorem and influence diagrams. Day 2 covers propagation algorithms, demonstrating how evidence is propagated through a sample chain network. Day 3 will cover learning from data and using continuous variables and software. The overview outlines propagation algorithms for singly and multiply connected graphs.

1. Introduction to Bayesian Networks A Tutorial for the 66th MORS Symposium 23 - 25 June 1998 Naval Postgraduate School Monterey, California Dennis M. Buede Joseph A. Tatman Terry A. Bresnick

14. Sample Factored Joint Distribution X 1 X 3 X 2 X 5 X 4 X 6 p(x 1 , x 2 , x 3 , x 4 , x 5 , x 6 ) = p(x 6 | x 5 ) p(x 5 | x 3 , x 2 ) p(x 4 | x 2 , x 1 ) p(x 3 | x 1 ) p(x 2 | x 1 ) p(x 1 )

16. Arc Reversal - Bayes Rule p(x 1 , x 2 , x 3 ) = p(x 3 | x 1 ) p(x 2 | x 1 ) p(x 1 ) p(x 1 , x 2 , x 3 ) = p(x 3 | x 2 , x 1 ) p(x 2 ) p( x 1 ) p(x 1 , x 2 , x 3 ) = p(x 3 | x 1 ) p(x 2 , x 1 ) = p(x 3 | x 1 ) p(x 1 | x 2 ) p( x 2 ) p(x 1 , x 2 , x 3 ) = p(x 3 , x 2 | x 1 ) p( x 1 ) = p(x 2 | x 3 , x 1 ) p(x 3 | x 1 ) p( x 1 ) is equivalent to is equivalent to X 1 X 3 X 2 X 1 X 3 X 2 X 1 X 3 X 2 X 1 X 3 X 2

17. Inference Using Bayes Theorem Tuber- culosis Lung Cancer Tuberculosis or Cancer Dyspnea Bronchitis Lung Cancer Tuberculosis or Cancer Dyspnea Bronchitis Lung Cancer Tuberculosis or Cancer Dyspnea Lung Cancer Dyspnea Lung Cancer Dyspnea The general probabilistic inference problem is to find the probability of an event given a set of evidence This can be done in Bayesian nets with sequential applications of Bayes Theorem

20. Introduction to Bayesian Networks A Tutorial for the 66th MORS Symposium 23 - 25 June 1998 Naval Postgraduate School Monterey, California Dennis M. Buede Joseph A. Tatman Terry A. Bresnick

24. Singly Connected Networks (or Polytrees) Definition : A directed acyclic graph (DAG) in which only one semipath (sequence of connected nodes ignoring direction of the arcs) exists between any two nodes. Do not satisfy definition Polytree structure satisfies definition Multiple parents and/or multiple children

27. An Example: Simple Chain p(Paris) = 0.9 p(Med.) = 0.1 M TO|SM = M AA |TO = Ch Di Pa Me No Ce So Ch Di Paris Med. Chalons Dijon North Central South Strategic Mission Tactical Objective Avenue of Approach .8 .2 .1 .9 [ ] .5 .4 .1 .1 .3 .6 [ ]

28. Sample Chain - Setup (1) Set all lambdas to be a vector of 1’s; Bel(SM) =   (SM)   (SM)  (SM) Bel(SM)  (SM) Paris 0.9 0.9 1.0 Med. 0.1 0.1 1.0 (2)  (TO) =  (SM) M TO|SM ; Bel(TO) =   (TO)   (TO)  (TO) Bel(TO)  (TO) Chalons 0.73 0.73 1.0 Dijon 0.27 0.27 1.0 (3)  (AA) =  (TO) M AA|TO ; Bel(AA) =   (AA)   (AA)  (AA) Bel(AA)  (AA) North 0.39 0.40 1.0 Central 0.35 0.36 1.0 South 0.24 0.24 1.0 M AA|TO = M TO|SM = Strategic Mission Tactical Objective Avenue of Approach .8 .2 .1 .9 [ ] .5 .4 .1 .1 .3 .6 [ ]

29. Sample Chain - 1st Propagation   t TR T   0 5  ( ) . 1 .6   t = 0 (lR) = .8 .2  t = 1  (SM) =  (IR)  (SM) Bel(SM)  (SM) Paris 0.8 0.8 1.0 Med. 0.2 0.2 1.0  (TO) Bel(TO)  (TO) Chalons 0.73 0.73 1.0 Dijon 0.27 0.27 1.0  (AA) Bel(AA)  (AA) North 0.39 0.3 0.5 Central 0.35 0.5 1.0 South 0.24 0.2 0.6 t = 1  (AA) =  (TR) Intel. Rpt. Troop Rpt. Strategic Mission Tactical Objective Avenue of Approach

30. Sample Chain - 2nd Propagation   t TR T   0 5  ( ) . 1 .6   t = 0 (lR) = .8 .2   (SM) Bel(SM)  (SM) Paris 0.8 0.8 1.0 Med. 0.2 0.2 1.0 t = 2  (TO) =  (SM) M TO|SM  (TO) Bel(TO)  (TO) Chalons 0.66 0.66 0.71 Dijon 0.34 0.34 0.71 t = 2  (TO) = M AA|TO  (SM)  (AA) Bel(AA)  (AA) North 0.39 0.3 0.5 Central 0.35 0.5 1.0 South 0.24 0.2 0.6 Intel. Rpt. Troop Rpt. Strategic Mission Tactical Objective Avenue of Approach

31. Sample Chain - 3rd Propagation  (SM) Bel(SM)  (SM) Paris 0.8 0.8 0.71 Med. 0.2 0.2 0.71 t = 3  (SM) = M TO|SM  (TO)  (TO) Bel(TO)  (TO) Chalons 0.66 0.66 0.71 Dijon 0.34 0.34 0.71 t = 3  (AA) =  (TO) M AA|TO  (AA) Bel(AA)  (AA) North 0.36 0.25 0.5 Central 0.37 0.52 1.0 South 0.27 0.23 0.6 Intel. Rpt. Troop Rpt. Strategic Mission Tactical Objective Avenue of Approach

32. Internal Structure of a Single Node Processor Processor for Node X Message to Parent U Message from Parent U M X|U     M X|U  k  k (X) BEL =     Message to Children of X Message from Children of X  BEL(X)  1 (X)  BEL(X)  N (X) ... ...  (X)  (X)  X (U)  1 (X)  N (X)  X (U)  N (X)  1 (X)

37. Situation Assessment Bayesian Net Initial Conditions Given Emergency

38. Situation Assessment Bayesian Net Steam Line Radiation Alarm Goes High

39. Situation Assessment Bayesian Net Steam Line Radiation Alarm Goes Low

40. Simulation of SGTR Scenario Event Timeline

43. Introduction to Bayesian Networks A Tutorial for the 66th MORS Symposium 23 - 25 June 1998 Naval Postgraduate School Monterey, California Dennis M. Buede, dbuede@gmu.edu Joseph A. Tatman, jatatman@aol.com Terry A. Bresnick, bresnick@ix.netcom.com http://www.gmu.edu - Depts (Info Tech & Eng) - Sys. Eng. - Buede

45. Building BN Structures Bayesian Network Bayesian Network Bayesian Network Problem Domain Problem Domain Problem Domain Expert Knowledge Expert Knowledge Training Data Training Data Probability Elicitor Learning Algorithm Learning Algorithm

47. Beta Distribution 1) n(n m/n) m(1 variance n m mean x) (1 x m) (n (m) (n) n) m, | (x p 1 m n 1 m Beta             

48. Multivariate Dirichlet Distribution 1) m ( m ) m / m (1 m state i the of variance m m state i the of mean ...x x x ) (m )... (m ) (m ) m ( ) m ,..., m , m | (x p N 1 i i N 1 i i N 1 i i i i th N 1 i i i th 1 m 1 - m 1 m N 2 1 N 1 i i N 2 1 Dirichlet N 2 1                     

54. Enemy Intent Trafficability of S. AA Trafficability of C. AA Trafficability of N. AA Troops @ NAI 1 Troops @ NAI 2 AA - Avenue of Approach NAI - Named Area of Interest Intelligence Reports Observations on Troop Movements Weather Forecast & Feasibility Analysis Strategic Mission Weather Tactical Objective Enemy’s Intelligence on Friendly Forces Avenue of Approach Deception Plan

55. Original Network

56. Learned Network with 1000 Cases Missing Arcs: 2 Added Arcs: 0 Arcs Misdirected: 5 Arcs Unspecified: 3 Missing Arcs: 2 Added Arcs: 0 Arcs Misdirected: 5 Arcs Unspecified: 3

57. Learned Network with 10,000 Cases Missing Arcs: 1 Added Arcs: 1 Arcs Misdirected: 4

58. Comparison of Learned Networks with Truth p(AoA) Truth 1 K 10 K Prior .37, .37, .26 .37, .35, .28 .38, .36, .26 “ Clear” .41, .37, .22 .38, .36, .26 .41, .36, .23 “ Rainy” .30, .36, .34 .35, .32, .33 .30, .36, .34 “ NAI 1 True” .15, .13, .71 .17, .12, .71 .16, .12, .71 “ Rain, NAI 1 True” .10, .11, .79 .15, .10, .75 .11, .11, .78 “ Rain, NAI 1 & 2 True” .56, .02, .43 .59, .05, .36 .56, .03, .40

61. Continuous Variables Example Entering values for the three discrete random variables shifts the sensor mean values

68. Backup

73. Tuberculosis XRay Result Tuberculosis or Cancer Lung Cancer Dyspnea Bronchitis

75. Sample Chain - Setup (1) Set all lambdas to be a vector of 1’s; Bel(SM) =   (SM)   (SM)  (SM) Bel(SM)  (SM) Paris 0.9 0.9 1.0 Med. 0.1 0.1 1.0 (2)  (TO) =  (SM) M TO|SM ; Bel(TO) =   (TO)   (TO)  (TO) Bel(TO)  (TO) Chalons 0.73 0.73 1.0 Dijon 0.27 0.27 1.0 (3)  (AA) =  (TO) M AA|TO ; Bel(AA) =   (AA)   (AA)  (AA) Bel(AA)  (AA) North 0.39 0.73 1.0 Central 0.35 0.27 1.0 South 0.24 0.24 1.0 Strategic Mission Tactical Objective Avenue of Approach M AA|TO = M TO|SM =