Bayesian network

1. BAYESIAN
NETWORKS
BY-
Ahmad Ali Al Taweel

2. CONDITIONAL PROBABILITY
Events A and B
• P(A|B)- Probability that event A occurs given that event B has already
occurred.
Example:
There are 2 baskets. B1 has 2 red ball and 5 blue ball. B2 has 4 red ball and 3
blue ball. Find probability of picking a red ball from basket 1?
• The question above wants P(red ball | basket 1).
• The answer intuitively wants the probability of red ball from only the sample
space of basket 1.
• So the answer is 2/7
• The equation to solve it is:
P(A|B) = P(A∩B)/P(B) [Product Rule]
P(A,B) = P(A)*P(B) [ If A and B are independent ]
How do you solve P(basket2 | red ball) ???

3. BAYESIAN THEOREM
• A special case of
Bayesian Theorem:
P(A∩B) = P(B) x P(A|B)
P(B∩A) = P(A) x P(B|A)
Since P(A∩B) = P(B∩A),
P(B) x P(A|B) = P(A) x
P(B|A)
=> P(A|B) = [P(A) x P(B|A)]
/ P(B)
A B
       ABPAPABPAP
ABPAP
BP
ABPAP
BAP
||
)|()(
)(
)|()(
)|(



4. BAYESIAN THEOREM
Solution to P(basket2 | red ball) ?
P(basket 2| red ball) = [P(b2) x P(r | b2)] / P(r)
= (1/2) x (4/7)] / (6/14)
= 0.66

5. • A Bayesian network is a graph-based model for conditional
independence assertions and hence for compact specification
of full joint distributions.
• A Bayesian network B is defined as a pair B = (G, P), where G
= (V (G),A(G)) is an acyclic directed graph with a set of
vertices (or nodes) V (G) = {X1,X2, . . . ,Xn}
• And a set of arcs A(G) ⊆ V (G) × V (G), and where P is a joint
probability distribution defined on the variables
corresponding to the vertices V (G).
Bayesian Network

6. A Bayesian Network consists of [Jensen, 1996]:
A set of variables and a set of direct edges between
variables.
1. Each variables has a finite set of mutually exclusive
states
2. The variable and direct edge form a DAG (directed
acyclic graph)
3. To each variable A with parents B1, B2 ..Bn there is
attached a conditional probability table P(A| B1, B2 ..
Bn)

7. p(A,B,C) = p(C|A,B)p(A)p(B)
A B
C
FORMS OF THE BAYESIAN NETWORKS
A CB Marginal Independence:
p(A,B,C) = p(A) p(B) p(C)
A Directed Acyclic Graph

8. A
CB
Conditionally independent effects:
p(A,B,C) = p(B|A)p(C|A)p(A)
B and C are conditionally independent
Given A
Each node in these graphs is a
random variable
Informally, an arrow from node X to
node Y means X has a direct
influence on Y
A CB
Markov dependence:
p(A,B,C) = p(C|B) p(B|A)p(A)

9. A node X is a parent of another node Y
if there is an arrow from node X to
node Y eg. A is a parent of B & C
P(A,B,C,D,E,F) = P(F|C,D,E)P(A,B,C,D,E)
= P(F|C,D,E)P(C|A,E)P(D|B)P(E|B)P(B,A)
= P(F|C,D,E)P(C|A,E)P(D|B)P(E|B)P(B|A)P(A)

10. EXAMPLE

11.  Learning phase

12.  Testing phase (inference)

14. Weng-Keen Wong, Oregon
State University ©2005
14
Using a Bayesian Network Example
Using the network in the example, suppose you want to
calculate:
P(A = true, B = true, C = true, D = true)
= P(A = true) * P(B = true | A = true) *
P(C = true | B = true) P( D = true | B = true)
= (0.4)*(0.3)*(0.1)*(0.95) A
B
C D

15. Weng-Keen Wong, Oregon
State University ©2005
15
Using a Bayesian Network Example
Using the network in the example, suppose you want to
calculate:
P(A = true, B = true, C = true, D = true)
= P(A = true) * P(B = true | A = true) *
P(C = true | B = true) P( D = true | B = true)
= (0.4)*(0.3)*(0.1)*(0.95) A
B
C D
This is from the
graph structure
These numbers are from the
conditional probability tables

16. Weng-Keen Wong, Oregon
State University ©2005
16
Bayesian Networks
Two important properties:
1. Encodes the conditional independence
relationships between the variables in the graph
structure
2. Is a compact representation of the joint
probability distribution over the variables

17. Why Bayesian Networks?
• — Bayesian Probability represents the degree of
belief in that event while Classical Probability (or
frequents approach) deals with true or physical
probability of an event Bayesian Network
Handling of Incomplete Data Sets Learning about
Causal Networks Facilitating the combination of
domain knowledge and data Efficient and
principled approach for avoiding the over fitting of
data.

18. CONCLUSION
• Bayesian nets are a network-based framework for
representing and analyzing models involving uncertainty
• Used for the cross fertilization of ideas between the artificial
intelligence, decision analysis, and statistic communities
• People are using this nowadays because of the development
of propagation algorithms followed by availability of easy to
use commercial software.
• And growing number of creative applications.