Udemyの「ベイズ推定とグラフィカルモデル：コンピュータビジョン基礎1」（https://www.udemy.com/computervision/）を受講した際に取ったノートです。もとのレクチャースライドはこちら（http://www.computervisionmodels.com/）から入手可能です。

1. Computer vision: models, learning
and inference
Chapter 2
Introduction to probability
Please send errata to s.prince@cs.ucl.ac.uk

2. Random variables
• A random variable x denotes a quantity that is
uncertain
• May be result of experiment (flipping a coin) or a real
world measurements (measuring temperature)
• If observe several instances of x we get different
values
• Some values occur more than others and this
information is captured by a probability distribution
2Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
TAKE '¥±K¥k

3. Discrete Random Variables
3Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
tf.az#AkTak9zTf ER

4. Continuous Random Variable
4Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
ELIE Fifa If Ek

5. Joint Probability
• Consider two random variables x and y
• If we observe multiple paired instances, then some
combinations of outcomes are more likely than others
• This is captured in the joint probability distribution
• Written as Pr(x,y)
• Can read Pr(x,y) as “probability of x and y”
5Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
TH # ¥ BHI 'T

6. Joint Probability
6Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Knight
&Ef¥Me
tapas
Apk
like
teak EL
Tak EE
like The
tggqttpkh.be Heifetz ¥pHz4akKI

7. Marginalization
We can recover probability distribution of any variable in a joint distribution
by integrating (or summing) over the other variables
7Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Eh Ela
:
(
I
I
I
I
I
I
l
a a At EG
'
Fe CERE 9¥88 ) E ti K 's k R
.
!
X E An C ,
L 2
'
EEE Ffa FE it FEE B !
To 3 .
Is a 17k¥ G
'
if 2-
'
it . f a-
IEEE
IKER I
If a za FEE Be c 2 ,
K 2 ME He c E 2 I a Y
I
A
"
ER y
'
7 3 EPzE¥ E TE 2 IT .
!

8. Marginalization
We can recover probability distribution of any variable in a joint distribution
by integrating (or summing) over the other variables
8Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
tfafzFawkes t.BE is FE. & a Z
Ek 2 .
•
- - - - - - - - - - - - -
- -
-
.

9. Marginalization
We can recover probability distribution of any variable in a joint distribution
by integrating (or summing) over the other variables
9Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
M
=be;÷¥÷:÷¥*÷¥:::¥¥÷÷÷÷÷÷÷
. .
I 9
"
HEGE EE ka 2. FERta

10. Marginalization
We can recover probability distribution of any variable in a joint distribution
by integrating (or summing) over the other variables
Works in higher dimensions as well – leaves joint distribution between
whatever variables are left
10Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
77 ok K C 77 EIH ME IT TG E As fZe .
Ke 8 a his a¥ 8 top Z fi H 3 k on ,
R2 I Z Aztec ,
TashI TakaE a w z
-
K E. ER . KLEEATE n Za
'
FER'S -53 .

11. Conditional Probability
• Conditional probability of x given that y=y1 is relative
propensity of variable x to take different outcomes given that y
is fixed to be equal to y1.
• Written as Pr(x|y=y1)
11
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Ff 44 He E EThe tf
y ee f ,
k EHEEL k ZE . 7 I Y I a-
-
F s
2- Tn 32€ a
ok a
KkkEE 8 top
↳
-
L
.
¥KEE E FF ,
2 I I E "
IT 2- it Fife FFS Fp u is
z taken THAT Y G y g q .
.
.
22 e
"
22 5 .
life II I TM a KEKE if .
E 2
*Ep If 9 22 I e
-
-
A 3 a
"
.
= th 2- I H 's 22 u 5 'so
U is .
To h =
s KERA
FE 2 z
EEE's c E ZE k
Ia Etfd
"
I e-
A 3 F 's
a IE Bae
EM
c I 3 .
u F- A
"
,
2
* in e
'
Eai 'm
't
→
I c =
22 3 a
THE
's T2
" o
42 a no -
I
"
Fi
'
a 5 .
EREH 3 I FER To 2¥ By a
"
To 3 ,

12. Conditional Probability
• Conditional probability can be extracted from joint probability
• Extract appropriate slice and normalize
12Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
g
EST THE Fi
'
it E EH Y E
-
-
T e Be t a = k Fi
'
H
.
you
.
y
*
a 2€ a g a EA a 5¥ ↳
¥
Eaglets 3¥ ! f a
- -
ft eh 's
In:&,
'
Ian .
hEs
u EAin .
C
I a BIKE HE et
4¥.
8 to 3 2 I iz it ?
¥ f- sa k EE EE E Etty 4h c E 's FE EH n ME I F The Za
Azul
,
2 ,
EE EE TE h¥ a FER A 6 "
I k A 3 F 7 iz I ER he
t 3 La FE to
"
In 3

13. Conditional Probability
• More usually written in compact form
• Can be re-arranged to give
13Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
€.
:*"¥⇒÷÷:*
.
¥**±⇒=÷¥÷÷÷
--
-
EffectED of M¥715.5

14. Conditional Probability
• This idea can be extended to more than two
variables
14Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
-
Z a I I 55 Fp
-
YEAHHTT 5 R F- FA I 8 Fp
- -
-
K n y a Z I Ff htt ht IT 5 RE EH E 5 Ep

15. Bayes’ Rule
From before:
Combining
:
Re-
arranging:
15Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Eff4 Ft I B Fp a
"
An -
a 32 .
no a a
.
.
*
. .
g a
-
3 .
A
'
h K
'
a Be EBE
THE¥ EEE
If it K2 "
¥44
ft H 2 E L 2
.
EH4 tf it
2 E A E .
t
fi
'
a 5 Eg § a
Fet 3 .

16. Bayes’ Rule Terminology
Posterior – what we
know about y after
seeing x
Prior – what we know
about y before seeing x
Likelihood – propensity for
observing a certain value of
x given a certain value of y
Evidence –a constant to
ensure that the left hand
side is a valid distribution
16Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
fi Fk (
'
¥HffEEeEz¥)
* '
HER '
kit " k¥-7424 .
.
z¥¥, §
THE 'a5kE2E9ka¥H #Tt
( ¥ ¥,
↳q¥ it ← 5Th )
-iFex¥¥iETT{}
= ¥14184
-344784C ¥1467 BEGET
IS # HIER 3-kkn.tk#k2HBeFtK-3kcTiGczOk .
(Ef
4564k¥
)
737×4 KEEN 3913449 Kritz By 2- I 3 .

17. Independence
• If two variables x and y are independent then variable x tells us
nothing about variable y (and vice-versa)
17Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
HEE
← ya
"
x a EEE ¥4 -
a ¥4 94dg z
ELK a
.
IT a 4¥ IKE FEE 34
EINE ,
a e f it f)eh I 2 "
In 3
U '
'
so FEE Ff c
"
.
Y $
"
Y ,
a z E 't
) :*. :*'s.
¥:
" "
⇒ y it iceI'M
'
Fatih Et ¥ EH - .
.

18. Independence
• If two variables x and y are independent then variable x tells us
nothing about variable y (and vice-versa)
18Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
¥4993971 a FBA '
E E FAL
' '
.

19. Independence
• When variables are independent, the joint factorizes into a
product of the marginals:
19Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
-
Ah I a z I or L in Ffa FR e I ¥ EI en D
-
s .
K Fi
'
I a
ERK EE
-
a Ft v .
HE .
2 ARE a GEEK n A ② ¥ To Ep is
q th z
"
k EEE 'T a
ftp.t '
Eat 3 .

20. Expectation
Expectation tell us the expected or average value of some
function f [x] taking into account the distribution of x
Definition:
20Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Eta 4¥ HE
a a Tropez It B FT E % C 2 .
FIFE K f EX ] a K IF A 7 A .
I 2 u it
THE ET I th 2 f TE E fi k 3 .
um
K K a
'
4h 2 C 3 .
d to
"
Totalk Faklike
n
HIPFa
x g
ETE If ri EI H It IT c K f ex , 2 '
ItE. & a E ER 2 .
a
*
KLEENE
a
Eka Es
a AFF to

21. Expectation
Expectation tell us the expected or average value of some
function f [x] taking into account the distribution of x
Definition in two dimensions:
21Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
2 s KK a -438 Fr .
x e f n THE¥ Eth EE I
'
Fs H ht i J c Te f Lx . g ]
E = EE EEN, .
E Fil .
= a TH it K E I E FLEE ftp.
~
7 LE Y V

22. Expectation: Common Cases
22Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
AME 's
FIG
EEE . y at 'E9
E -
tst
KEG Yaba
"
FFF # Yakka
£ch¥fhk3 . an FIFE 3k
E -
fit
BEAK
EEFE
FIFE IEIFT.tn#aEEa2ct22iz .
k£4123
¥B¥A

23. Expectation: Rules
Rule 1:
Expected value of a constant is the constant
23Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Reiko Kk a Hai EE the es k Za E a .

24. Expectation: Rules
Rule 2:
Expected value of constant times function is constant times
expected value of function
24Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
k
f = = a k x k .
2 .
ItEe
-
naw
IFF Sos II EI K FEE 47 E
'
'
a
-
's .
FEEL k He E C 2 E IF o E "
K S 7 t E I 2 .

25. Expectation: Rules
Rule 3:
Expectation of sum of functions is sum of expectation of
functions
25Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
As f- He FEND a FREE H FEE ft E
-
a
-
s .

26. Expectation: Rules
Rule 4:
Expectation of product of functions in variables x and y
is product of expectations of functions if x and y are independent
26Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
He 8 a
"
Heh EHF Box Es a 4
= a fly a Bfa 43 .

27. Conclusions
27Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Rules of probability are compact and simple
• Concepts of marginalization, joint and conditional
probability, Bayes rule and expectation underpin all of the
models in this book
• One remaining concept – conditional expectation –
discussed later
¥2 If RFI H a U L ,
D ' '
I 93 a fit IT 3 I 2 ,
-
Eff4 tf E FEE 4¥ The et KK 2
.
43 .