2. 1. Define a hierarchy of models.
2. Express the likelihood functions for
the models.
3. Find the parameters that maximize
the likelihood functions.
4. Compare the values of likelihood to
decide which model is best.
4. 2.1.1 A hierarchy of models
Y ∼ Binomial(N, p)
－只有一個model，故不用說明model的階層性
5. 2.1.2 Express the likelihood function
→以投擲吐司的pmf: Pr(Y = y)為例
N y
p (1− p )N − y
, y = 0,..., N
y
f ( y; p ) =
, otherwise
0
6. pmf N p y (1− p )N − y , y = 0,..., N
y
f ( y; p ) =
, otherwise
0
Likelihood Function 相同！
N y
L( p; y ) = p (1 − p )
N−y
y
7. Likelihood Function
N y
L( p; y ) = p (1 − p )
N−y
y
→ L是參數p的likelihood function
8. Likelihood Function
For a discrete random variable, the likelihood
function is the probability mass function
expressed as a function of the parameters.
- Definition 26
9. 5 successes from 10 flips 50 successes from 100 flips
7 successes from 10 flips 70 successes from 100 flips
10. p=0:.01:1
like1 = bilike(p,5,10) %圖A的binominal likelihood function
like2 = bilike(p,50,100) %圖B的binominal likelihood function
like3 = bilike(p,7,10) %圖C的binominal likelihood function
like4 = bilike(p,70,100) %圖D的binominal likelihood function
plot(p,like1,p,like2,p,like3,p,like4) %畫出圖A~D
11. 5 successes from 10 flips 50 successes from 100 flips
In Panel A, the maximum of the likelihood (about .25)
is far smaller than that in Panel B.
In most applications, the actual value of likelihood is
not important.
14. 1. Define a hierarchy of models.
2. Express the likelihood functions for
the models.
3. Find the parameters that maximize
the likelihood functions.
4. Compare the values of likelihood to
decide which model is best.
16. Maximum Likelihood Estimate
A maximum likelihood (ML) estimate is the
parameter value that maximizes the likelihood
function for a given set of data.
- Definition 27（與前提）
23. Log likelihood Function
The log likelihood function is the
natural logarithm of the likelihood
function.
N
log + y log p + ( N − y ) log(1 − p )
y
- Definition 28
26. 2.1.4 Calculus Methods
to find MLEs
The calculus methods are limited in their
application and many problems must be
solved with numerical methods.
27. 我們的目標是找到能最大化log likelihood
function（i.e., l(p; y)）的p值
N
l(p; y) = log + y log p + ( N − y ) log(1 − p )
y
微分！
28. Step 1. 對p進行微分
∂l ( p; y ) ∂ N
= log + y log p + (N − y ) log(1 − p )
∂p ∂p y
∂ N ∂ ∂
= log + [ y log p ] + [( N − y ) log(1 − p )]
∂p y ∂p
∂p
y N−y
= 0+ −
p 1− p
y N−y
= -
p 1− p
29. Step 2.
y N−y
使上式等於0， - = 0,
p 1− p
求p的解 y N−y
= ,
p 1− p
(1 − p ) y = (N − y ) p,
y − py = Np − yp,
y = Np,
y
p=
ˆ
N
30. y
p=
observed successes
ˆ
N
For a binomial, the proportion of
successes is the maximum
likelihood estimator of parameter p.
31. Problem 2.1.2 (Your Turn)
Using calculus methods, derive the
maximum likelihood estimator of
parameter λ in the Poisson distribution
(see Eq. 2.3).
