1. Statistical efficiency compares two unbiased estimators by calculating their relative variance. The estimator with the lower variance is more efficient. 2. The Cramer-Rao inequality provides a lower bound (CRLB) for the variance of an unbiased estimator. It states that the variance of an estimator must be greater than or equal to the inverse of the expected information. 3. An estimator that achieves the Cramer-Rao lower bound is considered statistically efficient, as its variance reaches the theoretical minimum. In the example given, the maximum likelihood estimator of the mean of a normal distribution is shown to be efficient.

1. Definitionof
StatisticalEFFICIENCY
Presented By :
Ruhul
AminMd. Osman
GoniMd. Ikram
Hossain

2. Outline
1. Definition
2. To find efficiency using Cramer-Rao Inequality
3. Cramer-Rao Inequality
4. Efficiency calculation

3. Definition:
Suppose there are two unbiased estimator 𝑡1 and 𝑡2
With parameter ‘θ’ then 𝑡1 will be more efficient
estimator than 𝑡2. The relative efficiency of 𝑡1compared
to 𝑡2 is given by the ratio.
Ef=
𝑉𝑎𝑟(𝑡2)
𝑉𝑎𝑟(𝑡1)
If Ef=1 both have same efficient
If Ef<1 𝑡1 less efficient
If Ef>1 𝑡1 more efficient
Here sample mean must be unbiased

4. EX. Suppose (𝑥1,𝑥2,…,𝑥5) be a rs of size 5 is drawn
from a normal distribution with unknown mean
µ.Consider the following estimator to estimate µ:
① 𝑡1 =
𝑥1+𝑥2+𝑥3+𝑥4+𝑥5
5
② 𝑡2 =
𝑥1+𝑥2
2
+ 𝑥3
the estimator which is best from 𝑡1 and 𝑡2?

5. Here,
E(𝑥𝑖)=µ,
=
1
5
σ2
=
1
25
{𝑉 𝑥1) + 𝑉(𝑥2) + 𝑉(𝑥3) + 𝑉(𝑥4) + 𝑉(𝑥5 }
V(𝑥𝑖)=σ2
Now,
V(𝑡1)=v(
𝑥1+𝑥2+𝑥3+𝑥4+𝑥5
5
)
V(𝑡2)=v(
𝑥1+𝑥2
2
+ 𝑥3)
=
1
4
{𝑉 𝑥1 + 𝑥2 + 𝑉 𝑥3
=
1
2
σ2 + σ2
Since var(𝑡1) < var(𝑡2) ,so 𝑡1 is the efficient estimator of µ .

6. To findefficiencyusing Cramer-RaoInequality
Efficiency=
𝐶𝑅𝐿𝐵
𝑉(𝛉)
Where CRLB is Cramer-Rao Lower Bound and 𝑉(𝛉)

7. Cramer-RaoInequality:
Let (𝑥1,𝑥2,…,𝑥 𝑛) be a random sample of size n drawn from the
density f(x,𝛉),LetT=t(𝑥1,𝑥2,…,𝑥 𝑛) be an unbiased estimator of 𝜏(𝛉),
afunctionofparameter.Weconsiderthefollowingcasescalled
regularitycondition:-
1.
𝜕 log f(x,𝛉)
𝜕 𝛉
exists for𝑎𝑙𝑙 𝑥 𝑎𝑛𝑑 𝑓𝑜𝑟 𝑎𝑙𝑙 𝛉𝜖𝜴
2.
𝜕
𝜕𝛉
... 𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛 = ...
𝜕
𝜕𝛉
𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛
3.
𝜕
𝜕𝛉
... t(𝑥1,𝑥2,…,𝑥 𝑛)𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛= ... t(𝑥1,𝑥2,…,𝑥 𝑛)
𝜕
𝜕𝛉
𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛
4. O<E{
𝜕 log f(x,𝛉)
𝜕 𝛉
}2<∞,forall 𝛉𝜖𝜴.

8. Under the above assumption, the variance of estimator t is given by ,
V(t)≥
{𝜏′ 𝛉 }2
−𝐸(
𝜕2 log 𝐿 𝛉
𝜕𝛉2 )
………………(1)
Where,
T=t(𝑥1,𝑥2,…,𝑥 𝑛) is an unbiased estimator of 𝜏(𝛉).Equation(1)is
calledcramer–rao inequality and RHS is called CRLB for the variance of
an unbiased estimator of 𝜏(𝛉).

9. Proof:
The likelihood function is given by,
L(𝛉)= 𝑖=1
𝑛
𝑓 𝑥, 𝛉 = 𝑓 𝑥2,𝛉 , 𝑓(𝑥2,𝛉)…, 𝑓(𝑥 𝑛,𝛉)
Or, ... 𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛 = ... 𝑓 𝑥1, 𝛉 ,…, 𝑓(𝑥 𝑛, 𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛
Or,
𝜕
𝜕𝛉
... 𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛 =0
Or, ...
𝜕
𝜕𝛉
𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛 =0………………….(1)
Or, ...
1
𝐿
𝜕
𝜕𝛉
𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛 =0
Or, ...
𝜕
𝜕𝛉
log𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛 =0……………(2)
Or,E{
𝜕
𝜕𝛉
log𝐿(𝛉)}=0…………………………………………(3)
nowdifferentiating(2),weget,
... (
𝜕
𝜕𝛉
log𝐿(𝛉)
𝜕
𝜕𝛉
𝐿(𝛉)+
𝜕2 log 𝐿 𝛉
𝜕𝛉2 L)d𝑥1,d𝑥2,…,d𝑥 𝑛 =0
Or, ... (
𝜕
𝜕𝛉
log𝐿(𝛉)
1
𝐿
𝜕
𝜕𝛉
𝐿(𝛉)L+
𝜕2 log 𝐿 𝛉
𝜕𝛉2 𝐿)d𝑥1,d𝑥2,…,d𝑥 𝑛 =0

10. 𝑜𝑟, ...
𝜕
𝜕𝛉
log𝐿(𝛉)
𝜕
𝜕𝛉
log𝐿(𝛉)Ld𝑥1,d 𝑥2,…,d 𝑥𝑛=- ...
𝜕2 log 𝐿 𝛉
𝜕𝛉2 𝐿d𝑥1,d 𝑥2,…,d 𝑥𝑛
𝑜𝑟, ... (
𝜕
𝜕𝛉
log𝐿(𝛉))
2
𝐿d𝑥1,d 𝑥2,…,d 𝑥𝑛 =- ...
𝜕2 log 𝐿 𝛉
𝜕𝛉2 𝐿d𝑥1,d 𝑥2,…,d 𝑥𝑛
Therefore,E(
𝜕
𝜕𝛉
log𝐿(𝛉))
2
=−𝐸(
𝜕2 log 𝐿 𝛉
𝜕𝛉2 )……………(4)
Since,T=𝑡(𝑥1,𝑥2,…,𝑥𝑛)beanunbiasedestimatorof 𝜏(𝛉),
Thus,E(t)=𝜏(𝛉)
... 𝑡(𝑥1,𝑥2,…,𝑥𝑛)𝐿(𝛉)d𝑥1,d 𝑥2,…,d 𝑥𝑛= 𝜏(𝛉)
Or,
𝜕
𝜕𝛉
... 𝑡(𝑥1,𝑥2,…,𝑥𝑛)𝐿(𝛉)d𝑥1,d 𝑥2,…,d 𝑥𝑛=𝜏’(𝛉)
Or, ... 𝑡(𝑥1,𝑥2,…,𝑥𝑛)
𝜕
𝜕𝛉
𝐿(𝛉)d𝑥1,d 𝑥2,…,d 𝑥𝑛=𝜏’(𝛉)…………………….(5)

11. Or, 𝜏’(𝛉)=E[{t(𝑥1,𝑥2,…,𝑥 𝑛) −𝜏(𝛉)}
𝜕
𝜕𝛉
𝑙 𝑜 𝑔𝐿(𝛉)]
Therefore,{𝜏’(𝛉)}2= {E[{t(𝑥1,𝑥2,…,𝑥 𝑛) − 𝜏(𝛉)}
𝜕
𝜕𝛉
𝑙 𝑜 𝑔𝐿(𝛉)]}2

12. Appling Cauchy-Schwarz inequality, we may write
𝐸(𝑥𝑦)2 ≤ 𝐸(𝑥2)𝐸(𝑦2)
Or,{𝜏’(𝛉)}2≤ 𝐸([{t(𝑥1, 𝑥2,…, 𝑥 𝑛) − 𝜏(𝛉)}2)𝐸(
𝜕
𝜕𝛉
𝑙𝑜𝑔𝐿(𝛉))2
Or, {𝜏’(𝛉)}2≤ V(t) 𝐸(
𝜕
𝜕𝛉
𝑙𝑜𝑔𝐿(𝛉))2
or, V(t)≥
{ 𝜏’(𝛉)}2
𝐸(
𝜕
𝜕𝛉
𝑙𝑜𝑔𝐿(𝛉))2
Therefore, , V(t)≥
{ 𝜏’(𝛉)}2
−𝐸(
𝜕2 log 𝐿 𝛉
𝜕𝛉2 )
; [using (4)]

13. EX. f(x,θ)=exp(-θ)
θ 𝑥
𝑥!
, 𝑥 ≥ 0. 𝐹𝑖𝑛𝑑 𝑡ℎ𝑒 𝑒𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑐𝑦 𝑜𝑓 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑜𝑟?
Solution: Here , f(x,θ)=
exp(−θ)θ 𝑥
𝑥!
; 𝑥 ≥ 0
The LF is L(θ)= 𝑖=1
𝑛
f(𝑥𝑖,θ) = 𝑖=1
𝑛 exp(−θ)θ 𝑥𝑖
𝑥𝑖!
=
exp(−nθ)θΣ𝑥𝑖
1
𝑛 𝑥𝑖!
Taking log on the both sides,
logL(θ)=-nθ+Σ𝑥𝑖logθ- log 1
𝑛
𝑥𝑖!
Therefore,
𝜕 log L(𝛉)
𝜕 𝛉
=-n+ Σ𝑥𝑖/θ
The equating estimator,
𝜕 log L(𝛉)
𝜕 𝛉
= 0

14. Or,-n+ Σxi/θ=0
Therefore, θ=x, is the MLE of θ
Now , V(θ)=V(x)=
1
n2 V(xi) =
θ
n
Now, ,
𝜕 log L(𝛉)
𝜕 𝛉
=-n+ Σxi/θ=
θ
n
Σxi
n
− θ
=A(n,θ)[t-𝜏(𝛉)]
Where,A(n,θ)=
θ
n
, 𝜏(𝛉)= θ ,and
t=
Σxi
n
is the MLE of θ.
Now ,
𝜕 log L(𝛉)
𝜕 𝛉
=-n+ Σxi/θ
or,
𝜕2 log L θ
𝜕θ2 = -Σ
xi
θ2

15. Therefore,CRLB is , V(t)≥
{ 𝜏’(𝛉)}2
−𝐸(
𝜕2 log 𝐿 𝛉
𝜕𝛉2 )
=
1
𝑛
θ
=
θ
𝑛
Since the variance of estimated coinside with CRLB,
hence the estimator is an efficient estimator of θ.
Now , Efficiency =
𝐶𝑅𝐿𝐵
𝑉(θ)
=
θ
𝑛
θ
𝑛
= 1
or, 𝐸(
𝜕2 log 𝐿 𝛉
𝜕𝛉2 )=-E(
𝑥 𝑖
θ2)
Therefore, -𝐸(
𝜕2 log 𝐿 𝛉
𝜕𝛉2 )=
𝑛
θ

16. Thank you