statistical efficiency is commonly used and basic topic in statistics.

1. Definitionof
StatisticalEFFICIENCY
Presented By :
Ruhul
AminMd. Osman
GoniMd. Ikram
Hossain

2. Outline
1. Definition
2. To find efficiency using Cramer-Rao Inequality
3. Cramer-Rao Inequality
4. Efficiency calculation

3. Definition:
Suppose there are two unbiased estimator 𝑡1 and 𝑡2
With parameter ‘θ’ then 𝑡1 will be more efficient
estimator than 𝑡2. The relative efficiency of 𝑡1compared
to 𝑡2 is given by the ratio.
Ef=
𝑉𝑎𝑟(𝑡2)
𝑉𝑎𝑟(𝑡1)
If Ef=1 both have same efficient
If Ef<1 𝑡1 less efficient
If Ef>1 𝑡1 more efficient
Here sample mean must be unbiased

4. EX. Suppose (𝑥1,𝑥2,…,𝑥5) be a rs of size 5 is drawn
from a normal distribution with unknown mean
µ.Consider the following estimator to estimate µ:
① 𝑡1 =
𝑥1+𝑥2+𝑥3+𝑥4+𝑥5
5
② 𝑡2 =
𝑥1+𝑥2
2
+ 𝑥3
the estimator which is best from 𝑡1 and 𝑡2?

5. Here,
E(𝑥𝑖)=µ,
=
1
5
σ2
=
1
25
{𝑉 𝑥1) + 𝑉(𝑥2) + 𝑉(𝑥3) + 𝑉(𝑥4) + 𝑉(𝑥5 }
V(𝑥𝑖)=σ2
Now,
V(𝑡1)=v(
𝑥1+𝑥2+𝑥3+𝑥4+𝑥5
5
)
V(𝑡2)=v(
𝑥1+𝑥2
2
+ 𝑥3)
=
1
4
{𝑉 𝑥1 + 𝑥2 + 𝑉 𝑥3
=
1
2
σ2 + σ2
Since var(𝑡1) < var(𝑡2) ,so 𝑡1 is the efficient estimator of µ .

6. To findefficiencyusing Cramer-RaoInequality
Efficiency=
𝐶𝑅𝐿𝐵
𝑉(𝛉)
Where CRLB is Cramer-Rao Lower Bound and 𝑉(𝛉)

7. Cramer-RaoInequality:
Let (𝑥1,𝑥2,…,𝑥 𝑛) be a random sample of size n drawn from the
density f(x,𝛉),LetT=t(𝑥1,𝑥2,…,𝑥 𝑛) be an unbiased estimator of 𝜏(𝛉),
afunctionofparameter.Weconsiderthefollowingcasescalled
regularitycondition:-
1.
𝜕 log f(x,𝛉)
𝜕 𝛉
exists for𝑎𝑙𝑙 𝑥 𝑎𝑛𝑑 𝑓𝑜𝑟 𝑎𝑙𝑙 𝛉𝜖𝜴
2.
𝜕
𝜕𝛉
... 𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛 = ...
𝜕
𝜕𝛉
𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛
3.
𝜕
𝜕𝛉
... t(𝑥1,𝑥2,…,𝑥 𝑛)𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛= ... t(𝑥1,𝑥2,…,𝑥 𝑛)
𝜕
𝜕𝛉
𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛
4. O<E{
𝜕 log f(x,𝛉)
𝜕 𝛉
}2<∞,forall 𝛉𝜖𝜴.

8. Under the above assumption, the variance of estimator t is given by ,
V(t)≥
{𝜏′ 𝛉 }2
−𝐸(
𝜕2 log 𝐿 𝛉
𝜕𝛉2 )
………………(1)
Where,
T=t(𝑥1,𝑥2,…,𝑥 𝑛) is an unbiased estimator of 𝜏(𝛉).Equation(1)is
calledcramer–rao inequality and RHS is called CRLB for the variance of
an unbiased estimator of 𝜏(𝛉).

9. Proof:
The likelihood function is given by,
L(𝛉)= 𝑖=1
𝑛
𝑓 𝑥, 𝛉 = 𝑓 𝑥2,𝛉 , 𝑓(𝑥2,𝛉)…, 𝑓(𝑥 𝑛,𝛉)
Or, ... 𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛 = ... 𝑓 𝑥1, 𝛉 ,…, 𝑓(𝑥 𝑛, 𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛
Or,
𝜕
𝜕𝛉
... 𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛 =0
Or, ...
𝜕
𝜕𝛉
𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛 =0………………….(1)
Or, ...
1
𝐿
𝜕
𝜕𝛉
𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛 =0
Or, ...
𝜕
𝜕𝛉
log𝐿(𝛉)d𝑥1,d𝑥2,…,d𝑥 𝑛 =0……………(2)
Or,E{
𝜕
𝜕𝛉
log𝐿(𝛉)}=0…………………………………………(3)
nowdifferentiating(2),weget,
... (
𝜕
𝜕𝛉
log𝐿(𝛉)
𝜕
𝜕𝛉
𝐿(𝛉)+
𝜕2 log 𝐿 𝛉
𝜕𝛉2 L)d𝑥1,d𝑥2,…,d𝑥 𝑛 =0
Or, ... (
𝜕
𝜕𝛉
log𝐿(𝛉)
1
𝐿
𝜕
𝜕𝛉
𝐿(𝛉)L+
𝜕2 log 𝐿 𝛉
𝜕𝛉2 𝐿)d𝑥1,d𝑥2,…,d𝑥 𝑛 =0

10. 𝑜𝑟, ...
𝜕
𝜕𝛉
log𝐿(𝛉)
𝜕
𝜕𝛉
log𝐿(𝛉)Ld𝑥1,d 𝑥2,…,d 𝑥𝑛=- ...
𝜕2 log 𝐿 𝛉
𝜕𝛉2 𝐿d𝑥1,d 𝑥2,…,d 𝑥𝑛
𝑜𝑟, ... (
𝜕
𝜕𝛉
log𝐿(𝛉))
2
𝐿d𝑥1,d 𝑥2,…,d 𝑥𝑛 =- ...
𝜕2 log 𝐿 𝛉
𝜕𝛉2 𝐿d𝑥1,d 𝑥2,…,d 𝑥𝑛
Therefore,E(
𝜕
𝜕𝛉
log𝐿(𝛉))
2
=−𝐸(
𝜕2 log 𝐿 𝛉
𝜕𝛉2 )……………(4)
Since,T=𝑡(𝑥1,𝑥2,…,𝑥𝑛)beanunbiasedestimatorof 𝜏(𝛉),
Thus,E(t)=𝜏(𝛉)
... 𝑡(𝑥1,𝑥2,…,𝑥𝑛)𝐿(𝛉)d𝑥1,d 𝑥2,…,d 𝑥𝑛= 𝜏(𝛉)
Or,
𝜕
𝜕𝛉
... 𝑡(𝑥1,𝑥2,…,𝑥𝑛)𝐿(𝛉)d𝑥1,d 𝑥2,…,d 𝑥𝑛=𝜏’(𝛉)
Or, ... 𝑡(𝑥1,𝑥2,…,𝑥𝑛)
𝜕
𝜕𝛉
𝐿(𝛉)d𝑥1,d 𝑥2,…,d 𝑥𝑛=𝜏’(𝛉)…………………….(5)

11. Or, 𝜏’(𝛉)=E[{t(𝑥1,𝑥2,…,𝑥 𝑛) −𝜏(𝛉)}
𝜕
𝜕𝛉
𝑙 𝑜 𝑔𝐿(𝛉)]
Therefore,{𝜏’(𝛉)}2= {E[{t(𝑥1,𝑥2,…,𝑥 𝑛) − 𝜏(𝛉)}
𝜕
𝜕𝛉
𝑙 𝑜 𝑔𝐿(𝛉)]}2

12. Appling Cauchy-Schwarz inequality, we may write
𝐸(𝑥𝑦)2 ≤ 𝐸(𝑥2)𝐸(𝑦2)
Or,{𝜏’(𝛉)}2≤ 𝐸([{t(𝑥1, 𝑥2,…, 𝑥 𝑛) − 𝜏(𝛉)}2)𝐸(
𝜕
𝜕𝛉
𝑙𝑜𝑔𝐿(𝛉))2
Or, {𝜏’(𝛉)}2≤ V(t) 𝐸(
𝜕
𝜕𝛉
𝑙𝑜𝑔𝐿(𝛉))2
or, V(t)≥
{ 𝜏’(𝛉)}2
𝐸(
𝜕
𝜕𝛉
𝑙𝑜𝑔𝐿(𝛉))2
Therefore, , V(t)≥
{ 𝜏’(𝛉)}2
−𝐸(
𝜕2 log 𝐿 𝛉
𝜕𝛉2 )
; [using (4)]

13. EX. f(x,θ)=exp(-θ)
θ 𝑥
𝑥!
, 𝑥 ≥ 0. 𝐹𝑖𝑛𝑑 𝑡ℎ𝑒 𝑒𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑐𝑦 𝑜𝑓 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑜𝑟?
Solution: Here , f(x,θ)=
exp(−θ)θ 𝑥
𝑥!
; 𝑥 ≥ 0
The LF is L(θ)= 𝑖=1
𝑛
f(𝑥𝑖,θ) = 𝑖=1
𝑛 exp(−θ)θ 𝑥𝑖
𝑥𝑖!
=
exp(−nθ)θΣ𝑥𝑖
1
𝑛 𝑥𝑖!
Taking log on the both sides,
logL(θ)=-nθ+Σ𝑥𝑖logθ- log 1
𝑛
𝑥𝑖!
Therefore,
𝜕 log L(𝛉)
𝜕 𝛉
=-n+ Σ𝑥𝑖/θ
The equating estimator,
𝜕 log L(𝛉)
𝜕 𝛉
= 0

14. Or,-n+ Σxi/θ=0
Therefore, θ=x, is the MLE of θ
Now , V(θ)=V(x)=
1
n2 V(xi) =
θ
n
Now, ,
𝜕 log L(𝛉)
𝜕 𝛉
=-n+ Σxi/θ=
θ
n
Σxi
n
− θ
=A(n,θ)[t-𝜏(𝛉)]
Where,A(n,θ)=
θ
n
, 𝜏(𝛉)= θ ,and
t=
Σxi
n
is the MLE of θ.
Now ,
𝜕 log L(𝛉)
𝜕 𝛉
=-n+ Σxi/θ
or,
𝜕2 log L θ
𝜕θ2 = -Σ
xi
θ2

15. Therefore,CRLB is , V(t)≥
{ 𝜏’(𝛉)}2
−𝐸(
𝜕2 log 𝐿 𝛉
𝜕𝛉2 )
=
1
𝑛
θ
=
θ
𝑛
Since the variance of estimated coinside with CRLB,
hence the estimator is an efficient estimator of θ.
Now , Efficiency =
𝐶𝑅𝐿𝐵
𝑉(θ)
=
θ
𝑛
θ
𝑛
= 1
or, 𝐸(
𝜕2 log 𝐿 𝛉
𝜕𝛉2 )=-E(
𝑥 𝑖
θ2)
Therefore, -𝐸(
𝜕2 log 𝐿 𝛉
𝜕𝛉2 )=
𝑛
θ

16. Thank you