Detailed explanation about mean squared error with the concept of the consistent estimator and unbiased estimator

1. MEAN SQUARED ERROR (MSE) OF AN
ESTIMATOR
By
Suruchi Somwanshi
M.Sc. (Mathematics)
M.Sc. (Statistics)

2. CONTENT
Introduction
Mean Squared Error
Mean Squared Error Consistency
Consistent Estimator
Unbiased Estimator
Tips

3. INTRODUCTION
Let ෡
𝜃 be the estimator of the unknown parameter θ
from the random sample X1, X2, · · · , Xn.
Then clearly the deviation from ෡
𝜃 to the true value of
θ, | ෡
𝜃 − θ|, measures the quality of the estimator, or
equivalently, we can use { ෡
𝜃 - }2 for the ease of
computation.
Since ෡
𝜃 is a random variable, we should take average
to evaluation the quality of the estimator.

4. Thus, we introduce the following Definition:
The mean square error (MSE) of an estimator ෡
𝜃 of a
parameter θ is the function of θ defined by E{෡
𝜃 −}2, and this is
denoted as 𝑀𝑆𝐸෡
𝜃 .
This is also called the risk function of an estimator, with
{෡
𝜃 −}2 called the quadratic loss function.
The expectation is with respect to the random variables X1, · ·
· , Xn since they are the only random components in the
expression.

5. Notice that the MSE measures the average squared
difference between the estimator ෡
𝜃 and the parameter θ, a
somewhat reasonable measure of performance for an
estimator.
 In general, any increasing function of the absolute distance
| ෡
𝜃 − θ| would serve to measure the goodness of an
estimator (mean absolute error, E{ ෡
𝜃 −} 2 ), is a reasonable
alternative.
But MSE has at least two advantages over other distance
measures: First, it is analytically tractable and, secondly, it
has the interpretation

6. MEAN SQUARED ERROR (MSE)
The Mean Square Error (MSE) of an estimator for estimating  is
𝑀𝑆𝐸𝜃
መ
𝜃 = 𝐸( መ
𝜃 − 𝜃)2
= 𝑉𝑎𝑟 መ
𝜃 + (𝐵𝑖𝑎𝑠𝜃
መ
𝜃 )2
If 𝑀𝑆𝐸𝜃
መ
𝜃 is smaller, መ
𝜃 is a better estimator of 𝜃.
For two estimators, መ
𝜃1 and መ
𝜃2 𝑜𝑓𝜃,
If 𝑀𝑆𝐸𝜃
መ
𝜃1 < 𝑀𝑆𝐸𝜃
መ
𝜃2 , 𝜃 ∈ Ω
መ
𝜃1 is better estimator of 𝜃 than መ
𝜃2.

7. 𝑀𝑆𝐸෡
𝜃 = E{෡
𝜃 −}2
= Var(෡
𝜃 ) + {𝐸(෡
𝜃 )−}2
= Var(෡
𝜃 ) + {𝐵𝑎𝑖𝑠 𝑜𝑓 ෡
𝜃 }2
This is so because
E{෡
𝜃 −}2 = E(෡
𝜃 2 ) + E(2) − 2θE(෡
𝜃 )
= Var(෡
𝜃 ) + {𝐸(෡
𝜃 )}2 + −2 − 2θE(෡
𝜃 )
= Var(෡
𝜃 ) + {𝐸(෡
𝜃 )−}2

8. MEAN SQUARED ERROR CONSISTENCY
෡
𝜃 is called mean squared error consistent (or consistent in
quadratic mean) if,
E{෡
𝜃 - }2→ 0 as n → .
Theorem: ෡
𝜃 is consistent in MSE iff
i) Var(෡
𝜃 )→ 0 as n → .

 =

→
)
ˆ
(
lim
) E
ii
n
If E{෡
𝜃 −}2→ 0 as n → , ෡
𝜃 is also a CE of .

9. CONSISTENT ESTIMATOR (CE): An estimator which
converges in probability to an unknown parameter  for all
   is called a CE of .
ˆ .
p
 
⎯⎯
→
For large n, a CE tends to be closer to the unknown
population parameter.
MLEs are generally CEs.

10. UNBIASED ESTIMATOR (UE):
We know that, the bias of an estimator ෡
𝜃 of a parameter θ is
the difference between the expected value of ෡
𝜃 and θ; that is,
Bias (෡
𝜃 ) = E(෡
𝜃 )−θ.
And an estimator whose bias is identically equal to 0 is called
unbiased estimator and satisfies
E(෡
𝜃 ) = θ for all θ.
For an unbiased estimator ෡
𝜃 , we have
𝑀𝑆𝐸෡
𝜃 = E{෡
𝜃 −}2 = Var(෡
𝜃 ) + {𝐸(෡
𝜃 )−}2 = Var(෡
𝜃 )
and so, if an estimator is unbiased, its MSE is equal to its
variance.

11. TIPS
MSE has two components,
1. One measures the variability of the estimator (precision)
and
2. Other measures its bias (accuracy).
An estimator that has good MSE properties has small
combined variance and bias.
To find an estimator with good MSE properties, we need to
find estimators that control both variance and bias.

12. THANK YOU