11-06-0333-00-000t-introduction-to-measurement-uncertainty (1).ppt

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-Lindgren
Slide 1
Introduction to Measurement Uncertainty
Notice: This document has been prepared to assist IEEE 802.11. It is offered as a basis for discussion and is not binding on the contributing individual(s) or organization(s). The material in
this document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein.
Release: The contributor grants a free, irrevocable license to the IEEE to incorporate material contained in this contribution, and any modifications thereof, in the creation of an IEEE
Standards publication; to copyright in the IEEE’s name any IEEE Standards publication even though it may include portions of this contribution; and at the IEEE’s sole discretion to permit
others to reproduce in whole or in part the resulting IEEE Standards publication. The contributor also acknowledges and accepts that this contribution may be made public by IEEE 802.11.
Patent Policy and Procedures: The contributor is familiar with the IEEE 802 Patent Policy and Procedures <http:// ieee802.org/guides/bylaws/sb-bylaws.pdf>, including the statement
"IEEE standards may include the known use of patent(s), including patent applications, provided the IEEE receives assurance from the patent holder or applicant with respect to patents
essential for compliance with both mandatory and optional portions of the standard." Early disclosure to the Working Group of patent information that might be relevant to the standard is
essential to reduce the possibility for delays in the development process and increase the likelihood that the draft publication will be approved for publication. Please notify the Chair
<stuart.kerry@philips.com> as early as possible, in written or electronic form, if patented technology (or technology under patent application) might be incorporated into a draft standard being
developed within the IEEE 802.11 Working Group. If you have questions, contact the IEEE Patent Committee Administrator at <patcom@ieee.org>.
Date: 2006-3-02
Name Company Address Phone email
Dr. Michael D. Foegelle ETS-Lindgren 1301 Arrow Point Drive
Cedar Park, TX 78613
(512) 531-6444 Foegelle@ets-lindgren.com
Authors:

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 2
Abstract
This presentation introduces the common industry
concept of Measurement Uncertainty to represent the
quality of a measurement.
Other common terms such as accuracy, precision, error,
repeatability, and reliability are defined and their
relationship to measurement uncertainty is shown.
Basic directions on calculating uncertainty and an
example are included.

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 3
Overview
• Definitions
• Measurement Uncertainty
– Type A Evaluations
– Type B Evaluations
– Putting It All Together – RSS
– Reporting Uncertainty
– Special Cases
• Example Uncertainty Budget
• Summary
• References

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 4
Definitions
• Error – The deviation of a measured result from the
correct or accepted value of the quantity being
measured.
• There are two basic types of errors, random and
systematic.
Error

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 5
Definitions
• Random Errors – cause the measured result to deviate
randomly from the correct value. The distribution of
multiple measurements with only random error
contributions will be centered around the correct value.
• Some Examples
– Noise (random noise)
– Careless measurements
– Low resolution instruments
– Dropped digits Random Errors

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 6
Definitions
• Systematic Errors – cause the measured result to deviate
by a fixed amount in one direction from the correct
value. The distribution of multiple measurements with
systematic error contributions will be centered some
fixed value away from the correct value.
• Some Examples:
– Mis-calibrated instrument
– Unaccounted cable loss
Systematic Errors

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 7
Definitions
• Measurements typically contain some combination of
random and systematic errors.
• Precision is an indication of the level of random error.
• Accuracy is an indication of the level of systematic error.
• Accuracy and precision are typically qualitative terms.
Low Precision
Low Accuracy
Low Precision
High Accuracy
High Precision
Low Accuracy
High Precision
High Accuracy

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 8
Definitions
• Measurement Uncertainty combines these concepts into a
single quantitative value representing the total expected
deviation of a measurement from the actual value being
measured.
– Includes a statistical confidence in the resulting uncertainty.
– Contains contributions from all components of the measurement
system, requiring an understanding of the expected statistical
distribution of these contributions.
– By definition, measurement uncertainty does not typically contain
contributions due to the variability of the DUT.
• The “correct” value of a measurement is the value generated by the DUT
at the time it is tested.
• Variability of the DUT cannot be pre-determined.
• Still, the uncertainty of a particular measurement result will include this
variability.

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 9
Definitions
• Repeatability refers to the ability to perform the same
measurement on the same DUT under the same test
conditions and get the same result over time.
• By repeating the test setup between measurements of a
stable DUT, a statistical determination of System
Repeatability can be made. This is simply the level of
random error (precision) of the entire system, including
the contribution of the test operator, setup, etc.

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 10
Definitions
• Reproducibility typically refers to the stability of the
DUT and the ability to reproduce the same
measurement result over time using a system with a
high level of repeatability.
• More generally, it refers to achieving the same
measurement result under varied conditions.
– Different test equipment
– Different DUT
– Different Operator
– Different location/test lab

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 11
Definitions
• Reliability refers to producing the same result in
statistical trials. This would typically refer to the
stability of the DUT, and has connotations of
operational reliability of the DUT.
• Correction - value added algebraically to the
uncorrected result of a measurement to compensate for
systematic error.
• Correction Factor - numerical factor by which the
uncorrected result of a measurement is multiplied to
compensate for systematic error.
• Resolution – indicates numerical uncertainty of test
equipment readout. Actual uncertainty may be larger.

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 12
Measurement Uncertainty
• A measurement uncertainty represents a statistical level
encompassing the remaining unknown error in a
measurement.
• If the actual value of an error is known, then it is not
part of the measurement uncertainty. Rather, it should
be used to correct the measurement result.
• The methods for determining a measurement
uncertainty have been divided into two generic classes:
– Type A evaluation produces a statistically determined
uncertainty based on a normal distribution.
– Type B evaluation represents uncertainties determined
by any other means.

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 13
Type A Evaluations
• Uncertainties are determined through Type A evaluation
by performing repeated measurements and determining
the statistical distribution of the results.
• This approach works primarily for random
contributions.
– Repeated measurements with systematic deviations from a known
correct value gives an error value that should be corrected for.
• However, when evaluating the resulting measurement,
the effect of many systematic uncertainties combine with
random uncertainties in such a way that their effect can
be determined statistically.
– Eg. A systematic offset in temperature can cause an increase in the
random thermal noise in the measurement result.

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 14
Type A Evaluations
• Type A evaluation is based on the standard deviation of
repeat measurements, which for n measurements with
results qk and average value q, is approximated by:
• The standard uncertainty contribution ui of a single
measurement qk is given by:
• If n measurements are averaged together, this becomes:





n
k
k
k q
q
n
q
s
1
2
)
(
)
1
(
1
)
(
_
)
( k
i q
s
u 
n
q
s
q
s
u k
i
)
(
)
( 


Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 15
Type B Evaluations
• For cases where Type A evaluation is unavailable or
impractical, and to cover contributions not included in
the Type A analysis, a Type B analysis is used.
– Determine potential contributions to the total meas. uncertainty.
– Determine the uncertainty value for each contribution.
• Type A evaluation.
• Manufacturer’s datasheet.
• Estimate a limit value.
Note: Contribution must be in terms of the variation in the measured
quantity, not the influence quantity.
– For each contribution, choose expected statistical distribution and
determine its standard uncertainty.
– Combine resulting uis and calculate the expanded uncertainty.

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 16
Type B Evaluations
• There are a number of common distributions for
uncertainty contributions:
• Normal distribution:
• Examples:
– Results of Type A evaluations
– expanded uncertainties of components
-4s -3s -2s -1s 0 1s 2s 3s 4s
68%
99.7%
95%
k
U
u i
i 
where Ui is the expanded
uncertainty of the
contribution and k is the
coverage factor (k = 2
for 95% confidence).

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 17
-2ai -ai 0 ai 2ai
100%
Type B Evaluations
• Rectangular distribution – measurement result has an
equal probability of being anywhere within the range of
–ai to ai.
3
i
i
a
u 
• Examples:
– Equipment manufacturer ±
accuracy values (not from
standard uncertainty budget)
– Equipment resolution limits.
– Any term where only maximal
range or error is known.

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 18
Type B Evaluations
• U-shaped distribution –
measurement result has a higher
likelihood of being some value
above or below the median than
being at the median.
2
i
i
a
u 
• Examples:
– Mismatch (VSWR)
– Distribution of a sine wave
– 5% Resistors (Culling)
-2ai -ai 0 ai 2ai
-2ai -ai 0 ai 2ai

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 19
Type B Evaluations
• Triangular distribution – non-normal distribution with
linear fall-off from maximum to zero.
6
i
i
a
u 
• Examples:
– Alternate to rectangular or
normal distribution when
distribution is known to
peak at center and has a
known maximum
expected value.
-2ai -ai 0 ai 2ai
100%

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 20
Type B Evaluations
• Another Look
-2ai -ai 0 ai 2ai
100%
Normal Distribution U-Shaped Distribution Triangular Distribution
-2ai -ai 0 ai 2ai
-2ai -ai 0 ai 2ai
100%
-4 -3 -2 -1 0 1 2 3 4
68%
99.7%
95%
Rectangular Distribution

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 21
Putting It All Together - RSS
• Once standard uncertainties have been determined for
all components, including any Type A analysis, they are
combined into a total standard uncertainty (the
combined standard uncertainty, uc), for the resultant
measurement quantity using the root sum of squares
method:
where N is the number of standard uncertainty
components in the Type B analysis.
• The combined standard uncertainty is assumed to have
a normal distribution.



N
i
i
c u
u
1
2

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 22
Reporting Uncertainty
• The standard uncertainty is the common term used for
calculations. It represents a ±1s span (~68%) of a
normal distribution.
• Typically, measurement uncertainties are expressed as
an Expanded Uncertainty, U = k uc, where k is the
coverage factor.
• A coverage factor of k=2 is typically used, representing
a 95% confidence that the measured value is within the
specified measurement uncertainty.
• Reporting of expanded uncertainties must include both
the uncertainty value and either the coverage factor or
confidence interval in order to assure proper use.

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 23
Special Cases
• For Type A analyses with only a small number of
samples, the standard coverage factor is insufficient to
ensure that the expanded uncertainty covers the
expected confidence interval. Must use variable kp.
• RSS math works for values in dB! However,
distribution of a linear value may change when
converted to dB.
– Uncertainties typically always determined in measurement output
units.
N-1 1 2 3 4 5 6 7 8 10 20 50 
kp 14.0 4.53 3.31 2.87 2.65 2.52 2.43 2.37 2.28 2.13 2.05 2.00

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 24
Special Cases
• Not all distributions are symmetrical!
– Can develop asymmetrical uncertainties (+X/-Y) treating
asymmetric inputs separately.
– Can separate random portion of uncertainty from systematic
portion and apply a systematic error correction to measurement.
(Convert asymmetric uncertainty to symmetric uncertainty.)
error correction = (X+Y)/2, U = (X-Y)/2

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 25
Example Uncertainty Budget
•
Contribution Source Value Unit Distribution u_j (dB)
Mismatch: Transmit Side 0.00 dB U-Shaped 0.00
Analyzer Output Port Source Reflectivity Manufacturer -35.00 dB
Analyzer Output Port VSWR 1.04
Antenna Input Port VSWR (1775-2000) Measured 1.45
Antenna Input Port Reflectivity -14.72 dB
Cable Loss (S21 & S12) Measured 8.00 dB
Mismatch: Receive Side 0.01 dB U-Shaped 0.00
Analyzer Input Port Load Reflectivity Manufacturer -42.00 dB
Analyzer Input Port VSWR 1.02
Antenna Output Port VSWR Measured 1.35
Antenna Output Port Reflectivity -16.54 dB
Cable Loss (S21 & S12) Measured 3.00 dB
Network Analyzer Measurement Uncertainty Manufacturer 0.40 dB Rectangular 0.23
(Full Two-Port Calibration, 50 dB path loss, Wide Dynamic Range device)
Transmit Cable Loss Variation Measured 0.05 dB Rectangular 0.03
(Due to flexing, etc.)
Mounting Accuracy: Reference Antenna Calculated 0.00 Rectangular 0.00
Antenna Mounting (PLS Laser Aligned & Custom Mounts) 0.13 inches
Range length 14.50 feet
Reference Antenna Gain Uncertainty Manufacturer 0.22 dB Normal 0.11
Miscellaneous Uncertainty CTIA 2.1 G.13 0.20 dB Normal 0.10
Total Uncertainty, u_c Type B RSS 0.28
Expanded Uncertainty, U k = 2 0.55
Validity Range: 1775-2000 MHz

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 26
Summary
• This presentation gives common definitions for various
terms that have been used and misused in the TGT
draft.
• The concept of measurement uncertainty has been
introduced as the industry standard replacement for
terms such as accuracy, precision, repeatability, etc.
• Basic information has been given for a general
knowledge of the concepts and components of
measurement uncertainty.
• This document is not intended as a reference! Please
refer to the published documents referenced here.

Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Slide 27
References
• 1. NIST Technical Note 1297-1994, “Guidelines
for Evaluating and Expressing the Uncertainty of
NIST Measurement Results”, Barry N. Taylor and
Chris E. Kuyatt.
• 2. NIS-81, “The Treatment of Uncertainty in EMC
Measurements”, NAMAS
• 3. ISO/IEC Guide 17025, “General requirements
for the competence of testing and calibration
laboratories.”

11-06-0333-00-000t-introduction-to-measurement-uncertainty (1).ppt

Recommended

Recommended

More Related Content

Similar to 11-06-0333-00-000t-introduction-to-measurement-uncertainty (1).ppt

Similar to 11-06-0333-00-000t-introduction-to-measurement-uncertainty (1).ppt (20)

Recently uploaded

Recently uploaded (20)

11-06-0333-00-000t-introduction-to-measurement-uncertainty (1).ppt

Editor's Notes