Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
NEON’S APPROACH TO UNCERTAINTY
ESTIMATION FOR SE...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Why?
• An example: Temperature change (2013 - 20...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Why?
• It is crucial that uncertainties are iden...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Why?
• It is crucial that uncertainties are iden...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Why?
• It is crucial that uncertainties are iden...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Why?
• It is crucial that uncertainties are iden...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Why?
• It is crucial that uncertainties are iden...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Why?
• It is crucial that uncertainties are iden...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Standardized – Traceable – Transparent
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
• Following the Joint Committee for Guides in Me...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
• Following the Joint Committee for Guides in Me...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
• Following the Joint Committee for Guides in Me...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Data & Uncertainty Flow Example: temperature
CAL...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Data & Uncertainty Flow Example: temperature
CAL...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Data & Uncertainty Flow Example: temperature
CAL...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Data & Uncertainty Flow Example: temperature
CAL...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Data & Uncertainty Flow Example: temperature
CAL...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Data & Uncertainty Flow Example: temperature
The...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Data & Uncertainty Flow Example: temperature
The...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
The Uncertainty Budget
Source of
uncertainty
Sta...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Further Propagation…
Pressure corrected to Sea L...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
Identifying (and quantifying?)
Precipitation: Ti...
© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.
The National Ecological Observatory Network is a...
Upcoming SlideShare
Loading in …5
×

Roberti: NEON's approach to uncertainty estimation for sensor-based measurements.

1,479 views

Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

Roberti: NEON's approach to uncertainty estimation for sensor-based measurements.

  1. 1. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. NEON’S APPROACH TO UNCERTAINTY ESTIMATION FOR SENSOR-BASED MEASUREMENTS Joshua A Roberti Jeffrey R Taylor Henry W Loescher Janae L Csavina Derek E Smith 5 August 2013 Ecological Society of America 98th Annual Meeting
  2. 2. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Why? • An example: Temperature change (2013 - 2042) • It is crucial that uncertainties are identified and quantified to determine statistical interpretations about mean quantity and variance structure; both are important when constructing higher level data products and modeled processes.
  3. 3. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Why? • It is crucial that uncertainties are identified and quantified to determine statistical interpretations about mean quantity and variance structure; both are important when constructing higher level data products and modeled processes.
  4. 4. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Why? • It is crucial that uncertainties are identified and quantified to determine statistical interpretations about mean quantity and variance structure; both are important when constructing higher level data products and modeled processes.
  5. 5. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Why? • It is crucial that uncertainties are identified and quantified to determine statistical interpretations about mean quantity and variance structure; both are important when constructing higher level data products and modeled processes.
  6. 6. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Why? • It is crucial that uncertainties are identified and quantified to determine statistical interpretations about mean quantity and variance structure; both are important when constructing higher level data products and modeled processes.
  7. 7. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Why? • It is crucial that uncertainties are identified and quantified to determine statistical interpretations about mean quantity and variance structure; both are important when constructing higher level data products and modeled processes.
  8. 8. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Why? • It is crucial that uncertainties are identified and quantified to determine statistical interpretations about mean quantity and variance structure; both are important when constructing higher level data products and modeled processes.
  9. 9. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Standardized – Traceable – Transparent
  10. 10. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. • Following the Joint Committee for Guides in Metrology’s (JCGM 100:2008) Guide to the Expression of uncertainty in measurement (GUM). This is an updated version of the International Organization for Standardization’s (ISO 1995) GUM Standardized – Traceable – Transparent
  11. 11. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. • Following the Joint Committee for Guides in Metrology’s (JCGM 100:2008) Guide to the Expression of uncertainty in measurement (GUM). This is an updated version of the International Organization for Standardization’s (ISO 1995) GUM “The evaluation of uncertainty is neither a routine task nor a purely mathematical one; it depends on detailed knowledge of the nature of the measurand and of the measurement method and procedure used. The quality and utility of the uncertainty quoted for the result of a measurement therefore ultimately depends on the understanding, critical analysis, and integrity of those who contribute to the assignment of its value.” (Eurachem-Citac 2000) Standardized – Traceable – Transparent
  12. 12. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. • Following the Joint Committee for Guides in Metrology’s (JCGM 100:2008) Guide to the Expression of uncertainty in measurement (GUM). This is an updated version of the International Organization for Standardization’s (ISO 1995) GUM “The evaluation of uncertainty is neither a routine task nor a purely mathematical one; it depends on detailed knowledge of the nature of the measurand and of the measurement method and procedure used. The quality and utility of the uncertainty quoted for the result of a measurement therefore ultimately depends on the understanding, critical analysis, and integrity of those who contribute to the assignment of its value.” (Eurachem-Citac 2000) • Algorithm Theoretical Basis Documents (ATBD) • Theory of measurement • Equations (converting from raw, uncalibrated data) • QA/QC; temporal averaging • Uncertainty estimates Standardized – Traceable – Transparent
  13. 13. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Data & Uncertainty Flow Example: temperature CALIBRATION Standards/ Procedures AD[08,10,14,15] Field measurement A S P I R A T I O N H E A T E R L1 DP: TEMPERATURE ± combined uncertainty Equations: 1: Ωi to ○ Ci 2: Averaging DAS Calibrated Field PRT Bridge Voltage Bridge Resistance Current Supply Noise Field PRT Bridge Voltage Bridge Resistance Current Supply Fig 1. Diagram outlining the data flow and potential sources of uncertainty associated with air temperature data
  14. 14. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Data & Uncertainty Flow Example: temperature CALIBRATION Standards/ Procedures AD[08,10,14,15] Field measurement A S P I R A T I O N H E A T E R L1 DP: TEMPERATURE ± combined uncertainty Equations: 1: Ωi to ○ Ci 2: Averaging DAS Calibrated Field PRT Bridge Voltage Bridge Resistance Current Supply Noise Field PRT Bridge Voltage Bridge Resistance Current Supply Uncertainties associated with PRTs and their calibration processes propagate into a combined uncertainty. This combined uncertainty represents i) the variation of an individual sensor from the mean of a sensor population, ii) uncertainty of the calibration procedures and iii) uncertainty of coefficients used to convert resistance to calibrated station temperature Fig 1. Diagram outlining the data flow and potential sources of uncertainty associated with air temperature data
  15. 15. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Data & Uncertainty Flow Example: temperature CALIBRATION Standards/ Procedures AD[08,10,14,15] Field measurement A S P I R A T I O N H E A T E R L1 DP: TEMPERATURE ± combined uncertainty Equations: 1: Ωi to ○ Ci 2: Averaging DAS Calibrated Field PRT Bridge Voltage Bridge Resistance Current Supply Noise Field PRT Bridge Voltage Bridge Resistance Current Supply • Air temperature measured with the aid of an aspirated shield are more accurate than those made with a naturally ventilated (passive) shield (World Meteorological Organization 2006) • wind + insolation = error Fig 1. Diagram outlining the data flow and potential sources of uncertainty associated with air temperature data
  16. 16. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Data & Uncertainty Flow Example: temperature CALIBRATION Standards/ Procedures AD[08,10,14,15] Field measurement A S P I R A T I O N H E A T E R L1 DP: TEMPERATURE ± combined uncertainty Equations: 1: Ωi to ○ Ci 2: Averaging DAS Calibrated Field PRT Bridge Voltage Bridge Resistance Current Supply Noise Field PRT Bridge Voltage Bridge Resistance Current Supply • Any measurements recorded during times of heating, and for a specified time after the heater is turned off, will be flagged. Fig 1. Diagram outlining the data flow and potential sources of uncertainty associated with air temperature data
  17. 17. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Data & Uncertainty Flow Example: temperature CALIBRATION Standards/ Procedures AD[08,10,14,15] Field measurement A S P I R A T I O N H E A T E R L1 DP: TEMPERATURE ± combined uncertainty Equations: 1: Ωi to ○ Ci 2: Averaging DAS Calibrated Field PRT Bridge Voltage Bridge Resistance Current Supply Noise Field PRT Bridge Voltage Bridge Resistance Current Supply Fig 1. Diagram outlining the data flow and potential sources of uncertainty associated with air temperature data
  18. 18. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Data & Uncertainty Flow Example: temperature The resulting value is multiplied by the partial derivative of the L1 DP. Since the DP is a temporal average, the partial derivative with respect to an individual measurement is simply: Where n represents the number of valid observations made during the averaging period. The absolute value of Eq. (2) is then multiplied by Eq. (1): (1) (2) (3) Finally, the combined uncertainty of the L1 mean DP is calculated via quadrature: (4)
  19. 19. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Data & Uncertainty Flow Example: temperature The resulting value is multiplied by the partial derivative of the L1 DP. Since the DP is a temporal average, the partial derivative with respect to an individual measurement is simply: Where n represents the number of valid observations made during the averaging period. The absolute value of Eq. (2) is then multiplied by Eq. (1): (1) (2) (3) Finally, the combined uncertainty of the L1 mean DP is calculated via quadrature: (4) SIGNAL : NOISE
  20. 20. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. The Uncertainty Budget Source of uncertainty Standard uncertainty component u(Xi) Value of standard uncertainty [○C] Degrees of Freedom L1 Temp. DP Eq. (11) -- -- Eq. (13) 1 Hz Temp. Eq. (8) Eq. (9) Eq. (10) Eq. (12) Sensor/calibration AD[15] 1 AD[15] AD[15] Noise (DAS) Eq. (4) [Ω] Eq. (5) Eq. (6) AD[15] Aspiration Eq. (7) 1 Eq. (7) 100 Table 1: Uncertainty budget for L1 mean temperature DPs. Shading denotes the order of uncertainty propagation (from lightest to darkest).
  21. 21. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Further Propagation… Pressure corrected to Sea Level • Air temperature data are used in the following equation: • Partial derivative with respect to temperature is: • Things get a bit messy…. May be better suited to solve with a Monte Carlo Method (JCGM 101:2008) (1) (2) (3) • And associated uncertainty propagates to the following equation:
  22. 22. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. Identifying (and quantifying?) Precipitation: Tipping buckets (also for throughfall) • Evaporative losses • Undercatchment • Splash-out • Wind • Wetting • Representativeness Fine-root turnover: Minirhizotrons • Proper quantification • Sampling frequency • Resolution of the sensor • Representativeness Fig 2. wind flow as a function of rain gauge size (Sevruk and Nespor 1994) Fig 3. Minirhizotrons at NEON headquarters
  23. 23. © 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED. The National Ecological Observatory Network is a project sponsored by the National Science Foundation and managed under cooperative agreement by NEON Inc. Contact: jroberti@neoninc.org • Traceable, standardized approach by which measurement uncertainties can be quantified. – Transparency! • It is our hope that current and future ecological networks will adapt this method, thereby strengthening ecological datasets while promoting interoperability. TAKE HOME: References Eurachem-Citac (2000) Quantifying uncertainty in analytical measurement. Technical Report. Second Edition Joint Committee for Guides in Metrology (JCGM) (100:2008) Evaluation of measurement data – Guide to the expression of uncertainty in measurement. JCGM (101:2008) Evaluation of measurement data – Supplement 1 to the “Guide of uncertainty in measurement” – Propagation of distributions using a monte carlo method International Organization for Standardization (ISO) (1995) Guide to the expression of uncertainty in measurement. Sevruk B. and Zahlavova L. (1994) Classification system of precipitation gauge site exposure: Evaluation and application. International Journal of Climatology, 14, pp. 681 – 689. World Meteorological Organization (WMO) (2006) Guide to meteorological instruments and methods of observation: Measurement of Temperature. WMO-No. 8.

×