Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
VARIOGRAM-DERIVED MEASURES FOR
QC PURPOSES
Markku Ohenoja
Control Engineering group
University of Oulu
1
10/15/2015Faculty...
15.10.2015
2
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
Petersen, L., Minkkinen, ...
BACKGROUND
• All measurements retain some amount of uncertainty, but also
sampling errors may affect on the result
• Utili...
OUTLINE
• What is Variogram and how it is calculated?
• Variogram-derived measures
• Examples within MMEA
15.10.2015
4
Fac...
VARIOGRAM
• Tool for empirical estimation of sampling errors incl. analytical
error
• Enables optimizing the sampling stra...
VARIOGRAM
• Collection of the data
• At least 30 samples with systematic sampling
• 1/5 smaller sampling interval than rou...
VARIOGRAM
15.10.2015
7
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
0 10 20 30
0
5
...
VARIOGRAM
15.10.2015
8
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
0 10 20 30
0
5
...
INDICES
15.10.2015
9
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
• Variogram-based...
INDICES
15.10.2015
10
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
• Variogram-base...
STANDARD ERROR OF THE MEAN
• Variance estimate of the sampling attained from variogram
• Standard error of the mean calcul...
STANDARD ERROR OF THE MEAN
15.10.2015
12
Month 2M 3M 4M 5M HalfYear Year All
0
0.5
1
1.5
2
2.5
3
2M
,%
Time frame for the...
STANDARD ERROR OF THE MEAN
15.10.2015TIEDEKUNTA TIEDEKUNTA / osasto osasto osaston osasto / Etuniminen
Sukuniminen-Sukunim...
DATA COMPARISON
• Multiple measurement sources with different sampling rates
• Data harmonization and comparison
• Based o...
WHAT SPARSE CANNOT SEE?
15.10.2015
15
0 5 10 15 20 25
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
Variograms for collective sam...
WHEN DENSE IS NOT REPRESENTATIVE?
15.10.2015TIEDEKUNTA TIEDEKUNTA / osasto osasto osaston osasto / Etuniminen
Sukuniminen-...
SUMMARY
15.10.2015Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
17
• Variogram can b...
Upcoming SlideShare
Loading in …5
×

Variogram-derived measures for QC purposes

The amount of environmental data is increasing, and the data would be valuable to the society if they are delivered to the right processes at the right time. In the seminar, we show examples of available data, how they are produced and processed, and how the data can be used in new innovative applications.

This presentation is part of the Environmental Data for Applications Seminar held on the 23rd of September 2015. The seminar was organised by the MMEA (Measurement, Measuring and Environmental Assessment) research programme under the Cleen Ltd (SHOK). The presentations are based on the research results related to environmental data interoperability. The participants included key players and partners in the field of environmental monitoring in Finland.

More info at www.mmea.fi

  • Login to see the comments

  • Be the first to like this

Variogram-derived measures for QC purposes

  1. 1. VARIOGRAM-DERIVED MEASURES FOR QC PURPOSES Markku Ohenoja Control Engineering group University of Oulu 1 10/15/2015Faculty of Technology / Control Engineering / Markku Ohenoja
  2. 2. 15.10.2015 2 Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi Petersen, L., Minkkinen, P. & Esbensen, K.H. 2005, "Representative sampling for reliable data analysis: Theory of Sampling", Chemometrics and Intelligent Laboratory Systems, vol. 77, no. 1–2, pp. 261-277. Time Meas. https://s-media-cache- ak0.pinimg.com/236x/64/46/7f/ 64467fa3382ac08d567d36b6aef05 13b.jpg
  3. 3. BACKGROUND • All measurements retain some amount of uncertainty, but also sampling errors may affect on the result • Utilization of different measurements collected with very different sampling rates requires evaluation of their information content • Environmental measurements are often periodic, sparsely collected and from various sources • Variographical analysis used for evaluating sampling errors and information content of the measurement 15.10.2015 3 Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi
  4. 4. OUTLINE • What is Variogram and how it is calculated? • Variogram-derived measures • Examples within MMEA 15.10.2015 4 Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi
  5. 5. VARIOGRAM • Tool for empirical estimation of sampling errors incl. analytical error • Enables optimizing the sampling strategy with respect to variance of the sampling error and number of samples takes • Provides an estimate of the standard error of the lot mean and the minimum possible error (MPE) of sampling 15.10.2015 5 Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi Semi-variogram Chrono-variogram Variographical analysis Geostatistics Kriging Variography Chronostatistics
  6. 6. VARIOGRAM • Collection of the data • At least 30 samples with systematic sampling • 1/5 smaller sampling interval than routine samples • Flowrate/sample weight should be included • Calculation of the heterogeneity of the data • Calculation of the experimental variogram v(j) • Relationship between the samples and the lag distance j • Estimation of the intercept v(0) (=MPE) • Graphically, separate experiment… • Auxiliary functions for comparing sampling strategies • Point-to-point calculation, algebraic modeling… 15.10.2015 6 Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi ℎ 𝑛 = 𝑎 𝑛 − 𝑎 𝐿 𝑎 𝐿 ∙ 𝑀 𝑛 𝑀 𝑛 𝑣 𝑗 = 1 2(𝑁 − 𝑗) ℎ 𝑛+𝑗 − ℎ𝑗 2 𝑁/2 𝑛=1 ≈
  7. 7. VARIOGRAM 15.10.2015 7 Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi 0 10 20 30 0 5 10 15 20 25 30 Variogram of 24h averaged online data Sampling interval (days) Relativestandarddeviationofthesamplingerror(%) 0 10 20 30 0 5 10 15 20 25 30 Variogram of daily sample Sampling interval (days) Relativestandarddeviationofthesamplingerror(%) Variogram Systematic sampling Random sampling Variogram Systematic sampling Random sampling σ2,σ,2σ,...
  8. 8. VARIOGRAM 15.10.2015 8 Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi 0 10 20 30 0 5 10 15 20 25 30 Variogram of 24h averaged online data Sampling interval (days) Relativestandarddeviationofthesamplingerror(%) 0 10 20 30 0 5 10 15 20 25 30 Variogram of daily sample Sampling interval (days) Relativestandarddeviationofthesamplingerror(%) Variogram Systematic sampling Random sampling Variogram Systematic sampling Random sampling 3x
  9. 9. INDICES 15.10.2015 9 Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi • Variogram-based indices applied for QC and PAT purposes • Standard error of the mean • MPE/σProcess • v(1)/σProcess
  10. 10. INDICES 15.10.2015 10 Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi • Variogram-based indices applied for QC and PAT purposes • Standard error of the mean • MPE/σProcess • v(1)/σProcess Process stability measure Bisgaard & Kulahci, Quality Engineering, 17(2), 2005 Drift estimation Paakkunainen et al., Chemometrics and Intelligent Laboratory Systems, 88(1), 2007 Fault diagnosis Kouadri et al., ISA Transactions, 51(3), 2012 Temporal uncertainty propagation Jalbert et al., Journal of Hydrology, 397(1-2), 2011 DQOs for control charts Minnit & Pitard, Journal of SAIMM, 108(2), 2008
  11. 11. STANDARD ERROR OF THE MEAN • Variance estimate of the sampling attained from variogram • Standard error of the mean calculated based on variance estimate and number of samples collected during a selected time frame • Recursive calculation possible for online measurements  moving average and its confidence intervals from the selected time frame 15.10.2015 11 Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi
  12. 12. STANDARD ERROR OF THE MEAN 15.10.2015 12 Month 2M 3M 4M 5M HalfYear Year All 0 0.5 1 1.5 2 2.5 3 2M ,% Time frame for the lot mean Online 17h average Online 12h average Online 8h average Online 6h average Online 4h average Online data Month 2M 3M 4M 5M HalfYear Year All 0 5 10 15 20 25 30 35 40 2M ,% Time frame for the lot mean Laboratory Calibrated online Raw online x 10 Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi
  13. 13. STANDARD ERROR OF THE MEAN 15.10.2015TIEDEKUNTA TIEDEKUNTA / osasto osasto osaston osasto / Etuniminen Sukuniminen-Sukuniminen 13 23-Nov-2009 12-Jan-2010 03-Mar-2010 22-Apr-2010 11-Jun-2010 31-Jul-2010 19-Sep-2010 08-Nov-2010 28-Dec-2010 16-Feb-2011 0 20 40 60 31-Dec-2010 7.341 7.3415 7.342 7.3425 7.343 7.3435 7.344 7.3445 7.345 7.3455 x 10 5 10 15 20 25 Lot mean and 2 M (%) for Three day average 7.341 7.3415 7.342 7.3425 7.343 7.3435 7.344 7.3445 7.345 7.3455 x 10 5 0 10 20 30 23-Nov-2009 12-Jan-2010 03-Mar-2010 22-Apr-2010 11-Jun-2010 31-Jul-2010 19-Sep-2010 08-Nov-2010 28-Dec-2010 16-Feb-2011 5 10 15 20 25 30 Lot mean and confidence intervals for Three day average
  14. 14. DATA COMPARISON • Multiple measurement sources with different sampling rates • Data harmonization and comparison • Based on MPE • Comparable averaging of the dense data around sparse samples, • Variographical analysis for whole averaged dense data mimicking more densely collected laboratory measurements • Information content evaluation based on v(1)/σProcess 15.10.2015 14 Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi
  15. 15. WHAT SPARSE CANNOT SEE? 15.10.2015 15 0 5 10 15 20 25 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 Variograms for collective samples Sampling interval Variance Variogram, Sparse meas. Variogram, Av. dense meas. 0 5 10 15 20 25 0 0.1 0.2 Variogram of sparse measurement Variance Sampling interval 0 200 400 600 800 1000 0 0.1 0.2 Variogram of averaged dense measurement Sampling interval Variance Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi
  16. 16. WHEN DENSE IS NOT REPRESENTATIVE? 15.10.2015TIEDEKUNTA TIEDEKUNTA / osasto osasto osaston osasto / Etuniminen Sukuniminen-Sukuniminen 16 26-May-2013 05-Jul-2013 14-Aug-2013 23-Sep-2013 02-Nov-2013 12-Dec-2013 0 10 20 30 40 Meas. Time series Dense meas. Sparse meas. 26-May-2013 05-Jul-2013 14-Aug-2013 23-Sep-2013 02-Nov-2013 12-Dec-2013 0 0.5 1 1.5 es / P Index Dense meas. Sparse meas. 26-May-2013 05-Jul-2013 14-Aug-2013 23-Sep-2013 02-Nov-2013 12-Dec-2013 -1 -0.5 0 0.5 1 Substracted index Index
  17. 17. SUMMARY 15.10.2015Faculty of Technology / Control Engineering / Markku Ohenoja markku.ohenoja@oulu.fi 17 • Variogram can be utilized for 1. Sampling error estimation 2. Sampling optimization 3. Moving average and confidence interval calculation 4. Information content evaluation • Recursive calculation enables e.g. monitoring, filtering, decision making • Information content evaluation allows comparison of measurement sources

×