On the representativeness of measurements

ON THE REPRESENTATIVENESS OF MEASUREMENTS
Markku Ohenoja
Control Engineering group
University of Oulu
1
10/28/2015Faculty of Technology / Control Engineering / Markku Ohenoja

BACKGROUND
• All measurements retain some amount of uncertainty, but also
sampling errors may affect on the result
• It is difficult to compare the information content of different
measurements, collected with very different sampling rates
• Online measurements
• Laboratory samples
• MOFI features
• MMEA/WP2
• Literature review à variogram
• Methods demonstrated with WWTP data
28.10.2015
2
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi

VARIOGRAM
• Tool for empirical estimation of sampling errors incl. analytical
error
• Enables optimizing the sampling strategy with respect to
variance of the sampling error and number of samples taken
• Provides an estimate of the standard error of the lot mean and
the minimum possible error (MPE) of sampling
28.10.2015
3
Semi-variogram
Chrono-variogram
Variographical
analysis
Geostatistics
Kriging
Variography Chronostatistics

EXAMPLES WITH WWTP DATA
• Error evaluation for MOFI
• Illustration of the benefits of dense data
• Monitoring of relation between sampling error and process
variation
• Model variable selection based on the latter one
28.10.2015Faculty of Technology / Control Engineering / Markku Ohenoja
4

ERROR EVALUATION FOR MOFI
Strong periodicity in the
data -> potentially biased
samples with systematic
sampling
5
Equivalent diameter with 52 data
points and sampling interval of 1h
0 5 10 15 20 25 30
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Lag / Time between samples, h
Relativestandarddeviation,%

6
0 5 10 15 20 25 30
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
sampling
Linear trend in data -> non-
stationary process

0 5 10 15 20 25 30
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Variogram
Random
7
sampling
stationary process

0 5 10 15 20 25 30
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Variogram
Random
Systematic
8
sampling
stationary process

0 5 10 15 20 25 30
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Variogram
Random
Systematic
2 = 2 = 1.45
2 ̅ = 2 = 0.78
Random sampling:
2 ̅ = 2
.
= 1.45%
Systematic sampling (lag=6):
2 ̅ = 2
.
= 0.78%
9
sampling
stationary process

DENSE VS. SPARSE DATA
28.10.2015
10
0 10 20 30
0
5
10
15
20
25
30
Variogram of 24h averaged online data
Sampling interval (days)
Relativestandarddeviationofthesamplingerror(%)
0 10 20 30
0
5
10
15
20
25
30
Variogram of daily sample
Sampling interval (days)
Relativestandarddeviationofthesamplingerror(%)
Variogram
Systematic sampling
Random sampling
Variogram
Systematic sampling
Random sampling
3x

11
26-May-2013 05-Jul-2013 14-Aug-2013 23-Sep-2013 02-Nov-2013 12-Dec-2013
0
10
20
30
40
Meas.
Time series
Dense meas.
Sparse meas.
0
0.5
1
1.5
e
s
/s
P
Index
Dense meas.
Sparse meas.
-1
-0.5
0
0.5
1
Substracted index
Index
MONITORING

MODELING
12
• Variable selection for WWTP model
• 71 variables with 48 samples -> model with 5 variables
• Model uses scaled values, scaling after variable selection
• Variable selection based on MPE/σ -ratio

MODELING
Markku.ohenoja@oulu.fi
13
• Variable selection for WWTP model
• 71 variables with 48 samples -> model with 5 variables
• Model uses scaled values, scaling after variable selection
• Variable selection based on MPE/σ -ratio
• First 5 variables
à model r2 = 0.65
• First 5 MOFI-features
à model r2 = 0.76
• Expert knowledge to reject some
variables à r2 = 0.76
• Tomperi et al. 2015,
à model r2 = 0.80
Variable name MPE/σP Matrix index
'Fe B0'
'Mean aspectr for objs > th'
'Mean roundness of objs > th'
'Cl T1'
'Fe L0'
'Cl L0'
'7. linjan lämpötila'
'Mean equiv.d for objs > th'
'Mean. area of objs > th'
'Median. area of objs > th'
'Number of all objects'
'Total fil length'
'Number of objs > th'
'Fil length/floc area'
'Number of small objs'
'Fe Mk'
'SO4 T1'
'PO4-P L0'
'Sähkönjohtavuus L0'
'Nitraattityppi Mk'
0.1871
0.2496
0.2505
0.2755
0.2764
0.2887
0.3140
0.3277
0.3483
0.3593
0.3756
0.3798
0.3886
0.4295
0.4732
0.4816
0.5208
0.5460
0.5562
0.5674
64
6
5
60
65
61
68
7
10
11
12
1
13
2
14
63
58
35
57
45
Tomperi J., Koivuranta E., Kuokkanen A., Juuso E., Leiviskä K. (2015)
Real-time optical monitoring of the wastewater treatment process,
Environmental Technology, DOI: 10.1080/09593330.2015.1069898

SUMMARY
14
Variogram
Sampling optimization
Empirically and recursively
evaluated
Monitoring Decision making
Information content
Comparison of measurements
Modeling Filtering
Moving average and confidence
intervals
Representativeness of measurements

On the representativeness of measurements

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (18)

Similar to On the representativeness of measurements

Similar to On the representativeness of measurements (20)

More from CLIC Innovation Ltd

More from CLIC Innovation Ltd (19)

Recently uploaded

Recently uploaded (20)

On the representativeness of measurements