1. Determining the Heterogeneity
of Reference Materials
Thomas Bagley, Dr. Cliff Stanley, Dr. John Murimboh
Depts. of Earth & Environmental Science and Chemistry
Acadia University
2. Acknowledgements
Grants Participating Laboratories &
CRM Manufacturers
• Acadia Faculty Research
Funding
• Canada Summer Jobs Funding
•MAC Student Research Funding
•SEG Student Research Funding
• ACME Analytical Labs, Vancouver
• Bureau Veritas, Perth
• African Mineral Standards,
Johannesburg Geostats Proprietary,
Vancouver
• CDN Resource Laboratories
• Ore Research/Exploration, Melbourne
• Rocklabs, Auckland
3. Background
• Certified Reference Materials (CRMs):
– Pulverized rock samples
– With accepted element concentrations
– With accepted standard deviations
– Used to monitor analytical quality
4.
5. Background
• How are CRMs used ?
– Analyzed in a batch of samples
• CRM measured concentrations are
compared with accepted concentrations to
monitor accuracy
– CRM measured concentrations are compared
with each other to monitor precision
– These provide an assessment of data quality
for the batch
7. Background
• How are CRMs prepared ?
– Pulverized
– Homogenized
– Sub-sampled
– Chemically analyzed
(in multi-lab round robin)
– Statistically analyzed
(to remove outliers)
8. Problem
• The variance observed in CRM concentrations is
not all laboratory error
• CRMs are heterogeneous, and thus exhibit
sampling error too
• But how heterogeneous are CRMs ?
10. Background
• What contributes to the total variance of a CRM ?
– Sampling Error
– Laboratory Error
– Inter-Laboratory Error
2222
InterLabSampTot σσσσ ++=
12. Background
• Inter-laboratory error and outliers are removed
using inferential statistical tests
• Observed variation is now only the sum of lab
error and sample heterogeneity
• Standard practice assumes that CRMs are
homogenous; suggesting that observed
variation is only lab error (NOT TRUE!)
222
LabSampTot σσσ +=
14. Strategy
• Employ Dr. Stanley’s method to measure CRM
lab and sampling errors simultaneously
– Measure CRM concentrations in small and large
samples
– Solve using 3 equations / 3 unknowns
2
.
2
.
22
.
2
,
22
.
2
,
SmSampSmLgSampLg
LabSmSampSmTot
LabLgSampLgTot
MM σσ
σσσ
σσσ
=
+=
+=
15. Methods – Analysis of CRMs
• Aqua Regia/ICP-MS for trace elements
• 9 * 2.00 g samples;
• 36 * 0.50 g samples;
• Data quality assessment samples
• Calculate large and small CRM sampling
errors and laboratory error
2
.
2
.
,
,
SmTotSm
LgTotLg
M
M
σ
σ
22
.
2
. LabSmSampLgSamp σσσ
16. Sources of Error in Co
0
2
4
6
8
10
12
14
16
2P
1M
4E
7F
12A
GS50
20B
BAS-1
P4B
BL-10
P6
QUA-1(1)
QUA-1(2)
%RSD
Small Sampling Error
Large Sampling Error
Analytical Error
Results%RelativeStanda
17. Fundamental Sampling Constant
• Unique to pulverized material
• Relates sample size to sampling error
(inversely proportional)
• For a given sample size, sampling error is
known
ΨσMσM SmSampSmLgSampLg == 2
,
2
,
18. Results
Variance Ratio Plot of Co in Reference Materials
0.1
1
10
2P
1M
4E
7F
12A
GS50
20B
BAS-1
P4B
BL-10
P6
QUA-1(1)
QUA-1(2)
Reference Material Batch
LargeVariance/SmallVariance
02
Lab <σ
0
and
0
2
Sm.Samp
2
Lg.Samp
<
<
σ
σ
19. Discussion
• The large & small sampling, and
laboratory variances exhibit error
• Sampling and Laboratory error
measurements are dependent on
adequate total variance estimates
• How do we achieve adequate estimates of
the total variance?
20. Discussion
• Standard error is dependent on the
sampling & analytical variances, and the
number of samples
• It is best to estimate total variances with
the same standard error
1n
2
s
1n
2
sSE
Lg
2
Lab
Lg
2
Lg.Samps2
Lg.Tot
−
+
−
=
1n
2
s
1n
2
sSE
Sm
2
Lab
Sm
2
Sm.Samps2
Sm.Tot
−
+
−
=
23. Discussion
• To determine the optimal sampling
strategy (equal standard errors)
1
)(
)1n()1(
n 2
S
2
L +
+
−+
=
κλ
λ
2
Lg.Samp
2
Lab
Sm
Lg
s
s
,
M
M
== λκ
26. Conclusions
• Reference material heterogeneity it is
not zero
• Fundamental sampling constants can be
used to estimate sampling error at
different sample masses
• CRM manufacturers should provide
fundamental sampling constants with their
accepted values
• Analytical procedures should be designed
to optimize standard error on the variance
27. Future work
• Investigate the controls on sampling
strategy, to maximize precision
• Develop QAQC methods that
accommodate reference material
heterogeneity
28. References
• Stanley, C.R. 2007. The Fundamental Relationship
Between Sample Mass and Sampling Variance in Real
Geological Samples and Corresponding Statistical
Models. Exploration and Mining Geology, 16: 109-123.
• Stanley, C.R., and Smee, B.W. 2007. Strategies for
Reducing Sampling Errors in Exploration and Resource
Definition Drilling Programmes for Gold Deposits.
Geochemistry: Exploration, Environment, Analysis, 7: 1-
12.
• Stanley, C.R, O'Driscoll, N., and Ranjan, P. 2010.
Determining the magnitude of true analytical error in
geochemical analysis. Geochemistry: Exploration,
Environment, Analysis, 10: 355–364
Editor's Notes
Geoscientists commonly use geochemistry to address geological and environmental problems. The geological samples are sent to the lab, and the results indicate the major or trace element compositions. But how do we know that the results are true to the sample?
Geoscientists send reference material with the batch of geological samples. Reference materials have an accepted concentration, so the quality of the analysis can be assessed from the result. My thesis concerns the heterogeneity of reference materials – how reproducible are the results from reference materials? Current practice assumes they are homogeneous, and attributes any variation to lab error.
Certified reference materials are pulverized rock or soil samples. They are useful in quality assessment/quality control (QAQC) because they are attributed with an accepted concentration & error.
Certified Reference Materials (CRMs) are used by the mining/resource and environmental industries to monitor analytical quality. Certified reference materials are sent with a batch of samples for analysis. To assess accuracy, the measured concentration can be compared to the accepted concentration. To measure precision, the results can be compared against themselves.
This is an example of a traditional control chart. Each CRM is plotted in the order that it was analyzed, from left to right.
The concentration is represented on the y axis. The control chart has tolerances above and below the mean (1, 2 and 3 SD). Samples are unlikely to have concentrations outside 2SD from the mean, and very unlikely to have concentrations outside 3SD from the mean. In this chart there is an acceptable variation in the results from an analysis. A sample beyond 2SD would probably not be an outlier, but multiple samples beyond 2 SD would be anomalous. There also aren’t any obvious patterns. Patterns can be cause for concern – they can indicate problems like contamination. If one of two crushers is contaminated then the results on a control chart would alternate high – low.
The rock or soil sample is pulverized in a ball mill.
The sample are blended. homogenized
Sub samples are taken from the population which has been ‘homogenized’.
The sub samples are analyzed at external labs in a ‘round robin’ analysis.
Statistical processes are applied to remove outliers from the round robin: t-test (iso 5725), z-test, iso/iec 43-1, iso 3207..
http://upload.wikimedia.org/wikipedia/commons/c/c2/Ball_mill.gif
Image of a ball mill from google images
http://bestservices.co.in/images/equipment/ball-mill-dry-type.jpg
Despite the best efforts of the manufacturer, the homogenization is imperfect – the CRMs are heterogeneous
Also, segregation can occur during shipping – graded bedding
Therefore, the mining industry cannot use CRMs to monitor analytical quality unless the magnitudes of their heterogeneity are known.
So, how heterogeneous are CRMs?
This diagram shows the results from a round robin CDN-GS-P4B. What contributes to the variation that we see?
Variance (gesture toward a sigma) is the standard deviation squared, it represents the spread of the data. Variation is high when replicate values differ significantly in concentration. Variance describes the magnitude of error. There are three sources of variation in a certification/round-robin:
Sampling error is the heterogeneity of the sample – this is relevant to all elements. Gold is especially prone to high sampling error (nuggets..)
Laboratory error is introduced during the analysis – variation in reagent quality, internal pressure in the mass spec, flame flicker..
Inter lab error is introduced when the labs in the round robin produce different distributions.
Here is our round-robin again:
The red labs produced results that differ from the other labs, they’re outlying labs. (different means)
The purple results from lab 14 differ substantially from most of lab 14’s results, and are outlying samples.
Statistical procedures are used to distinguish outliers in this sort of scenario, to improve the quality of the certification.
Also notice that the blue lab was much more precise than the other labs, this can happen if they analyze using a larger sample mass.
By removing the outliers we also remove the inter-lab error.
Since inter-lab error has been removed, the error equation becomes: (gesture toward the equation on ppt)
This is the point where it has generally been assumed that the reference material is homogeneous and that the total variation is lab error. THIS IS NOT TRUE, BUT IT IS STANDARD PRACTICE.
To improve the quality assessment methods, we must be able to determine the magnitude of the sampling error.
There is a statistical approach that was developed by my supervisor, Dr Cliff Stanley that’s separates sample and lab error.
It is known that the sample mass is proportional to sampling error. Assuming lab error to be constant then the sample error can be determined.
In this way, sampling error and lab error can both be determined.
The lab procedure was developed with Dr John Murimboh. This is the summary of the procedure that we use for each batch of samples. There are large and small samples, measured in replicate to estimate the variance. The procedure blank + control sample are for QAQC purposes
Using this procedure we estimate each error:
Small sampling error
Large sampling error
Lab error
The method allows us to determine the sampling error for both the large and small samples, and the analytical error. These graphs illustrates the relative proportions of analytical and sampling error for large and small samples. The small samples have much more sampling error! No surprise there! Samples that have no bars plotted did not produce valid estimates of sampling error and analytical error (to be discussed in detail later on)
More background:
In the last graphs we saw that the sampling error was smaller for large samples. It turns out that the variance is inversely proportional to sample mass. That is, they multiply to define a constant, the fundamental sampling constant (FSC). The FSC can be used to compare homogeneity, and it allows us to CALCULATE THE ERROR AT ANY SAMPLE MASS – very practical information for QAQC.
The ratio of the total variances determines if the sampling and analytical errors can be decomposed. The lower tolerance is equal to the mass ratio (Ms/ML = 0.25 in this case), the upper tolerance is 1: the large total variance equalling the small total variance. Above the upper tolerance, negative sampling errors result and below the lower tolerance we get negative analytical errors.
The cause of failed sampling & analytical error estimates is likely to be a low standard error on the total variance (caused by too few replicates or an ineffective sampling strategy). How can we achieve the best estimates?
Standard error on the total variance is a function of the sampling & analytical error, and the number of samples. The best strategy to maximize the precision of the analytical and sampling error estimates is to make the standard error of the large total variance equal to the standard error on the small total variance (Stanley & Smee 2007).
Analytical variance is equal to the small sampling variance in this diagram, the large sampling variance is ¼ of the small sampling variance for a sample mass 4 times the size (Stanley 2007).
Small samples need to be analyzed more times (as expected) to achieve the same standard error
2) The optimal sampling strategy uses the same standard error on the variance for both large and small samples; 36 and 15 in this case (we would have achieved more consistent results with fewer invalid estimates had we used 15 large samples instead of 9)
The previous example was for fixed sampling and analytical errors
The sampling strategy changes depending on the proportions of sampling and analytical error. This system models the number of small to large samples that it takes to achieve an equal standard error on the total variances.
The optimal proportion of small to large samples to analyze based on the % large sampling variance (of the large total variance). The orange line is the same case that we examined previously (50% sm. Sampling error), not that 15 large samples to approximately 36 small samples.
This diagram also illustrates that the sampling strategy for a reference material exhibiting no sampling error is an equal proportion of large and small samples, a case where there is only analytical error (slope = 1, blue line, 0.001%). The sampling strategy for a reference material that exhibits only sampling error (a perfect geochemical analysis!) would use a proportion of 1:16 large to small samples (slope of 16), when using a large sample mass 4 times the small sample mass.
By using samples with twice the difference (1:8 instead of 1:4 by mass), the number of large samples is greatly decreased – only 8 would need to be analyzed for 36 small samples.
