Download to read offline
![Acceptance Sampling[1]](https://cdn.slidesharecdn.com/ss_thumbnails/acceptancesampling1-1226078569232381-9-thumbnail.jpg?width=640&height=640&fit=bounds)
Bagley_HNRS_CRM_talk_2015
- 1. Determining the Heterogeneity ofReference Materials Thomas Bagley, Dr. Cliff Stanley, Dr. John Murimboh Depts. of Earth & Environmental Science and Chemistry Acadia University
- 2. Acknowledgements Grants Participating Laboratories& CRM Manufacturers • Acadia Faculty Research Funding • Canada Summer Jobs Funding •MAC Student Research Funding •SEG Student Research Funding • ACME Analytical Labs, Vancouver • Bureau Veritas, Perth • African Mineral Standards, Johannesburg Geostats Proprietary, Vancouver • CDN Resource Laboratories • Ore Research/Exploration, Melbourne • Rocklabs, Auckland
- 3. Background • Certified ReferenceMaterials (CRMs): – Pulverized rock samples – With accepted element concentrations – With accepted standard deviations – Used to monitor analytical quality
- 5. Background • How areCRMs used ? – Analyzed in a batch of samples • CRM measured concentrations are compared with accepted concentrations to monitor accuracy – CRM measured concentrations are compared with each other to monitor precision – These provide an assessment of data quality for the batch
- 6.
- 7. Background • How areCRMs prepared ? – Pulverized – Homogenized – Sub-sampled – Chemically analyzed (in multi-lab round robin) – Statistically analyzed (to remove outliers)
- 8. Problem • The varianceobserved in CRM concentrations is not all laboratory error • CRMs are heterogeneous, and thus exhibit sampling error too • But how heterogeneous are CRMs ?
- 9.
- 10. Background • What contributesto the total variance of a CRM ? – Sampling Error – Laboratory Error – Inter-Laboratory Error 2222 InterLabSampTot σσσσ ++=
- 11. Background Outlying sample Analysis of largersamples ?Outlying lab 0.45 0.4 0.5 0.55
- 12. Background • Inter-laboratory errorand outliers are removed using inferential statistical tests • Observed variation is now only the sum of lab error and sample heterogeneity • Standard practice assumes that CRMs are homogenous; suggesting that observed variation is only lab error (NOT TRUE!) 222 LabSampTot σσσ +=
- 13. Objective • Need todetermine the magnitude of sampling error in CRMs
- 14. Strategy • Employ Dr.Stanley’s method to measure CRM lab and sampling errors simultaneously – Measure CRM concentrations in small and large samples – Solve using 3 equations / 3 unknowns 2 . 2 . 22 . 2 , 22 . 2 , SmSampSmLgSampLg LabSmSampSmTot LabLgSampLgTot MM σσ σσσ σσσ = += +=
- 15. Methods – Analysisof CRMs • Aqua Regia/ICP-MS for trace elements • 9 * 2.00 g samples; • 36 * 0.50 g samples; • Data quality assessment samples • Calculate large and small CRM sampling errors and laboratory error 2 . 2 . , , SmTotSm LgTotLg M M σ σ 22 . 2 . LabSmSampLgSamp σσσ
- 16. Sources of Errorin Co 0 2 4 6 8 10 12 14 16 2P 1M 4E 7F 12A GS50 20B BAS-1 P4B BL-10 P6 QUA-1(1) QUA-1(2) %RSD Small Sampling Error Large Sampling Error Analytical Error Results%RelativeStanda
- 17. Fundamental Sampling Constant •Unique to pulverized material • Relates sample size to sampling error (inversely proportional) • For a given sample size, sampling error is known ΨσMσM SmSampSmLgSampLg == 2 , 2 ,
- 18. Results Variance Ratio Plotof Co in Reference Materials 0.1 1 10 2P 1M 4E 7F 12A GS50 20B BAS-1 P4B BL-10 P6 QUA-1(1) QUA-1(2) Reference Material Batch LargeVariance/SmallVariance 02 Lab <σ 0 and 0 2 Sm.Samp 2 Lg.Samp < < σ σ
- 19. Discussion • The large& small sampling, and laboratory variances exhibit error • Sampling and Laboratory error measurements are dependent on adequate total variance estimates • How do we achieve adequate estimates of the total variance?
- 20. Discussion • Standard erroris dependent on the sampling & analytical variances, and the number of samples • It is best to estimate total variances with the same standard error 1n 2 s 1n 2 sSE Lg 2 Lab Lg 2 Lg.Samps2 Lg.Tot − + − = 1n 2 s 1n 2 sSE Sm 2 Lab Sm 2 Sm.Samps2 Sm.Tot − + − =
- 21.
- 22.
- 23. Discussion • To determinethe optimal sampling strategy (equal standard errors) 1 )( )1n()1( n 2 S 2 L + + −+ = κλ λ 2 Lg.Samp 2 Lab Sm Lg s s , M M == λκ
- 24.
- 25.
- 26. Conclusions • Reference materialheterogeneity it is not zero • Fundamental sampling constants can be used to estimate sampling error at different sample masses • CRM manufacturers should provide fundamental sampling constants with their accepted values • Analytical procedures should be designed to optimize standard error on the variance
- 27. Future work • Investigatethe controls on sampling strategy, to maximize precision • Develop QAQC methods that accommodate reference material heterogeneity
- 28. References • Stanley, C.R.2007. The Fundamental Relationship Between Sample Mass and Sampling Variance in Real Geological Samples and Corresponding Statistical Models. Exploration and Mining Geology, 16: 109-123. • Stanley, C.R., and Smee, B.W. 2007. Strategies for Reducing Sampling Errors in Exploration and Resource Definition Drilling Programmes for Gold Deposits. Geochemistry: Exploration, Environment, Analysis, 7: 1- 12. • Stanley, C.R, O'Driscoll, N., and Ranjan, P. 2010. Determining the magnitude of true analytical error in geochemical analysis. Geochemistry: Exploration, Environment, Analysis, 10: 355–364
Editor's Notes
- #2 Geoscientists commonly use geochemistry to address geological and environmental problems. The geological samples are sent to the lab, and the results indicate the major or trace element compositions. But how do we know that the results are true to the sample? Geoscientists send reference material with the batch of geological samples. Reference materials have an accepted concentration, so the quality of the analysis can be assessed from the result. My thesis concerns the heterogeneity of reference materials – how reproducible are the results from reference materials? Current practice assumes they are homogeneous, and attributes any variation to lab error.
- #4 Certified reference materials are pulverized rock or soil samples. They are useful in quality assessment/quality control (QAQC) because they are attributed with an accepted concentration & error.
- #6 Certified Reference Materials (CRMs) are used by the mining/resource and environmental industries to monitor analytical quality. Certified reference materials are sent with a batch of samples for analysis. To assess accuracy, the measured concentration can be compared to the accepted concentration. To measure precision, the results can be compared against themselves.
- #7 This is an example of a traditional control chart. Each CRM is plotted in the order that it was analyzed, from left to right. The concentration is represented on the y axis. The control chart has tolerances above and below the mean (1, 2 and 3 SD). Samples are unlikely to have concentrations outside 2SD from the mean, and very unlikely to have concentrations outside 3SD from the mean. In this chart there is an acceptable variation in the results from an analysis. A sample beyond 2SD would probably not be an outlier, but multiple samples beyond 2 SD would be anomalous. There also aren’t any obvious patterns. Patterns can be cause for concern – they can indicate problems like contamination. If one of two crushers is contaminated then the results on a control chart would alternate high – low.
- #8 The rock or soil sample is pulverized in a ball mill. The sample are blended. homogenized Sub samples are taken from the population which has been ‘homogenized’. The sub samples are analyzed at external labs in a ‘round robin’ analysis. Statistical processes are applied to remove outliers from the round robin: t-test (iso 5725), z-test, iso/iec 43-1, iso 3207.. http://upload.wikimedia.org/wikipedia/commons/c/c2/Ball_mill.gif Image of a ball mill from google images http://bestservices.co.in/images/equipment/ball-mill-dry-type.jpg
- #9 Despite the best efforts of the manufacturer, the homogenization is imperfect – the CRMs are heterogeneous Also, segregation can occur during shipping – graded bedding Therefore, the mining industry cannot use CRMs to monitor analytical quality unless the magnitudes of their heterogeneity are known. So, how heterogeneous are CRMs?
- #10 This diagram shows the results from a round robin CDN-GS-P4B. What contributes to the variation that we see?
- #11 Variance (gesture toward a sigma) is the standard deviation squared, it represents the spread of the data. Variation is high when replicate values differ significantly in concentration. Variance describes the magnitude of error. There are three sources of variation in a certification/round-robin: Sampling error is the heterogeneity of the sample – this is relevant to all elements. Gold is especially prone to high sampling error (nuggets..) Laboratory error is introduced during the analysis – variation in reagent quality, internal pressure in the mass spec, flame flicker.. Inter lab error is introduced when the labs in the round robin produce different distributions.
- #12 Here is our round-robin again: The red labs produced results that differ from the other labs, they’re outlying labs. (different means) The purple results from lab 14 differ substantially from most of lab 14’s results, and are outlying samples. Statistical procedures are used to distinguish outliers in this sort of scenario, to improve the quality of the certification. Also notice that the blue lab was much more precise than the other labs, this can happen if they analyze using a larger sample mass.
- #13 By removing the outliers we also remove the inter-lab error. Since inter-lab error has been removed, the error equation becomes: (gesture toward the equation on ppt) This is the point where it has generally been assumed that the reference material is homogeneous and that the total variation is lab error. THIS IS NOT TRUE, BUT IT IS STANDARD PRACTICE.
- #14 To improve the quality assessment methods, we must be able to determine the magnitude of the sampling error.
- #15 There is a statistical approach that was developed by my supervisor, Dr Cliff Stanley that’s separates sample and lab error. It is known that the sample mass is proportional to sampling error. Assuming lab error to be constant then the sample error can be determined. In this way, sampling error and lab error can both be determined.
- #16 The lab procedure was developed with Dr John Murimboh. This is the summary of the procedure that we use for each batch of samples. There are large and small samples, measured in replicate to estimate the variance. The procedure blank + control sample are for QAQC purposes Using this procedure we estimate each error: Small sampling error Large sampling error Lab error
- #17 The method allows us to determine the sampling error for both the large and small samples, and the analytical error. These graphs illustrates the relative proportions of analytical and sampling error for large and small samples. The small samples have much more sampling error! No surprise there! Samples that have no bars plotted did not produce valid estimates of sampling error and analytical error (to be discussed in detail later on)
- #18 More background: In the last graphs we saw that the sampling error was smaller for large samples. It turns out that the variance is inversely proportional to sample mass. That is, they multiply to define a constant, the fundamental sampling constant (FSC). The FSC can be used to compare homogeneity, and it allows us to CALCULATE THE ERROR AT ANY SAMPLE MASS – very practical information for QAQC.
- #19 The ratio of the total variances determines if the sampling and analytical errors can be decomposed. The lower tolerance is equal to the mass ratio (Ms/ML = 0.25 in this case), the upper tolerance is 1: the large total variance equalling the small total variance. Above the upper tolerance, negative sampling errors result and below the lower tolerance we get negative analytical errors.
- #20 The cause of failed sampling & analytical error estimates is likely to be a low standard error on the total variance (caused by too few replicates or an ineffective sampling strategy). How can we achieve the best estimates?
- #21 Standard error on the total variance is a function of the sampling & analytical error, and the number of samples. The best strategy to maximize the precision of the analytical and sampling error estimates is to make the standard error of the large total variance equal to the standard error on the small total variance (Stanley & Smee 2007).
- #22 Analytical variance is equal to the small sampling variance in this diagram, the large sampling variance is ¼ of the small sampling variance for a sample mass 4 times the size (Stanley 2007). Small samples need to be analyzed more times (as expected) to achieve the same standard error
- #23 2) The optimal sampling strategy uses the same standard error on the variance for both large and small samples; 36 and 15 in this case (we would have achieved more consistent results with fewer invalid estimates had we used 15 large samples instead of 9)
- #24 The previous example was for fixed sampling and analytical errors The sampling strategy changes depending on the proportions of sampling and analytical error. This system models the number of small to large samples that it takes to achieve an equal standard error on the total variances.
- #25 The optimal proportion of small to large samples to analyze based on the % large sampling variance (of the large total variance). The orange line is the same case that we examined previously (50% sm. Sampling error), not that 15 large samples to approximately 36 small samples. This diagram also illustrates that the sampling strategy for a reference material exhibiting no sampling error is an equal proportion of large and small samples, a case where there is only analytical error (slope = 1, blue line, 0.001%). The sampling strategy for a reference material that exhibits only sampling error (a perfect geochemical analysis!) would use a proportion of 1:16 large to small samples (slope of 16), when using a large sample mass 4 times the small sample mass.
- #26 By using samples with twice the difference (1:8 instead of 1:4 by mass), the number of large samples is greatly decreased – only 8 would need to be analyzed for 36 small samples.