First Meeting of the Regional Soil
Laboratory Network for Eurasia and
Europe
Introduction to
Proficiency Testing
Winnie van Vark
Quality control in soil testing
Error sources
Quality in the laboratory
Performance characteristics of an analytical method.
• Precision
- Repeatability
- Reproducibility
• Accuracy Trueness, recovery
• Detection limit (LOD), quantification limit (LOQ), reporting
limit (RL)
• Measurement uncertainty
• Selectivity, robustness
Precision
• Repeatability
– Standard deviation in analytical result of a sample analyzed on 1 day
by one analyst in one batch on 1 instrument
– estimation of the “best” variance of the analytical result produced
by a lab
•Reproducibility (within lab)
– Same as repeatability, but sample analyzed by several analysts on
different days, on several instruments
– estimation of the “real life” variance of the analytical result over a
longer period by that lab
•Repeatability < Reproducibility
– Experience: [reproducibility ~1.5 –2.5 X repeatability]
Accuracy (trueness) and precision
Accuracy and precision
• CRM’s (Certified Reference Materials)
• Proficiency Testing - Comparable with other laboratories
Role of proficiency testing and use of results
• Set-up of analytical methods
to check the performance of the new method.
• Building up of experience
fine-tuning of the activities in the laboratory
• Proving experience
Results are close to the consensus value. Results are reliable.
• Accreditation
Results are used to convince the accreditation body
• Maintenance of performance
Results are used to maintain the performance of the laboratory
Proficiency Testing
• Every laboratory receives the same “item”
• Every laboratory analyses the sample
• (preferable without knowing it is a PT sample)
• Results are reported to the PT provider
• After assessment the results are reported to the
laboratories.
A simplified flow diagram of a
Proficiency test
Sample
preparation
Sample
distribution
Analysis by
laboratories
Reporting of
data by
laboratories
Statistical
assessment
of data
Performance
assessment
of
laboratories
Homogeneity and stability
tests
Analysis within QA/QC
system
Statistical evaluation
Performance
assessment
PT samples
• Each PT sample has to be equal
• Homogeneity is very important
• Mixing Soil
• Particles: difficult to mix
• Tendency to separate
• Only mixing (e.g. cement mixer) is not sufficient
• Partitioning is necessary
Pretreatment of the samples
• Soils
• Drying by air or at max. 40 degr C.
• Reduction to smaller particles
• Grinding ( - size??)
• Mixing
• Partioning
Partitioning of the samples (1)
• Cone and quarter technique:
Partitioning of the samples (2)
• Riffle box
Partitioning of the samples (3)
• Commercial available equipment
Partitioning of the samples (4)
• WEPAL
Homogeneity tests
• Carried out by organiser PT scheme:
• Properties and method:
small repeatability s.d. to detect any significant inhomogeneity
• Ratio repeatability s.d method < 0.5 proficiency assessment s.d.
• Following the ISO 13528 standard, 10 samples are taken
randomly and measured (at least) in duplicate under
repeatability conditions
Homogeneity tests
• Assumption: populations are normally distributed
with equal standard deviations
• Data analysed with a F-test (Anova single factor):
𝐹 − 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 =
𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝑜𝑓 𝑔𝑟𝑜𝑢𝑝 𝑚𝑒𝑎𝑛𝑠
𝑚𝑒𝑎𝑛 𝑜𝑓 𝑡ℎ𝑒 𝑤𝑖𝑡ℎ𝑖𝑛 𝑔𝑟𝑜𝑢𝑝 𝑣𝑎𝑟𝑖𝑎𝑛𝑐𝑒𝑠
• s.d. group < 0.3 * (s.d. historical PT tests)
Homogeneity tests
Example: s.d. Cu in PT – past 5 years
s.d. group < 0.3 * (s.d. historical PT tests)
Inhomogeneous sample
Stability of the samples
• Stable during the time of the PT
• Soil samples are stable for most parameters at
room temperature for many years
Reporting of data
• Preferably in a manner that copying by the PTP
(Proficiency Test Provider) is not necessary
• According to the instructions of the PTP
- singular, duplicate, triplicate?
- correct element - compound
- correct unit
- number of decimals
Evaluation of data
• Using robust statistics
- how to deal with outlying results?
(part of the practical exercise)
• Mean and/ or Median
• Standard deviation and / or MAD
• Relative standard deviation?
• No of participants
Performance Assessment
• Z-scores: indicates the distance to the assigned
value:
z-score: 𝑧 =
(𝑥 𝑙𝑎𝑏 𝑖
−𝑥 𝑝𝑡)
𝜎 𝑝𝑡
,
where
• 𝜎 𝑝𝑡 = the standard deviation for proficiency assessment ,
• 𝑥 𝑝𝑡 = the assigned value (consensus value)
Performance assessment
• Questionable performance: *
• Unsatisfactory performance: **
Z-scores at the laboratory
• Input from the participants
“Sheward chart” – Z-scores
Questions?
Thanks for your
attention

Introduction to proficiency testing

  • 1.
    First Meeting ofthe Regional Soil Laboratory Network for Eurasia and Europe Introduction to Proficiency Testing Winnie van Vark
  • 2.
    Quality control insoil testing Error sources
  • 3.
    Quality in thelaboratory Performance characteristics of an analytical method. • Precision - Repeatability - Reproducibility • Accuracy Trueness, recovery • Detection limit (LOD), quantification limit (LOQ), reporting limit (RL) • Measurement uncertainty • Selectivity, robustness
  • 4.
    Precision • Repeatability – Standarddeviation in analytical result of a sample analyzed on 1 day by one analyst in one batch on 1 instrument – estimation of the “best” variance of the analytical result produced by a lab •Reproducibility (within lab) – Same as repeatability, but sample analyzed by several analysts on different days, on several instruments – estimation of the “real life” variance of the analytical result over a longer period by that lab •Repeatability < Reproducibility – Experience: [reproducibility ~1.5 –2.5 X repeatability]
  • 5.
  • 6.
    Accuracy and precision •CRM’s (Certified Reference Materials) • Proficiency Testing - Comparable with other laboratories
  • 7.
    Role of proficiencytesting and use of results • Set-up of analytical methods to check the performance of the new method. • Building up of experience fine-tuning of the activities in the laboratory • Proving experience Results are close to the consensus value. Results are reliable. • Accreditation Results are used to convince the accreditation body • Maintenance of performance Results are used to maintain the performance of the laboratory
  • 8.
    Proficiency Testing • Everylaboratory receives the same “item” • Every laboratory analyses the sample • (preferable without knowing it is a PT sample) • Results are reported to the PT provider • After assessment the results are reported to the laboratories.
  • 9.
    A simplified flowdiagram of a Proficiency test Sample preparation Sample distribution Analysis by laboratories Reporting of data by laboratories Statistical assessment of data Performance assessment of laboratories Homogeneity and stability tests Analysis within QA/QC system Statistical evaluation Performance assessment
  • 10.
    PT samples • EachPT sample has to be equal • Homogeneity is very important • Mixing Soil • Particles: difficult to mix • Tendency to separate • Only mixing (e.g. cement mixer) is not sufficient • Partitioning is necessary
  • 11.
    Pretreatment of thesamples • Soils • Drying by air or at max. 40 degr C. • Reduction to smaller particles • Grinding ( - size??) • Mixing • Partioning
  • 12.
    Partitioning of thesamples (1) • Cone and quarter technique:
  • 13.
    Partitioning of thesamples (2) • Riffle box
  • 14.
    Partitioning of thesamples (3) • Commercial available equipment
  • 15.
    Partitioning of thesamples (4) • WEPAL
  • 16.
    Homogeneity tests • Carriedout by organiser PT scheme: • Properties and method: small repeatability s.d. to detect any significant inhomogeneity • Ratio repeatability s.d method < 0.5 proficiency assessment s.d. • Following the ISO 13528 standard, 10 samples are taken randomly and measured (at least) in duplicate under repeatability conditions
  • 17.
    Homogeneity tests • Assumption:populations are normally distributed with equal standard deviations • Data analysed with a F-test (Anova single factor): 𝐹 − 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 = 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝑜𝑓 𝑔𝑟𝑜𝑢𝑝 𝑚𝑒𝑎𝑛𝑠 𝑚𝑒𝑎𝑛 𝑜𝑓 𝑡ℎ𝑒 𝑤𝑖𝑡ℎ𝑖𝑛 𝑔𝑟𝑜𝑢𝑝 𝑣𝑎𝑟𝑖𝑎𝑛𝑐𝑒𝑠 • s.d. group < 0.3 * (s.d. historical PT tests)
  • 18.
    Homogeneity tests Example: s.d.Cu in PT – past 5 years s.d. group < 0.3 * (s.d. historical PT tests)
  • 19.
  • 20.
    Stability of thesamples • Stable during the time of the PT • Soil samples are stable for most parameters at room temperature for many years
  • 21.
    Reporting of data •Preferably in a manner that copying by the PTP (Proficiency Test Provider) is not necessary • According to the instructions of the PTP - singular, duplicate, triplicate? - correct element - compound - correct unit - number of decimals
  • 22.
    Evaluation of data •Using robust statistics - how to deal with outlying results? (part of the practical exercise) • Mean and/ or Median • Standard deviation and / or MAD • Relative standard deviation? • No of participants
  • 23.
    Performance Assessment • Z-scores:indicates the distance to the assigned value: z-score: 𝑧 = (𝑥 𝑙𝑎𝑏 𝑖 −𝑥 𝑝𝑡) 𝜎 𝑝𝑡 , where • 𝜎 𝑝𝑡 = the standard deviation for proficiency assessment , • 𝑥 𝑝𝑡 = the assigned value (consensus value)
  • 24.
    Performance assessment • Questionableperformance: * • Unsatisfactory performance: **
  • 25.
    Z-scores at thelaboratory • Input from the participants
  • 26.
  • 27.
  • 28.