This document discusses interlaboratory studies, including proficiency testing (PT) schemes and collaborative studies. It provides details on: - The difference between PT schemes and collaborative studies - The organization, benefits, and types of PT schemes - How assigned values, target ranges, and performance are determined in PT schemes - The typical sequence of events in collaborative studies - The interpretation of performance results in PT schemes

1. Benchmarking the
Laboratory

2. Interlaboratory Studies
• Two main types
– Proficiency Testing (PT) schemes
– Collaborative studies
• ISO/IEC 17025:2005 Clause 5.9 Assuring the quality
of test and calibration results
– “The laboratory shall have quality control procedures….This
monitoring shall be planned and reviewed and may include,
but not limited to the following:
…….
b) Participation in interlaboratory comparison or
proficiency-testing programmes.

3. Interlaboratory Studies
• What's the difference between
proficiency testing and collaborative
studies?
–PT tests the performance of the
participating laboratories
–Collaborative study is used to test
performance of a particular method

4. Proficiency Testing Schemes
• The primary aim of a PT scheme is to
provide laboratories with a framework for
obtaining a regular independent
assessment of their performance
–Performance expressed as a score
that can be readily interpreted

5. Proficiency Testing Schemes
Benefits of proficiency testing:
• Comparison of a lab’s performance with that of other
participating (peer) labs
• Monitoring a long-term facility performance
• Improvement in the performance of tests/calibrations
following investigation and identification of the
cause(s) of unsatisfactory PT performance, and the
introduction of corrective action to prevent re-
occurrence
• Staff education, training and competence monitoring

6. Proficiency Testing Schemes
Benefits of proficiency testing (continued):
• Evaluation of methods, including the establishment of
method precision and accuracy
• Estimation of measurement uncertainty
• Contribution to the facility’s overall risk management
system
• Confidence building with interested parties, e.g.
customers, accreditation bodies, regulators, specifiers

7. Benefits of proficiency testing (continued):
From first participation in any PT, the percentage of non-
acceptable scores decreases over several studies.
Proficiency Testing Schemes

8. Proficiency Testing Schemes
• Two main types:
– Those set up to assess the competence
of a group of laboratories to
undertake a very specific analysis
– Those used to evaluate the performance
of laboratories across a certain sector
for a particular type of analysis
• Not possible to assess performance against all
possible sample types

9. Proficiency Testing Schemes
• Each of the two previously motioned can be further
subdivided into 3 categories:
– Single sample distribution: randomly selected
subsamples from a bulk homogeneous supply of
material are distributed simultaneously to labs
– Split-sample testing: Samples of the product
are divided into two or more parts and each lab
tests a subsample of each part
– Measurement comparison scheme: Sample
to be tested circulated successively from one lab
to the next

10. Proficiency Testing Schemes

11. Proficiency Testing Schemes
• They are
many PT
schemes
providers
operated by
many
different
organizations

12. Proficiency Testing Schemes
• PT Providers
– EPTIS web-based Proficiency Testing
information system
– More than 1,300 PT schemes on EPTIS

13. Proficiency Testing Schemes

14. Proficiency Testing Schemes
• Organization of PT schemes:
– PT scheme organizer lays down rules for conduct
and interpretation of data
– Rules circulated to participants
– Organizer establish expert advisory group which
will address issues like:
• Test and types of samples
• How performance will be judged
• Technical difficulties arising

15. Proficiency Testing Schemes
• Organization of PT schemes continued:
– Material are chosen so that they are:
• Representative of the type normally analysed by labs
– In terms of matrix and concentration range
– Materials must be tested for homogeneity before
distribution
• Effective interpretation of all test results is based on
an assumption of homogeneity
• Each separate batch must be checked by a single
expert laboratory
• Precision important in sample preparation as all
subsamples is to have as near as possible the same
composition

16. Proficiency Testing Schemes
• Organization of PT schemes continued:
– Non-homogeneity is always possible after
distribution
• Due to sedimentation and separation
• Participant must re-homogenize the whole sample
before analysis
– No established optimum frequency for
distribution.
• Minimum about 4/year
– Once samples have been analysed lab report
results to PT organizer

17. Proficiency Testing Schemes
• Organization of PT schemes continued:
– Participants informed of outcome as soon
as possible
• Report that includes detailed information of
participants performance
• Participating labs are not disclosed in report.
Only lab identification numbers
• Labs can therefore identify their own
performance and compare it with others
without knowing their identities
• More info on organization of PT schemes can be found in ISO/IEC Guide
43 [2] , ILAC Guide 13 [3] and harmonized protocol produced by IUPAC

18. Proficiency Testing Schemes
• The ASSIGNED VALUE is the value attributed to the
quantity being measured
– “Target value” for participants
– Several different way to determine the assigned value.
• Formulation: synthetic test material with known concentration
of analyte (also called ‘fortified’ material)
• Certified Reference Values: obtained from certified reference
materials (CRM)
• Reference values: alternative to CRM where single expert
laboratory determines value using a primary method or validated
method calibrated with CRMs
• Consensus Values from Expert Laboratories: several used
to get data
• Consensus values from the Scheme Participants: Mean of
all results, outliers must be removed however when calculating
the mean.

19. Proficiency Testing Schemes
• The TARGET RANGE is standard deviation
used in the assessment of proficiency
– This value is usually set by organizer with specialist
advice.
– Target range represents acceptable variability of
particular analysis
– Same target range either in absolute terms or
proportional to assigned value should be used over
successive rounds of PT so performance scores ar
comparable over time

20. Proficiency Testing Schemes
• The TARGET RANGE is determined by several
ways:
– Prescribed value: may be derived from regulators etc..
– By Perception: determined level of performance
scheme organizer and participants would like laboratories
to achieve
– Results from collaborative study: repeatability and
reproducibility data from collaborative trial used.
– General Model: if collaborative data not available the
model like Horwitz used to estimate reproducibility.
– Data obtained from Proficiency Testing Round:
Participants data used to estimate standard deviation
with the rejection of outliers.

21. Proficiency Testing Schemes
• Performance Measure
– Most commonly used system is the z-score calculated
from the following:
x = result submitted by lab
X =assigned value
= Target range
– z-score based on normal distribution of data, 95% of
values expected to lie within ±2 Standard Deviation
– z-score ≤2 is considered satisfactory but between 2 and
3 is considered questionable.

22. Proficiency Testing Schemes
• Interpritation of performance
• Z-score ≤ 2.0
–(satisfactory" performance)
• Z-score between 2.0 to 3.0
–“questionable" performance
• Z-score ≥ 3.0
–"unsatisfactory" performance

23. Proficiency Testing Schemes
• Interpritation of performance
– Action should be considered in the
following situations
• When unsatisfactory results have been
obtained (requirement for labs accredited to
ISO/IEC 17025)
• When two consecutive questionable results
have been obtained for the same method
• When 9 consecutive results with the same bais
against the assigned value for the same
method has been obtained.

24. Collaborative Studies
• Each lab uses a define method to analyse
identical portions of homogeneous materials
• Assess performance characteristics of
method
• Used to develop standard methods

25. Collaborative Studies
• Typical sequence of events:
i. Text of proposed method sent to all participants
ii. Comments regarding the method are sent back to co-
ordinators
iii. Revised text of method and samples are sent to
participants
iv. Participants analyse sample
v. Results sent to co-ordinator to statistical analysis
vi. Report of study sent to participants
vii. Proposed method is made inconsultation with
participants.