Aacc Talk On Calibration Verification 2009

3,255 views

Published on

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,255
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
97
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Aacc Talk On Calibration Verification 2009

  1. 1. AACC Short Course 73214 How to Fulfill the CLIA Requirements for Calibration, Calibration Verification, and Reportable Range Anthony Killeen, MD, PhD William Castellani, MD (in conjunction with CAP)
  2. 2. This PowerPoint Slide Set • http://tinyurl.com/kmq8vu
  3. 3. Outline • Calibration Verification • Linearity • CVL Program at the CAP • Examples and Troubleshooting • Cases • Discussion of Selected CAP Checklist Questions
  4. 4. Calibration Verification • Periodic verification that the calibration of the analytical system remains valid • Typically assessed by comparing test results with appropriate target values • If the calibration changes, then patient test result values will also change
  5. 5. Linearity • From CLSI document EP6-A (2003) – A quantitative analytical method is linear when there exists a mathematically verified straight-line relationship between the observed values and the true concentrations or activities of the analyte. – …if the true amount of an analyte in a sample were to double, clinicians expect that would be reflected in a doubling of the measured value. – (Notice that there’s no reference to accuracy)
  6. 6. Calibration Verification vs. Linearity • Calibration verification is the process of verifying agreement between calibrators (or other materials of known analyte concentrations) and measured values • Calibrators should be traceable to a reference method to ensure accuracy • Linearity evaluation does not require knowledge of the “true” analyte concentration • “Linearity” does not appear in CLIA
  7. 7. Analytical Measurement Range vs. Linearity • AMR is the range of analyte values that a method can directly measure without any dilution, concentration, or other pretreatment of the specimen that’s not part of the usual assay process • A method should be linear across the AMR
  8. 8. Linearity as Defined in EP6-A vs. Instrument Response • Linearity typically refers to the final analytical result, not the raw instrument output. A plot of analyte concentration vs. the raw instrument output may not be linear (e.g., competitive immunoassay) • Linearity as used here is a straight-line relationship between “true” analyte concentrations and measured concentrations
  9. 9. Formal Calibration Verification/Linearity Program • College of American Pathologists – CVL Surveys • Other programs e.g., Maine Standards
  10. 10. Advantages to Participating in a CVL Program • Supplements proficiency testing (PT) • Linearity testing often has smaller absolute limits for error based on medically or analytically relevant criteria than does PT • Challenges the full calibration range • Can detect problems earlier than QC or PT • Calibration verification is required by CLIA, CVL program provides the materials and data analysis
  11. 11. Linearity Study Materials • Generally 5 samples that span the AMR • Preparation – Individually spiked samples – Serial dilutions of a high concentration sample – Admixtures of low and high concentrations (CAP) – At least 2 replicates at each level to assess random error
  12. 12. CAP Linearity Assessment • Uses a polynomial method described in EP6-A • For statistical approach see – EP6-A CLSI/NCCLS Document, Wayne, PA (2003) – Kroll MH et al., Arch Pathol Lab Med 124:1331-8, 2000
  13. 13. Calibration Verification Evaluation • Results reported as “verified” or “different” • If “verified”, then differences between your means and the peer-group means are within the allowable error limits in the range specified • If “different”, then at least one of your values exceeds the peer-group allowable error limits
  14. 14. Calibration Verification Report Example Result Tabular Data Graphical Data Peer Group Summary
  15. 15. Difference = 100 * (Your Mean – Peer Mean) / Peer Mean Allowable Error = larger of: One half of goal for total error OR minimum detectable difference
  16. 16. 100% 100%
  17. 17. Note that CLIA Allowable Error for PT is ± 10% for albumin
  18. 18. High biases Calibration not verified Need to re-calibrate
  19. 19. Proficiency Testing Data from Same Lab
  20. 20. CAP Linearity Evaluations • Standard • Extended Range • Dilution • Results – Linear – Nonlinear – Imprecise
  21. 21. Result Tabular Data Graphical Data
  22. 22. Extended Range Linearity Evaluation • Extended range evaluation is used if – Survey includes an additional high specimen that is spiked and not part of the admixture set – If there is a large gap in the relative concentrations of the admixture ratios such that a specimen(s) may have a large influence on the fitted regression line, the specimen(s) will be considered to be extended range
  23. 23. Scoring Extended Range Specimens • Specimens evaluated for linearity excluding the extended range specimen • If a range that includes the highest non- extended range specimen is linear, then the extended range specimen is evaluated • If the mean of the extended range specimen value falls on the extrapolated line (within allowable error) then the linear range will include the extended range specimen
  24. 24. Scoring Extended Range Specimens (2) • The extended range specimen is not used to calculate the best-fit line
  25. 25. Dilution Validation Linearity Evaluation • Performed if – Lab reports that it diluted at least one specimen – Enough undiluted specimens (normally 5) are reported for evaluation using the standard linearity evaluation
  26. 26. EXAMPLES AND TROUBLESHOOTING
  27. 27. Linearity Example 1: Standard Evaluation, specimens excluded from the linear range Linearity demonstrated for LN-01 to LN-05 only
  28. 28. Linearity Example 1: Standard Evaluation, specimens excluded from the linear range
  29. 29. Example 1: Troubleshooting • Does the linear range cover the AMR? If the high specimen is above the AMR, did you dilute? Was the dilution protocol followed? • If the linear range does not cover the AMR, then there may be problems with reagents, specimen handling, or the test system • Check QC, PT, calibration data • Address identified problems and re-run linearity • Consider adjusting AMR to cover the linear range
  30. 30. Example 2: Extended Range Evaluation, extended range sample not linear
  31. 31. Example 2: Extended Range Evaluation, extended range sample not linear
  32. 32. Example 2: Troubleshooting • Confirm that the linear range covers the AMR • Check the dilution protocol. Did you dilute? Dilution can introduce variability • If the linear range does not cover the AMR, then there may be problems with reagents, specimen handling, or the test system
  33. 33. Example 3: Standard Evaluation, nonlinear data (Best appreciated with graphical data)
  34. 34. Example 3: Standard Evaluation, nonlinear data
  35. 35. Example 3: Troubleshooting • Is the peer group generally linear? • If the peer group is generally linear then there may be problems with specimen handling or the test system • Review QC, calibration, PT data • Eliminate specimen or reagent handling errors • Diagnose fix any identified test system failures • Re-run linearity study
  36. 36. Example 4: Dilution Validation Evaluation, nonlinear trend in non-diluted samples
  37. 37. Example 4: Troubleshooting • (Because non-diluted samples are nonlinear, no evaluation of the diluted sample is performed • Is the peer group generally linear? • If the peer group is generally linear then there may be problems with specimen handling or the test system • Review QC, calibration, PT data • Eliminate specimen or reagent handling errors • Diagnose fix any identified test system failures • Re-run linearity study
  38. 38. Example 4: Standard Evaluation, large replicate imprecision
  39. 39. Example 5: Standard Evaluation, large replicate imprecision
  40. 40. Example 5: Troubleshooting • Pattern suggests pipetting problems which should be carefully investigated • Fix any identified test system failures • Re-run linearity study
  41. 41. Example 6: Standard Evaluation, only difference plot shows replicate imprecision (similar problem to Example 5)
  42. 42. Example 6: Standard Evaluation, only difference plot shows replicate imprecision (similar problem to Example 5) Looks OK?
  43. 43. Example 6: Troubleshooting • Pattern suggests pipetting problems which should be carefully investigated • Fix any identified test system failures • Re-run linearity study
  44. 44. Example 7: Standard Evaluation, Imprecision from poor fit
  45. 45. Example 7: Standard Evaluation, Imprecision from poor fit Here, the term “imprecision” refers to the inability to find a subset of data displaying linearity with adequate precision. Points zigzag around the best-fit line.
  46. 46. Example 7:Troubleshooting • Unusual pattern, may be difficult to diagnose • Is the peer group linear? If not, call Customer Service • Highest value may be diluted and a dilution error is masking loss of recovery at the high end of the AMR • Specimen handling problems may cause this
  47. 47. Example 8: Extended Range Evaluation, Imprecise in non-extended range, extended range sample(s) not evaluated
  48. 48. Example 8: Extended Range Evaluation, Imprecise in non-extended range, extended range sample(s) not evaluated Large gap indicates extended range Imprecise. E.R. specimen not evaluated
  49. 49. LN-24 • Creatinine linearity survey • Uses human serum spiked with pure crystalline creatinine • Accuracy assigned values (NIST) allow for accuracy based grading in addition to calibration verification and linearity checks
  50. 50. Case 1. Calibration Verification (AST)
  51. 51. Linearity study show the data were linear
  52. 52. Case 2. Calibration Verification (calcium)
  53. 53. Case 3. Calibration Verification
  54. 54. Case 4. Linearity (testosterone)
  55. 55. Executive Summary Page from CVL Survey
  56. 56. SOME CAP CHECKLIST QUESTIONS
  57. 57. CHM. 13000 • Are calibration procedures for each method adequate, and are the calibration results documented?
  58. 58. CHM.13100 • Are high quality materials with method- and matrix-appropriate target values used for calibration and calibration verification whenever possible?
  59. 59. CHM.13125 • Are all calibration materials used for non- FDA cleared assays documented as to quality? • NOTE: Standards used to prepare calibrators for non-FDA-cleared assays require certificates of purity from the vendor, or a check on purity as part of the initial assay validation process. The laboratory should document the accuracy of a new lot of calibrators by checking the new lot against the current lot.
  60. 60. CHM.13400 • Are criteria established for frequency of recalibration or calibration verification, and the acceptability of results? • • NOTE: Criteria typically include: • At changes of reagent lots for chemically or physically active or critical components, unless the laboratory can demonstrate that the use of different lots does not affect the accuracy of patient/client test results and the range used to report patient/client test data • QC fails to meet established criteria • After major maintenance or service • When recommended by the manufacturer • At least every 6 months
  61. 61. CHM.13500 • Is the method system recalibrated when calibration verification fails to meet the established criteria of the laboratory?
  62. 62. CHM.13600 • Is validation of the analytical measurement range (AMR) performed with matrix- appropriate materials which include the low, mid and high range of the AMR, and is the process documented?
  63. 63. CHM.13700 • Are criteria established for validating the analytical measurement range, and is compliance documented?
  64. 64. GEN.42085 • Is the reportable range verified/established for each analytic procedure before implementation? • NOTE: The reportable range includes all results that may be reliably reported, and embraces two types of ranges: • The ANALYTICAL MEASUREMENT RANGE (AMR) is the range of analyte values that a method can directly measure on the specimen without any dilution, concentration, or other pretreatment not part of the usual assay process • The CLINICALLY REPORTABLE RANGE (CRR) is the range of analyte values that a method can measure, allowing for specimen dilution, concentration, or other pretreatment used to extend the direct analytical measurement range
  65. 65. Summary of Topics Covered • Calibration Verification • Linearity • CVL Program at the CAP • Examples and Troubleshooting • Cases • Discussion of Selected CAP Checklist Questions

×