Aacc Talk On Calibration Verification 2009
Upcoming SlideShare
Loading in...5
×
 

Aacc Talk On Calibration Verification 2009

on

  • 3,436 views

 

Statistics

Views

Total Views
3,436
Views on SlideShare
3,433
Embed Views
3

Actions

Likes
3
Downloads
77
Comments
0

2 Embeds 3

http://www.slideshare.net 2
https://duckduckgo.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Aacc Talk On Calibration Verification 2009 Aacc Talk On Calibration Verification 2009 Presentation Transcript

  • AACC Short Course 73214 How to Fulfill the CLIA Requirements for Calibration, Calibration Verification, and Reportable Range Anthony Killeen, MD, PhD William Castellani, MD (in conjunction with CAP)
  • This PowerPoint Slide Set • http://tinyurl.com/kmq8vu
  • Outline • Calibration Verification • Linearity • CVL Program at the CAP • Examples and Troubleshooting • Cases • Discussion of Selected CAP Checklist Questions
  • Calibration Verification • Periodic verification that the calibration of the analytical system remains valid • Typically assessed by comparing test results with appropriate target values • If the calibration changes, then patient test result values will also change
  • Linearity • From CLSI document EP6-A (2003) – A quantitative analytical method is linear when there exists a mathematically verified straight-line relationship between the observed values and the true concentrations or activities of the analyte. – …if the true amount of an analyte in a sample were to double, clinicians expect that would be reflected in a doubling of the measured value. – (Notice that there’s no reference to accuracy)
  • Calibration Verification vs. Linearity • Calibration verification is the process of verifying agreement between calibrators (or other materials of known analyte concentrations) and measured values • Calibrators should be traceable to a reference method to ensure accuracy • Linearity evaluation does not require knowledge of the “true” analyte concentration • “Linearity” does not appear in CLIA
  • Analytical Measurement Range vs. Linearity • AMR is the range of analyte values that a method can directly measure without any dilution, concentration, or other pretreatment of the specimen that’s not part of the usual assay process • A method should be linear across the AMR
  • Linearity as Defined in EP6-A vs. Instrument Response • Linearity typically refers to the final analytical result, not the raw instrument output. A plot of analyte concentration vs. the raw instrument output may not be linear (e.g., competitive immunoassay) • Linearity as used here is a straight-line relationship between “true” analyte concentrations and measured concentrations
  • Formal Calibration Verification/Linearity Program • College of American Pathologists – CVL Surveys • Other programs e.g., Maine Standards
  • Advantages to Participating in a CVL Program • Supplements proficiency testing (PT) • Linearity testing often has smaller absolute limits for error based on medically or analytically relevant criteria than does PT • Challenges the full calibration range • Can detect problems earlier than QC or PT • Calibration verification is required by CLIA, CVL program provides the materials and data analysis
  • Linearity Study Materials • Generally 5 samples that span the AMR • Preparation – Individually spiked samples – Serial dilutions of a high concentration sample – Admixtures of low and high concentrations (CAP) – At least 2 replicates at each level to assess random error
  • CAP Linearity Assessment • Uses a polynomial method described in EP6-A • For statistical approach see – EP6-A CLSI/NCCLS Document, Wayne, PA (2003) – Kroll MH et al., Arch Pathol Lab Med 124:1331-8, 2000
  • Calibration Verification Evaluation • Results reported as “verified” or “different” • If “verified”, then differences between your means and the peer-group means are within the allowable error limits in the range specified • If “different”, then at least one of your values exceeds the peer-group allowable error limits
  • Calibration Verification Report Example Result Tabular Data Graphical Data Peer Group Summary
  • Difference = 100 * (Your Mean – Peer Mean) / Peer Mean Allowable Error = larger of: One half of goal for total error OR minimum detectable difference
  • 100% 100%
  • Note that CLIA Allowable Error for PT is ± 10% for albumin
  • High biases Calibration not verified Need to re-calibrate
  • Proficiency Testing Data from Same Lab
  • CAP Linearity Evaluations • Standard • Extended Range • Dilution • Results – Linear – Nonlinear – Imprecise
  • Result Tabular Data Graphical Data
  • Extended Range Linearity Evaluation • Extended range evaluation is used if – Survey includes an additional high specimen that is spiked and not part of the admixture set – If there is a large gap in the relative concentrations of the admixture ratios such that a specimen(s) may have a large influence on the fitted regression line, the specimen(s) will be considered to be extended range
  • Scoring Extended Range Specimens • Specimens evaluated for linearity excluding the extended range specimen • If a range that includes the highest non- extended range specimen is linear, then the extended range specimen is evaluated • If the mean of the extended range specimen value falls on the extrapolated line (within allowable error) then the linear range will include the extended range specimen
  • Scoring Extended Range Specimens (2) • The extended range specimen is not used to calculate the best-fit line
  • Dilution Validation Linearity Evaluation • Performed if – Lab reports that it diluted at least one specimen – Enough undiluted specimens (normally 5) are reported for evaluation using the standard linearity evaluation
  • EXAMPLES AND TROUBLESHOOTING
  • Linearity Example 1: Standard Evaluation, specimens excluded from the linear range Linearity demonstrated for LN-01 to LN-05 only
  • Linearity Example 1: Standard Evaluation, specimens excluded from the linear range
  • Example 1: Troubleshooting • Does the linear range cover the AMR? If the high specimen is above the AMR, did you dilute? Was the dilution protocol followed? • If the linear range does not cover the AMR, then there may be problems with reagents, specimen handling, or the test system • Check QC, PT, calibration data • Address identified problems and re-run linearity • Consider adjusting AMR to cover the linear range
  • Example 2: Extended Range Evaluation, extended range sample not linear
  • Example 2: Extended Range Evaluation, extended range sample not linear
  • Example 2: Troubleshooting • Confirm that the linear range covers the AMR • Check the dilution protocol. Did you dilute? Dilution can introduce variability • If the linear range does not cover the AMR, then there may be problems with reagents, specimen handling, or the test system
  • Example 3: Standard Evaluation, nonlinear data (Best appreciated with graphical data)
  • Example 3: Standard Evaluation, nonlinear data
  • Example 3: Troubleshooting • Is the peer group generally linear? • If the peer group is generally linear then there may be problems with specimen handling or the test system • Review QC, calibration, PT data • Eliminate specimen or reagent handling errors • Diagnose fix any identified test system failures • Re-run linearity study
  • Example 4: Dilution Validation Evaluation, nonlinear trend in non-diluted samples
  • Example 4: Troubleshooting • (Because non-diluted samples are nonlinear, no evaluation of the diluted sample is performed • Is the peer group generally linear? • If the peer group is generally linear then there may be problems with specimen handling or the test system • Review QC, calibration, PT data • Eliminate specimen or reagent handling errors • Diagnose fix any identified test system failures • Re-run linearity study
  • Example 4: Standard Evaluation, large replicate imprecision
  • Example 5: Standard Evaluation, large replicate imprecision
  • Example 5: Troubleshooting • Pattern suggests pipetting problems which should be carefully investigated • Fix any identified test system failures • Re-run linearity study
  • Example 6: Standard Evaluation, only difference plot shows replicate imprecision (similar problem to Example 5)
  • Example 6: Standard Evaluation, only difference plot shows replicate imprecision (similar problem to Example 5) Looks OK?
  • Example 6: Troubleshooting • Pattern suggests pipetting problems which should be carefully investigated • Fix any identified test system failures • Re-run linearity study
  • Example 7: Standard Evaluation, Imprecision from poor fit
  • Example 7: Standard Evaluation, Imprecision from poor fit Here, the term “imprecision” refers to the inability to find a subset of data displaying linearity with adequate precision. Points zigzag around the best-fit line.
  • Example 7:Troubleshooting • Unusual pattern, may be difficult to diagnose • Is the peer group linear? If not, call Customer Service • Highest value may be diluted and a dilution error is masking loss of recovery at the high end of the AMR • Specimen handling problems may cause this
  • Example 8: Extended Range Evaluation, Imprecise in non-extended range, extended range sample(s) not evaluated
  • Example 8: Extended Range Evaluation, Imprecise in non-extended range, extended range sample(s) not evaluated Large gap indicates extended range Imprecise. E.R. specimen not evaluated
  • LN-24 • Creatinine linearity survey • Uses human serum spiked with pure crystalline creatinine • Accuracy assigned values (NIST) allow for accuracy based grading in addition to calibration verification and linearity checks
  • Case 1. Calibration Verification (AST)
  • Linearity study show the data were linear
  • Case 2. Calibration Verification (calcium)
  • Case 3. Calibration Verification
  • Case 4. Linearity (testosterone)
  • Executive Summary Page from CVL Survey
  • SOME CAP CHECKLIST QUESTIONS
  • CHM. 13000 • Are calibration procedures for each method adequate, and are the calibration results documented?
  • CHM.13100 • Are high quality materials with method- and matrix-appropriate target values used for calibration and calibration verification whenever possible?
  • CHM.13125 • Are all calibration materials used for non- FDA cleared assays documented as to quality? • NOTE: Standards used to prepare calibrators for non-FDA-cleared assays require certificates of purity from the vendor, or a check on purity as part of the initial assay validation process. The laboratory should document the accuracy of a new lot of calibrators by checking the new lot against the current lot.
  • CHM.13400 • Are criteria established for frequency of recalibration or calibration verification, and the acceptability of results? • • NOTE: Criteria typically include: • At changes of reagent lots for chemically or physically active or critical components, unless the laboratory can demonstrate that the use of different lots does not affect the accuracy of patient/client test results and the range used to report patient/client test data • QC fails to meet established criteria • After major maintenance or service • When recommended by the manufacturer • At least every 6 months
  • CHM.13500 • Is the method system recalibrated when calibration verification fails to meet the established criteria of the laboratory?
  • CHM.13600 • Is validation of the analytical measurement range (AMR) performed with matrix- appropriate materials which include the low, mid and high range of the AMR, and is the process documented?
  • CHM.13700 • Are criteria established for validating the analytical measurement range, and is compliance documented?
  • GEN.42085 • Is the reportable range verified/established for each analytic procedure before implementation? • NOTE: The reportable range includes all results that may be reliably reported, and embraces two types of ranges: • The ANALYTICAL MEASUREMENT RANGE (AMR) is the range of analyte values that a method can directly measure on the specimen without any dilution, concentration, or other pretreatment not part of the usual assay process • The CLINICALLY REPORTABLE RANGE (CRR) is the range of analyte values that a method can measure, allowing for specimen dilution, concentration, or other pretreatment used to extend the direct analytical measurement range
  • Summary of Topics Covered • Calibration Verification • Linearity • CVL Program at the CAP • Examples and Troubleshooting • Cases • Discussion of Selected CAP Checklist Questions