Upcoming SlideShare
×

Like this presentation? Why not share!

# Aacc Talk On Calibration Verification 2009

## on Jul 17, 2009

• 3,436 views

### Views

Total Views
3,436
Views on SlideShare
3,433
Embed Views
3

Likes
3
77
0

### 2 Embeds3

 http://www.slideshare.net 2 https://duckduckgo.com 1

### Report content

• Comment goes here.
Are you sure you want to

## Aacc Talk On Calibration Verification 2009Presentation Transcript

• AACC Short Course 73214 How to Fulfill the CLIA Requirements for Calibration, Calibration Verification, and Reportable Range Anthony Killeen, MD, PhD William Castellani, MD (in conjunction with CAP)
• This PowerPoint Slide Set • http://tinyurl.com/kmq8vu
• Outline • Calibration Verification • Linearity • CVL Program at the CAP • Examples and Troubleshooting • Cases • Discussion of Selected CAP Checklist Questions
• Calibration Verification • Periodic verification that the calibration of the analytical system remains valid • Typically assessed by comparing test results with appropriate target values • If the calibration changes, then patient test result values will also change
• Linearity • From CLSI document EP6-A (2003) – A quantitative analytical method is linear when there exists a mathematically verified straight-line relationship between the observed values and the true concentrations or activities of the analyte. – …if the true amount of an analyte in a sample were to double, clinicians expect that would be reflected in a doubling of the measured value. – (Notice that there’s no reference to accuracy)
• Calibration Verification vs. Linearity • Calibration verification is the process of verifying agreement between calibrators (or other materials of known analyte concentrations) and measured values • Calibrators should be traceable to a reference method to ensure accuracy • Linearity evaluation does not require knowledge of the “true” analyte concentration • “Linearity” does not appear in CLIA
• Analytical Measurement Range vs. Linearity • AMR is the range of analyte values that a method can directly measure without any dilution, concentration, or other pretreatment of the specimen that’s not part of the usual assay process • A method should be linear across the AMR
• Linearity as Defined in EP6-A vs. Instrument Response • Linearity typically refers to the final analytical result, not the raw instrument output. A plot of analyte concentration vs. the raw instrument output may not be linear (e.g., competitive immunoassay) • Linearity as used here is a straight-line relationship between “true” analyte concentrations and measured concentrations
• Formal Calibration Verification/Linearity Program • College of American Pathologists – CVL Surveys • Other programs e.g., Maine Standards
• Advantages to Participating in a CVL Program • Supplements proficiency testing (PT) • Linearity testing often has smaller absolute limits for error based on medically or analytically relevant criteria than does PT • Challenges the full calibration range • Can detect problems earlier than QC or PT • Calibration verification is required by CLIA, CVL program provides the materials and data analysis
• Linearity Study Materials • Generally 5 samples that span the AMR • Preparation – Individually spiked samples – Serial dilutions of a high concentration sample – Admixtures of low and high concentrations (CAP) – At least 2 replicates at each level to assess random error
• CAP Linearity Assessment • Uses a polynomial method described in EP6-A • For statistical approach see – EP6-A CLSI/NCCLS Document, Wayne, PA (2003) – Kroll MH et al., Arch Pathol Lab Med 124:1331-8, 2000
• Calibration Verification Evaluation • Results reported as “verified” or “different” • If “verified”, then differences between your means and the peer-group means are within the allowable error limits in the range specified • If “different”, then at least one of your values exceeds the peer-group allowable error limits
• Calibration Verification Report Example Result Tabular Data Graphical Data Peer Group Summary
• Difference = 100 * (Your Mean – Peer Mean) / Peer Mean Allowable Error = larger of: One half of goal for total error OR minimum detectable difference
• 100% 100%
• Note that CLIA Allowable Error for PT is ± 10% for albumin
• High biases Calibration not verified Need to re-calibrate
• Proficiency Testing Data from Same Lab
• CAP Linearity Evaluations • Standard • Extended Range • Dilution • Results – Linear – Nonlinear – Imprecise
• Result Tabular Data Graphical Data
• Extended Range Linearity Evaluation • Extended range evaluation is used if – Survey includes an additional high specimen that is spiked and not part of the admixture set – If there is a large gap in the relative concentrations of the admixture ratios such that a specimen(s) may have a large influence on the fitted regression line, the specimen(s) will be considered to be extended range
• Scoring Extended Range Specimens • Specimens evaluated for linearity excluding the extended range specimen • If a range that includes the highest non- extended range specimen is linear, then the extended range specimen is evaluated • If the mean of the extended range specimen value falls on the extrapolated line (within allowable error) then the linear range will include the extended range specimen
• Scoring Extended Range Specimens (2) • The extended range specimen is not used to calculate the best-fit line
• Dilution Validation Linearity Evaluation • Performed if – Lab reports that it diluted at least one specimen – Enough undiluted specimens (normally 5) are reported for evaluation using the standard linearity evaluation
• EXAMPLES AND TROUBLESHOOTING
• Linearity Example 1: Standard Evaluation, specimens excluded from the linear range Linearity demonstrated for LN-01 to LN-05 only
• Linearity Example 1: Standard Evaluation, specimens excluded from the linear range
• Example 1: Troubleshooting • Does the linear range cover the AMR? If the high specimen is above the AMR, did you dilute? Was the dilution protocol followed? • If the linear range does not cover the AMR, then there may be problems with reagents, specimen handling, or the test system • Check QC, PT, calibration data • Address identified problems and re-run linearity • Consider adjusting AMR to cover the linear range
• Example 2: Extended Range Evaluation, extended range sample not linear
• Example 2: Extended Range Evaluation, extended range sample not linear
• Example 2: Troubleshooting • Confirm that the linear range covers the AMR • Check the dilution protocol. Did you dilute? Dilution can introduce variability • If the linear range does not cover the AMR, then there may be problems with reagents, specimen handling, or the test system
• Example 3: Standard Evaluation, nonlinear data (Best appreciated with graphical data)
• Example 3: Standard Evaluation, nonlinear data
• Example 3: Troubleshooting • Is the peer group generally linear? • If the peer group is generally linear then there may be problems with specimen handling or the test system • Review QC, calibration, PT data • Eliminate specimen or reagent handling errors • Diagnose fix any identified test system failures • Re-run linearity study
• Example 4: Dilution Validation Evaluation, nonlinear trend in non-diluted samples
• Example 4: Troubleshooting • (Because non-diluted samples are nonlinear, no evaluation of the diluted sample is performed • Is the peer group generally linear? • If the peer group is generally linear then there may be problems with specimen handling or the test system • Review QC, calibration, PT data • Eliminate specimen or reagent handling errors • Diagnose fix any identified test system failures • Re-run linearity study
• Example 4: Standard Evaluation, large replicate imprecision
• Example 5: Standard Evaluation, large replicate imprecision
• Example 5: Troubleshooting • Pattern suggests pipetting problems which should be carefully investigated • Fix any identified test system failures • Re-run linearity study
• Example 6: Standard Evaluation, only difference plot shows replicate imprecision (similar problem to Example 5)
• Example 6: Standard Evaluation, only difference plot shows replicate imprecision (similar problem to Example 5) Looks OK?
• Example 6: Troubleshooting • Pattern suggests pipetting problems which should be carefully investigated • Fix any identified test system failures • Re-run linearity study
• Example 7: Standard Evaluation, Imprecision from poor fit
• Example 7: Standard Evaluation, Imprecision from poor fit Here, the term “imprecision” refers to the inability to find a subset of data displaying linearity with adequate precision. Points zigzag around the best-fit line.
• Example 7:Troubleshooting • Unusual pattern, may be difficult to diagnose • Is the peer group linear? If not, call Customer Service • Highest value may be diluted and a dilution error is masking loss of recovery at the high end of the AMR • Specimen handling problems may cause this
• Example 8: Extended Range Evaluation, Imprecise in non-extended range, extended range sample(s) not evaluated
• Example 8: Extended Range Evaluation, Imprecise in non-extended range, extended range sample(s) not evaluated Large gap indicates extended range Imprecise. E.R. specimen not evaluated
• LN-24 • Creatinine linearity survey • Uses human serum spiked with pure crystalline creatinine • Accuracy assigned values (NIST) allow for accuracy based grading in addition to calibration verification and linearity checks
• Case 1. Calibration Verification (AST)
• Linearity study show the data were linear
• Case 2. Calibration Verification (calcium)
• Case 3. Calibration Verification
• Case 4. Linearity (testosterone)
• Executive Summary Page from CVL Survey
• SOME CAP CHECKLIST QUESTIONS
• CHM. 13000 • Are calibration procedures for each method adequate, and are the calibration results documented?
• CHM.13100 • Are high quality materials with method- and matrix-appropriate target values used for calibration and calibration verification whenever possible?
• CHM.13125 • Are all calibration materials used for non- FDA cleared assays documented as to quality? • NOTE: Standards used to prepare calibrators for non-FDA-cleared assays require certificates of purity from the vendor, or a check on purity as part of the initial assay validation process. The laboratory should document the accuracy of a new lot of calibrators by checking the new lot against the current lot.
• CHM.13400 • Are criteria established for frequency of recalibration or calibration verification, and the acceptability of results? • • NOTE: Criteria typically include: • At changes of reagent lots for chemically or physically active or critical components, unless the laboratory can demonstrate that the use of different lots does not affect the accuracy of patient/client test results and the range used to report patient/client test data • QC fails to meet established criteria • After major maintenance or service • When recommended by the manufacturer • At least every 6 months
• CHM.13500 • Is the method system recalibrated when calibration verification fails to meet the established criteria of the laboratory?
• CHM.13600 • Is validation of the analytical measurement range (AMR) performed with matrix- appropriate materials which include the low, mid and high range of the AMR, and is the process documented?
• CHM.13700 • Are criteria established for validating the analytical measurement range, and is compliance documented?
• GEN.42085 • Is the reportable range verified/established for each analytic procedure before implementation? • NOTE: The reportable range includes all results that may be reliably reported, and embraces two types of ranges: • The ANALYTICAL MEASUREMENT RANGE (AMR) is the range of analyte values that a method can directly measure on the specimen without any dilution, concentration, or other pretreatment not part of the usual assay process • The CLINICALLY REPORTABLE RANGE (CRR) is the range of analyte values that a method can measure, allowing for specimen dilution, concentration, or other pretreatment used to extend the direct analytical measurement range
• Summary of Topics Covered • Calibration Verification • Linearity • CVL Program at the CAP • Examples and Troubleshooting • Cases • Discussion of Selected CAP Checklist Questions