• Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide
  • Demonstrate acceptable performance in practice
  • As part of our assessment of genetic tests we are developing pilot projects to help us figure out how to monitor genetic testing. It is a difficult issue because we can’t use a population-based approach as we do with birth defect surveillance for example. Birth defects are fairly easy to ascertain on a population-basis because most birth defects are identified in the newborn period and most babies are born in hospitals. Genetic testing is more difficult. We could get some information about who is being tested through, clinics, labs and physicians but we won’t have any denominator data.
  • Needed information? Indication for testing, demographics, family history Offered appropriately? Right population, clinical background, family hx Reporting? Are labs able to convey information – risks - clearly?
  • All screening laboratories should maintain a minimum of data including: General Laboratory Information completeness of demographic/pregnancy-related information on the requisition slip sample rejection and assay failure rates turn-around times software/report components and validation Specific Information on each woman/couple screened date of test geographic location (as possible) race/ethnicity (in broad categories) gestational age at sampling result of test for positive tests, the mutation identified Specific Information on each couple testing positive whether partner accepted testing if yes, partner’s result was genetic counseling offered/accepted was diagnostic testing offered/accepted results of diagnostic testing gestational age when diagnostic results were available pregnancy outcome (termination, SAb, liveborn)
  • Pre-analytic Indication for CFTR testing Completeness of patient information Sample rejection rate Analytic Number of mutations tested Assay and test failures Number of women screened (by race/ethnicity) Turn-around time Carrier frequencies (by race/ethnicity) Partner uptake rates Number and rate of carrier couples detected Number and rate of affected pregnancies Pregnancy outcome (with genotype) Methods of payment Post-analytic Components and standardization of reports Software used Impact on birth prevalence (by race/ethnicity) Patient/health care provider satisfaction Miscellaneous ethical, legal and social issues
  • CFF – nearly double the number of CF patient chromosomes available for analysis in 2000
  • I148T does not cause classical CF by itself 3199del6 not common enough to add to panel
  • Key questions here
  • Public-private partnerships / data collaborations Sentinel laboratories Providers & health plans (HMO Research Network) Purchasers & payers Organizations / networks (AHRQ PC-Based Research Networks, HRSA Community Health Center Program) Surveys Proficiency testing Regulatory – CAP, CMS, states Voluntary Public health surveillance mechanisms


  • 1. Post Implementation Data Collection & Monitoring Linda A. Bradley, PhD Office of Genomics & Disease Prevention, CDC Public Health Assessment of Genetic Tests for Screening & Prevention September 27, 2004
  • 2. Public health assessment of genetic testing
    • Transition from research to clinical practice
    • Post-implementation period
      • Demonstrate acceptable performance in practice
      • Assess implementation success and public health impact
    Evidence-based evaluation needed at two key points:
  • 3.
    • Data collection and monitoring in the post-implementation period
    • Confirm or update performance estimates
    • Assess public health impact – including quality, acceptability, utilization, and access
    • Document implementation issues
    • Assess fit with healthcare delivery systems
    • Resolve gaps in knowledge
  • 4. Monitoring genetic testing
    • We know very little about genetic testing in the United States
    • Minimal assessment of genetic testing in clinical or public health practice setting
      • Who is being tested?
      • Who is ordering testing?
      • Why?
      • Where is testing being done?
      • What methods / technologies are used?
  • 5. Long-term monitoring can answer questions like:
    • Are providers & patients properly educated? Satisfied with the process?
    • Is the quality of laboratory service adequate?
      • Are labs able to obtain needed information?
      • Is the panel of mutations appropriate?
      • Is the test being offered appropriately?
      • How are laboratory test results being reported?
    • Are performance expectations and pilot trial results being confirmed?
    • Are there problems with implementation?
  • 6.
    • What actions are being taken? Are they appropriate?
    • Is there a discernable impact on outcomes?
    • Are there issues with reimbursement? Access?
    • Are safeguards in place to deal with ethical, legal and social issues?
      • Have additional ELSI been identified?
    • Are program costs acceptable?
    • Should the process or guidelines be modified? Discontinued?
    Long-term monitoring can answer questions like:
  • 7. Prenatal CF screening via carrier testing
  • 8. Collecting Long-term Monitoring Information: Experience of Three CFTR Laboratories Strom et Data Collected al, 2001 Clinigene FBR All CFTR Tests 20,103 3,324 4,260 Screening Tests Unk 3,298 4,260 Carriers Identified Unk 132 153 Carrier Rate (uncorr) Unk 1 in 25 1 in 28 Partners Tested Unk 55 * 153 Positive Couples Unk 3 * 7 Prenatal Diagnosis Unk Unk 6 Fetus Affected Unk Unk 3 * Known to be under-ascertained
  • 9. Cystic fibrosis carrier testing
    • ACOG/ACMG did not address evaluation
    • No group charged with coordinating data collection
      • Laboratories  expense, time, increasing difficulty & no CLIA requirement
      • Providers/payers lack access through claims data
    • Anecdotal reports on implementation issues
      • Access to & understanding of guidelines
      • Problems with patient information & result reporting
      • Small number of labs routinely reported 5T
      • Difficult to quantify problems
    • Addressing performance in practice
      • Proficiency testing
      • Reports on mutation frequencies
  • 10. ACMG Carrier Screening Work Group
    • 2001 recommended panel of mutations & variants
      • Grody WW et al. Laboratory standards and guidelines for population-based cystic fibrosis carrier screening.
      • Mutation should be present in at least 0.1% of CF patient chromosomes
    • 2002 review initiated
      • Information collected by laboratories
        • > 400,000 individuals tested
      • CF Foundation patient mutation database
        • 42,737 CF patient chromosomes
      • Reports of provider experience
  • 11. ACMG Work Group Questions
    • Has the observed frequency of any CF mutations changed significantly since 1999 ?
    • Any mutation with prevalence <0.1% should be removed from screening panel  1078delT removed
    • Future decisions based on benefits & costs of incremental gain in performance
      • 6 mutations at frequencies 0.1-0.17%  + 0.77%
      • Weigh against potential increase in error rate & adaptability of current methods / platforms  No additions at this time
      • Local demographics may suggest need to add ethnic-specific mutations
    • Watson MS et al. Cystic Fibrosis (CF) Couple Carrier Screening: 2004 Revision of ACMG’s Mutation Panel. Genetics in Medicine , in press.
  • 12. ACMG Work Group Questions
    • Is the prevalence of CF mutations in the general population the same as predicted from the frequency in CF patients ?
    • I148T occurs 50-100x more frequently in the general population (0.05%)
      • CFTR genes in CF patients also have 3199del6 (<0.1%)
      • Vast majority with I148T do not have 3199del6  I148T removed
    • Watson MS et al. Cystic Fibrosis (CF) Couple Carrier Screening: 2004 Revision of ACMG’s Mutation Panel. Genetics in Medicine , in press.
  • 13. ACMG Work Group Questions
    • Evidence of consistent & recurring challenges with interpretation of some mutations in the panel ?
    • Complexity of association between R117H & 5T variant
    • Frequency of R117H-5T appreciable  Retain R117H & use 5T as reflex only when R117H is present
      • Needs educational effort to ensure proper implementation
    • Watson MS et al. Cystic Fibrosis (CF) Couple Carrier Screening: 2004 Revision of ACMG’s Mutation Panel. Genetics in Medicine , in press.
  • 14. Reliability of CF Laboratory Reports
    • European Quality Assessment Scheme
      • Routinely collects laboratory reports as part of external proficiency testing
      • Found that 31% of CF reports had errors
    • Suggests that it might be important to monitor CF reporting in the US
      • Tulane University/CDC
        • Assess the variability of reports for CF and factor V Leiden testing
        • Assess the usefulness various report formats in interpreting genetic test results
  • 15. BRCA1/2 testing for susceptibility to breast and ovarian cancer
    • What are expected performance characteristics?
    • Who is being tested?
      • Geography, demographics, personal/family history
      • Comparison with guidelines
      • Observed mutation rates
    • Who orders tests?
      • Geography, demographics, specialty
    • Who pays for tests?
    • Routine testing since 1996
    • Coordination not the issue
      • Technology and data proprietary
    • Motivation for service provider?
    • Other sources of data?
  • 16. So, how do we do it?
    • Think programmatically
      • Patient-provider & provider-lab communication
      • Sampling & testing
      • Reporting & result communication ± genetic counseling
      • Facilitating & documenting appropriate follow-up
    • Develop plans & partnerships
      • Who is responsible for collecting / evaluating data?
      • What are the key pieces of information to be collected? Measurements / quality indicators to be evaluated?
      • How will the evaluation be funded?
      • What are pre-established expectations / goals that results will be compared to?
  • 17.
    • Public-private partnerships / data collaborations
      • Laboratories
      • Providers & health plans
      • Purchasers & payers
      • Organizations / networks
    • Utilization surveys
      • Proficiency testing
      • Regulatory – CAP, CMS, states
      • Voluntary
    • Surveillance mechanisms
      • Disease registries (CF Foundation)
      • Cancer registries
  • 18. Long term goals
    • Standardization of data collection formats
    • Identification of effective quality indicators
    • Coordination of care
    • Development & funding of research agenda
    • Support review of current programs / guidelines / recommendations based on new information
    • Create an expectation among providers, payers, policy makers that a certain level of review will occur
  • 19. Evaluation of Genomic Applications in Practice and Prevention (EGAPP): A Three-Year Model Project
    • Utilize
      • historical recommendations for action
      • knowledge gained from ACCE project & other CDC initiatives
      • existing processes for evaluation and appraisal
      • international health technology assessment experience
    • Establish & evaluate a systematic & sustainable mechanism for pre- & post-market evaluation of genomic applications in the US
  • 20. Elements of effective evaluation
    • Systematic review & integration of data on analytic and clinical validity
    • Unbiased assessment of clinical utility in comparison with outcomes obtained in the absence of testing or using alternative tests
    • Identification of ELSI
    • Appropriate dissemination of evidence summaries, guidelines, & recommendations to target audiences
    • Post-implementation data collection and updating of the knowledge base
  • 21. EGAPP Working Group Independent Non-Federal Multidisciplinary
    • 10-12 experts
      • Health care
      • Public health
      • HuGE
      • Health technology assessment
    • In-person meetings
    Support & coordination provided by RTI International / CDC
  • 22. Roles of Working Group
    • Define analytic framework for EBR
      • Transparent process - provide clear linkage between evidence and recommendations
    • Engage stakeholders
    • Develop criteria, select and prioritize topics
    • Request EBR  RTI
    • Develop recommendations based on evidence
    • Consider strategies for post-implementation monitoring & data collection  RTI
    • Address QA & technical issues that arise with general implementation  Appropriate groups
    • Take part in evaluation  RTI / CDC
  • 23. Review & interpret data without suggesting policy Specific recommendations about use in primary care Product Ad hoc approach for extracting maximum information Structured approach for inclusion/exclusion Grading Quality of Evidence 44+ targeted questions on ACCE elements plus disorder/ setting Collect, analyze, summarize data using tables & graphics Broader focus – “first look” at all elements Analytic framework with key questions that link preventions with outcomes Outcome tables on benefits and harms Focus on clinical utility Methodology Evaluate genetic tests before transition into clinical practice Identify gaps in knowledge Assess merit of preventive measures (screening tests) Identify research agenda Goals ACCE Model USPSTF
  • 24. Stakeholders
    • Identify & engage
    • Needs assessment
      • Specific topics for immediate consideration by WG
      • Content and format of information needed and useful from their perspectives
    • Content experts
    • Involvement in developing informational messages for key target audiences
    Health care providers Consumers Professional organizations Policy makers Public health Industry / biotechnology Health care payers & purchasers Laboratories Regulatory groups
  • 25. EGAPP Health care providers Consumers Professional organizations Policy makers Public health Industry / biotechnology Health care payers & purchasers Laboratories Regulatory groups STAKEHOLDERS EGAPP Working Group Nonfederal Multidisciplinary Independent Disseminate Recommendations, Reports, to Audiences Consumers Providers Policy Makers Payers/ purchasers Refer for appraisal USPSTF/AHRQ Community Guide Other agencies Stakeholder input on topics/priorities Recommend Pilot Data Collection Projects Request Evidence Based- Reviews Evidence Center Systematic reviews Identify gaps & data needed RTI
  • 26. Acknowledgments
    • The ACCE project was supported by a cooperative agreement with the CDC, Office of Genomics and Disease Prevention (CCU319352)
    • The EGAPP project is supported by the CDC, Office of Genomics and Disease Prevention through a contract with RTI International
      • (200-01-00123 TO36)