The document discusses strategies for developing a reimbursement case for molecular diagnostic tests early in development. It emphasizes designing analytic and clinical validity studies to demonstrate test accuracy and association with clinical conditions. Clinical utility studies should show how test results impact patient management, outcomes, and healthcare costs. Randomized controlled trials provide the strongest evidence but alternative study designs like prospective observational studies and decision modeling may also support reimbursement. The goal is to generate evidence of a test's medical necessity from the intended patient population to achieve coverage and payment.
One size does not fit all unique study management challenges for diagnostic ...Lyssa Friedman
When partnering with a CRO, diagnostic company needs are different from those of pharmaceutical and device companies. This presentation was delivered at Outsourcing in Clinical Trials West Coast 2015, February 3-4, Burlingame, California.
Emerging diagnostic technologies proving the clinical application through g...Lyssa Friedman
Next Generation Sequencing is an exciting new technology for diagnostics companies. But is it right for all products and for all companies? This presentation was delivered via Webinar for a IVD audience for Q1 Productions, March 25, 2014.
Strategies for Considerations Requirement Sample Size in Different Clinical T...IJMREMJournal
-------------------------------------------------------ABSTRACT ---------------------------------------------------
Usually the main problem face any investigation it how to determent a sample size, however, some
considerations required in sample size to conduct the efficacy and make realistic well-researched before began
study. This study aimed to determine the maximum possible sample size at different phases of clinical trials and
attempt to achieve the best accuracy of the results. To achieve that the maximum sample size in different phases
we found that the maximum sample size of phase I was (75) relies on largest response rate 20% and the minimal
clinically important difference (MCID) 15%, and because the participants are healthy often that means 15%
enough to show positive results of the transition to the second phase. for the phase II clinical trials; the
maximum sample size was (388) depend on the error 5% and largest response rate 50% when the response rate
should not be less than 20% according to the design used in this phase. Depend on the endpoint and hazard
ratio in phase III clinical trials when the probability of survival of the treatment group equal to median of the
probability of survival 50% we found that the maximum sample size (4796). For the phase IV the maximum
sample size in different phases of clinical trials does not affect whatever the large of the population size and
remains constant as large as possible size.
Protocol Design & Development: What You Need to Know to Ensure a Successful S...Brook White, PMP
Solid protocol design is critical to clinical development. No matter how well executed a clinical study is, if the underlying design is flawed, it wasn’t worth doing. In this presentation, Dr. David Shoemaker, SVP R&D, and Dr. Karen Kesler, AVP Operations, will walk through the process of developing a protocol, explain the major considerations, and point out common mistakes and challenges.
One size does not fit all unique study management challenges for diagnostic ...Lyssa Friedman
When partnering with a CRO, diagnostic company needs are different from those of pharmaceutical and device companies. This presentation was delivered at Outsourcing in Clinical Trials West Coast 2015, February 3-4, Burlingame, California.
Emerging diagnostic technologies proving the clinical application through g...Lyssa Friedman
Next Generation Sequencing is an exciting new technology for diagnostics companies. But is it right for all products and for all companies? This presentation was delivered via Webinar for a IVD audience for Q1 Productions, March 25, 2014.
Strategies for Considerations Requirement Sample Size in Different Clinical T...IJMREMJournal
-------------------------------------------------------ABSTRACT ---------------------------------------------------
Usually the main problem face any investigation it how to determent a sample size, however, some
considerations required in sample size to conduct the efficacy and make realistic well-researched before began
study. This study aimed to determine the maximum possible sample size at different phases of clinical trials and
attempt to achieve the best accuracy of the results. To achieve that the maximum sample size in different phases
we found that the maximum sample size of phase I was (75) relies on largest response rate 20% and the minimal
clinically important difference (MCID) 15%, and because the participants are healthy often that means 15%
enough to show positive results of the transition to the second phase. for the phase II clinical trials; the
maximum sample size was (388) depend on the error 5% and largest response rate 50% when the response rate
should not be less than 20% according to the design used in this phase. Depend on the endpoint and hazard
ratio in phase III clinical trials when the probability of survival of the treatment group equal to median of the
probability of survival 50% we found that the maximum sample size (4796). For the phase IV the maximum
sample size in different phases of clinical trials does not affect whatever the large of the population size and
remains constant as large as possible size.
Protocol Design & Development: What You Need to Know to Ensure a Successful S...Brook White, PMP
Solid protocol design is critical to clinical development. No matter how well executed a clinical study is, if the underlying design is flawed, it wasn’t worth doing. In this presentation, Dr. David Shoemaker, SVP R&D, and Dr. Karen Kesler, AVP Operations, will walk through the process of developing a protocol, explain the major considerations, and point out common mistakes and challenges.
Visioning the Next Decade: NIPTE-FDA CollaborationAjaz Hussain
NIPTE Seminar at US FDA, 16 March 2016.
QBR as an Organizing Principle for the Proposed NIPTE Center of Excellence for Pharmaceutical Formulations (CEPF)
We are defining the problem too narrowly. Our paradigm of pharmaceutical quality sifted long-ago. We have harmonized on a regulatory methodology for QbD (e.g., ICH Q8). However, with the prevailing ontological gaps (for example as illustrated in the continuing challenges posed with the current FDA’s Inactive Ingredient Database) - How good are the scientific explanations in regulatory submissions? Is quality risk-assessment - metaphysical or an epistemological category?
Improving Inclusion/Exclusion Criteria for Clinical TrialsBrook White, PMP
Adherence to inclusion and exclusion criteria is essential to the successful execution of a clinical trial. Deviations from these criteria must be avoided because they can jeopardize scientific integrity, regulatory acceptability, or the safety of subjects in the study. This presentation provides suggestions and advice on formulating inclusion and exclusion criteria to enhance the quality of subject selection, minimize protocol violations, and avoid protocol amendments.
Educating the Next Generation Pharmacist for Industry. The Panjab University ...Ajaz Hussain
The Panjab University Pharmaceutical Science Oration 2014: Educating the Next Generation Pharmacist for Industry.
“The dream begins with a teacher who believes in you, who tugs and pushes and leads you to the next plateau, sometimes poking you with a sharp stick called ‘truth’.“
Plato, the Republic
What are the most influential ideas, concepts, and developments introduced by ‘pharmaceutical scientists’ over the last 50 years?
How have these ideas/concepts introduced into practice?
How can we improve?
Tricks of the Trade: Patient Recruitment & Retention for Different Study TypesImperial CRS
In efforts to raise the bar for medical advancement, clinical trials are growing increasingly complex. This complexity, more often than not, leads to costly delays in enrollment. In this ebook, we'll take a look at 4 case studies for different study types, and examine the unique factors to consider during planning.
Get Your Development Program Started on the Right FootBrook White, PMP
You think you have a potential pharmaceutical or biotechnology product based on animal or in vitro data—what is the next step? Two documents you need at an early stage are the Target Product Profile (TPP) which defines expectations for your potential medicine and an Integrated Product Development Plan (IPDP) which describes the activities required through approval of your marketing application.
Feasibility Solutions to Clinical Trial Nightmaresjbarag
Slow patient recruitment and poor retention cause recurrent nightmares and perpetual problems often resulting in missing recruitment milestones. The cost of these delays represents hundreds of thousands of dollars for drug and device developers. By recognizing this issue, early detailed feasibility can provide planning and contingency solutions that are focused on reducing the impact of delayed recruitment. Furthermore understanding what motivates investigators and patients to actively participate in clinical studies and how patient recruitment strategies and materials can support all stakeholders to complete studies on time are critical aspects of clinical study delivery planning.
During this presentation, an experienced Premier Research feasibility and patient recruitment specialist, reviewed feasibility approaches to address protocol evaluation as well as addressed influences on country selection, site distribution and patient recruitment strategies to provide for more effective clinical trial planning and conduct.
For more information, go to http://www.premier-research.com.
Need for an Integrated approach to Formulation Research and Knowledge ManagementAjaz Hussain
1. Confidence in Generics: Need for an Integrated
approach to Formulation Research and Knowledge
Management (Ajaz Hussain)
2. Mechanism for an integrated approach to Formulation
Research, Knowledge Management, & Knowledge
sharing with FDA & Industry (Steve Byrn)
3. Integrated approach for evolving standards for
formulation design - case example NTI's (Ken Morris)
4. Integrated approach for evolving standard for analytical
characterization - case example excipient variability
(Eric Munson)
Visioning the Next Decade: NIPTE-FDA CollaborationAjaz Hussain
NIPTE Seminar at US FDA, 16 March 2016.
QBR as an Organizing Principle for the Proposed NIPTE Center of Excellence for Pharmaceutical Formulations (CEPF)
We are defining the problem too narrowly. Our paradigm of pharmaceutical quality sifted long-ago. We have harmonized on a regulatory methodology for QbD (e.g., ICH Q8). However, with the prevailing ontological gaps (for example as illustrated in the continuing challenges posed with the current FDA’s Inactive Ingredient Database) - How good are the scientific explanations in regulatory submissions? Is quality risk-assessment - metaphysical or an epistemological category?
Improving Inclusion/Exclusion Criteria for Clinical TrialsBrook White, PMP
Adherence to inclusion and exclusion criteria is essential to the successful execution of a clinical trial. Deviations from these criteria must be avoided because they can jeopardize scientific integrity, regulatory acceptability, or the safety of subjects in the study. This presentation provides suggestions and advice on formulating inclusion and exclusion criteria to enhance the quality of subject selection, minimize protocol violations, and avoid protocol amendments.
Educating the Next Generation Pharmacist for Industry. The Panjab University ...Ajaz Hussain
The Panjab University Pharmaceutical Science Oration 2014: Educating the Next Generation Pharmacist for Industry.
“The dream begins with a teacher who believes in you, who tugs and pushes and leads you to the next plateau, sometimes poking you with a sharp stick called ‘truth’.“
Plato, the Republic
What are the most influential ideas, concepts, and developments introduced by ‘pharmaceutical scientists’ over the last 50 years?
How have these ideas/concepts introduced into practice?
How can we improve?
Tricks of the Trade: Patient Recruitment & Retention for Different Study TypesImperial CRS
In efforts to raise the bar for medical advancement, clinical trials are growing increasingly complex. This complexity, more often than not, leads to costly delays in enrollment. In this ebook, we'll take a look at 4 case studies for different study types, and examine the unique factors to consider during planning.
Get Your Development Program Started on the Right FootBrook White, PMP
You think you have a potential pharmaceutical or biotechnology product based on animal or in vitro data—what is the next step? Two documents you need at an early stage are the Target Product Profile (TPP) which defines expectations for your potential medicine and an Integrated Product Development Plan (IPDP) which describes the activities required through approval of your marketing application.
Feasibility Solutions to Clinical Trial Nightmaresjbarag
Slow patient recruitment and poor retention cause recurrent nightmares and perpetual problems often resulting in missing recruitment milestones. The cost of these delays represents hundreds of thousands of dollars for drug and device developers. By recognizing this issue, early detailed feasibility can provide planning and contingency solutions that are focused on reducing the impact of delayed recruitment. Furthermore understanding what motivates investigators and patients to actively participate in clinical studies and how patient recruitment strategies and materials can support all stakeholders to complete studies on time are critical aspects of clinical study delivery planning.
During this presentation, an experienced Premier Research feasibility and patient recruitment specialist, reviewed feasibility approaches to address protocol evaluation as well as addressed influences on country selection, site distribution and patient recruitment strategies to provide for more effective clinical trial planning and conduct.
For more information, go to http://www.premier-research.com.
Need for an Integrated approach to Formulation Research and Knowledge ManagementAjaz Hussain
1. Confidence in Generics: Need for an Integrated
approach to Formulation Research and Knowledge
Management (Ajaz Hussain)
2. Mechanism for an integrated approach to Formulation
Research, Knowledge Management, & Knowledge
sharing with FDA & Industry (Steve Byrn)
3. Integrated approach for evolving standards for
formulation design - case example NTI's (Ken Morris)
4. Integrated approach for evolving standard for analytical
characterization - case example excipient variability
(Eric Munson)
IMS Health Clinical Trial Optimization SolutionsQuintilesIMS
IMS Health's Linda T. Drumright, general manager, Clinical Trial Optimization Solutions presents at the 3rd Annual Patient Recruitment & Retention Summit 2014 - San Francisco, CA
Big data, RWE and AI in Clinical Trials made simpleHadas Jacoby
Technology is slowly but surely penetrating the healthcare industry in general and the clinical trials sector in particular. New and advanced solutions offer a variety of possibilities aimed to both improving existing processes and creating new and more efficient ones. And on top of all stands the desire to make clinical trials more patient centric.
In all of this, even though some of the technologies have yet to mature enough to meet the high quality standards necessary, it is important to know them and begin imagining the promise they hold for clinical trials.
THE NEED FOR EVIDENCE-BASED PRACTICE
STEPS OF EVIDENCE-BASED PRACTICE
PICOT FORMAT IN EBP
RATING SYSTEM FOR THE HIERARCHY OF EVIDENCE: QUANTITATIVE QUESTIONS
ELEMENTS OF EVIDENCE-BASED ARTICLES
INTEGRATE THE EVIDENCE
EVALUATE THE OUTCOMES OF THE PRACTICE DECISION OR CHANGE
COMMUNICATE THE OUTCOMES OF THE EVIDENCE-BASED PRACTICE DECISION
SUSTAIN KNOWLEDGE USE
NURSING RESEARCH
TRANSLATION RESEARCH
5 PHASES OF TRANSLATION RESEARCH
OUTCOMES RESEARCH
SCIENTIFIC METHOD
CHARACTERISTICS OF SCIENTIFIC RESEARCH
NURSING AND THE SCIENTIFIC APPROACH
TYPES OF RESEARCH
TYPES OF RESEARCH APPROACH
RESEARCH PROCESS
RIGHTS OF HUMAN SUBJECT
COMPARISON OF STEPS OF THE NURSING PROCESS WITH THE RESEARCH PROCESS
Performance Improvement
Performance Improvement Programs
EXAMPLES OF PERFORMANCE IMPROVEMENT MODELS
THE RELATIONSHIP BETWEEN EBP, RESEARCH, AND PERFORMANCE IMPROVEMENT
SIMILARITIES AND DIFFERENCES AMONG EVIDENCE-BASED PRACTICE, RESEARCH, AND PERFORMANCE IMPROVEMENT
KEY ELEMENTS
MedicReS Winter School 2017 Vienna - Importance of Selection of Outcomes - Ma...MedicReS
Importance of Selection of Outcomes and Covariates in Comparative Effectiveness of Cancer ...
International Conference Good Biostatistical and Publication Practice in Cancer Research with “Real Work Data”
February 13-14th, Vienna
Mariana Chavez Mac GregorMD, MSc.
Assistant Professor, Health Services Research Department
Breast Medical Oncology Department
Evidence- based periodontology is a bridge from all the available literature to clinical practice. It is a tool which can be used for decision making from available evidence during clinical practice.It should be scientifically sound and patient focussed.
Methods for Observational Comparative Effectiveness Research on Healthcare De...Marion Sills
Research Objective: The SAFTINet project was funded by the AHRQ to build a distributed network of existing clinical and claims data that would support comparative effectiveness research (CER), with a focus on underserved populations and healthcare delivery system (HDS) characteristics. Observational research methods are appropriate, but require detailed protocols with a priori hypotheses and analytic plans. SAFTINet research specifically concerns the effects of a discrete set of HDS features (those often included in Patient-Centered Medical Home (PCMH) models) on health outcomes for primary care patients with asthma, hypertension, and hypercholesterolemia. Our objective is to present a description of this study’s measurement challenges, and to specify a priori hypotheses, analytic strategies, and plans for addressing bias and confounding for our asthma cohorts.
Study Design: An observational, longitudinal cohort study of primary care patients with asthma, with both secondary use of existing clinical and claims data and primary data collection for HDS features and patient- reported outcomes.
Population Studied: Our sample consists of 59 primary care practices in 5 healthcare organizations in Colorado, Utah and Tennessee; all practices serve underserved populations. These practices care for about 275,000 patients per year, of whom an estimated 22,000 have a diagnosis of asthma.
Principal Findings: We will present the processes used to define and measure the HDS features, covariates and asthma outcomes, along with planned analysis. Challenges include valid measurement of a multi-faceted HDS “exposure” variable, the inability to identify exposure onset, and the non-dichotomous nature of HDS characteristics. To measure HDS characteristics, we created a practice-level survey assessing 9 PCMH domains, including care coordination, specialty care and mental health integration, and patient-centeredness, as well as asthma-specific HDS characteristics (e.g., the use of asthma registries). Asthma outcomes included (1) those available as a result of routine electronic documentation of clinical care and claims administration (utilization indicative of an exacerbation), and (2) patient reported outcomes tools (Asthma Control Test). We used directed acyclic graphs to identify potential confounders of the relationship between HDS characteristics and asthma control, as well as other potential biases. The analytic plan is based on linear mixed effects models. Perspectives of the CER team, the technology team and the community engagement group were considered in the operationalization of all variables.
Conclusions: The design of rigorous observational CER observational CER should recognize the need for an intense planning phase. In accordance with good practice guidance for observational studies, an important component of the planning phase is to disseminate and obtain feedback on the research design in advance of its conduct.
Pine Biotech - a company that merges big -omics data analysis with clinical care and precision applications for Real World Evidence: research & development of new targets and therapeutics, stratified clinical trials, and development of biomarkers for early detection and companion diagnostics. We want to improve patient outcomes and provide tools for researchers and clinicians to have an impact on healthcare.
Similar to Developing the Reimbursement Story 2016-03-10 (20)
1. Developing the Reimbursement Story:
It’s Never Too Early
Commercialization of Molecular Diagnostics
23rd International Molecular Medicine Tri-Conference
March 10, 2016
1
2. For today’s discussion
March 10, 2016 Lyssa Friedman 2
ü Study design for technology assessment and
reimbursement dossier
ü Framework
Centers for Medicare and Medicaid Services (CMS)
Centers for Disease Control EGAPP Initiative/ACCE Model
ü LDTs/IVDs
!Companion diagnostics
!Regulatory trends
!Coding
!Pricing
EGAPP: Evaluation of Genomic Applications in Practice and Prevention
ACCE: Analytical validity, Clinical validity, Clinical utility, Ethical/legal/social implications
3. Achieving success
March 10, 2016 Lyssa Friedman 3
Novel
biomarkers
Robust assay
Clinical
relevance
Excellent
test
performance
Well
characterized
samples
• Test the market• Test the market
• Understand current practice/physician behavior
• Test the market
• Understand current practice/physician behavior
• Achieve literacy in applicable clinical guidelines
• Test the market
• Understand current practice/physician behavior
• Achieve literacy in applicable clinical guidelines
• Characterize the reimbursement strategy early
4. Why build the reimbursement case early?
March 10, 2016 Lyssa Friedman 4
Design studies
Plan sample accrual
Develop milestones
• Study completion
• Product launch
• Billing
Pricing / revenue / time to cash-positivity
Communicate with investors
Align with TPP / PRD / marketing strategy
TPP: Target Product Profile; PRD: Product Requirements Document
5. Align study design with commercial strategy
March 10, 2016 Lyssa Friedman 5
• Study design
• Test inputs (sample type,
requisition form)
• Test outputs (patient report)
• Market intelligence
• Commercial strategy
• Target product profile
• Product requirements
6. CMS pathway to reimbursement
March 10, 2016 Lyssa Friedman 6
Clinical
Utility
Analytic
Validity
Clinical
Validity
If needed:
Economic
Value
• Coverage
determination
• Non-coverage
determination
• Limited coverage
• Recommendation
to submit for
Coverage with
Data
Development
(CDD)
Palmetto GBA MolDX Program 2015
Technical
Assessment
• Publications
• Accepted for
publication
• Subject matter
expert white
papers
• Professional
association
guidance
• Abstracts
Publications
Unique Test
Identifier
(Code)
Registration
7. Technical Assessment
• Defined disorder/clinical setting
• Analytic validity
• Clinical validity
• Clinical utility
• ELSI safeguards
March 10, 2016 Lyssa Friedman 7
Centers for Disease Control Evaluation of Genomic
Applications in Practice and Prevention (CDC EGAPP)
• Independent, non-federal working group that assesses new tests and
publishes evidence reports
ACCE Model Process for Evaluating Genetic Tests
8. Building the reimbursement story
March 10, 2016 Lyssa Friedman 8
Set up why the test
matters
• Who tested/in what
circumstances
• Information provided that
is not available without
test
• Resulting actions
• Outcomes
• Affordability
• Medical necessity
Design and
execute studies
• Analytic validity
• Clinical validity
• Clinical utility
Generate evidence
• Study results
• Published data
• Abstracts
• Guidance from
professional societies
• Evidence-based
consensus reports
National Academies Planning Committee Roundtable on Translating Genomic-Based Research for Health, 2014
Frueh FW & Quinn B, 2014
9. Set up why the test matters
March 10, 2016 Lyssa Friedman 9
Set up why the test
matters
• Who tested/in what
circumstances
• Information provided that
is not available without
test
• Resulting actions
• Outcomes
• Affordability
• Medical necessity
Design and
execute studies
Generate evidence
Frueh FW & Quinn B, 2014
10. CMS: Demonstrate test is medically necessary
Falls within a defined Medicare benefit category
Not excluded from coverage by statute, regulation, National Coverage
Determination (NCD), or Local Coverage Determination (LCD)
Reasonable and necessary in the diagnosis or treatment of an illness or injury,
or to rule out or confirm a suspected diagnosis based on patient has sign(s)
and/or symptom(s)
Ordered by a treating physician
Provides data that will be used to manage a specific medical condition
Excludes investigational services
March 10, 2016 Lyssa Friedman 10
National Academies Planning Committee Roundtable on Translating Genomic-Based Research for Health, 2014
cms.gov
11. Define the clinical disorder
Specific condition
Clinical findings defining the condition
Clinical setting in which test will be performed
Tests currently in use for the condition
How patients are identified/screened to be determined appropriate for use
of the test
Stand-alone vs one of a series of tests
If one of a series, situations when all vs only some of tests are performed
March 10, 2016 Lyssa Friedman 11
CDC ACCE Model Process for Evaluating Genetic Tests
12. Design and execute studies
March 10, 2016 Lyssa Friedman 12
Set up why the test
matters
Design and
execute studies
• Analytic validity
• Clinical validity
• Clinical utility
Generate evidence
13. Design and execute studies
March 10, 2016 Lyssa Friedman 13
• Well characterized samples – clinically and analytically
• Cleanly defined standard use with samples from that population
• Clinical relevance verified by community, literature, guidelines
• Studies that demonstrate test is medically necessary and actionable
• Aligned with TPP, PRD, product launch/marketing strategies
Analytic Validity Clinical Validity Clinical Utility
How stable and robust is
the assay?
How reliably does the test
correlate to the clinical
condition?
What difference does the
test make?
• Accuracy and reliability of
measuring analyte
• Accuracy of ability to
diagnose, predict or
measure clinical condition
• Ability to change patient
management/outcomes
• Impact on cost of care
14. Analytic Validity
Test qualitative vs quantitative
Frequency of test-positivity when mutation/biomarkers present - sensitivity
Frequency of test-negativity when mutation/biomarkers not present - specificity
QC program
Repeated measurements on specimens
Within- and between-laboratory precision
Method for performing confirmatory testing to resolve inaccurate results in a timely manner
Range of specimens tested
Frequency of failure to deliver usable result
Methods for assessing from a clinical perspective the “clinical cost” of delivering a wrong result
and degree of allowable error
Similarity of results across laboratories and technologies, if applicable
Inputs (test requisition, specimens) and outputs (patient report) aligned with commercial plan
March 10, 2016 Lyssa Friedman 14
CDC ACCE Model Process for Evaluating Genetic Tests
Analytic Validity
How stable and robust
is the assay?
• Accuracy/reliability of
measuring analyt
15. Analytic Validation Studies
Validation Element Measures
Accuracy Method comparison, e.g., gold standard, target values
Specimen types
Matrix comparisons
Analytic sensitivity Limit of detection
Limits of quantitation, upper and lower, including analytically measurable and clinically
reportable ranges
Linearity and reportable range
Minimum input quantity and quality
Minimum tumor content
Analytic specificity Primer and probe specificity
Interfering substances
Precision Repeatability – single operator, instrument, lot, day, run
Intermediate precision – multiple operators, instruments, lots, days, runs within a lab
Reproducibility – multiple labs
Lot-to-lot reproducibility – multiple reagent, calibrator, controls lots
Reagent stability Closed – shelf life
Open – in use
Freeze-thaw
Reference intervals Specimens from healthy subjects in intended use population
Sample stability Shipping
Primary and intermediate samples (e.g., extracted RNA or DNA)
Freeze-thaw
Software (algorithm) Verification and validation
March 10, 2016 Lyssa Friedman 15
Palmetto GBA MolDX Program 2015; clsi.org
Analytic Validity
How stable and robust
is the assay?
• Accuracy/reliability of
measuring analyt
16. Clinical Validity
Demonstrate association between test result and clinical condition of
interest
Study population = intended use population – that which is intended to
benefit from decision guided by the test result
Obtain sufficient prior evidence from the intended use population from early
validation studies
Select appropriate gold standard or reference
Define terminology and concepts used
Provide measures of certainty, e.g., 95% confidence intervals
March 10, 2016 Lyssa Friedman 16
Center for Medical Technology Policy, 2013
Clinical Validity
How reliably does the test
correlate to the clinical
condition?
• Accuracy of ability to
diagnose, predict or
measure clinical condition
17. Clinical Validity, cont’d
Report strength of association between test and disease state using metrics
most useful to clinicians
Apply appropriate weight to false negatives vs false positives based on clinical
significance – optimize appropriately for NPV vs PPV
Use appropriate reporting standards
• Binary test: sensitivity, specificity, positive and negative predictive values
• Continuous variable (e.g., risk score): select threshold or cut-off to generate
binary score; when clinical outcome is binary, receiver operating
characteristic (ROC) curves can be used
• Continuous or time-to-event variable: regression methods to model
relationship between discrete or continuous test result and outcome
• Predictive marker: use appropriate control group
March 10, 2016 Lyssa Friedman 17
Center for Medical Technology Policy, 2013
Clinical Validity
How reliably does the test
correlate to the clinical
condition?
• Accuracy of ability to
diagnose, predict or
measure clinical condition
18. Sensitivity – frequency of test positive when disorder present
Specificity – frequency of test negative when disorder not present
Methods to resolve clinical false positive results in timely manner
Prevalence of disorder in the setting
Adequate validation on all populations to which it may be offered
Positive and negative predictive values (PPV, NPV)
Genotype/phenotype relationships
March 10, 2016 Lyssa Friedman 18
CDC ACCE Model Process for Evaluating Genetic Tests
Clinical Validity, cont’d
Clinical Validity
How reliably does the test
correlate to the clinical
condition?
• Accuracy of ability to
diagnose, predict or
measure clinical condition
19. Clinical Utility
Natural history of disorder
Impact of positive or negative test on patient care
Other diagnostic tests available
Measurable benefit or acceptable action as result of test with access to that
benefit/action
Documentation if test will be offered to socially vulnerable population
Quality assurance measures
Results of pilot trials
Health risk identified for follow-up testing and/or intervention
March 10, 2016 Lyssa Friedman 19
CDC ACCE Model Process for Evaluating Genetic Tests
Clinical Utility
What difference does the
test make?
• Ability to change patient
management/outcomes
• Impact on cost of care
20. Clinical Utility, cont’d
Financial costs associated with testing
Economic benefits resulting from testing
Facilities/personnel available
Educational materials validated and available
Informed consent requirements
Methods for long term monitoring
Guidelines for evaluating program performance
March 10, 2016 Lyssa Friedman 20
CDC ACCE Model Process for Evaluating Genetic Tests
Clinical Utility
What difference does the
test make?
• Ability to change patient
management/outcomes
• Impact on cost of care
21. Considerations for Clinical Utility
Inability to demonstrate clinical utility most cited reason for failure to obtain coverage
Additional pitfalls
• Poorly defined population
• Lack of evidence-based decisions
• Coding issues
• Lack of evidence to support medical necessity
If outcomes hard to capture, measure clinical behavior change as proximate to outcomes change
Start clinical utility studies early
Learn from others’ successes and failures
May need to involve private payers and providers early
Patient influences: some patients do not get better even when clinical care is done correctly
Provider influences: Providers practice differently, and occasionally poorly
Study design: appropriate and representative
March 10, 2016 Lyssa Friedman 21
A well-designed clinical utility study demonstrates that the test changes patient
management decisions and, ultimately, patient outcomes
Clinical Utility
What difference does the
test make?
• Ability to change patient
management/outcomes
• Impact on cost of care
Jeter E, Palmetto GBA
Peabody et al. Am J Managed Care 2014
22. Clinical Utility Study Design/Levels of Evidence
March 10, 2016 Lyssa Friedman 22
Design Description Level of
Evidence
Randomized, Prospectively
Controlled Trials (RCT)
• Demonstrates therapeutic intervention based on test results leads to
statistically and clinical significant improvement in patient outcomes versus
standard of care
• End points widely considered clinically appropriate
3A
Prospective-Retrospective
Trials (PRT)
• Uses archived samples from a previously reported RCT to demonstrate that
treatment based on test is associated with improved outcomes in a
statistically and clinically significant manner versus standard of care
• Samples and study design sufficiently characterized and powered to permit
definition of the indications for test use
3B
Prospective Observational
Studies (POS)
• Enrolls patients prospectively in registry
• Treatment according to defined pathway using test as part of care plan
• Demonstrates statistically and clinically significant improvement in
healthcare outcomes versus standard of care
2A
Results from at least one study with Level of Evidence 2A or above must be submitted for acceptance
for full clinical review
Retrospective Data Modeling
(RDM)
• Uses complex data modeling using large data sets to determine statistically
and clinically significant improvement in outcomes when test used to guide
treatment versus standard of care
2B
Retrospective Observational
Studies (ROS)
• Does not stipulate treatment pathways based on test result 1
Preclinical Studies (PS) • Preclinical data only, or related studies 0
Palmetto GBA MolDX Program 2015
$
23. Randomized Controlled Trials (RCTs)
Biomarker-stratified design
• Classic clinical trial design
• All comers randomized
• Large sample size
Enrichment design
• All patients tested
• Only test-positive continue to treatment/management
• Smaller sample size
Biomarker-strategy design
• Randomization to arm that uses test to direct therapy or control arm that does not
Prospective-Retrospective Analysis of Previously Conducted RCTs
• RCT design using existing sample cohort
• Faster and less expensive
March 10, 2016 Lyssa Friedman 23
Frieidlin B, McShane LM, Korn EL. J Natl Cancer Inst 2010
Simon RM, Paik S, Hayes DF. J Natl Cancer Inst 2009
Center for Medical Technology Policy, 2013
Clinical Utility
What difference does the
test make?
• Ability to change patient
management/outcomes
• Impact on cost of care
Time and cost of RCTs often exceeds capacity of
small/emerging company
24. Non-RCT Study Design
Single-arm studies
• Test developed to be used with a FDA-approved drug
• Adequate archived samples not available to conduct prospective-retrospective trial
• Feasible to use response as endpoint
• Comparable response data in comparative cohort exists
Prospective observational studies
• Patient registries
• Multiple group, pretest/posttest design
• Acceptable if compelling rationale for not doing RCT is addressed
Decision-analytic modeling
Clinical Performance and Value (CPV) vignettes
• In silico simulation of clinical behavior change
• Builds clinical utility case early – completion of validation not required
• Validated against actual practice
• Not affected by patient variation
Cost-effectiveness
• Can be compelling for payers
• May not be sufficient for coverage but should be employed as part of reimbursement strategy
March 10, 2016 Lyssa Friedman 24
Peabody et al. Am J Managed Care 2014
Center for Medical Technology Policy, 2013
Clinical Utility
What difference does the
test make?
• Ability to change patient
management/outcomes
• Impact on cost of care
25. Criteria
• Strong evidence analytic and clinical validity
• Potentially significant but unproven potential of clinical utility
• Potential to affect the management of a serious, prevalent disease within Medicare population
Submit study plans to support safety, diagnostic performance and clinical utility (effect on
health outcomes)
Can be realistically completed in 3-4 years or less
Adequate resources to complete study
Study protected from conflicts of interest
Study development guided by key stakeholders (e.g., patients, clinicians, professional
societies)
Study registered on www.ClinicalTrials.gov prior to enrollment
Specified method and timing of public release of results
March 10, 2016 Lyssa Friedman 25
Palmetto GBA MolDX Program 2015
Coverage with Data Development (CDD)
Clinical Utility
What difference does the
test make?
• Ability to change patient
management/outcomes
• Impact on cost of care
26. Successes | Failures
March 10, 2016 Lyssa Friedman 26
Positive coverage Negative coverage
Genomic Health Oncotype DX
• Strong validation studies
• Multiple large, multicenter retrospective and
prospective studies looking at multiple
outcomes
Tethys PreDX diabetes test
• Lack of clinical utility data
• Inability to justify test as screening vs test for
at-risk population
Crescendo Biosciences Vectra DA (RA)
• CPV vignettes in a randomized controlled
study, provider assessments and
comparative review of cases to compare
clinical assessment with test score
Berkeley HeartLab LPA-Aspirin genotype test
• Lack of clinical utility data
• Failure to demonstrate target population
CardioDx Corus CAD
• Large longitudinal study showing significant
and clinically relevant utility combined with
retrospective chart review showing patient
management impact
Agendia subtyping profile tool, breast CA
• Insufficient clinical utility data
Peabody et al. Am J Managed Care 2014
27. Clinical Utility Strategy
March 10, 2016 Lyssa Friedman 27
Physician
surveys
Clinical utility
secondary
outcomes as
part of clinical
validation study
Clinical
Performance
and Value
Vignettes
Economic
modeling
Pre-launch
Build evidence
Prospective
Observational
Study
Cost
effectiveness
study
Post-launch
Higher level
of evidence
28. Results
March 10, 2016 Lyssa Friedman 28
Set up why the test
matters
Design and
execute studies
Generate evidence
Dossier
Technical
assessment
Coverage decision