2nd CUTEHeart Workshop Constantine Gatsosis Presentation

The link of diagnostic tests and patient
outcomes: methodologic challenges
April 2016 CUTEheart Workshop
Constantine Gatsonis
Department of Biostatistics and
Center for Statistical Sciences
Brown University School of Public Health

Outline
1. The problem: Broadly shared concern about the
effectiveness of modalities for diagnosis and prediction
2. The solutions, classical and new:
a. Randomized studies
b. Observational studies:
– Large databases: EMR, Health Insurance Claims
– Large registries: The National Oncology PET Registry
studies
– Combining registries with administrative and insurance
databases
3. Evaluation of effectiveness of diagnostics revisited

Diagnostic testing is receiving much attention
and skepticism
1 ‐ Specificity
0.0 0.2 0.4 0.6 0.8 1.0
0.00.20.40.60.81.0
Sensitivity

CMS to Trim Spending on Diagnostic Lab Tests
The Centers for Medicare and Medicaid Services appear to be executing a two‐step
strategy to shrink the $8 billion annual price tag of clinical diagnostic laboratory tests.
On Sept. 25, CMS released a final payment determination for the Clinical Laboratory Fee
Schedule that cuts payment by over 90 percent for some tests
October 2, 2015
The Health Care Value Transparency Movement and Its Implications for
Radiology Daniel J. Durand, MD et al, JACR 2015
….radiology, along with other “ancillary services,” has been a major focus of early
efforts to enhance consumer price transparency. By contrast, radiology as a field
remains in the “middle of the pack” with regard to quality transparency.
Addressing Overutilization in Medical Imaging
Hendee et al, Radiology 2010
Key forces influencing overutilization .. include payment mechanisms and financial
incentives; the practice behavior of referring physicians; self‐referral, including referral
for additional radiologic examinations; defensive medicine; missed educational
opportunities when inappropriate procedures are requested; patient expectations;
and duplicate imaging studies.

Tests and therapeutic interventions
Fundamentally tests provide information for use  in
selecting course of care.
Both long‐ and short‐term effects of tests materialize
in context  of available health care options, including
therapeutic interventions.
Not possible to define and measure test effects
outside the particular health care context in which the
test will be used.
However, oftentimes diagnosis may be ahead of
therapy:  e.g. DCIS in breast cancer, Amyloid plaque
testing in Alzheimer’s

Main questions in test evaluation
Accurate? • Accuracy in detection
• Accuracy in prediction
Affects
Care ?
Affects
Outcome?
Process of care:
• Dx thinking/decision making
• Tx thinking/decision making
Patient outcomes:
• Quality of life, satisfaction, cost
• Mortality, morbidity

The challenge for diagnosis and prediction
Dx information
Outcomes
How can we illuminate the complex path
The main pathway goes through therapy
THERAPY

CUTEheart WorkshopApril 2016
A heuristic argument
highlighting key constraints

R
Test A
Test B
Tx1
Tx1
Tx2
Tx2
Outcome
Outcome
Outcome
Outcome
+
+
‐
‐
Simple randomized design, comparing two tests

Difference in success rates between two arms:
D=(r1-r2)xpx(SensA – SensB)
r1 and r2 =  success rates for therapeutic interventions Tx1 and Tx2, when
performed on cases that have the clinical condition (irrespectively of
which test detected them).
p = prevalence of the clinical condition
Sens= test sensitivity   (Specificities are assumed equal above).
Typically, overall effect D between arms will be much
smaller than difference  in efficacies of therapies r1-r2
If r1-r2 = 0 (therapy not effective)   or SensA = SensB
then overall effect of test D=0.
Key observations:

R
CT, 3 annual
screens
Workup
Therapy
No further
intervention
Lung Ca.
Mortality+
+
‐
‐
Example: Lung Cancer Screening Trial
X‐ray , 3 annual
screens No further
intervention
Lung Ca.
Mortality
Lung Ca.
Mortality
Lung Ca.
Mortality
N=53546

A randomized study of strategies for
managing stable angina patients
CUTEheart Workshop
http://www.acrin.org/PROTOCOLSUMMARYTABLE/ACRIN4701RESCUE/tabid/747/Default.aspx
April 2016

Randomized designs in practice
Key design question: Randomization to test only or to strategy that
includes the test?
Example 1: CT Angiography for patients with stable angina.
– Randomize to anatomic (CTA) vs functional test (e.g. SPECT)
without specified followup (as done in the PROMISE trial) or
– Randomize to strategies linking test results to patient
management ( is patient candidate for optimal medical therapy
(OMT)?) (RESCUE trial)
Example 2: Preoperative MRI in breast cancer
– Randomize to MRI or no MRI before surgery, without algorithms for
use of MRI results (COMICE trial, )
– Randomize to MRI or no MRI, with algorithms to guide use of test
results (Alliance A011104 / ACRIN 6694, active trial)

A more elaborate design
-Paired tests
-Randomized
discordant
pairs
Outcome
Tx2
Outcome
A B
+ +
Tx1
Tx1
Tx2
Outcome
Outcome
Outcome
Tx1
- -
Tx2 Outcome
+ -
- +
R
R

Randomized designs in practice, cont.
Paired test designs
– Efficient but rarely adopted. MINDACT trial
is example.
– Ethical issues‐ can positive or negative test
results be ignored?
– Implementation difficulties. Strong
potential for non‐compliance.

Beyond clinical trials
• Observational studies
– Registry studies
– Electronic Medical Records
– Claims datasets
– Cohort studies
• Research synthesis
– Systematic reviews and meta‐analysis
– Modeling

Use of large databases (e.g. EMR, Claims)
General approach:
– Construct longitudinal record including test and
outcome information.
– Use methods for observational data to make
comparisons.
Recent examples:
– Coronary calcium study using Medicare claims.
– Testing of ER patients with chest pain study using
EMR data.

Outcomes After Coronary Artery Calcium and Other
Cardiovascular Biomarker Testing Among Asymptomatic
Medicare Beneficiaries
Jacqueline Baras Shreibati, MD, MS; Laurence C. Baker, PhD; Michael V. McConnell, MD, MSEE; Mark A.
Hlatky, MD
Effectiveness of CAC and hs-CRP testing
Circ Cardiovasc Imaging.
2014;7:655‐662.)
20% random sample of
Medicare beneficiaries
without CVD claims in
previous 6 mos.
Propensity score
matching of patients
receiving CAC and high
sensitivity C-reactive
protein testing
Compared utilization and
outcomes

Testing of ER patients with chest pain
>420K ER patients.
Records from 2011
Marketscan
Patients classified
into 5 testing
strategies
Outcomes compared
using regression
analysis
JAMA Intern Med.
2015;175(3):428‐436.

Limitations of Claims and EMR data
EMR and claims data typically deficient in key
information about
Context of the test (indication, filtering)
Test interpretation and findings
Clinical characteristics of patient
Subsequent diagnostic and therapeutic
decisions
Diagnostic
Procedure
OutcomesTHERAPY
Clinical
Context

information about
decisions
Diagnostic
Procedure
OutcomesTHERAPY
Clinical
Context
Defensive
medicine

information about
decisions
Diagnostic
Procedure
OutcomesTHERAPY
Clinical
Context
Defensive
medicine
Creative Coding

National Oncologic PET Registry:
A Nationwide Collaborative Program
Sponsored by
Managed by
Advisor
Goal:
Assess the effect of PET on referring physicians’ plans of
intended patient management.
• across a wide spectrum of cancer indications for PET, not
covered currently by the Medicare program,
•in relation to cancer‐type, indication, performance status,
physician’s role in management, and scan type
http://www.cancerpetregistry.org
Biostatistics Center at
Brown University

NOPR Accrual (Cases Completed/Business Day)
More than 200,000 patients registered

PET Changed Intended Management in 36.5% of Cases
Non-Treat Treat 23.2 31.6 28.6 29.2 28.3
Treat Non-Treat 7.9 7.9 7.5 9.7 8.2
Patients with
change post-PET 31.1 39.5 36.1 39.0 36.5
Hillner et al J Clin Onc 2008
Clinical Indication for PET Study (Percent)
Pre-Pet
Plan
Post-PET
Plan
Dx
n=5,616
Staging
n=6,464
Restaging
n=5,607
Recurrence
n=5,388
All
n=22,975
Treat Same 16.0 46.5 15.8 20.4 25.5
Non-Treat Same 52.9 14.0 48.0 40.7 37.9

Summary of NOPR Results (Before 2009 NCD)
Overall Impact on Patient Management
– Diagnosis, Staging, Restaging, Recurrence
– Data on 22,975 scans from May 8, 2006 – May 7, 2007
– J Clin Oncol 2008; 26:2155‐61
Impact on Patient Management by Cancer Type
– Confirmed Cancers
– Staging, Restaging, Recurrence
– Data on 40,863 scans from May 8, 2006 – May 7, 2008
– J Nucl Med 2008; 49:1928‐35
Treatment Monitoring
– Data on 10,447 scans from May 8, 2006 – Dec 31, 2007
– Cancer 2009:115:410‐18

Do NOPR findings agree with those from
• other studies or
• other sources of information on the same
participants
How does registry information compare to other sources?

Combining registries with administrative and
insurance databases
– Linking NOPR and CMS claims data
Registry
record
CMS claims,
longitudinal
record

NOPR strengths
Provided extensive, timely, “real world” data, that are
very difficult and expensive to collect in clinical trials.
Prospectively organized and controlled data collection
Registry data can be effectively linked to data from
other information sources
Results consonant with more tightly controlled studies
(later in presentation)
Supports the Coverage with Evidence Decisions
approach of CMS

NOPR weaknesses (long list!)
Evidence documents change in intended management,
not actual management
No evaluation of whether management changes were
appropriate.
No information on whether the use of PET improved
long‐term outcomes
How should PET should be used in the flow of patient
care.
Plus the concerns about generalizability and bias in
data obtained from registries

Imaging Amyloid Plaques (PIB-PET)
A harder case: Diagnosis ahead of therapy
18F-florbetaben (Piramal) March 2014
(Neuraceq)
18F-florbetapir (Avid/Eli Lilly) April 2012
(Amyvid)
18F-flutemetamol (GE) October 2013
(Vizamyl)

CMS National Coverage Decision
Insufficient evidence of clinical utility to justify coverage
of Aβ PET
Reimbursement would be considered under coverage with
evidence development (CED) in clinical studies designed to:
• Develop better treatments or prevention strategies
for AD
• Identify subpopulations at risk for developing AD
• Resolve clinically difficult differential diagnoses (e.g.,
frontotemporal dementia versus AD)
• Must demonstrate Aβ PET improves health outcomes (short‐
term outcomes related to changes in management as well as
longer‐term dementia outcomes)

Combining registries with administrative and
insurance databases
– Amyloid plaque imaging: Comparison of
• Registry of participants undergoing Dx, and
• Matched Controls identified from CMS claims data
?
? ?
?

IDEAS Steering Committee
Core Science Team
Gil Rabinovici, UCSF
Maria Carrillo, Alzheimer’s Association
Constantine Gatsonis, Brown U
Bruce Hillner, VCU
Barry Siegel, Wash U
Rachel Whitmer, Kaiser Permanente
Additional committee members
William Abbot, Piramal Imaging
Rosemarie Hakim, CMS
Meredith Johnson, GE Healthcare
Mark Mintun, Avid Radiopharmaceuticals
Don Rosen, ACR
ACR Operations
Charlie Apgar
Cynthia Olson
MITA representative :Terri Wilson
Consultant: Brian Carey, FoleyHoag LLP
Biostatistics Center (Brown U)
Ilana Gareen
Roee Gutman
Erin Greco
Lucy Hanna
Benjamin Herman
Rajesh Makineni
Scientific and logistical support
Jim Hendrix, Alzheimer’s Association
IDEAS-Study@acr.org
IDEAS-Study.org

IDEAS (Imaging Dementia—Evidence for Amyloid Scanning)
Two main goals:
Aim 1: To assess the impact of amyloid PET on the
management of patients meeting Appropriateness Use
Criteria
– This aim will be addressed using a registry of
participants undergoing amyloid PET.
Aim 2: To compare hospital admissions and emergency
room visits within 12 months in patients enrolled in the
registry to matched controls (no amyloid PET), to be
identified from CMS claims data.
– This aim will be addressed using linked registry and
CMS claims data.
https://clinicaltrials.gov/ct2/show/NCT02420756?term=IDEAS&rank=1

Sample size considerations
Aim 1 Hypothesis: Determine whether amyloid PET imaging will
lead to a ≥ 30% change between intended and actual patient
management within 90 days in a composite measure consisting of
the following:
a) AD drug therapy; b) Other drug therapy; or
c) Counseling about safety and future planning
N=11,050 for 80% power (separate analyses for dementia and MCI
subgroups)
Aim 2 Hypothesis: Determine whether amyloid PET is associated
with a ≥ 10% relative reduction in registry patients in comparison
to matched controls in the following endpoints: a) Inpatient
hospital admissions over 12 months. b) Emergency room visits over
12 months
N= 18,500 for 90% power (separate analyses for each endpoint)

In closing

(Can we assess) how effective is diagnosis in health care?
Current approaches can be enhanced:
Linking tests to downstream outcomes is a
challenging undertaking.
Large secondary databases can be used, but
may lack necessary information.
Registries offer flexibility but typically have
narrow focus.
Combination of information from several
sources holds promise.

Solutions to the Problem: Coverage with Requirement for
Data Collection through a National Registry or Registries
1) Require Standardization of High Quality Testing
2) Compare Back to Companion Diagnostics Where Available
3) Collect Outcomes: High‐level clinical outcomes need to be collected
and compared back to the new testing standard.
4) Attach Testing to Existing and Future Clinical Trials
5) Aggregate All Data, Nationally and Internationally
Editorial by ASCO

New realities in health care:
Integrated Systems, ACOs, etc
Comment in “Modern Health Care”
The CMS stance on PET is an artifact of fee‐for‐service medicine.
In a few years, as insurers in both the public and private sectors
gravitate toward episode‐based and bundled care, the risk of
using expensive and unproven technologies that don't improve
outcomes won't fall on payers. It will fall on the purchasers of
those expensive technologies which, for PET scanners, are mostly
hospitals.
Decision Memo for Positron Emission Tomography (NaF-18) to Identify Bone
Metastasis of Cancer (CAG-00065R)
CMS has concluded that the evidence is not sufficient to determine that the results of
NaF-18 PE imaging to identify bone metastases improve health outcomes of
beneficiaries with cancer. Therefore we have determined that this use is not
reasonable and necessary under §1862(a)(1)(A) of the Social Security Act.

New realities in health care:
Integrated Systems, ACOs, Bundled Care etc
The CMS stance on PET is an artifact of fee‐for‐service medicine. In a few years, as insurers in both
the public and private sectors gravitate toward episode‐based and bundled care, the risk of using
expensive and unproven technologies that don't improve outcomes won't fall on payers. It will fall
on the purchasers of those expensive technologies which, for PET scanners, are mostly hospitals.
• New incentives for health care systems to developmand
expand their own research efforts.
• Realistic hope for better data from more effective and
efficient information systems.
• Continuous, integrated, enterprise‐wide data collection
• Effectiveness questions posed from a “systemic” perspective
in contrast to the current la carte evaluation of components
of health care.
• Metrics and endpoints would be adapted to new realities.
• Validity and Generalizability of findings may be a challenge.

Thank you!

Key challenges
Best target population (age range?; MCI vs. atypical
dementia?)
No established clinical management algorithms
based on scan results (unlike situation for FDG‐PET
in cancer)
No good historical control data
Some patient‐centered outcomes will take quite
long to be detectable (slowing of functional decline
vs. avoiding futile Rx)

NOPR linked to CMS claims data
Approach
Construct health care utilization history using CMS
claims data
Cohort of 8640 consenting registry participants,
age 65+, enrolled in 2007‐2008 undergoing PET for
cancer restaging.
Extensive database work involved to identify
patterns of care and classify into post‐PET
treatment categories.
(Med Care 2013;
51: 361–367)

Australian Prospective Studies of Impact of PET
Agreement in Post-PET Plan and Actual Care (2003-2006)
Cancer Pts Centers Indication Change
in Plan
F/U
(mo.
)
Agree-
ment
Ovarian 90 3 SR 58.9 6 67.8
Esophagus 129 5 IS 38.0 12 53.2
Lymphoma 74 6 IS 34.0 6 74.3
Colorectal 93 4 SR 65.6 6 62.0
Colorectal 98 4 Resect
Hepatic
49.0 6 70.1
Head/Neck 71 3 IS 33.8 3 74.7
SR: Suspected Recurrence. IS: Initial Staging

Referring MD
requests amyloid
PET
Patient consent.
Visit 1
Pre‐PET
assessment and
form completed
Visit 2
PET done, interpreted
& reported.
MD communicates
results to participant
Visit 3
90‐day Post‐PET
Assessment and
form completed
Ongoing
patient
management
IDEAS Registry Workflow (main steps)

Additional material

Impact of PET Used for Treatment Monitoring
• Chemotherapy 82%, chemoRT 12%, RT 6%
• Ovarian, pancreas, NSCLC, SCLC most frequent
• Metastatic disease in 54%
• PET findings led to:
– Switch to another therapy in 26%
– Adjust dose or duration of therapy in 17%
– Switch from therapy to observation/supportive care in 6%
• Management change more often if post-PET
prognosis worse rather than improved/unchanged
(70% vs. 40%)
Hillner et al., Cancer 2009; 115:410

NOPR Analysis by Cancer Types
Two years of data (40,863 scans)
Confirmed Cancers
Initial staging, restaging, suspected recurrences
All Cases Change in
Management
95% C.I. Imaging adjusted
impact
Initial staging 39.8 39.0-40.6 13.4
Restaging 35.9 35.1-36.7 12.6
Suspected Recurrences 38.5 37.6-39.3 14.7
All Cases Initial Staging Restaging Suspected Recurrence
Prostate 32.0 34.0 39.4
Ovary 43.1 37.7 44.5
Bladder 39.9 36.4 36.7
Pancreas 39.2 38.3 39.3

Changes in Intended Management (%)
Stratified by Pre-PET Plan
Image
n=9,518
Biopsy
n=3,552
Watch
n=2,199
Treatm.
n=7,706
Post-PET Plan
Image 5.8 6.0 4.6 3.0
Biopsy 9.5 24.0 9.0 6.8
Watch 37.2 33.6 62.3 15.6
Same Rx NA NA NA 42.4
New or Major
Change in Rx
47.6 36.3 24.1 8.7
Minor change
Rx
NA NA NA 23.5
Pre-PET Plan
Hillner et al., J Clin Oncol 2008; 26:2155

Patient presentation
Treatment Decisions
Diagnostic workup
Patient outcomes

The study of modalities for diagnosis and Prediction:
Conceptual Framework
Fundamentally tests provide information for use  in
selecting course of care.
Both long‐ and short‐term effects of tests materialize
in context  of available health care options, including
therapeutic interventions.
Not possible to define and measure test effects
outside the particular health care context in which the
test will be used.
However, oftentimes diagnosis may be ahead of
therapy: Diagnosis of DCIS;  Amyloid plaque imaging
for Alzheimer’s.

Pre‐PET Form: Intended Patient Management
Observation (with close follow‐up)
Additional imaging (CT, MRI) or other non‐invasive diagnostic tests
Tissue biopsy (surgical, percutaneous, or endoscopic).
Treatment (if treatment is selected, then also complete the following)
Treatment Goal: (check one) Curative Palliative
Type(s): (check all that apply)
– Surgical Chemotherapy (including biologic modifiers)
– Radiation Other Supportive care
If PET were not available, your current management strategy
would be (select one)?
Intended management, given PET findings, asked on post-PET form

Cohort Profile
• First year of NOPR
(5/8/06 to 5/7/07)
• 22,975 “consented”
cases from 1,519
facilities
• Technology profile
– 84% PET/CT
– 71% non-hospital
– 76% fixed sites
Hillner et al., J Clin Oncol 2008

Intended vs Inferred Care after PET for initial staging
J Nucl Med 2013
54:2024–2031
Cohort of 4661 consenting registry participants, age 65+, enrolled in 2007‐2008. PET for
initial cancer restaging. N PPV Agreement Kappa
Any systemic tx
All 3,030 79.3 74.1 0.40
Bladder 676 79.0 73.0 0.42
Ovarian 223 86.1 82.5 0.52
Stomach 648 67.6 67.6 0.34
Any radiation
All 1,293 64.7 80.8 0.53
Bladder 337 68.8 83.3 0.60
Ovarian 23 30.4 91.6 0.32
Stomach 250 48.8 79.0 0.39
Any surgery
All 1,031 63.6 77.9 0.43
Bladder 248 56.9 75.8 0.36
Ovarian 57 52.6 75.9 0.31
Stomach 458 71.4 75.7 0.50
Conclusion
Better concordance
than in restaging PET

Clark et al. JAMA 2011, Lancet Neurol 2012
Pathology Validation: Florbetapir PET
Median 5 raters
vs. autopsy
Sensitivity 92%
(69%-95%)
Specificity 95%
(90%-100%)
FDA approval:
• Florbetapir
(AmyvidTM)
2012
• Flutemetamol
(VizamylTM)
2013
• Florbetaben
(NeuraceqTM)
2014

Amyloid vs. FDG-PET in Differential Diagnosis of
AD vs. FTD
Rabinovici et al. Neurology 2011
Rabinovici et al. AAN 2014
AD (N=62, age 65, MMSE 22)
FTD (N=45, age 65, MMSE 22)
Amyloid (PIB) PET visual reads
90% sensitivity, 83% specificity
Inter-rater agreement κ=0.96
FDG-PET visual reads
78% sensitivity*, 84% specificity
Inter-rater agreement κ=0.72*
47 autopsy-proven cases
PIB: Sensitivity 100%, Specificity 90%
FDG: Sensitivity 87%, Specificity 78%
* ‐ p<0.05 vs. PIB

IDEAS study (under development)
Organization:
Working collaboration of researchers from NOPR Working
Group, Alzheimer’s Association, SNMMI, American College
of Radiology Imaging Network, and Brown Biostatistics.
Stakeholders from industry and professional organizations.
Support from industry and from CMS (coverage of scans)

Control group construction: Selection of superset
Criterion Factors/Examples Time
1 New Diagnosis of MCI or
Dementia*
CMS claims related to dementia and MCI
in the 24 mo. prior to IDEAS Study
initiation
≥24 mo.
1A MCI CODE 331.83 Mild cognitive impairment
-other codes to be selected in sensitivity
analysis
1B DEMENTIA CODE:
Prior to structural imaging
Inconsistent claims patterns*
≥2 different categories of dementia codes
or
>2 claims with exclusively limited to non-
specific dementia codes) occurring within
a two year (24 month) time interval
2 Minimal Structural Brain Tests
Minimal Blood Laboratory w/u
Head MRI or CT (+/-)
CBC, standard blood chemistry profile,
TSH, vitamin B12
<12 mo.
3 Exclude cases with
unanticipated brain pathology
on intake MRI/CT
Such as primary or metastatic cancers at
time of match and at baseline for
contemporaneous cohort
4 Exclude cases with:
‐ All Non-Skin Cancers,
Lymphomas, Hematologic
Malignancies
‐ Hip/Pelvic Fracture
<12 mo.

PET Amyloid Imaging
Imaging biomarkers for amyloid deposits have been
approved by FDA:
– 18F‐florbetapir in April 2012,
– 18F‐flutemetamol in October 2013
– 18F‐florbetaben in March 2014
CMS National Coverage Decision in September 2013
Coverage for one study per patient, but only under CED to:
– Develop better treatments or prevention strategies for AD, or,
as a strategy to identify subpopulations at risk for developing
AD
– Resolve clinically difficult differential diagnoses (e.g., FTD vs.
AD) with goal of improving health outcomes (including short
term outcomes related to changes in management as well as
longer term dementia outcomes).

Control group construction: Matching criteria
Criterion Factors/Examples Time
1 Age Minimal age of 66 to 67 years +/-24mo.
2 Gender N.A.
3 Marital status N.A.
4 Ethnicity N.A.
5 Location/Service Area Hospital Referral Region (n=306) N.A.
6 Match Chronic
Condition Warehouse
per table below: Focus
is on conditions related
to hospitalizations
Match on number or specifics of
associated CMS Codes. Non-CHF
heart disease is an aggregate of
Acute MI, A fib,, HTN, and
CAD/IHD), CKD, COPD, diabetes
<24 mo.
7 Match for Chronic
Neurological
Conditions, not
included in Warehouse
Prior stroke, TIA, PD, MS,
epilepsy, TBI
<24 mo.
8 Hospitalization/Emergen
cy Room Visits
Dementia or delirium <12 mo.
9 Medicare/Medicaid
Eligibility
Single or dual eligibility N.A.

rA
1= rate of favorable outcomes for pts randomized to test A
rB
2= rate of favorable outcomes for discordant cases randomized to be treated according to
result of A
Difference between arms in simple randomized design
Difference between arms in randomized of discordant cases design
D1= rA
1 ‐ rB
1 =
(r21-r11) x p x (SeB – SeA)+(r22-r12) x (1-p) x (SpA – SpB)
D2= rA
2 – rB
2 =
[(r21-r11) x p x (SeB – SeA)+(r22-r12) x (1-p) x (SpA – SpB)]
= D1/ f
where f = fraction of discordant cases

Collect Registry participant data
Select superset of potential matches from
national CMS claims data using
inclusion/exclusion criteria
Select matching cases from superset
Streamlined process flow of selection of controls

The Recent Reversal of the Growth Trend in
MRI: A Harbinger of the Future?
Richard Sharpe et al. JACR 2013
JACR 2012
October 2, 2015

Referring MD
requests PET
Pre-PET
Form
PET
done
PET
interpreted
& reported
Post-PET
Form sent,
including question for
referring MD consent
Post-PET Form
completed.
Claim submitted
Ongoing
patient
management
NOPR Workflow
Ask patient
for consent

Agreement of Post‐PET Plan and Claims‐inferred Actions at
30 Days
Bladder Ovary Stomach
Patients (total), n 1127 2075 632
Systemic tx only planned, n 261 811 134
PPV (%) 66.7 65.0 51.5
Raw agreement (%) 71.2 71.1 65.7
k 0.33 0.40 0.17
Radiation only planned, n 71 46 34
PPV (%) 74.6 67.4 35.3
Raw agreement (%) 89.4 96.9 95.3
k 0.42 0.48 0.23
Biopsy planned, n 150 176 85
PPV (%) 64.7 55.1 48.2
Raw agreement (%) 74.4 80.5 75.2
K 0.27 0.23 0.21
Watching, n 473 804 320
PPV (%) 70.8 74.1 66.3
Raw agreement (%) 71.0 69.1 60.8
k 0.41 0.38 0.21
Med Care 2013;51: 361–367
Conclusion: Moderate concordance

National coverage of NOPR registry (2006-2009)

Randomized designs in practice, cont.
Randomization to test only or to strategy with test?
– Tradeoff between generalizability and efficiency
• Test only: “noisy” (hence larger sample sizes) but may be
more generalizable
• Test as part of strategy : Less “noise” but results apply to
specific strategies.
– Compliance is important factor to study.
Paired test designs
– Efficient but rarely adopted (see recent report in JClinEpi).
MINDACT trial is example.
– Ethical issues‐ can positive or negative test results be
ignored?
– Implementation difficulties.

2nd CUTEHeart Workshop Constantine Gatsosis Presentation

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to 2nd CUTEHeart Workshop Constantine Gatsosis Presentation

Similar to 2nd CUTEHeart Workshop Constantine Gatsosis Presentation (20)

More from LBNicolau

More from LBNicolau (7)

Recently uploaded

Recently uploaded (20)

2nd CUTEHeart Workshop Constantine Gatsosis Presentation