Your SlideShare is downloading. ×
Randomized Control Clinical Trial
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Randomized Control Clinical Trial

493
views

Published on


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
493
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
39
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. 542-10-#1 Statistics 542Statistics 542 Introduction to Clinical TrialsIntroduction to Clinical Trials Issues in Analysis ofIssues in Analysis of Randomized Clinical TrialsRandomized Clinical Trials
  • 2. 542-10-#2 Issues in Analysis ofIssues in Analysis of Randomized Clinical TrialsRandomized Clinical Trials • Reference: May, DeMets et al (1981) Circulation 64:669-673 Peto et al (1976) British Journal of Cancer
  • 3. 542-10-#3 Sources of BiasSources of Bias 1. Patient selection 2. Treatment assignment 3. Patient Evaluation 4. Data Analysis Methods to Minimize Bias 1. Randomized Controls 2. Double blind (masked) 3. Analyze what is randomized
  • 4. 542-10-#4 What Data Should Be Analyzed?What Data Should Be Analyzed? • Basic Intention-to-Treat Principle – Analyze what is randomized! – All subjects randomized, all events during follow-up • Randomized control trial is the “gold” standard” • Definitions Exclusions – Screened but not randomized – Affects generalizability but validity OK Withdrawals from Analysis – Randomized, but not included in data analysis – Possible to introduce bias!
  • 5. 542-10-#5 Patient CloseoutPatient Closeout • ICH E9 Glossary – “Intention-to-treat principle - …It has the consequence that subjects allocated to a treatment group should be followed up, assessed, and analyzed as members of that group irrespective of their compliance with the planned course of treatment.”
  • 6. 542-10-#6 Intention To Treat (ITT)Intention To Treat (ITT) PrinciplePrinciple • Analyze all subjects randomized & all events • Beware of “look alikes” – Modified ITT: Analyze subjects who get some intervention – Per Protocol: Analyze subjects who comply according to the protocol
  • 7. 542-10-#7 Patient Withdrawn in Analysis (1)Patient Withdrawn in Analysis (1) • Common Practice - 1980s – Over 3 years, 37/109 trials in New England Journal of Medicine published papers with some patient data not included • Typical Reasons Given a. Patient ineligible (in retrospect) b. Noncompliance c. Competing events d. Missing data
  • 8. 542-10-#8 Patient Withdrawn in Analysis (2)Patient Withdrawn in Analysis (2) A. Patient INELIGIBLE – After randomization, discover some patients did not in fact meet entry criteria – Concern ineligible patients may dilute treatment effect – Temptation to withdraw ineligibles – Withdrawl of ineligible patients, post hoc, may introduce bias
  • 9. 542-10-#9 Betablocker Heart Attack TrialBetablocker Heart Attack Trial (JAMA, 1982)(JAMA, 1982) • 3837 post MI patients randomized • 341 patients found by Central Review to be ineligible • Results % Mortality Propranolol Placebo Eligible 7.3 9.6 Ineligible 6.7 11.3 Best Total 7.2 9.8 ⇒ In the ineligible patients, treatment works best
  • 10. 542-10-#10 Anturane Reinfarction Trial (1980)Anturane Reinfarction Trial (1980) NEJMNEJM • Randomized, double blind, placebo controlled Anturane Placebo Total Randomized 813 816 1629 Ineligible 38 33 71 • Reasons for ineligible 1/3 - time since MI: < 25 days or > 35 days 1/3 - enzymes not elevated 1/3 - other: age, enlarged heart, prolonged hospitalization, .… • Number ineligible about the same in each treatment group BUT
  • 11. 542-10-#11 Anturane Reinfarction Trial (1980)Anturane Reinfarction Trial (1980) • 1629 patients randomized – 1631 entered, but two patients randomized twice – Need to delete 03013, 17008 – Use first randomization! • Declared post hoc 71 “ineligible” patients
  • 12. 542-10-#12 Anturane Reinfarction TrialAnturane Reinfarction Trial (1980)(1980) Placebo Anturane Total All 817 812 1629 Ineligible 33 38 71 Eligible 784 774 1558 < 7 day rule 15 15 30 Analyzable subjects 769 759 1528 (Table 3) Analyzable Deaths - Within 7 days of being off drug
  • 13. 542-10-#13 1980 Anturane Mortality1980 Anturane Mortality ResultsResults Anturane Placebo P-Value Randomized 74/813 (9.1%) 89/816 (10.9%) 0.20 “Eligible” 64/775 (8.3%) 85/783 (10.9%) 0.07 “Ineligible” 10/38 (26.3%) 4/33 (12.1%) 0.12 P-Values for 0.0001 0.92 eligible vs ineligible Reference: Temple & Pledger (1980) NEJM, p. 1488
  • 14. 542-10-#14 1980 Anturane Mortality1980 Anturane Mortality ResultsResults Anturane Placebo Withdrawn 10 4 • “Early” discontinuation 4 3 • “Late” discontinuation 6 1
  • 15. 542-10-#15 Total MortalityTotal Mortality Anturane Reinfarction Trial (1980)Anturane Reinfarction Trial (1980) I. All Pts - 1629 All Deaths Random Deaths * NEJM accounts for only Anturane 812 74 106 Placebo 817 89 43 Total 1629 *163 149 P = 0.20 (Table 3+6) II. 1558 Subjects - Exclude 71 Non-eligibles All Deaths Random Deaths “71" Anturane 774 64 (10) Placebo 784 85 (4) Total 1558 149 P = 0.07
  • 16. 542-10-#16 Total MortalityTotal Mortality Anturane Reinfarction Trial (1980)Anturane Reinfarction Trial (1980) III. 1528 Subjects - Exclude 71 Non-eligibles + 30<7 days All Deaths Random Deaths Anturane 759 60 Placebo 769 80 Total 1528 140 IV. 1528 Subjects Analyzable Deaths Random Deaths Anturane 759 44 Placebo 769 62 Total 1528 106 P = 0.076
  • 17. 542-10-#17 Total MortalityTotal Mortality Anturane Reinfarction Trial (1980)Anturane Reinfarction Trial (1980) Consider Patients Excluded I. < 7 day rule - 30 pts Alive Dead Total Placebo 10 5 15 Anturane 11 4 15 Total 21 9 30
  • 18. 542-10-#18 Total MortalityTotal Mortality Anturane Reinfarction Trial (1980)Anturane Reinfarction Trial (1980) II. 71 "ineligibles" Alive Dead Total Placebo 29 4 33 Anturane 28 10 38 Total 57 14 71
  • 19. 542-10-#19 Anturane Sudden Death (SD)Anturane Sudden Death (SD) I. All Patients (N = 1629) Randomized NDA-SDs NEJM-SDs Placebo 816 41 49 8 Anturane 813 27 30 3 Total 1629 68 79 P-value 0.08 0.03 11 additional SD's were defined from submission of NDs to publications II. Exclude 71 Protocol Violators (N = 1558) Randomized NDA-SDs NEJM-SDs Placebo 783 40 46 Anturane 775 26 28 Total 1558 66 64 P-value 0.08 0.03 Difference of 8 SD's
  • 20. 542-10-#20 Anturane Sudden Death (SD)Anturane Sudden Death (SD) for Total Follow-upfor Total Follow-up III. Exclude 71 Protocol Violators & 30 7 Day Rule Violators (N = 1528) Randomized NDA-SDs NEJM-SDs Placebo 768 37 37 Anturane 760 24 22 Total 1528 61 59 P-value 0.09 0.04 * Information not necessarily given in NEJM articlebut used to prepare tables presented
  • 21. 542-10-#21 Anturane AnalysisAnturane Analysis Percent Mortality Patient Group Anturane Placebo P-value 1620 5.6 (45/806)+ 7.5 (61/814) 0.10 1567 5.2 (41/781) 7.4 (58/786) 0.07 1547 4.6 (35/768) 7.4 (58/779) 0.01 1475 4.4 (32/733) 7.1 (53/742) 0.02 Article '78 3.4 (25/733) 5.9 (44/742) 0.016 + Number of deaths/number at risk Table D - 1978 Article Comparison of the Mortality Experience for the 4 Patient Groups
  • 22. 542-10-#22 ART (NEJM, 1978)ART (NEJM, 1978) Comparisons of the Mortality Experience for the 73 Patients with "Objective" and "Subjective" Baseline Exclusions Groups Compared % Mortality Placebo vs. Anturane in the 73** 8.6 (3/35)* 26.3 (10/38) 73 vs. 1547*** 17.8 (13/73) 6.0 (93/1547) (Both Treatment Groups) 73 vs. 1547 (Anturane Group) 26.3 (10/38) 4.6 (35/768) 73 vs. 1547 (Placebo Group) 8.6 (3/35) 7.5 (58/779)* Number of deaths/number at risk ** 73 refers to the group of 73 patients with "objective" or "subjective" reasons at baseline for exclusion *** 1547 refers to the total group of randomized patients with the 73 patients with objective and subjective baseline exclusions removed
  • 23. 542-10-#23 ART (NEJM, 1978)ART (NEJM, 1978) P-Values Using Two Techniques for Survival Curve Comparisons of the Groups P-Values Groups Compared Mantel-Haenszel Gehan Method Method Placebo vs. Anturane in the 73 0.045 0.052 73 vs. 1547 (Both Treatment Groups) 0.0009 0.0003 73 vs. 1547 (Anturane Only) < 0.0001 < 0.0001 73 vs. 1547 (Placebo Only) 0.91 0.98
  • 24. 542-10-#24 Acceptable PoliciesAcceptable Policies For Ineligible SubjectsFor Ineligible Subjects 1. Delay randomization, confirm eligibility and allow no withdrawals (e.g. AMIS) (Chronic Studies) 2. Accept ineligibles, allow no withdrawals (e.g. BHAT, MILIS) (Acute Studies) 3. Allow withdrawals if: a. Procedures defined in advance b. Decision made early (before event) c. Decision independent and blinded d. Use baseline covariates only (two subgroups) e. Analysis done with and without
  • 25. 542-10-#25 B. WITHDRAWL FOR NON-COMPLIANCE References: Sackett & Gent (1979) NEJM, p. 1410 Coronary Drug Project (1980) NEJM, p. 1038 • Two Types of Trials 1. Management - "Intent to Treat" Principle - Compare all subjects, regardless of compliance 2. Explanatory - Estimate optimum effect, understand mechanism - Analyze subjects who fully comply WITHDRAWALS FOR NON-COMPLIANCE MAY LEAD TO BIAS!
  • 26. 542-10-#26 Breast Cancer Adjuvant TherapyBreast Cancer Adjuvant Therapy Probability of Disease Free Survival forProbability of Disease Free Survival for Years Post Mastectomy (Method I)Years Post Mastectomy (Method I) Redmond et al (1983) Cancer Treatment Report doseprotocoltotal receiveddose IMethod =
  • 27. 542-10-#27 Breast Cancer Adjuvant TherapyBreast Cancer Adjuvant Therapy Probability of Disease Free Survival forProbability of Disease Free Survival for Years Post Mastectomy (Method II)Years Post Mastectomy (Method II) (possible) studyonwhiledose receiveddose IIMethod = Redmond et al (1983) Cancer Treatment Report
  • 28. 542-10-#28
  • 29. 542-10-#29 Breast Cancer Adjuvant TrialBreast Cancer Adjuvant Trial • Results using stratification by compliance analysis can be re-ordered according to definition • Both previous graphs are for the placebo arm • Lesson: Compliance is an outcome & analysis of one outcome, stratified by another, is highly vulnerable to bias
  • 30. 542-10-#30 Cancer Trial (5-FU & Radiation)Cancer Trial (5-FU & Radiation) Gastric CarcinomaGastric Carcinoma • Reference:Moertel et al. (Journal of Clinical Oncology, 1984) • 62 patients randomized – No surgical adjuvant therapy vs. – 5-FU and radiation • 5 year survival results Randomized Percent (%) Treatment 23% P < 0.05 No Treatment 4%
  • 31. 542-10-#31 Cancer Trial (5-FU & Radiation)Cancer Trial (5-FU & Radiation) Gastric CarcinomaGastric Carcinoma • According to treatment received 5 year survival Received % Survival Treatment 20% Refused Treatment 30% NS Control 4%
  • 32. 542-10-#32 Example: Coronary Drug ProjectExample: Coronary Drug Project 5-Year Mortality5-Year Mortality Clofibrate Placebo N % Deaths N % Deaths Total (as reported) 1103 20.0 2782 20.9 By Compliance 1065 18.2 2695 19.4 < 80% 357 24.6 882 28.2 > 80% 708 15.0 1813 15.1 • Adjusting for 40 covariates had little impact • Compliance is an outcome Compliers do better, regardless of treatment
  • 33. 542-10-#33 Example: Coronary Drug ProjectExample: Coronary Drug Project 2-Year Mortality2-Year Mortality Compliance Assessed Estrogen Placebo N % Deaths N % Deaths Total 903 6.2 2361 5.7 < 80% 488 6.1 436 9.9 > 80% 415 6.3 1925 4.8 Comments • Higher % of estrogens patients did not comply • Beneficial to be randomized to estrogen & not take it • (6.1% vs. 9.9%) • Best to be randomized to placebo & comply (4.8%)
  • 34. 542-10-#34 Example: Wilcox et al (1980) Trial,Example: Wilcox et al (1980) Trial, BMJBMJ 6-Week Mortality6-Week Mortality Propranolol Atenolol Placebo N % Deaths N % Deaths N % Deaths Total 132 7.6 127 8.7 129 11.6 Compliers 88 3.4 76 2.6 89 11.2 Non-compliers 44 15.9 51 17.6 40 12.5 Comments • Compliers did better than placebo • Treatment non-compliers did worse than placebo • Placebo non-compliers only slightly worse than compliers • Analysis by compliers overestimates benefit
  • 35. 542-10-#35 Aspirin Myocardial InfarctionAspirin Myocardial Infarction Study (AMIS)Study (AMIS) % Mortality Compliance Aspirin Placebo Good 6.1 5.1 Poor 21.9 22.0 Total 10.9 9.7
  • 36. 542-10-#36 Summary of ComplianceSummary of Compliance • No consistent pattern Example Non-compliance Did Worse CDP Clofibrate, AMIS Both Treatment & Control CDP Estrogen Control Only Beta-blocker, Wilcox Two Treatments, Not Control • Compliance an outcome, not always independent of treatment • Withdrawal of non-compliers can lead to bias • Non-compliers dilute treatment • Try hard not to randomize non-compliers
  • 37. 542-10-#37 II. Competing EventsII. Competing Events • Subject may be censored from primary event by some other event (e.g. cancer vs. heart disease) • Must assume independence • If cause specific mortality used, should also look at total death • If non-fatal event is primary, should also look at total death and non-fatal event • Problem for some response measures
  • 38. 542-10-#38 COMPANIONCOMPANION Example of Competing EventsExample of Competing Events • COMPANION trial was a device trial in CHF patients • Best care vs pacemaker vs pacemaker+defibrillator • Another device approved during trial • Patients in best care arm withdrew consent; censored follow-up • Differential censoring biases analysis
  • 39. 542-10-#39 Optimal Pharmacological Therapy (OPT) (OPT) + CRT (CONTAK TR® /EASYTRAK® ) (OPT) + CRT + ICD (CONTAK CD® /EASYTRAK® ) COMPANION (COMPANION (COCOmparison ofmparison of MMedicaledical Therapy,Therapy, PPacing,acing, ANANd Defibrillatd DefibrillatIONION inin Heart Failure): Study DesignHeart Failure): Study Design Patient Enrollment OPT Alone Randomize Baseline Testing OPT + CRT OPT + CRT-D Target Time to Implant ≤ 2 days from randomization Randomization stratifications: by site, +/- β-blocker therapy Patients randomized 1:2:2 to the following three arms: 1 2 2 HFSA Late-Breaker Sept 24, 2003
  • 40. 542-10-#40 COMPANION: Endpoints (1)COMPANION: Endpoints (1) • Primary Endpoint: –Composite of time to first all-cause mortality or all-cause hospitalization analyzed from randomization • Hospital emergency or outpatient (unscheduled) administration of IV inotropes or vasoactive drugs for more than 4 hours were considered a hospitalization primary event HFSA Late-Breaker Sept 24, 2003
  • 41. 542-10-#41 COMPANION: Endpoints (2)COMPANION: Endpoints (2) • Highest order secondary endpoint: – All-Cause Mortality • Other outcomes analyzed: – Combined mortality or CV, heart failure hospitalizations HFSA Late-Breaker Sept 24, 2003
  • 42. 542-10-#42 COMPANION: Statistical PlanCOMPANION: Statistical Plan • Intention to treat, endpoint data collection begins with randomization; open-label (ethical reasons) • Steering and Endpoints Committees, Exercise Core Laboratory, Sponsor were blinded • Alpha allocation: OPT vs. CRT = 0.02; OPT vs. CRT-D = 0.03 • Sample size assumptions and calculations: – Primary endpoint: 12 month event rate of 40% in OPT arm, 25% reduction in either device arm would require 2200 patients followed for ≥12 month (would translate to 1000 primary events), power ≥90% – Mortality (secondary endpoint): 12 month event rate of 24% in the OPT arm, 25% reduction in either device arm; power = 80% HFSA Late-Breaker Sept 24, 2003
  • 43. 542-10-#43 COMPANION: Data UpdateCOMPANION: Data Update • ACC March 2003 (Preliminary Data) – Data indicated a disproportionate withdrawl rate among OPT, CRT and CRT-D (13%, 2%,2% w/o prior PEPs) – After deliberations with the independent SDAC and DSMB, a decision was made by the Steering Committee to: • Re-consent withdrawn patients to collect endpoint data and vital status • Not count elective device admissions as hospitalization EPs • HFS 2003 (Final Data) The process of collecting endpoint data and vital status on patients that withdrew prior to 12/01/02 is complete: – OPT = 95%, CRT = 99%, and CRT-D = 99% – Median follow-up times (days) are 442 for OPT, 495 for CRT (p = .03), and 479 for CRT-D (p = .13) HFSA Late-Breaker Sept 24, 2003
  • 44. 542-10-#44 COMPANION: Primary EndpointCOMPANION: Primary Endpoint HFSA Late-Breaker Sept 24, 2003
  • 45. 542-10-#45 COMPANION: Secondary EndpointCOMPANION: Secondary Endpoint of All-Cause Mortalityof All-Cause Mortality HFSA Late-Breaker Sept 24, 2003
  • 46. 542-10-#46 CONCLUSIONSCONCLUSIONS When added to optimal pharmacological therapy in patients with modern-severe LV dysfunction, NYHA class III or IV symptoms and QRS lengthening: • CRT or CRT-D reduces mortality + hospitalization • CRT-D reduces mortality – 2/3 of the effect size can be attributed to CRT HFSA Late-Breaker Sept 24, 2003
  • 47. 542-10-#47 III. Problem of DefinitionsIII. Problem of Definitions Classification Anturane Placebo P-value ART 30/812 48/817 0.03 Another Committee 28/812 39/817 0.17 • Cause specific definitions hard to apply • Example: Anturane Reinfarction Trail (ART) (NEJM, 1980) Sudden Death
  • 48. 542-10-#48 Anturane Reinfarction TrialAnturane Reinfarction Trial Sudden DeathSudden Death Category Source Placebo Anturane P-value All patients & all NEJM 48/817 30/812 0.03 sudden deaths AC 39/817 28/812 0.17 "Eligible" patients & NEJM 46/785 28/775 0.03 all sudden deaths AC 37/782 25/773 0.12 • Problem of cause specific definitions • AC = Another review committee
  • 49. 542-10-#49 IV. "Wrong", Inconsistent,IV. "Wrong", Inconsistent, Outlying DataOutlying Data • "Wrong" or "outlying" data may in fact be real • Decisions must be made blind of group assignment • All modifications or withdrawals must be documented
  • 50. 542-10-#50 V. Missing Outcome DataV. Missing Outcome Data • Design with zero – missingness may be associated with treatment • for analysis, data are not missing at random • even if same number missing, missing may be for different reason in each treatment group • Implement with minimum possible • Missing data happens! • Analyze exploring different approaches – If all, or most, agree, then more persuasive – Sensitivity / robustness analysis
  • 51. 542-10-#51 Common SolutionsCommon Solutions • Last observation carried forward (LOCF) • Best Case – Worst Case • Multiple Imputation Methods
  • 52. 542-10-#52 ““Best” and “Worst”Best” and “Worst” Case AnalysesCase Analyses Treatment Control Total Events 170 220 Lost to Follow-up 30 10 "Best" Case 170 230 "Worst" Case 200 220
  • 53. 542-10-#53 Multiple Imputation (Rubin, 2006)Multiple Imputation (Rubin, 2006) MI is well established as a valid method of dealing with missing data when realistic MI model is used • Each missing datum is replaced my multiple values • Reflects uncertainty about the correct value to impute • Allows standard complete-data methods of analysis • MI proposed by Rubin in the 1970s and has a very large number of evaluations supporting its validity and robustness to modeling assumptions • Standard MI assumes “ignorable” missingness • “ignorable” in a particular technical sense
  • 54. 542-10-#54 Multiple Imputation (Rubin, 2006)Multiple Imputation (Rubin, 2006)
  • 55. 542-10-#55 VI: Poor Quality DataVI: Poor Quality Data • Not all data need to be of same quality • Focus on key data (eligibility, primary outcome, SAEs • Poor quality data, like missing data, may be treatment related, not at random • Best solution is prevention, minimize occurrence of poor quality data for key outcomes • Conduct sensitivity analysis to see possible impact • Use rank statistics
  • 56. 542-10-#56 VII. Poor Clinic Performance inVII. Poor Clinic Performance in a Multicenter Studya Multicenter Study • If randomization was stratified by clinic, then withdrawal of a clinic is theoretically valid • Withdrawal must be done independent of the outcome at that clinic
  • 57. 542-10-#57 Mortality in Aspirin MyocardialMortality in Aspirin Myocardial Infarction Study (AMIS)Infarction Study (AMIS) Aspirin Placebo P-value All 30 Centers 246/2267 219/2257 0.99 7 “Selected” Centers 39 66 < 0.01 • In “selected” centers, aspirin showed superiority
  • 58. 542-10-#58 Mortality in Beta-BlockerMortality in Beta-Blocker Heart Attack Trial (BHAT)Heart Attack Trial (BHAT) Propranolol Placebo P-value All 32 Centers 138/1916 188/1921 < 0.01 Cox adjusted Z = 3.05 6 “Selected” Centers 43 26 < 0.05 • In “selected” centers, propranolol worse
  • 59. 542-10-#59 VIII. Special Counting RulesVIII. Special Counting Rules • Events beyond a specified number of days after treatment stopped not counted "non-analyzable" • Examples 1. "7 Day Rule" Anturane (1978) NEJM 2. "28 Day Rule" Timolol (1981) NEJM 3. “14 Day Rule” Approve (2005) NEJM • If used, must – Specify in advance – Be a long period to insure termination not related to outcome – Must analyze results both ways
  • 60. 542-10-#60 APPROVE TrialAPPROVE Trial NEJM (2005)NEJM (2005) • Trial of COX II inhibitor (Vioxx) in patients with colon polyps • Reduce risk of recurrent polyps • Randomized, multicenter, placebo controlled trial • 2586 patients randomized • Follow-up terminated 14 days after patient going off study medication
  • 61. 542-10-#61 APPROVE TrialAPPROVE Trial NEJM (2005)NEJM (2005) • DMC terminated trial early for excess thrombotic events (46 vs. 26) • Separation of KM curves after 18 months • Initial paper claimed risk the same for 18 months • Controversy followed
  • 62. 542-10-#62 APPROVE TrialAPPROVE Trial NEJM (2005)NEJM (2005) Kaplan-Meier Estimates of the Cumulative Incidence of Confirmed Serious Thrombotic Events
  • 63. 542-10-#63 APPROVE TrialAPPROVE Trial with additional follow-upwith additional follow-up NEJM, Nissen (2006)NEJM, Nissen (2006) Kaplan-Meier Estimates of the Cumulative Incidence of Confirmed Thrombotic Cardiovascular Events in the Rofecoxib and Placebo Groups, According to the Intention-to-Treat Principle
  • 64. 542-10-#64 APPROVE LessonsAPPROVE Lessons • Use of “14 Day” rule was not justified • Likely that going off study medication was related to patient risk • Caused a bias in assessment of risk • In general, censoring rules such as this should be avoided
  • 65. 542-10-#65 IX. Fishing orIX. Fishing or Dichotomizing OutcomesDichotomizing Outcomes • Common practice to define a response (S,F) from a non-dichotomous variable • By changing our definition, we can alter results • Thus, definitions stated in advance • Definitions should be based on external data
  • 66. 542-10-#66 Dichotomizing OutcomesDichotomizing Outcomes Heart Rate Trt A Trt B Subject Pre Post ∆ Pre Post ∆ 1 72 72 0 72 702 2 74 73 1 71 68 3 ... 25 73 73 0 79 79 0 Mean 74.0 73.2 0.8 74.4 74.0 0.4 Example
  • 67. 542-10-#67 Three Possible Analyses (1)Three Possible Analyses (1) Change ∆ Treatment A Treatment B P-Value 1.F = < 7 23 25 0.49 S = > 7 2 0
  • 68. 542-10-#68 Three Possible Analyses (2)Three Possible Analyses (2) Change ∆ Treatment A Treatment B P-Value 1.F = < 7 23 25 0.49 S = > 7 2 0 2.F = < 5 19 25 0.02 S = > 5 6 0
  • 69. 542-10-#69 Three Possible Analyses (3)Three Possible Analyses (3) Change ∆ Treatment A Treatment B P-Value 1.F = < 7 23 25 0.49 S = > 7 2 0 2.F = < 5 19 25 0.02 S = > 5 6 0 3.F = < 3 17 18 0.99 S = > 3 8 7
  • 70. 542-10-#70 X. Time Dependent CovariateX. Time Dependent Covariate AdjustmentAdjustment • Classic covariate adjustment uses baseline prognostic factors only – Adjust for Imbalance – Gain Efficiency • Adjustment by time dependent variates not recommended in clinical trials (despite Cox time dependent regression model) • Habit from epidemiology studies
  • 71. 542-10-#71 Coronary Drug ProjectCoronary Drug Project 5-Year Mortality5-Year Mortality Baseline Cholesterol % Deaths Cholesterol Change Clofibrate Placebo < 250mg%* Fall 16.0 21.2 < 250 Rise 25.5 18.7 > 250 mg% Fall 18.1 20.2 > 250 ** Rise 15.5 21.3 • Little change in placebo group • Best to have a. Low cholesterol getting lower * b. High cholesterol getting higher ** Example
  • 72. 542-10-#72 Example: Cancer TrialsExample: Cancer Trials • A common practice to compare survival on patients with a tumor response • Problem is that patient must first survive to be a responder length - bias sampling
  • 73. 542-10-#73 Cancer Trials (1)Cancer Trials (1) Advanced Breast Cancer: Surgery vs. Medicine Santen et al. (1981) NEJM (Letter to editor, Paul Meier, U of Chicago) • A randomized clinical trial comparing surgical adrenalectomy vs. drug therapy in women with advanced breast cancer • 17 pts withdrawn from surgery group 10 pts withdrawn from medical group
  • 74. 542-10-#74 Cancer Trials (2)Cancer Trials (2) • Reasons – Medical group (10 pts) 2 stopped taking their drugs 5 drug toxicity – Surgical group (17 pts) 7 later refused surgery 8 rapid progression precluding surgery • No follow-up data on these 27 pts presented
  • 75. 542-10-#75 XI. Subgroup AnalysesXI. Subgroup Analyses
  • 76. 542-10-#76 False Positive RatesFalse Positive Rates The greater the number of subgroups analyzed separately, the larger the probability of making false positive conclusions. No. of Subgroups False Positive Rate 1 .05 2 .08 3 .11 4 .13 5 .14 10 .19
  • 77. 542-10-#77 Subgroup AnalysesSubgroup Analyses • Focusing on a particular “significant” subgroup can be risky – Due to chance – Results not consistent • Estimates not precise due to small sample size
  • 78. 542-10-#78 MERIT Total MortalityMERIT Total Mortality
  • 79. 542-10-#79 MERITMERIT
  • 80. 542-10-#80 MERITMERIT (AHJ, 2001)
  • 81. 542-10-#81 Praise IPraise I Ref: NEJM, 1996 • Amlodipine vs. placebo • NYHA class II-III • Randomized double-blind • Mortality/hospitalization outcomes • Stratified by etiology (ischemic/non-ischemic) • 1153 patients
  • 82. 542-10-#82 PRAISE IPRAISE I
  • 83. 542-10-#83 PRAISE I - InteractionPRAISE I - Interaction • Overall P = 0.07 • Etiology by Trt Interaction P = 0.004 • Ischemic P = NS • Non-Ischemic P < 0.001
  • 84. 542-10-#84 PRAISE I - IschemicPRAISE I - Ischemic
  • 85. 542-10-#85 PRAISE I – Non- IschemicPRAISE I – Non- Ischemic
  • 86. 542-10-#86 PRAISE IIPRAISE II • Repeated non-ischemic strata • Amlodipine vs. placebo • Randomized double-blind • 1653 patients • Mortality outcome • RR ≅ 1.0
  • 87. 542-10-#87 PRAISE I vs PRAISE IIPRAISE I vs PRAISE II Placebo armsPlacebo arms
  • 88. 542-10-#88 Three ViewsThree Views • Ignore subgroups and analyze only by treatment groups. • Plan for subgroup analyses in advance. Do not “mine” data. • Do subgroup analyses However view all results with caution.
  • 89. 542-10-#89 Analysis Issues SummaryAnalysis Issues Summary • Important not to introduce bias into the analysis • ITT principle is critical • Important to have “complete” follow- up • Off treatment is not off study
  • 90. 542-10-#90 Henry MallHenry Mall