This document discusses key concepts in determining sample size and statistical power for clinical trials. It provides examples of calculating sample sizes for dichotomous and continuous outcomes. The importance of choosing an appropriate primary outcome and estimating event rates in control and intervention groups is emphasized. Methods for randomization in clinical trials like simple randomization and blocked randomization are also covered.

1. DESIGN OF CLINICAL TRIALS EPIDEMIOLOGY 2181 “ SAMPLE SIZE DETERMINATION AND STATISTICAL POWER IN CLINICAL TRIALS” S.F.Kelsey/class2181/lecture 4-sample size October 30, 2008 Lecture 4 SHERYL F. KELSEY, PhD Department of Epidemiology

2. S.F.Kelsey/class2181/lecture 4-sample size QUIZ Assume 90% Power, a = 0.05 two-sided (x) more with A (y) more with B (z) the same 1 . Mortality 20% vs 10% 40% vs 30% 2. Mortality 20% vs 10% 20% vs 15% 3. Diastolic 80 vs 85 mmHg 90 vs 95 mmHg BP 4. Diastolic 80 vs 85 mmHg 80 vs 85 mmHg BP A B (x) more with A (y) more with B (z) the same (x) more with A (y) more with B (z) the same (x) more with A (y) more with B (z) the same (St Dev 10) (St Dev 10) (St Dev 10) (St Dev 8) How many subjects?

3. S.F.Kelsey/class2181/lecture 4-sample size 1. More with B 2. More with B 3. The same 4. More with A ANSWERS Variance of the binomial bigger 50% smaller 0% 100% Small difference need more subjects Only standard deviation matters Bigger standard deviation more subjects

4. S.F.Kelsey/class2181/lecture 4-sample size SHALL WE COUNT THE LIVING OR THE DEAD? 40% vs 20% 20% 50% “reduction” in mortality lower mortality 20% vs 10% 10% 50% “reduction” in mortality lower mortality 10% vs 5% 05% 50% “reduction” in mortality lower mortality 60% vs 80% 20% 33% “improvement” in survival higher mortality 80% vs 90% 10% 2.5% “improvement” in survival higher mortality 90% vs 95% 05% 5.6% “improvement” in survival higher mortality Absolute Relative

5. S.F.Kelsey/class2181/lecture 4-sample size Even more confusing with continuous variables Blood pressure (St Dev 10) 5.9% “reduction” 80 vs 85 mmHg 5.3% “reduction” 90 vs 95 mmHg

6. S.F.Kelsey/class2181/lecture 4-sample size Intervention A has “increased” survival No: Intervention A has longer, better, greater survival “ Increase” should be used for changes over time

7. S.F.Kelsey/class2181/lecture 4-sample size PERCENTS Absolute difference Relative difference Percents as continuous measures Proceed with caution

8. S.F.Kelsey/class2181/lecture 4-sample size Legal null hypothesis: innocent until proven guilty Scientific null hypothesis: no difference in response between treatment groups Innocent Guilty Innocent Guilty Truth Decision of Judge/Jury ok ok guilty goes free type II error (  ) hang the innocent type I error (  ) Treatment Different Treatment Same Truth Same Different Observed Data ok ok miss good treatment type II error (  ) promote worthless Tx type I error (  )

9. FUNDAMENTAL POINT S.F.Kelsey/class2181/lecture 4-sample size Clinical trials should have sufficient statistical power to detect differences between groups considered to be of clinical interest. Therefore, calculation of sample size with provision for adequate levels of significance and power is an essential part of planning.

10. S.F.Kelsey/class2181/lecture 4-sample size THE RAW INGREDIENTS What is your question, precisely? What is your outcome, precisely? Who will be measured? Type 1 and type 2 error rates Variability

11. PRIMARY COMPARISONS S.F.Kelsey/class2181/lecture 4-sample size Dichotomous Response Variables The event rates in the intervention group (P i ) and the control group (P c ) are compared Continuous Response Variables The true but unknown mean value in the intervention group is compared with the true, but unknown mean value in the control group. Survival Data A hazard rate is often compared for the two study groups or at least is used for sample size estimation.

12. SAMPLE SIZE ISSUES S.F.Kelsey/class2181/lecture 4-sample size Choice of outcome – primary endpoint Change from baseline only if correlation > .5 when in doubt don’t The difference____________ to detect you want you believe is clinically meaningful you believe is biologically credible you can afford to

13. ASSESSMENTS OF EVENT RATE IN THE CONTROL AND INTERVENTION GROUPS S.F.Kelsey/class2181/lecture 4-sample size The estimate for the control group event rate is usually obtained from a previous study of similar people. Good data base desirable The investigator must choose the difference in event rate based on preliminary evidence of the potential effectiveness of the intervention or be willing to specify some minimum difference. Calculation of several sample sizes based on a range of estimates helps one to assess how sensitive the sample size is to these estimates.

14. S.F.Kelsey/class2181/lecture 4-sample size To Plan with continuous endpoints:  Clinical difference worth detecting 1–   Power Probability of obtaining a significant result if  is true difference  Significance level, must specify one or two-tailed test (Z   Z  ) 2 Multiplier which depends on level of significance  and Power 1-  n Sample size for each of two groups For continuous measures:  Standard deviation With a little algebra Z  =1.96 for  =.05, two-sided (solve for power) (Solve for difference)

15. S.F.Kelsey/class2181/lecture 4-sample size For two proportions P 1 vs P 2,  = P 1 - P 2 With a little algebra Z  = 1.64 for  .05, one-sided Z  = 1.96 for  .05, two-sided

16. S.F.Kelsey/class2181/lecture 4-sample size TABLE (Z  + Z  ) 2 Needed to determine the size of each sample (Z 2  2.32 1.645 1.28) Desired Two-Tailed Tests One-Tailed Tests Power Level Level Z  P 0.01 0.05 0.10 0.01 0.05 0.10 Two groups of unequal size: Calculate the harmonic mean This n is what is needed for 2 groups of equal size. Note that equal sized groups are the most efficient, that is the harmonic mean is less than the arithmetic mean. References: Snedecor and Cochran, 7th Edition Statistical Methods , 1980, pp 102-1- 5, 120, 130. Fleiss, JL. Statistical Methods for Rates and Proportions, 1981, Chapter 3 & Tables. Schlesselman, JJ. Case Control Studies, 1981, Chapter 6 & Tables. (Z  2.576 1.96 1.645) 0.84 0.80 11.7 7.9 6.2 10.0 6.2 4.5 1.28 0.90 14.9 10.5 8.6 13.0 8.6 6.6 1.645 0.95 17.8 13.0 10.8 15.8 10.8 8.6

17. S.F.Kelsey/class2181/lecture 4-sample size Example: Compare .10 vs .05  = .05 one sided Power 80% arcsin arcsin So total study: 334 x 2 = 668 .10 vs .05 with 200 patients in each group Power = 61% with 100 patients Z  = .28 39% power 50 patients Z  = .68 25% power | | .0963| - 1.64 = .286 | — Z  arcsin

18. FURTHER SAMPLE SIZE CONSIDERATIONS outcome is survival time discount for noncompliance/dropout projections subgroups more than 2 treatment groups unequal group sizes - less efficient - can get more information on new treatment computer packages (PASS, SAS, MINITAB) ASSUMPTIONS ABOUT EVENT RATES PARAMOUNT sensitivity analysis S.F.Kelsey/class2181/lecture 4-sample size

19. ONE-SIDED VERSUS TWO-SIDED TESTS S.F.Kelsey/class2181/lecture 4-sample size I Drug A side effects/expensive Drug B no side effects/cheap A more efficacious A&B the same B more efficacious II X Nutrition Intervention Strategy-Group sessions Y Nutrition Intervention Strategy-Individual program X reduce sodium intake more X&Y the same Y reduce sodium intake more

20. SAMPLE SIZE FOR TESTING “EQUIVALENCY” OF INTERVENTIONS S.F.Kelsey/class2181/lecture 4-sample size The problem in designing positive control studies is that there is no statistical method to demonstrate complete equivalence. Computing a sample size assuming no difference results in an infinite sample size. One approach is to specify a value for difference in response such that interventions with differences that are less than this might be considered equally effective or equivalent.

21. T = Innovative Therapy S = Standard Therapy S.F.Kelsey/class2181/lecture 4-sample size “ Superiority” H 0 : death rate T = death rate S H alt :death rate T < death rate S Equivalence H 0 : death rate T  death rate S +  H alt :death rate T < death rate S +  In general equivalence studies require more patients

22. S.F.Kelsey/class2181/lecture 4-sample size Patients: Acute MI Treatment: Double bolus vs accelerated Alteplace Outcome: 30 day mortality COBALT Equivalence Death rate within 0.4% GUSTO III Superiority Double bolus reduce mortality by 20% WARE AND ANTMAN EDITORIAL

23. MORTALITY RESULTS S.F.Kelsey/class2181/lecture 4-sample size COBALT GUSTO III N 7169 15059 Double bolus 7.98% 7.47% Accelerated 7.53% 7.24% Difference 0.45% 0.23% 95% CI Approx. (-.85%, 1.66%) (-.66%, 1.10%) Action reject equivalence accept null not significantly different from zero

24. DESIGN OF CLINICAL TRIALS EPIDEMIOLOGY 2181 RANDOMIZATION IN CLINICAL TRIALS S.F.Kelsey/class2181/lecture 4-sample size SHERYL F. KELSEY, PH.D

25. WHY RANDOMIZE? S.F.Kelsey/class2181/lecture 4-sample size Best way to assure comparability In the long run balance of factors known unknown Statistical hypothesis test based on random assignment Selection is impartial: “dice not trying to prove a point” - must convince others of validity of comparison

26. RANDOMIZATION S.F.Kelsey/class2181/lecture 4-sample size FIXED ALLOCATION: Assigns with pre-specified probability (not necessarily, though usually, equal) ADAPTIVE: Changes probabilities during study Baseline adaptive: on basis of number per group on basis of variables Responsive adaptive: depends on prior outcome assumes: rapid response stable population source

27. STEPS IN THE RANDOMIZATION OF A PATIENT Check eligibility Informed consent Formal identification RANDOMIZE Confirmation of patient entry S.F.Kelsey/class2181/lecture 4-sample size

28. HOW RANDOM TREATMENT ASSIGNMENTS ARE MADE S.F.Kelsey/class2181/lecture 4-sample size Model: Slips in a hat or flipping a coin Masked drugs numbered and given in order: pharmacy, drug manufacturer Envelopes Telephone to central unit Microcomputer at the site Central computer – internet access

29. HOW TO DO THE SCHEME S.F.Kelsey/class2181/lecture 4-sample size Simple randomization Biased coin, urn models Example: Start with 2 balls, one black and one white Draw-replace and add one of opposite color Prevents imbalance with high probability early on Random permuted block Balance at the end of block Could predict with unmasked trial

30. BLOCKS OF SIZE 4 S.F.Kelsey/class2181/lecture 4-sample size 1) 1100 2) 1010 3) 1001 4) 0110 5) 0101 6) 0011

31. HOW TO USE BLOCKS WHEN TREATMENT IS NOT MASKED S.F.Kelsey/class2181/lecture 4-sample size Choose the block sizes at random, too Example: 2 treatment, equal allocation Block sizes 4, 6, and 8 Balance in each block

32. SHOULD YOU STRATIFY? S.F.Kelsey/class2181/lecture 4-sample size Clinical sites - generally yes Prognostic variables - generally no Size Practical considerations Often governed by custom rather than statistical justification Stratified ANALYSIS is generally preferable

33. MINIMIZATION S.F.Kelsey/class2181/lecture 4-sample size Advantages: Balance several prognostic factors Balance marginal treatment totals Good for small trials (<100 patients) Computer makes this fairly easily Disadvantages: Can’t prepare treatment assignment Scheme in advance Need up-to-date record Not really random - could predict but can introduce random element by using say 3/4, 1/4

34. S.F.Kelsey/class2181/lecture 4-sample size TABLE 5.7. - TREATMENT ASSIGNMENTS BY THE FOUR PATIENT FACTORS FOR 80 PATIENTS IN AN ADVANCED BREAST CANCER TRIAL Factor Level No. on each Next treatment patient A B Performance status Ambulatory 30 31 Non-ambulatory 10 9 Age <50 18 17  50 22 23 Disease-free interval <2 years 31 32  2 years 9 8 Dominant metastatic lesion Visceral 19 21 Osseous 8 7 Soft tissue 13 12 Pocock S. Clinical Trials: A Practical Approach. John Wiley & Sons, Chichester, England, 1991, p. 85. Thus, for A this sum = 30 + 18 + 9 + 19 = 76 while for B this sum = 31 + 17 + 8 + 21 = 77

35. S.F.Kelsey/class2181/lecture 4-sample size INTERNAL VALIDITY compare treatments External Validity/ Generalizability extrapolate to other patients Not realistic to find a random sample of patients for recruitment (at the very least they have to consent) More important to establish efficacy of treatment before deciding if it can be broadly applied