The document outlines the ICH E9 guideline on statistical principles for clinical trials, emphasizing the importance of proper statistical practices in trial design, conduct, analysis, and evaluation. It addresses minimizing bias, maximizing precision, and ensuring robust protocols while distinguishing between exploratory and confirmatory trials. The guideline serves as a roadmap for statisticians and researchers to apply appropriate statistical methods to ensure the validity of trial outcomes.

1. Overview of ICH E9: Statistical Principles for Clinical Trials Presented by Jeff Davidson

2. ICH E9 cover page INTERNATIONAL CONFERENCE ON HARMONISATION OF TECHNICAL REQUIREMENTS FOR REGISTRATION OF PHARMACEUTICALS FOR HUMAN USE ICH HARMONISED TRIPARTITE GUIDELINE STATISTICAL PRINCIPLES FOR CLINICAL TRIALS E9 Current Step 4 version dated 5 February 1998 This Guideline has been developed by the appropriate ICH Expert Working Group and has been subject to consultation by the regulatory parties, in accordance with the ICH Process. At Step 4 of the Process the final draft is recommended for adoption to the regulatory bodies of the European Union, Japan and USA.

3. The great tragedy of science - the slaying of a beautiful hypothesis by an ugly fact. - Aldous Huxley The most important science in the whole world: for upon it depends the practical application of every other science and of every art: the one science essential to all political and social administration, all education, all organization based on experience, for it only gives results of our experience. - Florence Nightingale

4. Focus on statistical principles Gives direction to researchers in design, conduct, analysis, and evaluation of trials Does not address use of specific statistical tests Emphasis on later phase, confirmatory trials Target audience: individuals from a broad range of scientific disciplines Statisticians, clinicians, pharmacologists, epidemiologists Scope and Direction

5. Trial Statistician: Responsible for all the statistical work associated with the trial Ensures statistical principles are appropriately applied Has the proper training and experience to implement the principles in this guidance Unjustified statistics are like smiling cats - not to be trusted. Scope and Direction

6. Minimizing bias Systematic tendency of any factors associated with design, conduct, analysis and interpretation To lead to an estimate of treatment effect Different from the true value Maximizing precision Obtaining small standard errors and narrow confidence intervals Evaluating robustness Sensitivity of overall conclusions to various limitations of the data, assumptions, analysis procedures used Scope and Direction

7. Controlling the type I error Ensuring that the chance of declaring a treatment efficacious when it in fact does not work is low (e.g., α ≤ 0.05) “ Multiplicity” refers to having more than one opportunity to detect a difference between drugs (e.g., interim analyses, multiple endpoints of interest) Scope and Direction

8. Principal features of statistical analysis should be clearly specified in the protocol Protocol (and amendments) should be approved by a trained statistician A detailed Analysis Plan should be written before data analysis begins Protocol and Analysis Plan

9. Types of Trials Exploratory Trial Confirmatory Trial

10. Exploratory Trial Clear and precise objectives, however, tests of hypothesis may be data dependent Such trials cannot be the basis of the formal proof of efficacy

11. Confirmatory Trial An adequately controlled trial in which the hypotheses are stated in advance and evaluated Key hypothesis of interest Follows directly from the trial’s primary objective Is always pre-defined Is the hypothesis that is subsequently tested when the trial is complete Adherence to protocols and SOPs is particularly important

12. Population Earlier phases may focus on a very narrow subgroup Confirmatory trials should more closely mirror target population of the therapy under study Issues of Generalizability Clear Inclusion/Exclusion criteria

13. Outcome Variables Primary variable(s) Directly related to the primary objective Preferable to specify only one reliable and validated variable Used in earlier studies or in published literature Used when estimating the sample size Secondary variables Either supportive measurements related to the primary objective, or measurements of effects related to the secondary objectives

14. Avoiding Bias by Design: Blinding Blinding limits the occurrence of conscious and unconscious bias arising from the influence one’s knowledge of treatment may have on Recruitment and allocation of subjects Their subsequent care Attitudes of subjects to the treatments Assessment of endpoints Handling of withdrawals Exclusion of data from analysis Choice of analysis methods

15. Avoiding Bias by Design: Randomization Introduces a deliberate element of chance into the assignment of treatments Provides a sound statistical basis for the comparison of treatment groups Tends to produce treatment groups with distributions of prognostic factors (measured and unmeasured) are similar

16. Parallel group: random assignment to A vs B Randomization Design Configuration Population A B

17. Crossover: random assignment to AB or BA subject serves as own control Randomization Washout period Design Configuration Population A B B A

18. Factorial Designs: Design Configuration Population A (A1, A2) B (B1,B2) A1, B1 A1, B2 A2, B1 A2, B2

19. Other Study Designs Cohort Studies Case-Control Studies Descriptive Studies (e.g., surveys) Registries Adaptive Etc., etc…

20. Type of Comparison Trials to show “superiority” New treatment vs. placebo, or new treatment vs. active control Test (and hopefully reject) the null hypothesis that there is no difference in outcomes between groups vs. the alternative hypothesis that there is a difference between the groups One-sided or two-sided

21. Type of Comparison Trials to show “ equivalence ” New treatment vs. active control Test (and hopefully reject) the null hypothesis that the new treatment performs differently than the active control by at least some small amount vs. the alternative hypothesis that the difference between the groups is no greater than this small amount An amount that is sufficiently small that the treatments are considered equivalent for all practical purposes if the difference between the treatments is smaller than this amount (equivalence margin)

22. Type of Comparison Trials to show “noninferiority” New treatment vs. active control Test (and hopefully reject) the null hypothesis that the active control performs better than the new treatment by at least some small amount vs. the alternative hypothesis that the new treatment does not perform worse than the active control by more than this small amount

24. Type of comparison Trials to show “equivalence” e.g., test null hypothesis that % cured with active Control is ≥ 5% more than the % cured with the new treatment and vice versa Trials to show “noninferiority” e.g., test null hypothesis that % cured with active Control is ≥ 5% than the % cured with the new treatment

25. Sample Size Determine based on: A primary endpoint The null hypothesis The test statistic (e.g., t-test, chi-square test, logrank test) The treatment difference to be detected (the “alternative hypothesis”) Significant level (Type I error) Desired power (Type II error) Variability assumptions The plan for handling treatment withdrawals and protocol violations

26. Data Monitoring Oversight of Trial Quality Monitoring of Treatment Effects

27. Oversight of Trial Quality Checks performed in a blinded manner: Whether the protocol is being followed The acceptability of data being accrued The success of planned accrual targets The appropriateness of the design assumptions Success in keeping patients in the trials Has no impact on Type I error

28. Monitoring Treatment Effects: Interim Analysis Usually for serious outcomes Requires unblinded access to treatment group summary data Should only be done if included in the protocol Goal, stop the trial early if: Superiority of the new treatment is clear Future demonstration of a treatment effect is unlikely Unacceptable adverse effects are apparent

29. Monitoring Treatment Effects: Interim Analysis May require a DMC, which should approve interim plans Independent Statistical Team Study Team IRB Sponsor DMC

30. Monitoring Treatment Effects: Interim Analysis Repeated testing of outcome data increases the chance of a Type I error Test for difference in proportions failing in two groups without adjustment for multiple testing Decision Rule : Reject null if |Z| ≥1.96 Overall Type I Error Rate Single test at end of study -----> 0.05 Two tests, equally spaced -----> 0.08 Five tests, equally spaced -----> 0.14 -(Friedman, Furberg and DeMets, 1996)

31. Data Analysis Include main features of analysis in protocol For confirmatory trial, include statistical methods to be used for the primary variable(s) For exploratory trials, include general principles and directions Additional ‘statistical analysis plan’ Detailed procedures for primary and secondary variables Do blind review of data, record date of breaking blind

32. Analysis Sets Disposition of participants enrolled, summary of protocol violations Degree of compliance and missing data lead to different Analysis Sets: Full Analysis Set Per Protocol Set Rationale: Minimize bias (Analysis Sets defined a priori) Demonstrate lack of sensitivity

33. Full Analysis Set ‘ Full analysis set’ = the analysis set which is as complete as possible and as close as possible to the intention-to-treat ideal of including all randomized subjects, it may exclude, for example: Participants who failed to meet a major entry criteria Participants who lack any data post randomization

34. Full Analysis Set If specified in the plan, subjects who fail to meet an entry criterion may be excluded without the possibility of introducing bias under the following circumstances: The entry criterion was measured prior to randomization The detection of the relevant eligibility violations can be made completely objectively All subjects receive equal scrutiny for eligibility violations All detected violations of the particular entry criterion are excluded

35. Per Protocol Set ‘ Per protocol set’ = subset of the participants in the full analysis set who are more compliant with the protocol Complete a certain pre-specified minimal exposure to the treatment regimen Have some minimum number of measurements of the primary variable(s) Have no major protocol violations May give overly optimistic results in superiority trials May be the more conservative analysis set for equivalence or non-inferiority trials

36. Estimation and Confidence Intervals Not just p-values, include confidence intervals for estimated treatment effects Prespecify any covariates to be controlled for in primary or secondary analysis To improve precision To adjust for potential imbalances To account for stratified designs Never adjust for post-randomization variables Prespecify interactions and subgroups of interest if treatment effect is likely to vary by baseline factors (e.g., gender)

37. Evaluation of Safety Choice of variables: Laboratory tests, vital signs, adverse events Safety Analysis Set: Usually those who received at least one dose of the investigational drug Statistical Analysis

38. Example of an unacceptable adverse event

39. Evaluation of Safety is critical to the success of a new drug

40. Reporting Document deviations from analysis plan, when and why they occurred Account for all subjects who entered the study Describe all reasons for exclusion from analysis dataset and all protocol violations Summarize measurements of all important variables Consider the effect of loss of subjects, violations and missing data on analysis results Describe participants lost, withdrawn, etc.

41. Thank you! Questions?