Vaccine trials in the age of COVID-19
•Download as PPTX, PDF•
0 likes•699 views
The response to the COVID-19 crisis by various vaccine developers has been extraordinary, both in terms of speed of response and the delivered efficacy of the vaccines. It has also raised some fascinating issues of design, analysis and interpretation. I shall consider some of these issues, taking as my example, five vaccines: Pfizer/BioNTech, AstraZeneca/Oxford, Moderna, Novavax, and J&J Janssen but concentrating mainly on the first two. Among matters covered will be concurrent control, efficient design, issues of measurement raised by two-shot vaccines and implications for roll-out, and the surprising effectiveness of simple analyses. Differences between the five development programmes as they affect statistics will be covered but some essential similarities will also be discussed.
More Related Content
What's hot
What's hot (20)
Similar to Vaccine trials in the age of COVID-19
Similar to Vaccine trials in the age of COVID-19 (20)
More from Stephen Senn
More from Stephen Senn (11)
Recently uploaded
Recently uploaded (20)
Vaccine trials in the age of COVID-19
- 1. Vaccine trials in the age of COVID-19: issues and inferences Stephen Senn stephen@senns.uk S Senn Vaccine Trials 1 http://www.senns.uk/Blogs.html
- 2. Talk outline • Background • Infectious processes • Reporting data • Herd immunity? • COVID Mortality • Five vaccine trials • General overview • The AZ/Oxford study • The Pfizer study • Some general design issues • Conclusions? S Senn Vaccine Trials 2 This is being prepared for a 2nd edition with an extra COVID chapter. The material for this talk comes from that.
- 3. Background What’s happening and why vaccines matter S Senn Vaccine Trials 3
- 4. Simulated toy epidemic R0 < 1 S Senn Vaccine Trials 4 Thick black line is median Thick dashed red lines are 2.5% and 97.5% quantiles
- 5. Simulated toy epidemic R0 > 1 S Senn Vaccine Trials 5 Thick black line is median Thick dashed red lines are 2.5% and 97.5% quantiles
- 6. Pay attention to reporting issues Daily deaths UK S Senn Vaccine Trials 6
- 7. Deaths in Minnesota (Data kindly provided by Walter Kremers) S Senn Vaccine Trials 7
- 8. How do you like your Bayes? S Senn Vaccine Trials 8
- 9. Five vaccine programmes Similarities and differences S Senn Vaccine Trials 9
- 10. Design of five large trials Control rates are per 6 month. (Novavax rate not given in protocol J & J has a rather complicated story.) S Senn Vaccine Trials 10 Sponsor or organiser Shots Number treatment Number control Events targeted Assumed control rate % H0 efficacy % H1 efficacy % Looks Pfizer/ BioNTech 2 21,999 21,999 164 0.65 30 60 5 AZ/ Oxford 2 20,000 10,000 150 0.80 30 60 2 Moderna 2 15,000 15,000 151 0.75 30 60 3 Novavax 2 7,500 7,500 100 30 60 2 J & J Janssen 2 20000 20000 154 ? 30 60 Continu ous
- 11. A surprisingly effective simple analysis • Condition on the total cases • Use • Calculate exact binomial confidence limits • Transform to vaccine efficacy scale 2 1 1 p v p VE VE Vaccine infection rate Placebo infection rate S Senn Vaccine Trials 11 , ˆ , , v v p v v p v v p Y n v p v v v p Y Y n n v p Y if n n Y Bin Y Y Y Y
- 12. Contours of efficacy S Senn Vaccine Trials 12
- 13. Results of five large vaccine trials S Senn Vaccine Trials 13 Sponsor Vaccine Subjects Control Subjects Vaccine Cases Control Cases Pfizer/ BioNTech 18198 18325 8 162 AZ/ Oxford 21632 10817 62 128 Moderna 14134 14073 11 185 Novavax 7500 7500 6 56 J & J Janssen 19514 19544 116 348
- 14. The AZ/Oxford programme Various issues S Senn Vaccine Trials 14
- 15. AZ/Oxford Phase III study • H0: vaccine efficacy 30% • H1: vaccine efficacy 60% • 2:1 randomisation • Great uncertainty as to how many cases in total will occur • However, for analysis this is largely irrelevant • It is the split of cases vaccine v placebo that matters S Senn Vaccine Trials 15
- 16. Some trial simulations S Senn Vaccine Trials 16
- 17. The AZ/Oxford Registration Story • Registration was not sought using the single large planned study • It was based on two incomplete smaller studies • As a result of an error some subjects were given a lower dose • A curious finding was that efficacy was apparently greater for the lower dose • However the dosing interval was larger for the lower dose and this provides a plausible explanation S Senn Vaccine Trials 17
- 18. AZ/Oxford Registration Data S Senn Vaccine Trials 18
- 19. Comparison of results by interval S Senn Vaccine Trials 19
- 20. Pfizer/BioNTech First to report S Senn Vaccine Trials 20
- 21. Pfizer’s strange choice of prior distribution S Senn Vaccine Trials 21 Gives mean 0.4118, which transforms to 0.3 (30%) on vaccine efficacy scale
- 22. Posterior distributions S Senn Vaccine Trials 22
- 23. Pfizer/BioNTech Confidence intervals S Senn Vaccine Trials 23
- 24. Confidence or credibility? Type Point estimate Lower Limit Upper Limit Pfizer/BioNTech full model credible 95.0% 90.3% 97.6% Simple credible (mean ) 94.7% 90.4% 97.6% Simple credible (mode ) 95.2% Simple credible (median ) 94.9% Simple confidence 95.0% 90.0% 97.9% S Senn Vaccine Trials 24
- 25. General Design Issues And also analysis issues And also practical matters S Senn Vaccine Trials 25
- 26. The value of concurrent control S Senn Vaccine Trials 26 AZ results based on the two studies used to register the vaccine not the subsequent phase III study
- 27. The Value of Blinding General Points • If we run a trial double blind then we not only prevent subjective bias but we support the randomisation process • If we run a parallel group trial we make it impossible for clustering to occur • However even if we design a parallel group trial running it as open may lead to clustering Difficulties with open trials • Subjects who are chosen to be vaccinated are invited to attend a clinic to receive their vaccine. • A team of health workers is assigned for vaccination and another team is assigned for assessing controls. • Blood samples are collected by the vaccinating clinic. • Subsequent blood samples (say after 28 days) are also collected in the vaccinating clinic. • Samples are sent in batches to the laboratory to be analysed. • Control subjects are visited by nurses at home to collect blood samples. S Senn Vaccine Trials 27
- 28. How auto-correlation can lead to clustering S Senn Vaccine Trials 28
- 29. What policy for vaccination? Timing of the second jab S Senn Vaccine Trials 29
- 30. Conclusions Report card S Senn Vaccine Trials 30
- 31. Lessons Good thing done • Some brilliant work by the life scientists • Big trials done rapidly • Regulators worked fast • Subsequent roll out impressive For discussion • Pfizer’s bizarre choice of prior distribution • Oxford/AstraZeneca’s unwise choice of allocation ratio • Improbable success of analysis based on cases only? • Importance of concurrent control • Importance of using an appropriate scale S Senn Vaccine Trials 31
Editor's Notes
- The response to the COVID-19 crisis by various vaccine developers has been extraordinary, both in terms of speed of response and the delivered efficacy of the vaccines. It has also raised some fascinating issues of design, analysis and interpretation. I shall consider some of these issues, taking as my example, five vaccines: Pfizer/BioNTech, AstraZeneca/Oxford, Moderna, Novavax, and J&J Janssen but concentrating mainly on the first two. Among matters covered will be concurrent control, efficient design, issues of measurement raised by two-shot vaccines and implications for roll-out, and the surprising effectiveness of simple analyses. Differences between the five development programmes as they affect statistics will be covered but some essential similarities will also be discussed.
- LHS has stopping boundaries as a function of the number of cases in both groups. The similarity of the boundaries is striking. Four of the protocols proposed a frequentist analysis. Pfizer proposed a Bayesian one. Four of the trials used 1:1 randomisation. AZ/Oxford had 2:1 J & J Janssen had continuous monitoring and a sequential probability ratio test. The boundary is not shown
- More generally, given knowledge of the allocation ratio this can be used in a modification of the above procedure when randomisation is not 1:1
- The analysis carried out be me only uses the number of cases on each arm and in addition, for the AZ/Oxford vaccine, the allocation ratio of 2:1.
- The RHS set of contour lines give the expected probabilities for joint case numbers under H0. The LHS gives them under H1
- LD = low dose, SD= standard dose
- The prior distribution is for the value of , the proportionate infect rate, previously discussed. Note that one of the parameters of the beta distribution is given to 6 significant figures: 0.700102. This is to be very precise about a vague prior distribution! Other prior distributions provided for comparison. The distribution was chosen to have a mean of 0.4118, which, when transformed to the vaccine efficacy scale, was equal to 30%. It seems that Pfizer started with a Beta(1,1) and considered how the first parameter had to be changed to make the mean equal to 0.4118. Also considered on the RHS iis what the two values should be to make the mean 0.4118 and the variance 1/12, which is the value for a Beta(1,1)
- The RHS gives the one corresponding to the Pfizer/BioNTech prior
- Only three of the sponsors are shown here. For AZ/Oxford I have used the results from the two smaller trials they use to support registration rather than the subsequently completed USA/Chile/Peru Phase III trial. LHS plots vaccine incidence against control incidence. Note the different scales. The RHS illustrates what would happen if you used control arms from other studies to judge vaccine efficacy.
- Many of these are discussed in blogs of mine. See http://www.senns.uk/Blogs.html