Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Fixed-effect and random-effects models in meta-analysis

Systematic review and meta-analysis

Source & acknowledgement

https://www.terripigott.com/meta-analysis
https://mason.gmu.edu/~dwilsonb/ma.html
https://www.campbellcollaboration.org/research-resources/training-courses.html
https://handbook-5-1.cochrane.org/
https://www.youtube.com/user/Biostat100
https://www.youtube.com/user/collaborationtube
https://www.meta-analysis.com/pages/videotutorials.php?cart=BWJJ3522355

  • Login to see the comments

Fixed-effect and random-effects models in meta-analysis

  1. 1. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Fixed-effect vs. Random-effects models in meta-analysis Dr. S. A. Rizwan M.D., Public Health Specialist & Lecturer, Saudi Board of Preventive Medicine – Riyadh, Ministry of Health, KSA 25.11.2019 1 With thanks to Michael Borenstein
  2. 2. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Outline • Provide a description of fixed and of random effects models • Outline the underlying assumptions of these two models in order to clarify the choices a reviewer has in a meta-analysis • Discuss how to estimate key parameters in the model 25.11.2019 2
  3. 3. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Our choice between the two depends on • Our assumption about how the effect sizes vary in our meta-analysis • The two models are based on different assumptions about the nature of the variation among effect sizes in our research synthesis 25.11.2019 3
  4. 4. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Sampling assumptions • Fixed-effect • Sampling takes place at one level only • Any between-study variance will be ignored when assigning weights • Random-effects • Sampling takes place at two levels • Any between-study variance will be used when assigning weights 25.11.2019 4
  5. 5. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Fixed-effect • When there is reason to believe that all the studies are functionally identical • When our goal is to compute the common effect size, for the studies in the analysis • Example of drug company has run five studies to assess the effect of a drug • Note that the effect size from each study estimate a single common mean – the fixed-effect • We know that each study will give us a different effect size, but each effect size is an estimate of a common mean, designated in the prior picture as θ 25.11.2019 5
  6. 6. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 6
  7. 7. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Random-effects • We assume two components of variation: • – Sampling variation as in our fixed-effect model assumption • – Random variation because the effect sizes themselves are sampled from population of effect sizes • In a random effects model, we know that our effect sizes will differ because they are sampled from an unknown distribution • Our goal in the analysis will be to estimate the mean and variance of the underlying population of effect sizes • All the studies are functionally equivalent • Example of studies culled from publications 25.11.2019 7
  8. 8. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Random-effects • We see in the picture that each distribution has its own mean that is sampled from the underlying population distribution of effect sizes • That underlying population distribution also has its own variance, τ2, commonly called the variance component • Thus, each effect size has two components of variation, one due to sampling error, and one from the underlying distribution 25.11.2019 8
  9. 9. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 9
  10. 10. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Definition of combined effect • Fixed effect model • There is one true effect • Summary effect is estimate of this value • Random effects model • There is a distribution of effects • Summary effect is mean of distribution 25.11.2019 10
  11. 11. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course How weights are assigned? 25.11.2019 11
  12. 12. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course W = 1/ (V1) Fixed-effects model 25.11.2019 12
  13. 13. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Random-effects model 25.11.2019 13 1W = 1/ (V +T 2 )
  14. 14. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 190 How weights shift? • If within-study variance only, W=1/V • If between-study variance only, W=1/T2 • If both, W=1/(V+T2) 25.11.2019 14
  15. 15. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 15
  16. 16. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 16
  17. 17. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Random vs. Fixed 25.11.2019 17
  18. 18. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Random vs. Fixed 25.11.2019 18 RE weights are more balanced
  19. 19. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Random vs. Fixed 25.11.2019 19 RE confidence interval is wider
  20. 20. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Large study has less impact under RE 25.11.2019 20
  21. 21. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Small study has more impact under RE 25.11.2019 21
  22. 22. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course FE 25.11.2019 22
  23. 23. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Summary effect in Fixed model 25.11.2019 23 244.215 101.171 = 0.414FE
  24. 24. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Variance of summary effect in fixed model 25.11.2019 24 1 244.215 = 0.004FE
  25. 25. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course RE 25.11.2019 25
  26. 26. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Summary effect in random model 25.11.2019 26 90.284 32.342 = 0.358RE
  27. 27. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Variance of summary effect in random model 25.11.2019 27 1 90.284 = 0.011RE
  28. 28. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Why does it matter? • One matches the sampling • One does not match the sampling • Wrong model yields incorrect weights • Estimate of mean is wrong • Estimate of CI is wrong • MUST choose based on sampling model • The meaning of the ES is different • Relative weights are closer under RE (effect size will shift) • Absolute weights are smaller under RE (CI will become wider) • p-value will change (less significant in long run but can go either way) 25.11.2019 28
  29. 29. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Test of null 25.11.2019 29 M M Z = SE M* M* Z* = SE Fixed Random Two sources of error One source of error
  30. 30. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Implications of the choice • If you expect that the effect size from each study arises from a different population then use random effects • Random effects model is more likely to give non-significant estimates • Recall that all our analyses in a meta-analysis are weighted by the inverse of the variance of the effect size, i.e., by the precision of the effect size estimate • Because we have more variation assumed in a random effects model, our weights for each study will be more equal to one another • In other words, in a fixed effect model, we will more heavily weight larger studies. In a random effects model, the larger studies will not be weighted as heavily 25.11.2019 30
  31. 31. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Question! • Suppose we had four studies, each with • N =1,000,000, and a true (mean) effect size of 50. • Under the two models, • What would the forest plot look like? • What would the diamond look like? 25.11.2019 31
  32. 32. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 32
  33. 33. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 33
  34. 34. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Which model should we use? • Base decision on the model that matched the way the data were collected • Not on test of homogeneity 25.11.2019 34
  35. 35. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course What you may hear • Fixed-effect is simple model • Random-effects is more complicated 25.11.2019 35
  36. 36. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Actually • Fixed-effect is more restricted model • Random-effects makes less assumptions 25.11.2019 36
  37. 37. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course An alternate view • Random-effects model only makes sense if we have a clear picture of the sampling frame • Otherwise, we should report the mean and CI for the studies in our sample without attempt to generalize to a larger universe 25.11.2019 37
  38. 38. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Goals in a fixed effect model • In a fixed effect model, we will be most interested in estimating our common effect, θ, and its standard error • We will also want to know if there is heterogeneity present by computing Q • HOWEVER, WE WILL NOT decide on the basis of a significant Q that we should really do a random effects model • WE WILL make a decision about fixed effect versus random effects models because of our substantive knowledge of the area of the systematic review 25.11.2019 38
  39. 39. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Goals in a random effects model • We also want to estimate a mean effect size, but now this is the mean effect size from the underlying population, μ • We also want to estimate the variance of the underlying effect size, τ2 • We will test heterogeneity by testing whether τ2 is different from 0. • Biggest difficulty: how to estimate τ2 25.11.2019 39
  40. 40. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Estimating the variance component, τ2 • There are two main methods for computing τ2 • Variously called the method of moments, the DerSimonian/Laird estimate • Restricted maximum likelihood • The method of moments estimator is easy to compute and is based on the value of Q, the homogeneity statistic • Restricted maximum likelihood requires an iterative solution 25.11.2019 40
  41. 41. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Method of moments estimator 25.11.2019 41
  42. 42. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Restricted maximum likelihood estimator • Many statisticians do not like the method of moments estimator • Can estimate τ2 using HLM, SAS Proc Mixed, or R 25.11.2019 42
  43. 43. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Differences between the two models 25.11.2019 43
  44. 44. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course ES assumptions Fixed effect model • We assume that the treatment effect is the same in all trials. • We use only the sampling variation within the trials. Random effects model • We assume that the treatment effect is the not same in all trials. • The trials are a sample from a population of possible of trials where the treatment effect varies. • We use the sampling variation within the trials and the sampling variation between trials. 25.11.2019 44
  45. 45. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Sampling assumptions Fixed effect model • If the treatment effect is the same in all trials, it is more powerful and easier. • No assumption about representativeness. Random effects model • Less powerful because P values are larger and confidence intervals are wider. • The trials are a sample from a population of possible of trials where the treatment effect varies. • They must be a representative or random sample. • Very strong assumption. 25.11.2019 45
  46. 46. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Variance Fixed effect model • Variance of treatment effect in trial = standard error squared. • Weight = 1/variance = 1/SE2 Random effects model • Variance of treatment effect in trial = standard error squared plus inter- trial variance • Weight = 1/variance. • 1 = -------------------------------- SE2 + inter-trial variance • Inter-trial variance has degrees of freedom given by number of trials minus one. • Typically small. 25.11.2019 46
  47. 47. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course When heterogeneity exists Fixed effect model • When heterogeneity exists we get: • a pooled estimate which may give too much weight to large studies, • a confidence interval which is too narrow, • a P-value which is too small. Random effects model • When heterogeneity exists we get: • possibly a different pooled estimate with a different interpretation, • a wider confidence interval, • a larger P-value. 25.11.2019 47
  48. 48. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course When heterogeneity doesn’t exist Fixed effect model • When heterogeneity does not exists: • a pooled estimate which is correct, • a confidence interval which is correct, • a P-value which is correct. Random effects model • When heterogeneity does not exist: • a pooled estimate which is correct, • a confidence interval which is too wide, • a P-value which is too large. 25.11.2019 48
  49. 49. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 49
  50. 50. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Thank you Kindly email your queries to sarizwan1986@outlook.com 25.11.2019 50

×