Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
Upcoming SlideShare
Loading in …5
×

0

Share

Download to read offline

RSS local 2012 - Software challenges in meta-analysis

Download to read offline

Software and model selection challenges in meta-analysis: MetaEasy, model assumptions and homogeneity

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

RSS local 2012 - Software challenges in meta-analysis

  1. 1. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary Software and model selection challenges in meta-analysis MetaEasy, model assumptions and homogeneity Evan Kontopantelis David Reeves Health Sciences Primary Care Research Centre University of Manchester RSS Primary Care Study Group Errol Street, 2 July 2012 Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  2. 2. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary Outline 1 Meta-analysis overview The heterogeneity issue More challenges 2 A practical guide the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 3 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  3. 3. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary The heterogeneity issue More challenges Heterogeneity The big bad wolf When the effect of the intervention varies significantly from one study to another. It can be attributed to clinical and/or methodological diversity. Clinical: variability that arises from different populations, interventions, outcomes and follow-up times. Methodological: relates to differences in trial design and quality. Detecting quantifying and dealing with heterogeneity can be very hard. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  4. 4. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary The heterogeneity issue More challenges Absence of heterogeneity Assumes that the true effects of the studies are all equal and deviations occur because of imprecision of results. Analysed with the fixed-effects method. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  5. 5. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary The heterogeneity issue More challenges Presence of heterogeneity Assumes that there is variation in the size of the true effect among studies (in addition to the imprecision of results). Analysed with random-effects methods. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  6. 6. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary The heterogeneity issue More challenges Challenges with meta-analysis Heterogeneity is common and the fixed-effect model is under fire. Methods are asymptotic: accuracy improves as studies increase. But what if we only have a handful, as is usually the case? Almost all random-effects models (except Profile Likelihood) do not take into account the uncertainty in ˆτ2. Is this, practically, a problem? DerSimonian-Laird is the most common method of analysis, since it is easy to implement and widely available, but is it the best? Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  7. 7. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary The heterogeneity issue More challenges Challenges with meta-analysis ...continued Can be difficult to organise since... outcomes likely to have been disseminated using a variety of statistical parameters appropriate transformations to a common format required tedious task, requiring at least some statistical adeptness Parametric random-effects models assume that both the effects and errors are normally distributed. Are methods robust? Sometimes heterogeneity is estimated to be zero, especially when the number of studies is small. Good news? Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  8. 8. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Based on our original work... Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  9. 9. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Organising Data initially collected using data extraction forms. A spreadsheet is the next logical step to summarise the reported study outcomes and identify missing data. Since in most cases MS Excel will be used we developed an add-in that can help with most processes involved in meta-analysis. More useful when the need to combine differently reported outcomes arises. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  10. 10. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 What it can do Help with the data collection using pre-formatted worksheets. Its unique feature, which can be supplementary to other meta-analysis software, is implementation of methods for calculating effect sizes (& SEs) from different input types. For each outcome of each study... it identifies which methods can be used calculates an effect size and its standard error selects the most precise method for each outcome Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  11. 11. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 What it can do ...continued Creates a forest plot that summarises all the outcomes, organised by study. Uses a variety of standard and advanced meta-analysis methods to calculate an overall effect. a variety of options is available for selecting which outcome(s) are to be meta-analysed from each study Plots the results in a second forest plot. Reports a variety of heterogeneity measures, including Cochran’s Q, I2, HM 2 and ˆτ2 (and its estimated confidence interval under the Profile Likelihood method). Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  12. 12. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Advantages Free (provided Microsoft Excel is available). Easy to use and time saving. Extracted data from each study are easily accessible, can be quickly edited or corrected and analysis repeated. Choice of many meta-analysis models, including some advanced methods not currently available in other software packages (e.g. Permutations, Profile Likelihood, REML). Unique forest plot that allows multiple outcomes per study. Effect sizes and standard errors can be exported for use in other meta-analysis software packages. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  13. 13. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Installing Latest version available from www.statanalysis.co.uk Compatible with Excel 2003, 2007 and 2010. Manual provided but also described in: Kontopantelis E and Reeves D. MetaEasy: A Meta-Analysis Add-In for Microsoft Excel. Journal of Statistical Software, 30(7):1-25, 2009. Play video clip Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  14. 14. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Stata implementation MetaEasy methods implemented in Stata under: metaeff, which uses the different study input to provide effect sizes and SEs metaan, which meta-analyses the study effects with a fixed-effect or one of five available random-effects models To install, type in Stata: ssc install <command name> help <command name> Described in: Kontopantelis E and Reeves D. metaan: Random-effects meta-analysis. The Stata Journal, 10(3):395-407, 2010. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  15. 15. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Many random-effects methods which to use? DerSimonian-Laird (DL): Moment-based estimator of both within and between-study variance. Maximum Likelihood (ML): Improves the variance estimate using iteration. Restricted Maximum Likelihood (REML): an ML variation that uses a likelihood function calculated from a transformed set of data. Profile Likelihood (PL): A more advanced version of ML that uses nested iterations for converging. Permutations method (PE): Simulates the distribution of the overall effect using the observed data. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  16. 16. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Performance evaluation our approach Simulated various distributions for the true effects: Normal. Skew-Normal. Uniform. Bimodal. Created datasets of 10,000 meta-analyses for various numbers of studies and different degrees of heterogeneity, for each distribution. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  17. 17. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Performance evaluation our approach Compared all methods in terms of: Coverage, the rate of true negatives when the overall true effect is zero. Power, the rate of true positives when the true overall effect is non-zero. Confidence Interval performance, a measure of how wide the (estimated around the effect) CI is, compared to its true width. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  18. 18. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Homogeneity Zero between study variance 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE zero between study variance Coverage 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE zero between study variance Power (25th centile) 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE zero between study variance Overall estimation performance Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  19. 19. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Coverage performance Small and large heterogeneity under various distributional assumptions 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=0, ku=3 H²=1.18 − I²=15% (normal distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=2, ku=9 H²=1.18 − I²=15% (skew−normal distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk1=0, ku1=3, var1=.1, sk2=0, ku2=3, var2=.1 H=1.18 − I=15% (bimodal distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE H=1.18 − I=15% (uniform distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=0, ku=3 H²=2.78 − I²=64% (normal distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=2, ku=9 H²=2.78 − I²=64% (skew−normal distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk1=0, ku1=3, var1=.1, sk2=0, ku2=3, var2=.1 H=2.78 − I=64% (bimodal distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE H=2.78 − I=64% (uniform distribution) Coverage Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  20. 20. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Power performance Small and large heterogeneity under various distributional assumptions 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=0, ku=3 H²=1.18 − I²=15% (normal distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=2, ku=9 H²=1.18 − I²=15% (skew−normal distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk1=0, ku1=3, var1=.1, sk2=0, ku2=3, var2=.1 H=1.18 − I=15% (bimodal distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE H=1.18 − I=15% (uniform distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=0, ku=3 H²=2.78 − I²=64% (normal distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=2, ku=9 H²=2.78 − I²=64% (skew−normal distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk1=0, ku1=3, var1=.1, sk2=0, ku2=3, var2=.1 H=2.78 − I=64% (bimodal distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE H=2.78 − I=64% (uniform distribution) Power (25th centile) Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  21. 21. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 CI performance Small and large heterogeneity under various distributional assumptions 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=0, ku=3 H²=1.18 − I²=15% (normal distribution) Confidence Interval performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=2, ku=9 H²=1.18 − I²=15% (skew−normal distribution) Confidence Interval performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk1=0, ku1=3, var1=.1, sk2=0, ku2=3, var2=.1 H=1.18 − I=15% (bimodal distribution) Overall estimation performance (medians) 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE H=1.18 − I=15% (uniform distribution) Overall estimation performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=0, ku=3 H²=2.78 − I²=64% (normal distribution) Confidence Interval performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=2, ku=9 H²=2.78 − I²=64% (skew−normal distribution) Confidence Interval performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk1=0, ku1=3, var1=.1, sk2=0, ku2=3, var2=.1 H=2.78 − I=64% (bimodal distribution) Overall estimation performance (medians) 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE H=2.78 − I=64% (uniform distribution) Overall estimation performance Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  22. 22. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Coverage by method Large heterogeneity across various between-study variance distributions 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% FE − Coverage 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% DL − Coverage 0.70 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% ML − Coverage 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% REML − Coverage 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% PL − Coverage 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% PE − Coverage Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  23. 23. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Power by method Large heterogeneity across various between-study variance distributions 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% FE − Power (25th centile) 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% DL − Power (25th centile) 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% ML − Power (25th centile) 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% REML − Power (25th centile) 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% PL − Power (25th centile) 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% PE − Power (25th centile) Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  24. 24. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 CI performance by method Large heterogeneity across various between-study variance distributions 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% FE − Overall estimation performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% DL − Overall estimation performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% ML − Overall estimation performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% REML − Overall estimation performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% PL − Overall estimation performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% PE − Overall estimation performance Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  25. 25. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Which method then? Within any given method, the results were consistent across all types of distribution shape. Therefore methods are highly robust against even severe violations of the assumption of normality. Choose PE if the priority is an accurate Type I error rate (false positive). But low power makes it a poor choice when control of the Type II error rate (false negative) is also important and it cannot be used with less than 6 studies. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  26. 26. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Which method then? For very small study numbers (≤5) only PL gives coverage >90% and an acccurate CI. PL has a ‘reasonable’ coverage in most situations, especially for moderate and large heterogeneiry, giving it an edge over other methods. REML and DL perform similarly and better than PL only when heterogeneity is low (I2 < 15%) The computational complexity of REML is not justified. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  27. 27. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Bring on the champagne? Does not necessarily mean homogeneity. Most methods use biased estimators and not uncommon to get a negative ˆτ2 which is set to 0 by the model. We identified a large percentage of cases where the estimators failed to identify existing heterogeneity. In our simulations, for 5 studies and I2 ≈ 29%: 30% of the meta-analyses were erroneously estimated to be homogeneous under the DL method. 32% for REML and 48% for ML-PL. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  28. 28. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 What does it mean? In these cases coverage was substandard and was over 10% lower than in cases where ˆτ2 > 0, on average. The problem becomes less profound as the number of studies and the level of heterogeneity increase. Better estimators are needed. There might be a large number of meta-analyses of ‘homogeneous’ studies which have reached a wrong conclusion. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  29. 29. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide Summary What to take home MetaEasy can help you organise your meta-analysis and can be especially useful if you need to combine continuous and binary outcomes. Methods implemented in Stata under metaeff and metaan. A zero ˆτ2 is a reason to worry. Heterogeneity might be there but we cannot measure or account for in the model. If ˆτ2 > 0, even if very small, use a random-effects model. The DL method works reasonably well, under all distributions, especially for low levels of heterogeneity. Profile likelihood, which takes into account the uncertaintly in ˆτ2, works better when I2 ≥ 15%. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  30. 30. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Appendix Thank you! Key references Comments, suggestions: e.kontopantelis@manchester.ac.uk Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  31. 31. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Appendix Thank you! Key references References Kontopantelis E, Reeves D. MetaEasy: A Meta-Analysis Add-In for Microsoft Excel. Journal of Statistical Software, 30(7):1-25, 2009. Kontopantelis E, Reeves D. metaan: Random-effects meta-analysis. The Stata Journal, 10(3):395-407, 2010. Kontopantelis E, Reeves D. Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: A simulation study. Stat Methods Med Res, published online Dec 9 2010. Kontopantelis, Reeves Software and model selection challenges in meta-analysis

Software and model selection challenges in meta-analysis: MetaEasy, model assumptions and homogeneity

Views

Total views

502

On Slideshare

0

From embeds

0

Number of embeds

9

Actions

Downloads

13

Shares

0

Comments

0

Likes

0

×