Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

One-stage meta-analysis in Stata: power issues and analyses with ipdforest

No Downloads

Total views

450

On SlideShare

0

From Embeds

0

Number of Embeds

6

Shares

0

Downloads

12

Comments

4

Likes

1

No notes for slide

- 1. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary One-stage meta-analysis in Stata power issues and analyses with ipdforest Evan Kontopantelis Centre for Primary Care Institute of Population Health Faculty of Medicine University of Manchester Amsterdam, 5 Nov 2012 Kontopantelis ipdforest
- 2. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary Outline 1 Meta-analysis overview 2 A practical guide 3 ipdforest methods examples 4 Power 5 Summary Kontopantelis ipdforest
- 3. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary Timeline ‘Meta’ is a Greek preposition meaning ‘after’, so meta-analysis =⇒ post-analysis Efforts to pool results from individual studies back as far as 1904 The ﬁrst attempt that assessed a therapeutic intervention was published in 1955 In 1976 Glass ﬁrst used the term to describe a "statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the ﬁndings" Kontopantelis ipdforest
- 4. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary Meta-analysing reported study results A two-stage process the relevant summary effect statistics are extracted from published papers on the included studies these are then combined into an overall effect estimate using a suitable meta-analysis model However, problems often arise papers do not report all the statistical information required as input papers report a statistic other than the effect size which needs to be transformed with a loss of precision a study might be too different to be included (population clinically heterogeneous) Kontopantelis ipdforest
- 5. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary Individual Patient Data IPD These problems can be avoided when IPD from each study are available outcomes can be easily standardised clinical heterogeneity can be addressed with subgroup analyses and patient-level covariate controlling Can be analysed in a single- or two-stage process mixed-effects regression models can be used to combine information across studies in a single stage this is currently the best approach, with the two-stage method being at best equivalent in certain scenarios Kontopantelis ipdforest
- 6. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary Forest plot One advantage of two-stage meta-analysis is the ability to convey information graphically through a forest plot study effects available after the ﬁrst stage of the process, and can be used to demonstrate the relative strength of the intervention in each study and across all informative, easy to follow and particularly useful for readers with little or no methodological experience key feature of meta-analysis and always presented when two-stage meta-analyses are performed In one-stage meta-analysis, only the overall effect is calculated and creating a forest-plot is not straightforward Enter ipdforest Kontopantelis ipdforest
- 7. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary The hypothetical study Individual patient data from randomised controlled trials For each trial we have a binary control/intervention membership variable baseline and follow-up data for the continuous outcome covariates Assume measurements consistent across trials and standardisation is not required We will explore linear random-effects models with the xtmixed command; application to the logistic case using xtmelogit should be straightforward In the models that follow, in general, we denote ﬁxed effects with ‘γ’s and random effects with ‘β’s Kontopantelis ipdforest
- 8. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary Model 1 ﬁxed common intercept; random treatment effect; ﬁxed effect for baseline Yij = γ0 + β1jgroupij + γ2Ybij + ij ij ∼ N(0, σ2 j ) β1j = γ1 + u1j u1j ∼ N(0, τ1 2) i: the patient j: the trial Yij: the outcome γ0: ﬁxed common intercept β1j: random treatment effect for trial j γ1: mean treatment effect groupij: group membership γ2: ﬁxed baseline effect Ybij: baseline score u1j: random treatment effect for trial j τ1 2: between trial variance ij: error term σ2 j : within trial variance for trial j Kontopantelis ipdforest
- 9. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary Model 1 ﬁxed common intercept; random treatment effect; ﬁxed effect for baseline Possibly the simplest approach In Stata it can be expressed as xtmixed Y i.group Yb || studyid:group, nocons where studyid, the trial identiﬁer group, control/intervention membership Y and Yb, endpoint and baseline scores note that the nocons option suppresses estimation of the intercept as a random effect Kontopantelis ipdforest
- 10. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary Model 2 ﬁxed trial speciﬁc intercepts; random treatment effect; ﬁxed trial-speciﬁc effects for baseline Common intercept & ﬁxed baseline difﬁcult to justify A more accepted model allows for different ﬁxed intercepts and ﬁxed baseline effects for each trial: Yi j = γ0j + β1j groupi j + γ2j Ybi j + i j β1j = γ1 + u1j where γ0j the ﬁxed intercept for trial j γ2j the ﬁxed baseline effect for trial j In Stata expressed as: xtmixed Y i.group i.studyid Yb1 Yb2 Yb3 Yb4 || studyid:group, nocons where Yb‘i’=Yb if studyid=‘i’ and zero otherwise Kontopantelis ipdforest
- 11. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary Model 3 random trial intercept; random treatment effect; ﬁxed trial-speciﬁc effects for baseline Another possibility, althought contentious, is to assume trial intercepts are random (e.g. multi-centre trial): Yi j = β0j + β1j groupi j + γ2j Ybi j + i j β0j = γ0 + u0j β1j = γ1 + u1j wiser to assume random effects correlation ρ = 0: i j ∼ N(0, σ2 j ) u0j ∼ N(0, τ2 0 ) u1j ∼ N(0, τ2 1 ) cov(u0j , u1j ) = ρτ0τ1 In Stata expressed as: xtmixed Y i.group Yb1 Yb2 Yb3 Yb4 || studyid:group, cov(uns) cov(uns): allows for distinct estimation of all RE variance-covariance components Kontopantelis ipdforest
- 12. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary Model 4 random trial intercept; random treatment effect; random effects for baseline The baseline could also have been modelled as a random-effect: Yi j = β0j + β1j groupi j + β2j Ybi j + i j β0j = γ0 + u0j β1j = γ1 + u1j β2j = γ2 + u2j as before, non-zero random effects correlations: u0j ∼ N(0, τ2 0 ) u1j ∼ N(0, τ2 1 ) u2j ∼ N(0, τ2 2 ) cov(u0j , u1j ) = ρ1τ0τ1 cov(u0j , u2j ) = ρ2τ0τ2 cov(u1j , u2j ) = ρ3τ1τ2 In Stata expressed as: xtmixed Y i.group Yb || studyid:group Yb, cov(uns) Kontopantelis ipdforest
- 13. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary Model 5 Interactions and covariates A covariate or an interaction term can be modelled as a ﬁxed or random effect Assuming continuous and standardised variable age we can expand Model 2 to include ﬁxed effects for both age and its interaction with the treatment: Yi j = γ0j +β1j groupi j +γ2j Ybi j +γ3agei j +γ4groupi j agei j + i j β1j = γ1 + u1j In Stata expressed as: xtmixed Y i.group i.studyid Yb1 Yb2 Yb3 Yb4 age i.group#c.age || studyid:group, nocons If modelled as a random effect, non-convergence issues more likely to be encountered Kontopantelis ipdforest
- 14. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary methods examples General ipdforest is issued following an IPD meta-analysis that uses mixed effects two-level regression, with patients nested within trials and a linear model (xtmixed) or logistic model (xtmelogit) Provides a meta-analysis summary table and a forest plot Trial effects are calculated within ipdforest Can calculate and report both main and interaction effects Overall effect(s) and variance estimates are extracted from the preceding regression Kontopantelis ipdforest
- 15. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary methods examples Process ipdforest estimates individual trial effects and their standard errors using one-level linear or logistic regressions Following xtmixed, regress is used and following xtmelogit, logit is used, for each trial ipdforest controls these regressions for ﬁxed- or random-effects covariates that were speciﬁed in the preceding two-level regression User has full control over included covariates in the command (e.g. speciﬁcation as ﬁxed- or random-effects) But we strongly recommend using the same speciﬁcations Kontopantelis ipdforest
- 16. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary methods examples Estimation details In the estimation of individual trial effects, ipdforest controls for a random-effects covariate (i.e. allowing the regression coefﬁcient to vary by trial) by including the covariate as an independent variable in each regression Control for a ﬁxed-effect covariate (regression coefﬁcient assumed constant across trials and given by the coefﬁcient estimated under two-level model) is a little more complex. Not possible to specify a ﬁxed value for a regression coefﬁcient under regress and the continuous outcome variable is adjusted by subtracting the contribution of the ﬁxed covariates to its values in a ﬁrst step prior to analysis For a binary outcome the equivalent is achieved through use of the offset option in logit Kontopantelis ipdforest
- 17. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary methods examples Heterogeneity part I Between-trial variability τ2 in the treatment effect, known as heterogeneity, arises from differences in trial design, quality, outcomes or populations For continuous outcomes, ipdforest reports, I2 and H2 M, based on the xtmixed output For binary outcomes, an estimate of the within-trial variance is not reported under xtmelogit and hence heterogeneity measures cannot be computed Between-trial variability estimate ˆτ2 and its conﬁdence interval is reported under both models. Kontopantelis ipdforest
- 18. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary methods examples Heterogeneity part II We are not calculating an IPD version of Cochran’s Q, the orthodox χ2 k−1 homogeneity test, considering its poor performance when the number of trials k is small Besides, taking into account even low levels of τ2 by adopting a random-effects model is a more conservative approach than the ﬁxed-effect one When between-trial variance is estimated to be close to zero, results with the two approaches converge Kontopantelis ipdforest
- 19. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary methods examples Depression intervention We apply the ipdforest command to a dataset of 4 depression intervention trials Complete information in terms of age, gender, control/intervention group membership, continuous outcome baseline and endpoint values for 518 patients Results not published yet; we use fake author names and generated random continuous & binary outcome variables, while keeping the covariates at their actual values Introduced correlation between baseline and endpoint scores and between-trial variability Logistic IPD meta-analysis, followed by ipdforest Kontopantelis ipdforest
- 20. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary methods examples Dataset . use ipdforest_example.dta, . describe Contains data from ipdforest_example.dta obs: 518 vars: 17 6 Feb 2012 11:14 size: 20,202 storage display value variable name type format label variable label studyid byte %22.0g stid Study identifier patid int %8.0g Patient identifier group byte %20.0g grplbl Intervention/control group sex byte %10.0g sexlbl Gender age float %10.0g Age in years depB byte %9.0g Binary outcome, endpoint depBbas byte %9.0g Binary outcome, baseline depBbas1 byte %9.0g Bin outcome baseline, trial 1 depBbas2 byte %9.0g Bin outcome baseline, trial 2 depBbas5 byte %9.0g Bin outcome baseline, trial 5 depBbas9 byte %9.0g Bin outcome baseline, trial 9 depC float %9.0g Continuous outcome, endpoint depCbas float %9.0g Continuous outcome, baseline depCbas1 float %9.0g Cont outcome baseline, trial 1 depCbas2 float %9.0g Cont outcome baseline, trial 2 depCbas5 float %9.0g Cont outcome baseline, trial 5 depCbas9 float %9.0g Cont outcome baseline, trial 9 Sorted by: studyid patid Kontopantelis ipdforest
- 21. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary methods examples ME logistic regression model - continuous interaction ﬁxed trial intercepts; ﬁxed trial effects for baseline; random treatment and age effects . xtmelogit depB group agec sex i.studyid depBbas1 depBbas2 depBbas5 depBbas9 i > .group#c.agec || studyid:group agec, var nocons or Mixed-effects logistic regression Number of obs = 518 Group variable: studyid Number of groups = 4 Obs per group: min = 42 avg = 129.5 max = 214 Integration points = 7 Wald chi2(11) = 42.06 Log likelihood = -326.55747 Prob > chi2 = 0.0000 depB Odds Ratio Std. Err. z P>|z| [95% Conf. Interval] group 1.840804 .3666167 3.06 0.002 1.245894 2.71978 agec .9867902 .0119059 -1.10 0.270 .9637288 1.010403 sex .7117592 .1540753 -1.57 0.116 .4656639 1.087912 studyid 2 1.050007 .5725516 0.09 0.929 .3606166 3.057303 5 .8014551 .5894511 -0.30 0.763 .189601 3.387799 9 1.281413 .6886057 0.46 0.644 .4469619 3.673735 depBbas1 3.152908 1.495281 2.42 0.015 1.244587 7.987253 depBbas2 4.480302 1.863908 3.60 0.000 1.982385 10.12574 depBbas5 2.387336 1.722993 1.21 0.228 .5802064 9.823007 depBbas9 1.881203 .7086507 1.68 0.093 .8990569 3.936262 group#c.agec 1 1.011776 .0163748 0.72 0.469 .9801858 1.044385 _cons .5533714 .2398342 -1.37 0.172 .2366472 1.293993 Random-effects Parameters Estimate Std. Err. [95% Conf. Interval] studyid: Independent var(group) 8.86e-21 2.43e-11 0 . var(agec) 5.99e-18 4.40e-11 0 . Kontopantelis ipdforest
- 22. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary methods examples ipdforest modelling main effect and interaction . ipdforest group, fe(sex) re(agec) ia(agec) or One-stage meta-analysis results using xtmelogit (ML method) and ipdforest Main effect (group) Study Effect [95% Conf. Interval] % Weight Hart 2005 2.118 0.942 4.765 19.88 Richards 2004 2.722 1.336 5.545 30.69 Silva 2008 2.690 0.748 9.676 8.11 Kompany 2009 1.895 0.969 3.707 41.31 Overall effect 1.841 1.246 2.720 100.00 One-stage meta-analysis results using xtmelogit (ML method) and ipdforest Interaction effect (group x agec) Study Effect [95% Conf. Interval] % Weight Hart 2005 0.972 0.901 1.049 19.88 Richards 2004 0.995 0.937 1.055 30.69 Silva 2008 0.987 0.888 1.098 8.11 Kompany 2009 1.077 1.015 1.144 41.31 Overall effect 1.012 0.980 1.044 100.00 Heterogeneity Measures value [95% Conf. Interval] I^2 (%) . H^2 . tau^2 est 0.000 0.000 . Kontopantelis ipdforest
- 23. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary methods examples Forest plots main effect and interaction Overall effect Kompany 2009 Silva 2008 Richards 2004 Hart 2005 Studies 0 2 3 4 5 6 7 8 9 101 Effect sizes and CIs (ORs) Main effect (group) Overall effect Kompany 2009 Silva 2008 Richards 2004 Hart 2005 Studies 0 .2 .4 .6 .8 1.2 1.41 Effect sizes and CIs (ORs) Interaction effect (group x agec) Kontopantelis ipdforest
- 24. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary methods examples ME logistic regression model - binary interaction ﬁxed trial intercepts; ﬁxed trial effects for baseline & age cat; random treatment effect . xtmelogit depB group i.agecat sex i.studyid depBbas1 depBbas2 depBbas5 depBba > s9 i.group#i.agecat || studyid:group, var nocons or Mixed-effects logistic regression Number of obs = 518 Group variable: studyid Number of groups = 4 Obs per group: min = 42 avg = 129.5 max = 214 Integration points = 7 Wald chi2(11) = 42.67 Log likelihood = -326.24961 Prob > chi2 = 0.0000 depB Odds Ratio Std. Err. z P>|z| [95% Conf. Interval] group 1.931763 .5702338 2.23 0.026 1.083151 3.445233 1.agecat .7389353 .213031 -1.05 0.294 .4199616 1.300179 sex .7173044 .154959 -1.54 0.124 .469698 1.095439 studyid 2 1.029638 .5608887 0.05 0.957 .3539959 2.994823 5 .828577 .6082301 -0.26 0.798 .1965599 3.492777 9 1.266728 .6812765 0.44 0.660 .4414556 3.634794 depBbas1 3.196139 1.515334 2.45 0.014 1.261999 8.094542 depBbas2 4.625802 1.923028 3.68 0.000 2.047989 10.44832 depBbas5 2.354493 1.692149 1.19 0.233 .5756364 9.630454 depBbas9 1.902359 .7183054 1.70 0.089 .9075907 3.987446 group#agecat 1 0 .9277371 .3558053 -0.20 0.845 .4374944 1.967331 1 1 1 (omitted) _cons .6175744 .2800575 -1.06 0.288 .253914 1.502076 Random-effects Parameters Estimate Std. Err. [95% Conf. Interval] studyid: Identity var(group) 2.24e-19 1.24e-10 0 . Kontopantelis ipdforest
- 25. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary methods examples ipdforest modelling main effect and interaction . ipdforest group, fe(sex i.agecat) ia(i.agecat) or One-stage meta-analysis results using xtmelogit (ML method) and ipdforest Main effect (group), agecat=0 Study Effect [95% Conf. Interval] % Weight Mead 2005 3.594 1.133 11.402 19.88 Willemse 2004 2.814 1.034 7.657 30.69 Lovell 2008 3.750 0.585 24.038 8.11 Meyer 2009 1.010 0.475 2.147 41.31 Overall effect 1.792 1.079 2.976 100.00 One-stage meta-analysis results using xtmelogit (ML method) and ipdforest Main effect (group), agecat=1 Study Effect [95% Conf. Interval] % Weight Mead 2005 1.166 0.376 3.617 19.88 Willemse 2004 2.566 1.007 6.543 30.69 Lovell 2008 1.935 0.340 11.003 8.11 Meyer 2009 3.551 1.134 11.126 41.31 Overall effect 1.932 1.083 3.445 100.00 Heterogeneity Measures value [95% Conf. Interval] I^2 (%) . H^2 . tau^2 est 0.000 0.000 . Kontopantelis ipdforest
- 26. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary methods examples Forest plots main effect for each age group Overall effect Meyer 2009 Lovell 2008 Willemse 2004 Mead 2005 Studies 0 2 4 6 8 10 12 14 16 18 20 22 24 261 Effect sizes and CIs (ORs) Main effect (group), agecat=0 Overall effect Meyer 2009 Lovell 2008 Willemse 2004 Mead 2005 Studies 0 2 4 6 8 10 121 Effect sizes and CIs (ORs) Main effect (group), agecat=1 Kontopantelis ipdforest
- 27. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary Power calculations to detect a moderator effect The best approach is through simulations As always, numerous assumptions need to be made effect sizes (main and interaction) exposure and covariate distributions correlation between variables within-study (error) variance between-study (error) variance - ICC Generate 100s of data sets using the assumed model(s) Estimate what % of these give a signiﬁcant p-value for the interaction Can be trial and error till desired power level achieved Kontopantelis ipdforest
- 28. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary Research on power calculations with simulations The Stata Journal (2002) 2, Number 2, pp. 107–124 Power by simulation A. H. Feiveson NASA Johnson Space Center alan.h.feiveson1@jsc.nasa.gov Abstract. This paper describes how to write Stata programs to estimate the power of virtually any statistical test that Stata can perform. Examples given include the t test, Poisson regression, Cox regression, and the nonparametric rank-sum test. Keywords: st0010, power, simulation, random number generation, postﬁle, copula, sample size 1 Introduction Statisticians know that in order to properly design a study producing experimental data, one needs to have some idea of whether the scope of the study is suﬃcient to give a reasonable expectation that hypothesized eﬀects will be detectable over experimental error. Most of us have at some time used published procedures or canned software to obtain sample sizes for studies intended to be analyzed by t tests, or even analysis of variance. However, with more complex methods for describing and analyzing both continuous and discrete data (for example, generalized linear models, survival models, selection models), possibly with provisions for random eﬀects and robust variance es- timation, the closed-form expressions for power or for sample sizes needed to achieve a certain power do not exist. Nevertheless, Stata users with moderate programming ability can write their own routines to estimate the power of virtually any statistical test that Stata can perform. 2 General approach to estimating power 2.1 Statistical inference Before describing methodology for estimating power, we ﬁrst deﬁne the term “power” and illustrate the quantities that can aﬀect it. In this article, we restrict ourselves to classical (as opposed to Bayesian) methodology for performing statistical inference on the eﬀect of experimental variable(s) X on a response Y . This is accomplished by calculating a p-value that attempts to probabilistically describe the extent to which the data are consistent with a null hypothesis, H0, that X has no eﬀect on Y . We would then reject H0 if the p-value is less than a predesignated threshold α. It should be pointed out that considerable criticism of the use of p-values for statistical inference has COMMENTARY Open Access Simulation methods to estimate design power: an overview for applied research Benjamin F Arnold1*† , Daniel R Hogan2† , John M Colford Jr1 and Alan E Hubbard3 Abstract Background: Estimating the required sample size and statistical power for a study is an integral part of study design. For standard designs, power equations provide an efficient solution to the problem, but they are unavailable for many complex study designs that arise in practice. For such complex study designs, computer simulation is a useful alternative for estimating study power. Although this approach is well known among statisticians, in our experience many epidemiologists and social scientists are unfamiliar with the technique. This article aims to address this knowledge gap. Methods: We review an approach to estimate study power for individual- or cluster-randomized designs using computer simulation. This flexible approach arises naturally from the model used to derive conventional power equations, but extends those methods to accommodate arbitrarily complex designs. The method is universally applicable to a broad range of designs and outcomes, and we present the material in a way that is approachable for quantitative, applied researchers. We illustrate the method using two examples (one simple, one complex) based on sanitation and nutritional interventions to improve child growth. Results: We first show how simulation reproduces conventional power estimates for simple randomized designs over a broad range of sample scenarios to familiarize the reader with the approach. We then demonstrate how to extend the simulation approach to more complex designs. Finally, we discuss extensions to the examples in the article, and provide computer code to efficiently run the example simulations in both R and Stata. Conclusions: Simulation methods offer a flexible option to estimate statistical power for standard and non- traditional study designs and parameters of interest. The approach we have described is universally applicable for evaluating study designs used in epidemiologic and social science research. Keywords: Computer Simulation, Power, Research Design, Sample Size Background Estimating the sample size and statistical power for a study is an integral part of study design and has profound consequences for the cost and statistical precision of a study. There exist analytic (closed-form) power equations for simple designs such as parallel randomized trials with treatment assigned at the individual level or cluster (group) level [1]. Statisticians have also derived equations to estimate power for more complex designs, such as designs with two levels of correlation [2] or designs with two levels of correlation, multiple treatments and attrition [3]. The advantage of using an equation to esti- mate power for study designs is that the approach is fast and easy to implement using existing software. For this reason, power equations are used to inform most study designs. However, in our applied research we have routi- nely encountered study designs that do not conform to conventional power equations (e.g. multiple treatment interventions, where one treatment is deployed at the group level and a second at the individual level). In these situations, simulation techniques offer a flexible alterna- tive that is easy to implement in modern statistical software. Here, we provide an overview of a general method to* Correspondence: benarnold@berkeley.edu Arnold et al. BMC Medical Research Methodology 2011, 11:94 http://www.biomedcentral.com/1471-2288/11/94 Kontopantelis ipdforest
- 29. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview A practical guide ipdforest Power Summary What to take home A few different approaches exist for conducting one-stage IPD meta-analysis Stata can cope through the xtmixed and the xtmelogit commands The ipdforest command aims to help meta-analysts calculate trial effects display results in standard meta-analysis tables produce familiar and ‘expected’ forest-plots The best way to calculate power to detect complex effects is through simulations Kontopantelis ipdforest
- 30. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Appendix Thank you! Comments, suggestions: e.kontopantelis@manchester.ac.uk Kontopantelis ipdforest

No public clipboards found for this slide

Login to see the comments