Download to read offline
![
Establish Null Hypothesis that Effect
Sizes Are Equal in All of the Studies
[FAIL TO REJECT NULL]
Tested By Using Q-statistic
Q-statistic is distributed as chi-square
distribution with degree of freedom =
n-1 where n = number of studies
Tests of Homogeneity](https://image.slidesharecdn.com/howtoconductmetaanalysis-180331110545/75/How-to-conduct-meta-analysis-17-2048.jpg)
Uploaded byDr.Junaid Nazar
How to conduct meta analysis
This document outlines the steps for conducting a meta-analysis, including identifying relevant studies, evaluating study quality, abstracting data, performing statistical analysis using fixed and random effects models, and conducting sensitivity analysis and addressing publication bias. The objectives are to synthesize previous studies, provide a summary estimate, and identify different types of models for calculating summary effects and interpreting meta-analysis results.
More Related Content
Similar to How to conduct meta analysis
How to conduct meta analysis
- 1. Dr.Chaudhary Muhammad JunaidNazarDr.Chaudhary Muhammad Junaid Nazar MBBS, MPH, FRSPH, MD INTERNAL MEDICINE,MSC NEPHROLOGY, FWAMSMBBS, MPH, FRSPH, MD INTERNAL MEDICINE,MSC NEPHROLOGY, FWAMS Registrar Nephrologist/Clinical ResearcherRegistrar Nephrologist/Clinical Researcher Bharia International HospitalBharia International Hospital
- 2. Following the course,you will be able to: Define Meta-analysis Select Studies for a Meta-analysis Identify different types of Models Calculate Summary Effects Interpret Results of a Meta-analysis Objectives
- 3. Synthesis ofprevious studies Providing a Summary estimate Steps Identify studies Define Eligibility Criteria Abstract Data Do Statistical Analysis What is Meta-analysis?
- 4. Steps: SearchYour Personal Files First Search Electronic Databases Review Reference Lists of Articles Contact Experts and Researchers Retrieve New Articles Evaluate Quality of the Studies Set Up Eligibility Criteria Identify Studies: Overview
- 5. First, Definea Search Strategy Limitations of Databases incomplete and imperfect queries language problems problems with fugitive literature Publication Bias is important What is publication bias How to deal with publication bias Searching Electronic Databases
- 6. Define StudyQuality Criteria Early Set Up A Good Scoring System Develop A Form for Assessment Calculate Quality for each Study Use this for Sensitivity Analysis stratify studies according to quality Evaluating Study Quality
- 7. Select Eligible StudiesBased On: Study Designs Years of Publication Language Choice among multiple articles Sample-size or follow-up issues Similarity of Exposure and/or Rx Completeness of information Defining Eligibility of Studies
- 8. Steps: Identify RelevantArticles Sort out Eligible Articles Set up a Form for Abstraction Enter the Eligible Studies Use this as your database Statistical Analysis is next... Abstract Data - Review!
- 9. Select AnEstimate of Effect Choose An Effects Measure Select An Effects Model For Each Model: Calculate Summary Effect Size Calculate Confidence Intervals Calculate Q-statistic for Homogeneity Perform Sensitivity Analysis Statistical Analysis - Overview
- 10. Choose OnlyOne Estimate For RCTS, choose the one with Once randomized always randomized For nonrandomized trials, choose: estimate adjusted only for age that, and for a known confounder the “most adjusted” estimate estimate presented in the abstract Selecting Estimate of Effect
- 11. RCTs orCohort Studies Rate Difference between Treatment and Control Groups Ratio of Disease Rates Case Control Studies Odds’ Ratio Rate Ratio Choosing An Effect Measure
- 12. Available Types: Fixed Effects Model Random Effects Model Difference Between the Two Special Cases: When Outcomes are not binary Methods to be Used for them Selecting An Effects Model
- 13. Methods: MantelHaenszel Method Peto’s Method General Variance Based Methods For Rate Difference For Rate Ratios When only RR and 95 CI given Tests of Homogeneity Calculation of Q Statistic Fixed Effects Model
- 14. Download SpreasheetCalculator Strength of Mantel Haenszel Very powerful Widely Used Limitation Cannot Control For Confounding! Mantel Haenszel Method
- 15. Similar toMantel Haenszel Download Calculator Simpler Computation No Control for Confounding Good for RCTs Requires 2 X 2 Table Peto’s Method
- 16. Download CalculatorsFor: Rate Difference For only Relative Risk and 95% CI Strengths and Limitations Good For Rate Differences Computationally Intensive Variance-Based Methods
- 17. Establish NullHypothesis that Effect Sizes Are Equal in All of the Studies [FAIL TO REJECT NULL] Tested By Using Q-statistic Q-statistic is distributed as chi-square distribution with degree of freedom = n-1 where n = number of studies Tests of Homogeneity
- 18. Download TheCalculator! Strengths and Limitations: Can Generalize the Conclusions Computationally Intensive Random Effects Model
- 19. Measurement Scale:Continuous Outcome Measured in Same Scale Download Spreadsheet Calculator! Essentially Extension of ANOVA Useful For Integrating Social Science Research Data Continuous Outcomes
- 20. Conduct SensitivityAnalysis Stratify studies by their quality rating Compare Fixed and Random Effects Deal With Publication Bias Construct A Funnel Plot Sensitivity and Publication Bias
- 21. Presentation ofMeta-analysis or Literature Synthesis Download The Graph Maker More Resources... Conclusion
Editor's Notes
- #3 This is an introductory tutorial on how to quickly search, review and analyze informaiton contained in health care literature. You can conduct meta-analysis at a number of levels: from a full scale study to for very personal reasons to summarize results across studies. Generally has four steps: Develop a research question and identify studies Select Studies Select between fixed effects model and random effects model Calculate a summary effect and interpret the results in the light of findings. In this tutorial, it is assumed that you already have a research question in mind, and you are familiar with using spreadsheets.
- #4 Meta-analysis is essentially synthesis of available literature about a topic. Ideally, synthesis of randomized trials to arrive at a single summary estimate is used. Other types of studies are used for meta-analysis as well. Begin with a systematic process of identifying your studies. Search your personal files and computerized databases, for example, Medline. Write down the search strategy. After identifying all studies, define the ones you want to keep for the meta-analysis. This will help another researcher faced with the same body of literature applying the same criteria to find and work with the same studies. Now create structured formats to key in information taken from the selected studies. Finally, combine the data to arrive at a summary estimate of the effect, it’s 95% confidence interval, and a test of homogeneity of the studies.
- #5 Begin your search in your personal files archive before moving on to electronic databases. First, read their titles and abstracts. Eliminate those studies that do not match with your requirements. Then, for the remaining articles, read their reference lists, retrieve the abstracts of articles you think are important to your research, and continue the process. You should aim to retrieve all relevant articles. You also need to search experts and researchers working in the area of your interest. It’s a good idea to search newsgroups and mailing lists, in addition to contacting individual authors.
- #6 Identify documents in electronic databases by searching for words that appear in the title, in the abstract or in the body of the text. Medline can also be searched by using the name of the author or the source journals. Remember, records retrieved from electronic databases depend both on the structure of these databases and the way you query the database. Also, if you limit the studies to only those published in english language, you will leave out important studies in other languages. Finally, several documents are published in dissertation theses, or government documents which are not archived in these databases. These are oftent termed as fugitive literature. Publication Bias refers to the greater likelihood of research with statistically significant results to be reported in comparison to those with null or nonsignificant results. Acknowledge the problem in the study report Try to retrieve all studies Construct a Funnel Plot with effect size in the X axis and Sample Size in the Y axis. If the plot resembles a funnel with base down, shows that publication bias is minimum.
- #7 Decide which studies you want to include based on the quality of the research reported in them. Decide the study quality standards early on, preferably before beginning the analysis. A standardized form helps a lot. Give a score to each study and analyze the studies based on the quality scores of the studies.
- #8 If you are reviewing both randomized and non-randomized studies in the same meta-analysis, report the effect sizes separately for the RCTS and non-randomized studies. This is because the effect of a new treatment is likely to be larger in studies that employ a non-randomized design. Try to be as up-to-date as possible. Mention the cut-off dates in the analysis so that it becomes clear that studies that were published before and after the time mentioned were not missed but not included in the study as part of the design. If you are including only English language articles in the study, mention the rationale for doing so. Importantly, if you are faced with multiple reports published from the same study on similar topic, include only one study so that information from the same study population should contribute ONLY ONCE to the analysis. Try to exclude studies with small sample size to avoid overemphasizing small studies. Similarly, make an early decision about the period of follow up of studies. More of these in the sensitivity analysis. Decide early on whether you are going to include studies with similar exposures or outcomes in your analysis. Base this decision on your experience and understanding of the research question. Remember, if you are too restrictive, you gain face validity, though you may end up omitting important studies.
- #9 Before you proceed on to the statistical analyses, make sure you have the following: All your relevant articles on hand. You have prepared a format to enter the studies. You have tabulated the results of all eligible studies systematically. Now proceed with the statistical analysis.
- #10 The purpose of conducting a statistical analysis is to determine a summary estimate of effect. The effect measures could be rate difference, relative risk estimate (e.g. an odds’ ratio or relative risk) or a rate ratio. Selecting an effects model is important. Usually the choice is between fixed effects model, which indicates that the conclusions derived in the meta-analysis are valid for the studies included in the analysis, and random effects model, which assume that the studies included in the meta-analysis belong to a random sample of a universe of such studies. When the studies are found to be homogeneous, random and fixed effects models are indistinguishable. Finally, conduct a sensitivity analysis to test that the model you propose is a good model.
- #11 Usually most studies come with many estimates of effects and sub-analyses. Which ones should you accept? If you are studying randomized control trials (RCTs), take only that estimate where the rule of randomization is strictly followed. All other effect estimates are not important. For studies that use non-randomized study designs, look for those estimates that are adjusted for age. For models that include more than one independent variable, take the estimate where age and additionally, effect of a well known confounders (on theoretical grounds) are used for adjustment. Alternatively, use the effect measure in a model where most variables are used for adjustment. A good rule of the thumb is to use the effect measure that is presented in the abstract.
- #12 For Randomized Control Trials or Cohort Studies, difference in the rates of the disease in the treatment group (or the group subjected to exposure) and that in the control group is usually measured. The rates could be cumulative incidence rates or incidence-density rates. Use Odds’ Ratio for case control studies.
- #13 Fixed Effects Model: Answers the question whether the studies included in the meta-analysis show that the treatment or exposure produced the effect on average Random Effects Model: Answers the question, on the basis of the studies that are examined, is it possible to comment that the treatment or the exposure will produce a result? A random effects model is computationally more intense than a fixed effects model. Also, if the studies are homogeneous, fixed effects and random effects models are similar. In this tutorial, we will also examine special situations when the outcomes are on a continuous scale, rather than discrete counting of events. Finally, check out this table about the methods to be used for different types of models.
- #14 Both Mantel Haenszel and Peto’s methods are popularly used. Mantel-Haenszel Method is a method of stratified analysis of data. In Meta-analysis, this is used to pool together the effects of individual studies. Peto’s Method is similar to Mantel Haenszel’s but computationally simpler. Use Calculators for General Variance Based Methods for Rate Difference and for the situation when only Relative Risk Estimate and 95% Confidence Interval are provided.
- #15 The Mantel Haenszel Method is a widely used method to calculate stratified summary effects. In this tutorial, a spreadsheet calculator is provided to calculate the summary effect from a group of studies. Ideal for a variety of study designs including RCTs. However, calculation of summary effect using the Mantel Haenszel Method needs a data matrix that can be entered into a 2*2 matrix. Besides, it cannot control for the effects of confounding factors.
- #16 Peto’s method is very similar to Mantel Haenszel method, except is’s computationally simpler. Like Mantel Haenszel, it requires a 2*2 table for computation of summary effect. It cannot control for confounding.
- #17 Two separate calculators are provided for estimating the summary effects using the variance based methods. The first one can be used in the situation where difference in the rates of even occurrence is used as an effect measure. The second calculator can be used to calculate a summary relative risk estimate when, for all the studies listed, only relative risk estimate (e.g. odds’ ratio or relative risk) is provided with 95% confidence interval.
- #18 A statistic that is particularly useful for assessing whether the studies are all homogenous is the Q-statistic. The hypotheses we test here are: The null hypothesis states that the studies are homogeneous The purpose of this test is to see if the null hypothesis holds. For each model, Q-statistic is calculated. The Q-statistic has a chi-square distribution with degree of freedom N where N = (Number of Studies - 1). For homogeneous studies, there is essentially no difference between a fixed effects model and the random-effects model.
- #19 Random effects model is based on the assumption that the studies are part of a large universe of similar studies. The relative risk estimate can be used to answer the question that will the intervention or the exposure be associated with the outcome studied (compare this with fixed effects model where the question was: did the studies show on average that the exposure or treatment associated with the outcome of interest?). However, a random effects model is computationally more intensive than a fixed effects model.
- #20 For several measures, for example blood pressure, or biochemical estimates or scale scores, a binary value for the outcome (present/absent, or yes/no) is not applicable. It helps if one can measure the effects in a continuous scale. This is particularly useful for several different kinds of medical and social science data. Use this spreadsheet to calculate the summary effects. You may note that in cases where outcome measures of all the studies are measured in the same unit, finding the summary effect is more like a one-way analysis of variance procedure with studies as independent variable.
- #21 Based on the quality of the studies you selected, divide them into two or more groups. Then, conduct separate analyses for each group. Also, compare the results of fixed and random effects model for the studies. A funnel plot will help you to identify presence of publication bias. In the absence of publication bias, if a large number of studies are compared, the distribution of the studies in the funnel plot resembles a funnel looked at sideways with the wider end down. In the presence of publication bias, either end of the base of the funnel will appear truncated.
- #22 The results of meta-analysis is presented in the form of graphs (graphs are plotted with studies against relative risk estimates). In this tutorial, a small spreadsheet application is presented to construct graphs for the presentation of the results of the meta-analysis. Download the graph maker. This tutorial presents a general introduction to the process of meta-analysis. The web is replete with software and tutorials on meta-analysis. Some of them are listed in the following page.