Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PowerPoint Presentation

1,066 views

Published on

Published in: Technology, Economy & Finance
  • Be the first to comment

  • Be the first to like this

PowerPoint Presentation

  1. 1. Ben A. Dwamena, MD Department of Radiology, University of Michigan Medical School Nuclear Medicine Service, VA Ann Arbor Health Care System Ann Arbor, Michigan METAGRAPHITI
  2. 2. Statistical Graphics For Interpretation, Exploration And Presentation Of Meta-analysis Data METAGRAPHITI
  3. 3. VISUOGRAPHIC FRAMEWORK FOR  Exploring distributional assumptions  Testing and correcting for publication bias  Investigating heterogeneity  Summary of Data and Sensitivity Analyses METAGRAPHITI
  4. 4.  Avoid potential misrepresentation by faulty distributional and other statistical assumptions.  Facilitates greater interaction between the researcher and the data by highlighting interesting and unusual aspects of the quantitative data. METAGRAPHITI
  5. 5.  User-friendlier summaries of large, complicated quantitative data sets  Preliminary exploration before definite data synthesis  Effective emphasis of important features rather than details of data METAGRAPHITI
  6. 6. CONTINGENCY TABLE FOR SINGLE STUDY
  7. 7.  True Positives =Experimental Group With Outcome Present (a).  False Positives = Control Group With Outcome Present (b).  False Negatives=Experimental Group With Outcome Absent (c).  True Negatives= Control Group With Outcome Absent (d). DIAGNOSTIC VERSUS TREATMENT TRIAL
  8. 8. Odds Ratio (OR) =(a x d)/(b x c). Relative risk in experimental group {[a/(a + c)]/[b/(b+ d)]} =Likelihood Ratio for a Positive Test. Relative Risk in Control Group = Likelihood Ratio for a Negative Test. DIAGNOSTIC VERSUS TREATMENT TRIAL
  9. 9. Box plots Normal quantile plots Stem-and-Leaf plots DISTRIBUTION PLOTS
  10. 10.  Displays important characteristics of the dataset based on the five-number summary of the data.  “Box” covers inter-quartile range.  “Beltline” of box represents the median value.  “Whiskers” include all but outlier observations. BOX AND WHISKER PLOT
  11. 11. 123456 BOX AND WHISKER PLOT
  12. 12. 1** | 47 1** | 81,93 2** | 20,48 2** | 51,86 3** | 04,22 3** | 59,81,85 4** | 10 4** | 59,67,67,68 5** | 24,34,48 5** | 57,58,67 6** | 06 rounded to nearest multiple of .01 plot in units of .01 STEM-AND-LEAF PLOT
  13. 13.  Plot of standardized effect size, Ei/√Vi vs. normal distribution.  Deviations from linearity ⇒ deviations from normality.  Slope of regression line =standard deviation of data= 1 for effect size if the studies from a single population and have large samples.   The y-intercept of the regression =the mean. NORMAL QUANTILE PLOT
  14. 14. NORMAL QUANTILE PLOT
  15. 15. NORMAL QUANTILE PLOT
  16. 16. NORMAL QUANTILE PLOT
  17. 17. NORMAL QUANTILE PLOTS
  18. 18. Selective publication of articles showing certain types of results over those of showing other types of results Commonly, tendency to publish only studies with statistical significant results PUBLICATION BIAS
  19. 19. Published studies do not represent all studies on a specific topic.  Trend towards publishing statistically significant (p < 0.05) or clinically relevant results.  Publication bias assessed by examining asymmetry of funnel plots of estimates of odds ratios vs. precision. INVESTIGATING PUBLICATION BIAS
  20. 20. Funnel plot Begg’s rank correlation plot Egger’s regression plot Harbord’s modified radial plot INVESTIGATING PUBLICATION BIAS
  21. 21. A funnel diagram (a.k.a. funnel plot, funnel graph, bias plot) Special type of scatter plot with an estimate of sample size on one axis vs. effect-size estimate on the other axis FUNNEL PLOT
  22. 22. Based on statistical principle that sampling error decreases as sample size increases Used to search for publication bias and to test whether all studies come from a single population FUNNEL PLOT
  23. 23. FUNNEL PLOTS
  24. 24. metafunnel ldor seldor, xlab(0(2)8) xtitle (Log odds ratio) ytitle(Standard error of log OR) saving(zfunnel, replace) metafunnel ldor seldor, xlab(0(2)8) xtitle(Log odds ratio) ytitle (Standard error of log OR) egger saving (eggerfunnel, replace) FUNNEL PLOT: STATA SYNTAX
  25. 25. FUNNEL PLOT: STATA DIALOG
  26. 26. 0.511.52 StandarderroroflogOR 0 2 4 6 8 Log odds ratio Funnel plot with pseudo 95% confidence limits FUNNEL PLOT: EXAMPLE
  27. 27. 0.511.52 StandarderroroflogOR 0 2 4 6 8 Log odds ratio Funnel plot with pseudo 95% confidence limits FUNNEL PLOT: WITH REGRESSION LINE
  28. 28. An adjusted rank correlation method to assess the correlation between effect estimates and their variances. Deviation of Spearman's rho from zero=estimate of funnel plot asymmetry. Positive values=a trend towards higher levels of effect sizes in studies with smaller sample sizes BEGG’S BIAS TEST
  29. 29. metabias LogOR seLogOR, graph(b) saving(beggplot, replace) BEGG’S BIAS TEST: STATA SYNTAX
  30. 30. BEGG’S BIAS TEST: STATA DIALOG
  31. 31. Begg's funnel plot with pseudo 95% confidence limits logOR s.e. of: logOR 0 .5 1 1.5 2 0 5 10 BEGG’S BIAS PLOT
  32. 32. Adjusted Kendall's Score (P-Q) = 26 Std. Dev. of Score = 40.32 Number of Studies = 24 z = 0.64 Pr > |z| = 0.519 z = 0.62 (continuity corrected) Pr >|z| = 0.53(continuity corrected) BEGG’S BIAS TEST: STATISTICS
  33. 33.  Assesses potential association b/n effect size and precision.  Regression equation: SND = A + B x SE(d)-1. 1. SND=standard normal deviate (effect, d divided by its standard error SE(d)); 2. A =intercept 3. B=slope. . EGGER’S REGRESSION TEST
  34. 34. The intercept value (A) = estimate of asymmetry of funnel plot Positive values (A > 0) indicate higher levels of effect size in studies with smaller sample sizes. EGGER’S REGRESSION METHOD
  35. 35. EGGER’S BIAS PLOT
  36. 36. metabias logOR selogOR, graph(e) saving(eggerplot, replace) EGGER’S BIAS TEST: STATA SYNTAX
  37. 37. EGGER’S BIAS TEST: STATA DIALOG
  38. 38. Egger's publication bias plot standardizedeffect precision 0 1 2 3 4 0 2 4 6 8 EGGER’S BIAS PLOT: EXAMPLE
  39. 39.   ------------------------------------------------------------- Std_Eff | Coef. P>|t| [95% CI] -------------+----------------------------------------------- slope | 1.737492 0.001 .8528166 2.622168 bias | 1.796411 0.002 .7487423 2.84408 ------------------------------------------------------------- EGGER’S BIAS TEST: STATISTICS
  40. 40. Test for funnel-plot asymmetry Regresses Z/sqrt(V) vs. sqrt (V), where Z is the efficient score and V is Fisher's information (the variance of Z under the null hypothesis). Modified Galbraith plot of Z/sqrt(V) vs. sqrt(V) with the fitted regression line and a confidence interval around the intercept. MODIFIED BIAS TEST(HARBORD)
  41. 41. metamodbias tp fn fp tn, graph z(Z) v(V) mlabel(index) saving(HarbordPlot, replace) MODIFIED BIAS TEST: STATA SYNTAX
  42. 42. 8 18 213 13 7 15 6 4 9 22 11 10 12 24 17 1 2 14 16 19 5 23 20 051015 Z/sqrt(V) 0 1 2 3 4 sqrt(V) Study regression line 90% CI for intercept MODIFIED BIAS PLOT
  43. 43. ----------------------------------------------------------------------------- ZoversqrtV | Coef. Std. Err. P>|t| [90% Conf. Interval] --+-------------------------------------------------------------------------- sqrtV| 2.406756 .3464027 0.000 1.811933 3.00158 bias| .9965934 .6383554 0.133 -.0995549 2.092742 ----------------------------------------------------------------------------- MODIFIED BIAS TEST: STATISTICS
  44. 44.  A rank-based data augmentation technique  used to estimate the number of missing studies and to produce an adjusted estimate of test accuracy by imputing suspected missing studies.  Both random and fixed effect models may be used to assess the impact of model choice on publication bias. TRIM-AND-FILL METHOD
  45. 45. • metatrim LogOR seLogOR, eform funnel print graph id(author)saving(tweedieplot, replace) TRIM-AND-FILL TEST: STATA SYNTAX
  46. 46. TRIM AND FILL: STATA DIALOG
  47. 47. TRIM-AND-FILL BIAS PLOT Filled funnel plot with pseudo 95% confidence limits theta,filled s.e. of: theta, filled 0 .5 1 1.5 2 -2 0 2 4 6
  48. 48.  When effect sizes differences are attributable to only sampling error, studies are homogeneous.  Heterogeneity means that there is between-study variation and variability in effect sizes exceeds that expected from sampling error. HETEROGENEITY
  49. 49. Potential sources of heterogeneity: 1. Characteristics of study population 2. Variation in study design 3. Statistical methods 4. Covariates adjusted for (if relevant) HETEROGENEITY
  50. 50. DEALING WITH HETEROGENEITY  Use analysis of variance with the log odds ratio as dependent variable and categorical variables for subgroups as factors to look for differences among subgroups
  51. 51. DEALING WITH HETEROGENEITY  Repeat analysis after excluding outliers  Conduct analysis with predefined subgroups  Construct multivariate models to search for the independent effect of study characteristics
  52. 52. Standardized effect vs. reciprocal of the standard error. Small studies/less precise results appear on the left side and the largest trials on the right end . GALBRAITH’S PLOT
  53. 53. A regression line , through the origin, represents the overall log-odds ratio. Lines +/- 2 above regression line =95 per cent boundaries of the overall log-odds ratio.  The majority of points within area of +/- 2 in the absence of heterogeneity. GALBRAITH’S PLOT
  54. 54. galbr LogOR seLogOR, id(index) yline(0) saving(gallplot, replace) GALBRAITH’S PLOT: STATA SYNTAX
  55. 55. GALBRAITH PLOT: STATA DIALOG
  56. 56. b/se(b) 1/se(b) b/se(b) Fitted values 0 3.80729 -2-2 0 2 13.1045 . 7 8 1315 11 18 321 1222 1419 17 6 9 2 24 1 4 10 23 16 5 20 GALBRAITH PLOT: EXAMPLE
  57. 57. This plots the event rate in the experimental (intervention) group against the event rate in the control group  Visual aid to exploring the heterogeneity of effect estimates within a meta-analysis. L’ABBE PLOT
  58. 58. labbe tp fn fp tn, s(O) xlab(0,0.25,0.50,0.75,1) ylab(0,0.25,0.50,0.75,1) l1("TPR) b2("FPR") saving(flabbeplot, replace) L’ABBE PLOT: STATA SYNTAX
  59. 59. L’ABBE PLOT: STATA DIALOG
  60. 60. TPR FPR 0 .25 .5 .75 1 0 .25 .5 .75 1 L’ABBE PLOT: EXAMPLE
  61. 61. twoway (rcap dorlo dorhi Study, horizontal blpattern(dash))(scatter Study dor, ms(O)msize(medium) mcolor(black))(scatter DOR_with_CIs eb_dor, yaxis(2) msymbol(i) msize(large) mcolor(black))(scatteri 26 83, msymbol(diamond) msize(large)), ylabel(1(1)25 26 "OVERALL", valuelabels angle(horizontal)) xlabel(0 10 100 1000 10000) xscale(log) ylabel(1(1)25 26 "Pooled Estimate", valuelabels angle(horizontal) axis(2)) legend(off) xtitle(Odds Ratio) xline(83, lstyle(foreground)) saving(OddsForest, replace) DATA SUMMARY: STATA SYNTAX
  62. 62. metan tp fn fp tn, or random nowt sortby(year) label(namevar=author, yearvar=year) t1(Summary DOR, Random Effects) b2(Diagnostic Odds Ratio) saving(SDORRE, replace) force xlabel(0,1,10,100,1000) DATA SYNTHESIS: RANDOM EFFECTS
  63. 63. • metan tp fn fp tn, or fixed nowt sortby(year) label(namevar=author, yearvar=year) t1(Summary DOR, Fixed Effects) b2(Diagnostic Odds Ratio) saving(SDORFE, replace) force xlabel(0,1,10,100,1000) DATA SYNTHESIS: FIXED EFFECTS
  64. 64. 106.33(6.40 - 1765.75 107.23(40.31 - 285.28 107.67(6.57 - 1763.99 12.33(2.04 - 74.41) 17.50(2.65 - 115.66) 175.00(28.22 - 1085.2 189.00(11.36 - 3143.3 191.67(12.02 - 3055.9 21.00(1.43 - 308.21) 240.88(68.75 - 843.96 25.00(1.72 - 364.11) 262.66(24.39 - 2829.1 266.00(33.46 - 2114.7 32.82(5.43 - 198.37) 33.00(8.83 - 123.26) 4.33(1.05 - 17.90) 429.00(14.65 - 12562. 45.00(3.75 - 539.38) 46.93(17.48 - 125.96) 56.08(4.62 - 680.33) 6.10(3.95 - 9.42) 6.86(1.81 - 26.01) 7.09(1.06 - 47.42) 9.00(0.58 - 140.71) 98.80(15.24 - 640.50) Pooled Estimate DOR_with_CIs Adler 1997 Avril 1996 Bassa 1996 Danforth 2002 Greco 2001 Guller 2002 Hubner 2000 Inoue 2004 Lin 2002 Nakamoto(a) 2002 Nakamoto(b) 2002 Noh 1998 Ohta 2000 Palmedo 1997 Rostom 1999 Scheidhauer 1996 Schirrmeister 2001 Smith 1998 Tse 1992 Utech 1996 Wahl 2004 Yang 2001 Yutani 2001 Zornoza 2004 van Hoeven 2002 OVERALL 0 10 100 1000 10000 Odds Ratio FOREST PLOT: STATA GRAPHICS
  65. 65. Assumes homogeneity of effects across the studies being combined. There is a common true effect size for all studies. In the summary estimate, only the variance of each study is taken into account. FIXED EFFECTS META-ANALYSIS
  66. 66. Summary DOR, Fixed Effects Odds ratio .1 1 100 1000 10000 Study Odds ratio (95% CI) 36.27 (4.27,308.02)Adler (1997) 98.80 (10.66,916.11)Avril (1996) 21.00 (0.86,515.50)Bassa (1996) 4.33 (0.80,23.49)Danforth (2002) 107.23 (33.42,344.07)Greco (2001) 12.00 (1.23,117.41)Guller (2002) 429.00 (7.67,23982.81)Hubner (2000) 189.00 (6.63,5384.60)Lin (2002) 17.50 (1.84,166.04)Nakamoto (2002) 6.86 (1.40,33.57)Nakamoto (2002) 208.33 (7.72,5621.57)Noh (1998) 60.23 (3.09,1174.51)Ohta (2000) 106.33 (3.74,3023.90)Palmedo (1997) 289.00 (15.83,5276.04)Rostom (1999) 107.67 (3.85,3013.13)Scheidhauer (1996) 46.93 (14.47,152.17)Schirrmeister (2001) 266.00 (22.50,3145.19)Smith (1998) 9.00 (0.34,238.21)Tse (1992) 262.66 (15.47,4459.70)Utech (1996) 6.10 (3.64,10.24)Wahl (2004) 25.00 (1.03,608.09)Yang (2001) 45.00 (2.33,867.81)Yutani (2001) 240.88 (54.08,1072.94)Zornoza (2004) 12.33 (1.45,104.97)van Hoeven (2002) 19.85 (14.16,27.82)Overall (95% CI) FIXED EFECTS META-ANALYSIS
  67. 67. Heterogeneity is incorporated into the pooled estimate by including a between study component of variance.  Assumes sample of studies included in the analysis is drawn from a population of studies.  Each sample of studies has a true effect size. RANDOM EFFECTS META-ANALYSIS
  68. 68. Summary DOR, Random Effects Odds ratio .1 1 100 1000 10000 Study Odds ratio (95% CI) 36.27 (4.27,308.02)Adler (1997) 98.80 (10.66,916.11)Avril (1996) 21.00 (0.86,515.50)Bassa (1996) 4.33 (0.80,23.49)Danforth (2002) 107.23 (33.42,344.07)Greco (2001) 12.00 (1.23,117.41)Guller (2002) 429.00 (7.67,23982.81)Hubner (2000) 189.00 (6.63,5384.60)Lin (2002) 17.50 (1.84,166.04)Nakamoto (2002) 6.86 (1.40,33.57)Nakamoto (2002) 208.33 (7.72,5621.57)Noh (1998) 60.23 (3.09,1174.51)Ohta (2000) 106.33 (3.74,3023.90)Palmedo (1997) 289.00 (15.83,5276.04)Rostom (1999) 107.67 (3.85,3013.13)Scheidhauer (1996) 46.93 (14.47,152.17)Schirrmeister (2001) 266.00 (22.50,3145.19)Smith (1998) 9.00 (0.34,238.21)Tse (1992) 262.66 (15.47,4459.70)Utech (1996) 6.10 (3.64,10.24)Wahl (2004) 25.00 (1.03,608.09)Yang (2001) 45.00 (2.33,867.81)Yutani (2001) 240.88 (54.08,1072.94)Zornoza (2004) 12.33 (1.45,104.97)van Hoeven (2002) 42.54 (20.88,86.68)Overall (95% CI) RANDOM EFFECTS META-ANALYSIS
  69. 69.  Process of “prospectively” performing a new or updated analysis every time another trial is published  Provides answers regarding effectiveness of an intervention at the earliest possible date in time CUMULATIVE META-ANALYSIS
  70. 70.  Studies are sequentially pooled by adding each time one new study according to an ordered variable.  For instance, the year of publication; then, a pooling analysis will be done every time a new article appears. CUMULATIVE META-ANALYSIS
  71. 71.  In theory, the effect of any continuous or ordinal study-related variable can be assessed  Ex: sample size, study quality score, baseline risk etc CUMULATIVE META-ANALYSIS
  72. 72. metacum LogOR seLogOR, eform id(author) effect(f) graph cline saving(year_fcummplot, replace) CUMULATIVE META-ANALYSIS: SYNTAX
  73. 73. CUMULATIVE META-ANALYSIS: DIALOG 1
  74. 74. CUMULATIVE META-ANALYSIS: DIALOG 2
  75. 75. LogOR 7.67387 23982.8 Wahl 2004 Yang 2001 Yutani 2001 Zornoza 2004 Inoue 2004 Nakamoto(b) 2002 Nakamoto(a) 2002 van Hoeven 2002 Adler 1997 Rostom 1999 Avril 1996 Guller 2002 Schirrmeister 2001 Smith 1998 Utech 1996 Greco 2001 Ohta 2000 Hubner 2000 Bassa 1996 Tse 1992 Noh 1998 Scheidhauer 1996 Palmedo 1997 Lin 2002 Danforth 2002 CUMULATIVE META-ANALYSIS: PLOT
  76. 76.  Studies are pooled according to influence of a trial on overall effect defined as the difference between the effect estimated with and without the trial INFLUENCE ANALYSIS
  77. 77. metaninf tp fn fp tn, id(author) saving(influplot, replace) save(infcoeff, replace) INFLUENCE ANALYSIS: STATA SYNTAX
  78. 78. INFLUENCE ANALYSIS: STATA DIALOG
  79. 79. INFLUENCE ANALYSIS: STATA DIALOG
  80. 80. INFLUENCE ANALYSIS: PLOT 4.75 6.535.41 7.88 10.30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Lower CI Limit Estimate Upper CI Limit Meta-analysis estimates, given named study is omitted
  81. 81. INFLUENCE ANALYSIS: STATA DIALOG
  82. 82. A scatter plot of true positive fraction (sensitivity) vs. false positive fraction (1- specificity) Aids in visualization of range of results from primary studies ROC PLANE
  83. 83. twoway (scatter TPF FPF, sort ) (lfit uTPR FPF, sort range(0 1) clcolor(black) clpat(dash) clwidth(vthin) connect(direct)) (lfit sTPR FPF, sort range(0 1) clcolor(black) clpat(dot) clwidth(vthin) connect(direct)), ytitle(Sensitivity) ylabel(0(.1)1, grid) xtitle(1-Specificity) xlabel(0(.1)1, grid) title(ROC Plot of SENSITIVITY vs. 1-SPECIFICITY, size(medium)) legend(pos(3) col(1) lab(1 "Observed Data") lab(2 "Uninformative Test") lab(3 "Symmetry Line")) saving(ROCplot, replace) plotregion(margin(zero)) ROC PLANE: STATA SYNTAX
  84. 84. 0.1.2.3.4.5.6.7.8.91 Sensitivity 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1 1-Specificity Observed Data Uninformative Test Symmetry Line ROC Plot of SENSITIVITY vs. 1-SPECIFICITY ROC PLANE: PLOT
  85. 85.  ORDINARY LEAST SQUARES METHOD: Studies are weighted equally  WEIGHTED LEAST SQUARES METHOD: Weighted by the inverse variance weights of the odds ratio, or simply the sample size  ROBUST-RESISTANT METHOD: Minimizes the influence of outliers SROC: LINEAR REGRESSION MODELS
  86. 86.  Logit transformations of the TP rate (sensitivity) and FP rate (1 - specificity). D=ln(DOR) =logit(TPR) – logit(FPR)  Differences in logit transformations, D, regressed on sums of logit transformations, S. S=logit(TPR)+logit(FPR)  Logit(TPR)=natural log odds of a TP result and logit(FPR) =natural log of the odds of a FP test result. SROC: LOGIT TRANSFORMATION
  87. 87. twoway (scatter D S, sort msymbol(circle)) (lfit tfitted S, clcolor(black) clpat(solid) clwidth(thin) connect(direct))(lfit wfitted S, clcolor(black) clpat(dash) clwidth(thin) connect(direct)), ytitle(Discriminatory Power/D) xtitle(Diagnostic Threshold/S) title(REGRESSION PLOT) legend(lab(1 "Observed Data")lab(2 "EWLSR")lab(3 "VWLSR"))saving(regplot, replace) xline(0) yscale(noline) ACCURACY/THRESHOLD PLOT
  88. 88. 123456 DiscriminatoryPower/D -4 -2 0 2 4 Diagnostic Threshold/S Observed Data EWLSR VWLSR REGRESSION PLOT ACCURACY/THRESHOLD PLOT
  89. 89.  Back transformation of logistic regression to conventional axes of sensitivity [TPR] vs. (1 – specificity) [FPR]) with the equation  TPR = 1/{1 + exp[- a/(1 - b )]} [(1 - FPR)/(FPR)](1+b)/(1-b) .  Slope b and intercept a are obtained from the linear regression analyses SUMMARY ROC CURVE
  90. 90. twoway (scatter TPF FPF, sort msymbol(circle) msize(medium) mcolor(black))(fpfit tTPR FPF, clpat(dash)clwidth(medium) connect(direct ))(fpfit wTPR FPF, clpat(solid)clwidth(medium) connect(direct ))(lfit uTPR FPF, sort range(0 1) clcolor(black) clpat(dash) clwidth(thin) connect(direct)) (lfit sTPR FPF, sort range(0 1) clcolor(black) clpat(dot) clwidth(medium) connect(direct)), ytitle(Sensitivity/TPF) yscale(range(0 1)) ylabel( 0(.2)1,grid ) xtitle(1- Specificity/FPF) xscale(range(0 1)) xlabel(0(.2)1, grid) legend(lab(1 "Observed Data")lab(2 "EWLSR")lab(3 "VWLSR")lab(4 "RRLSR")lab(5 "Uninformative Test") lab(6 "Symmetry Line") pos(3) col(1)) title(SUMMARY ROC CURVES) graphregion(margin(zero)) saving(aSROCplot, replace) SUMMARY ROC CURVE: STATA SYNTAX
  91. 91. 0.2.4.6.81 Sensitivity/TPF 0 .2 .4 .6 .8 1 1-Specificity/FPF Observed Data EWLSR VWLSR RRLSR Uninformative Test SUMMARY ROC CURVES SUMMARY ROC CURVE: EXAMPLE
  92. 92. SUMMARY ROC : SUBGROUP ANALYSIS
  93. 93. STATA 8.2 (Stata Corp, College Station, Texas, USA) SOFTWARE

×