Upcoming SlideShare
×

# 2 stat critical_1

187 views

Published on

stat critical

Published in: Education, Technology
0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
187
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
7
0
Likes
0
Embeds 0
No embeds

No notes for slide

### 2 stat critical_1

1. 1. CABstat Critical Appraisal-based Statistical Training Bandit Thinkhamrop, Ph.D.(Statistics) Department of Biostatistics and Demography Faculty of Public Health Khon Kaen University
2. 2. Retrospective Statistical Practices (Read a published research article and do a critical appraisal) Begin at the conclusion Identify the primary research question Identify the primary study outcome Identify type of the study outcome Identify type of the study design Generate a data set that would've been used Identify type of the main statistical goal List choices of the statistical methods Select the most appropriate statistical method Perform the data analysis using a software Report and interpret the results from the outputs Summarize problems faced and lessons learned
3. 3. Retrospective Statistical Practices (Read a published research article and do a critical appraisal) Begin at the conclusion Identify the primary research question Identify the primary study outcome Identify type of the study outcome Identify type of the study design Generate a data set that would've been used Identify type of the main statistical goal List choices of the statistical methods Select the most appropriate statistical method Perform the data analysis using a software Report and interpret the results from the outputs Summarize problems faced and lessons learned
4. 4. Begin at the conclusion 7
5. 5. Retrospective Statistical Practices (Read a published research article and do a critical appraisal) Begin at the conclusion Identify the primary research question Identify the primary study outcome Identify type of the study outcome Identify type of the study design Generate a data set that would've been used Identify type of the main statistical goal List choices of the statistical methods Select the most appropriate statistical method Perform the data analysis using a software Report and interpret the results from the outputs Summarize problems faced and lessons learned
6. 6. Identify the primary research question Where to find the research question? – Title of the study – The objective(s) – The conclusion(s) If more than one, find the primary aim. Try to make the question “quantifiable”
7. 7. Retrospective Statistical Practices (Read a published research article and do a critical appraisal) Begin at the conclusion Identify the primary research question Identify the primary study outcome Identify type of the study outcome Identify type of the study design Generate a data set that would've been used Identify type of the main statistical goal List choices of the statistical methods Select the most appropriate statistical method Perform the data analysis using a software Report and interpret the results from the outputs Summarize problems faced and lessons learned
8. 8. Identify the primary study outcome It is the “primary” dependence variable It is the main finding that was used as the basis for the conclusion of the study It is the target of the statistical inference It is the basis for sample size calculation It resided in the : – – – – – – Title Research question Objective Sample size calculation Main finding in the RESULTS section of the report Conclusion
9. 9. Retrospective Statistical Practices (Read a published research article and do a critical appraisal) Begin at the conclusion Identify the primary research question Identify the primary study outcome Identify type of the study outcome Identify type of the study design Generate a data set that would've been used Identify type of the main statistical goal List choices of the statistical methods Select the most appropriate statistical method Perform the data analysis using a software Report and interpret the results from the outputs Summarize problems faced and lessons learned
10. 10. Type of the study outcome: Key for selecting appropriate statistical methods Study outcome – Dependent variable or response variable – Focus on primary study outcome if there are more Type of the study outcome – Continuous – Categorical (dichotomous, polytomous, ordinal) – Numerical (Poisson) count – Even-free duration
11. 11. Continuous outcome Parameters: – Mean (SD) – Median (Min:Max) – Correlation coefficient: r and ICC Modeling: – Linear regression The model coefficient = Mean difference – Quantile regression The model coefficient = Median difference Example: – Outcome = Weight, BP, score of ?, level of ?, etc. – RQ: Factors affecting birth weight
12. 12. Categorical outcome Parameters: – Proportion or Risk Modeling: – Logistic regression The model coefficient = Odds ratio (OR) Example: – Outcome = Disease (y/n), Dead(y/n), cured(y/n), etc. – RQ: Factors affecting low birth weight
13. 13. Numerical (Poisson) count outcome Parameters: – Incidence rate (e.g., rate per person time) Modeling: – Poisson regression The model coefficient = Incidence rate ratio (IRR) Example: – Outcome = Total number of falls Total time at risk of falling – RQ: Factors affecting tooth elderly fall
14. 14. Event-free duration outcome Parameters: – Median survival time Modeling: – Cox regression The model coefficient = Hazard ratio (HR) Example: – Outcome = Overall survival, disease-free survival, progression-free survival, etc. – RQ: Factors affecting survival
15. 15. The outcome determine statistics Continuous Mean Median Categorical Proportion (Prevalence Or Risk) Linear Reg. Count Survival Rate per “space” Median survival Risk of events at T(t) Logistic Reg. Poisson Reg. Cox Reg.
16. 16. Statistics quantify errors for judgments Parameter estimation [95%CI] Hypothesis testing [P-value] 7
17. 17. Statistics quantify errors for judgments 7
18. 18. Statistics quantify errors for judgments Parameter estimation [95%CI] Hypothesis testing [P-value] 7
19. 19. Retrospective Statistical Practices (Read a published research article and do a critical appraisal) Begin at the conclusion Identify the primary research question Identify the primary study outcome Identify type of the study outcome Identify type of the study design Generate a data set that would've been used Identify type of the main statistical goal List choices of the statistical methods Select the most appropriate statistical method Perform the data analysis using a software Report and interpret the results from the outputs Summarize problems faced and lessons learned
20. 20. Types of Research Quantitative Qualitative Phenomenology Grounded Theory Ethnography Description Observational Experimental Quasi-experimental Descriptive Analytical Clinical trial Field trial Community intervention trial Cross-sectional descriptive Prevalence survey Poll Cross-sectional Case-control Prevalence case-control Nested case-control Case-cohort case-control Randomized-controlled Cohort Parallel or Cross-over or factorial Fixed length or group sequential With or without baseline Prospective cohort Retrospective cohort Ambi-spective cohort Systematic review Meta-analysis
21. 21. Caution about biases 7
22. 22. Caution about biases Selection bias Information bias Confounding bias 7 Research Design -Prevent them -Minimize them
23. 23. Caution about biases Selection bias (SB) Information bias (IB) Confounding bias (CB) If data available: 7 SB & IB can be assessed CB can be adjusted using multivariable analysis
24. 24. Retrospective Statistical Practices (Read a published research article and do a critical appraisal) Begin at the conclusion Identify the primary research question Identify the primary study outcome Identify type of the study outcome Identify type of the study design Generate a data set that would've been used Identify type of the main statistical goal List choices of the statistical methods Select the most appropriate statistical method Perform the data analysis using a software Report and interpret the results from the outputs Summarize problems faced and lessons learned
25. 25. Generate a data set that would've been used General format of the data layout id 1 2 3 4 5 … n y x1 x2 X3
26. 26. Generate a data set that would've been used Continuous outcome example id 1 2 3 4 5 … n y 2 2 0 2 14 x1 1 0 1 0 1 x2 21 12 4 89 0 X3 22 19 20 21 18 6 0 45 21 Mean (SD)
27. 27. Generate a data set that would've been used Continuous outcome example id 1 2 3 4 5 … n y 1 1 0 0 0 x1 1 0 1 0 1 x2 21 12 4 89 0 X3 22 19 20 21 18 0 0 45 21 n, percentage
28. 28. Retrospective Statistical Practices (Read a published research article and do a critical appraisal) Begin at the conclusion Identify the primary research question Identify the primary study outcome Identify type of the study outcome Identify type of the study design Generate a data set that would've been used Identify type of the main statistical goal List choices of the statistical methods Select the most appropriate statistical method Perform the data analysis using a software Report and interpret the results from the outputs Summarize problems faced and lessons learned
29. 29. Common types of the statistical goals Single measurements (no comparison) Difference (compared by subtraction) Ratio (compared by division) Prediction (diagnostic test or predictive model) Correlation (examine a joint distribution) Agreement (examine concordance or similarity between pairs of observations)
30. 30. Retrospective Statistical Practices (Read a published research article and do a critical appraisal) Begin at the conclusion Identify the primary research question Identify the primary study outcome Identify type of the study outcome Identify type of the study design Generate a data set that would've been used Identify type of the main statistical goal List choices of the statistical methods Select the most appropriate statistical method Perform the data analysis using a software Report and interpret the results from the outputs Summarize problems faced and lessons learned
31. 31. Dependency of the study outcome required special statistical methods to handle it Example of dependency or correlated data: – – – – Before-after or Pre-post design Measuring paired organs i.e., ears, eyes, arms, etc. Longitudinal data, repeated measurement Clustered data, many observation unit within a cluster Choices of approaches: – Ignore it => use ordinary analysis as independency not save – Simplify it => use summary measure then analyze the data as it is independent – not efficient – Handle it => Mixed model, multilevel modeling, GEE recommended
32. 32. Dependency of the study outcome required special statistical methods to handle it Continuous Mean Median Categorical Proportion (Prevalence Or Risk) Linear Reg. Count Survival Rate per “space” Median survival Risk of events at T(t) Logistic Reg. Poisson Reg. Mixed model, multilevel model, GEE Cox Reg.
33. 33. Back to the conclusion Continuous Categorical Count Survival Appropriate statistical methods Mean Median Proportion (Prevalence or Risk) Rate per “space” Median survival Risk of events at T(t) Magnitude of effect 95% CI Answer the research question based on lower or upper limit of the CI P-value
34. 34. Always report the magnitude of effect and its confidence interval Absolute effects: – Mean, Mean difference – Proportion or prevalence, Rate or risk, Rate or Risk difference – Median survival time Relative effects: – Relative risk, Rate ratio, Hazard ratio – Odds ratio Other magnitude of effects: – – – – Correlation coefficient (r), Intra-class correlation (ICC) Kappa Diagnostic performance Etc.
35. 35. Touch the variability (uncertainty) to understand statistical inference id A 1 2 2 2 3 4 0 2 5 14 20 Sum (Σ) Mean(X) SD Median 4 (x-X) (x- X ) 2 -2 4 -2 4 -4 16 -2 10 4 100 0 0 128 32.0 2+2+0+2+14 = 20 2+2+0+2+14 = 20 = 4 5 5 0 2 2 2 14 Variance = SD2 5.66 2 Standard deviation = SD
36. 36. Touch the variability (uncertainty) to understand statistical inference id A 1 2 2 2 3 4 0 2 5 14 20 Sum (Σ) Mean(X) SD Median 4 (x-X) (x- X ) 2 -2 4 -2 4 -4 16 -2 10 4 100 0 0 128 32.0 5.66 2 Measure of central tendency Measure of variation
37. 37. Standard deviation (SD) = The average distant between each data item to their mean  (X − X) ∑ SD =  n −1  2 Degree of freedom    
38. 38. Same mean BUT different variation id A id B 1 2 2 2 1 2 4 3 3 4 0 2 3 4 5 4 5 14 20 5 4 20 Sum (Σ) Mean SD Median 4 5.66 Sum (Σ) Mean SD 4 0.71 2 Median 4 Heterogeneous data Homogeneous data
39. 39. Facts about Variation Because of variability, repeated samples will NOT obtain the same statistic such as mean or proportion: – Statistics varies from study to study because of the role of chance – Hard to believe that the statistic is the parameter – Thus we need statistical inference to estimate the parameter based on the statistics obtained from a study Data varied widely = heterogeneous data Heterogeneous data requires large sample size to achieve a conclusive finding
40. 40. The Histogram id A id B 1 2 1 4 2 2 2 3 3 0 3 5 4 2 4 4 5 14 5 4 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
41. 41. The Frequency Curve id A id B 1 2 1 4 2 2 2 3 3 0 3 5 4 2 4 4 5 14 5 4 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
42. 42. Area Under The Frequency Curve id A id B 1 2 1 4 2 2 2 3 3 0 3 5 4 2 4 4 5 14 5 4 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
43. 43. Central Limit Theorem Right Skew X1 Symmetry X2 Left Skew X3 X1 XX µ Xn Normally distributed
44. 44. Central Limit Theorem X1 Distribution of the raw data X2 X3 X1 XX µ Xn Distribution of the sampling mean
45. 45. Central Limit Theorem Distribution of the raw data X1 XX µ Xn Distribution of the sampling mean Large sample (Theoretical) Normal Distribution
46. 46. Central Limit Theorem Many X, X , SD X1 XX Xn µ Standard deviation of the sampling mean Standard error (SE) Estimated by SE = SD √n Many X , XX , SE Large sample Standardized for whatever n, Mean = 0, Standard deviation = 1
47. 47. (Theoretical) Normal Distribution
48. 48. (Theoretical) Normal Distribution
49. 49. 99.73% of AUC Mean ± 3SD
50. 50. 95.45% of AUC Mean ± 2SD
51. 51. 68.26% of AUC Mean ± 1SD
52. 52. Sample n = 25 X = 52 SD = 5 Population Parameter estimation [95%CI] Hypothesis testing [P-value]
53. 53. SD SE = n 5 SE = 25 5 5 = 1
54. 54. Report and interpret p-value appropriately Example of over reliance on p-value: – Real results: n=5900; ORDrug A vs Drug B = 1.02 (P<0.001) – Inappropriate: Quote p-value as < 0.05 or put * or **** (star) to indicate significant results – Wrong: Drug A is highly significantly better than Drug B (P<0.001) – What if 95%CI: 1.001 to 1.300? – This is no clinical meaningful at all….!
55. 55. Report and interpret p-value appropriately Example of over reliance on p-value: – Real results: n=30; ORDrug A vs Drug B = 9.2 (P=0.715) – Inappropriate: Quote p-value as > 0.05 – Wrong: There is no statistical significant difference of the treatment effect (P<0.05). Thus Drug A is as effective as Drug B – What if 95%CI: 0.99 to 28.97? – This is study indicated a low power, NOT suggested an equivalence…! – Correct: There was no sufficient information to concluded that . . . => inconclusive findings
56. 56. P-value is the magnitude of chance NOT magnitude of effect P-value < 0.05 = Significant findings Small chance of being wrong in rejecting the null hypothesis If in fact there is no [effect], it is unlikely to get the [effect] = [magnitude of effect] or more extreme Significance DOES NOT MEAN importance Any extra-large studies can give a very small Pvalue even if the [magnitude of effect] is very small
57. 57. P-value is the magnitude of chance NOT magnitude of effect P-value > 0.05 = Non-significant findings High chance of being wrong in rejecting the null hypothesis If in fact there is no [effect], the [effect] = [magnitude of effect] or more extreme can be occurred chance. Non-significance DOES NOT MEAN no difference, equal, or no association Any small studies can give a very large P-value even if the [magnitude of effect] is very large
58. 58. P-value vs. 95%CI (1) An example of a study with dichotomous outcome A study compared cure rate between Drug A and Drug B Setting: Drug A = Alternative treatment Drug B = Conventional treatment Results: Drug A: n1 = 50, Pa = 80% Drug B: n2 = 50, Pb = 50% Pa-Pb = 30% (95%CI: 26% to 34%; P=0.001)
59. 59. P-value vs. 95%CI (2) Pa > Pb Pb > Pa Pa-Pb = 30% (95%CI: 26% to 34%; P< 0.05)
60. 60. P-value vs. 95%CI (3) Adapted from: Armitage, P. and Berry, G. Statistical methods in medical research. 3rd edition. Blackwell Scientific Publications, Oxford. 1994. page 99
61. 61. Tips #6 (b) P-value vs. 95%CI (4) Adapted from: Armitage, P. and Berry, G. Statistical methods in medical research. 3rd edition. Blackwell Scientific Publications, Oxford. 1994. page 99 There were statistically significant different between the two groups.
62. 62. Tips #6 (b) P-value vs. 95%CI (5) Adapted from: Armitage, P. and Berry, G. Statistical methods in medical research. 3rd edition. Blackwell Scientific Publications, Oxford. 1994. page 99 There were no statistically significant different between the two groups.
63. 63. P-value vs. 95%CI (4) Save tips: – Always report 95%CI with p-value, NOT report solely p-value – Always interpret based on the lower or upper limit of the confidence interval, p-value can be an optional – Never interpret p-value > 0.05 as an indication of no difference or no association, only the CI can provide this message.
64. 64. Retrospective Statistical Practices (Read a published research article and do a critical appraisal) Begin at the conclusion Identify the primary research question Identify the primary study outcome Identify type of the study outcome Identify type of the study design Generate a data set that would've been used Identify type of the main statistical goal List choices of the statistical methods Select the most appropriate statistical method Perform the data analysis using a software Report and interpret the results from the outputs Summarize problems faced and lessons learned
65. 65. The outcome determine statistics Continuous Mean Median Categorical Proportion (Prevalence Or Risk) Linear Reg. Count Survival Rate per “space” Median survival Risk of events at T(t) Logistic Reg. Poisson Reg. Cox Reg.
66. 66. Dependency of the study outcome required special statistical methods to handle it Continuous Mean Median Categorical Proportion (Prevalence Or Risk) Linear Reg. Count Survival Rate per “space” Median survival Risk of events at T(t) Logistic Reg. Poisson Reg. Mixed model, multilevel model, GEE Cox Reg.
67. 67. Back to the conclusion Continuous Categorical Count Survival Appropriate statistical methods Mean Median Proportion (Prevalence or Risk) Rate per “space” Median survival Risk of events at T(t) Magnitude of effect 95% CI Answer the research question based on lower or upper limit of the CI P-value
68. 68. Retrospective Statistical Practices (Read a published research article and do a critical appraisal) Begin at the conclusion Identify the primary research question Identify the primary study outcome Identify type of the study outcome Identify type of the study design Generate a data set that would've been used Identify type of the main statistical goal List choices of the statistical methods Select the most appropriate statistical method Perform the data analysis using a software Report and interpret the results from the outputs Summarize problems faced and lessons learned
69. 69. Perform the data analysis using a software Use the data being generated similar to what should have been used in the research article Analyze the way that the article did Analyze the way it should be based on your own judgments Try to understand the computer output Always ask yourself – does the output answer the research question?
70. 70. Retrospective Statistical Practices (Read a published research article and do a critical appraisal) Begin at the conclusion Identify the primary research question Identify the primary study outcome Identify type of the study outcome Identify type of the study design Generate a data set that would've been used Identify type of the main statistical goal List choices of the statistical methods Select the most appropriate statistical method Perform the data analysis using a software Report and interpret the results from the outputs Summarize problems faced and lessons learned
71. 71. Report and interpret the results from the outputs Interpret the results based on the confidence interval rather than the p-value
72. 72. Retrospective Statistical Practices (Read a published research article and do a critical appraisal) Begin at the conclusion Identify the primary research question Identify the primary study outcome Identify type of the study outcome Identify type of the study design Generate a data set that would've been used Identify type of the main statistical goal List choices of the statistical methods Select the most appropriate statistical method Perform the data analysis using a software Report and interpret the results from the outputs Summarize problems faced and lessons learned
73. 73. Summarize problems faced and lessons learned Write in your own words Evaluate what you wrote by the new round of steps stated previously Keep on doing this way Eventually you will find statistics a logical and intuitive tools for you
74. 74. Q&A Thank you