Successfully reported this slideshow.

×

## Check these out next

Primer on the application of statistical significance testing for business research purposes.
1) How to use statistics to make more informed decisions (and when not to use).
2) Highlight differences between statistics in science vs business.
3) Highlight assumptions, limitations and best practices.

Primer on the application of statistical significance testing for business research purposes.
1) How to use statistics to make more informed decisions (and when not to use).
2) Highlight differences between statistics in science vs business.
3) Highlight assumptions, limitations and best practices.

## More Related Content

### Similar to Statistics for Business Decision-making (20)

1. 1. 1 STATISTICAL SIGNIFICANCE & BUSINESS RELEVANCE How to apply statistics for business decision-making.
2. 2. Is Superman stronger than Charlie Brown? 2 Some tests are easy to analyze Don’t need statistical testing
3. 3. Is Superman stronger than Batman? 3 Data does not show a clear overwhelming winner Use statistical significance to determine if their findings are valid.
4. 4. Use statistics only if valuable. • If statistics can benefit me, I will use it. • If statistics poses a threat, then I will not. 4
5. 5. Lets take an example… 5 Research Question Determine the effectiveness of share-a-coke campaigns. Business Question Coke needs to select an ad to go to market with a \$10M investment.
6. 6. Research needs 3 Things… 1. Who are Coke Users/Non-Users? – Sample to recruit to include in study 2. What will we compare the adcepts to? – Independent Variable 3. What is a valid & reliable measure of effectiveness? – Outcome Measure / Dependent Variable 6
7. 7. 1. SAMPLING: Who are Coke Users/Non-Users? 7 populationconclusions based on the sample sample generalization to the population hypotheses
8. 8. SAMPING BIAS: The greater the variation in the underlying population, the larger the sampling error. 8 Gallo, 2016; HBR
9. 9. 2. INDEPENDENT VARIABLE: What will we compare the adcepts to determine the effectiveness? 9 CONTROL No-Adcept Control Previous Best Competitor
10. 10. Compare Superman to: Result Interpretation Superman is significantly stronger; Lets choose him. Superman is not significantly stronger; (no statistical differences); Lets compare the differences in strength We may keep looking The comparison is KEY for interpreting data in order to make relevant business decisions 10 Business Question: Should we choose Superman to help us win?
11. 11. Compare to: Hypothetical Result: Confidence in Making Decision Will it perform better than nothing? Significant Not as confident Wil perform better than our previous best? Significant More confidence Wil perform better than our competition? Significant More confidence 11 Business Question: What adcept will be most effective campaign? CONTROL The comparison is KEY for interpreting data in order to make relevant business decisions
12. 12. Choosing Comparisons Take Aways… 1. Choose comparisons that are grounded in the business decision context. 2. Instead of thinking in terms of statistical significance, p values (.05), and confidence intervals (which are limited for business application) think in terms of methods to increase subjective confidence to make the decision (better comparisons, additional comparisons, better dependent variables, using effect sizes). 12
13. 13. The Two Hypothesis! Null Hypothesis Treatment There is NO difference between the two groups There is a difference between the two groups = no effect = there is an effect 13 Sig Differences?
14. 14. When you perform a test of statistical significance you usually reject or do not reject the Null Hypothesis (H0). The null hypothesis – no difference between treatment effects or – no association between variables 14
15. 15. Significance Testing Statistically significant mean difference at p < .05 tells us that if we sampled many pairs of groups from the same hypothetical population, we would expect to get a difference as large as the observed result or larger with no more than 5% of the groups as the result of sampling error, given that the null hypothesis 15
16. 16. SO WHAT? 16
17. 17. Statistical Significance • P values, or significance levels, measure the strength of the evidence against the null hypothesis Significance can only tell the likelihood that a relationship exists It can’t tell whether or not it’s important. 17 Sterne, 2001
18. 18. Much value in making business decisions is with effect sizes. That is, how much stronger is Superman than Batman? 18 SIGNIFIGANCE A P value describes the likelihood of a true relationship between X (Superman) and Y (Batman) MAGNITUDE & EFFECT SIZE Effect size show the magnitude or size of the relationship between X (Superman) and Y (Batman)?
19. 19. What is statistical significance… • A result has statistical significance when it is very unlikely to have occurred given the null hypothesis. • More precisely, the significance level defined for a study, α, is the probability of the study rejecting the null hypothesis, given that it were true 19
20. 20. Effect Size is a more practical for business purposes. 20 CONTROL EFFECT SIZE How much more effective? SIGNIFIGANCE Is there a relationship? ?
21. 21. What are the limitations of inferential statistics? 21
22. 22. People overvalue the role of statistics Statistical significance testing is often misused: 1. Endow them with capabilities they do not have. 2. Utilize them as the sole approach to analyzing data. We should… 1. Become aware of the limitations of most inferential statistics. 2. Augmenting statistics with other information and research approaches. 22 Sawyer & Peter, 1983
23. 23. What is important to a Manager may not be statistically significant. Alternatively, what is not important to a Manger may be statistically significant. 23 People overvalue the role of statistics
24. 24. Successful use of statistics is different… 24 Academics Business Prove a point Make a decision Statistical Significance & High Confidence intervals Practical Significance & Subjective confidence (Decision certainty)
25. 25. Statistical Tests are NOT completely objective. The statistical significance level obtained is strongly influenced by subjective decisions by the researcher. 25
26. 26. 26 1. 1 or 2-tailed test 2. Level of significance 3. Number of observations 1. Standard deviation 2. Amount of deviation from the null hypothesis Controlled by researcher Influenced by researcher Sawyer & Peter, 1983 Statistical Tests are NOT Completely Objective.
27. 27. 4 Misinterpretations of Significant Results 1. Probability of the Null Hypothesis • Probability that the results occurred because of chance 2. Probability of the Results being Replicated • Probability that results will be replicated in the future 3. Probability of Results Being Valid • Probability that the alternative hypothesis is true 4. Sample Size and Probability of the Research Hypothesis • Confusion about the sample size and level of statistical significance 27 Sawyer & Peter, 1983
28. 28. 5 Ways to turn “Non-significant” into “Significant” 1. Increasing the sample size 2. Increasing the reliability of the measures 3. Changing post-hoc the acceptable level of statistical significance 4. Changing from 2-tailed to 1 tailed test 5. Obtaining better control over non-manipulated variables 28 Sawyer & Peter, 1983
29. 29. 3 Common Misunderstandings with sample size and statistical significance 1. Relationship between sample size and level of statistical significance implied that more confidence should accompany the result of the study had a large sample size rather than a small one. 2. Larger samples do reduce likely sampling error because their estimates more closely approximate the population parameters, but it should also be clear that differences in the amount of sampling error are included explicitly in the computation of statistical significance tests. 3. There should NOT be a bias against statistically significant results obtained from properly selected small samples. 29 Sawyer & Peter, 1983
30. 30. Statistics Humor… 30
31. 31. Critics of statistics… • "Reliance on merely refuting the null hypothesis...is basically unsound, poor scientific strategy, and one of the worst things that ever happened in the history of psychology.” • "Can you articulate even one legitimate contribution that significance testing has made (or makes) to the research enterprise (i.e., any way in which it contributes to the development of cumulative scientific knowledge)?” • Is there any study wherein statistical significance improves decision- making? 31 Schmidt, 1996; Meehl, 1978
32. 32. Citations 1. Gallo, A. (2016). A Refresher on Statistical Significance. Harvard business review. 2. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of consulting and clinical Psychology 3. Sawyer, A. G., & Peter, J. P. (1983). The significance of statistical significance tests in marketing research. Journal of marketing research. 4. Schmidt, F. L. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. 5. Sterne, J. A., & Smith, G. D. (2001). Sifting the evidence—what's wrong with significance tests?. Physical Therapy. 32
33. 33. Jason Martuscello 33 jason@prospectionsciences.com