This presentation discusses the following topics: Hypothesis Test Potential Outcomes in Hypothesis Testing Significance level P-value Sampling Errors Type I Error What causes Type I errors? What causes Type II errors? 4 possible outcomes

1. Basics of RESEARCH
Dr. C.V. Suresh Babu
(CentreforKnowledgeTransfer)
institute
testing

2. Discussion Topics • Hypothesis Test
• Potential Outcomes in Hypothesis
Testing
• Significance level
• P-value
• Sampling Errors
• Type I Error
• What causes Type I errors?
• What causes Type II errors?
• 4 possible outcomes

3. Hypothesis tests
• Hypothesis tests are inferential procedures.
• These tests determine whether your sample evidence is strong
enough to suggest that an effect exists in an entire population.
Example:
Suppose you’re comparing the means of two groups. Your sample data show that there
is a difference between those means.
• Does the sample difference represent a difference between the two populations?
OR
• is that difference likely due to random sampling error?
That’s where hypothesis tests come in!

4. Hypothesis tests (cont..)
• Your sample data provide evidence for an effect.
• The significance level is a measure of how strong the sample
evidence must be before determining the results are
statistically significant.
• Because we’re talking about evidence !!!

5. Potential Outcomes in Hypothesis Testing
Hypothesis testing is a procedure in inferential statistics that assesses two
mutually exclusive theories about the properties of a population. For a
generic hypothesis test, the two hypotheses are as follows:
• Null hypothesis: There is no effect
• Alternative hypothesis: There is an effect.
• The sample data must provide sufficient evidence to reject the null
hypothesis and conclude that the effect exists in the population.
• Ideally, a hypothesis test fails to reject the null hypothesis when the
effect is not present in the population, and it rejects the null hypothesis
when the effect exists.

6. Significance level
• The significance level, also denoted as alpha or α, is a measure
of the strength of the evidence that must be present in your
sample before you will reject the null hypothesis and conclude
that the effect is statistically significant.
• The researcher determines the significance level before
conducting the experiment

7. • The significance level is the probability of rejecting the null
hypothesis when it is true.
For example,
• a significance level of 0.05 indicates a 5% risk of concluding
that a difference exists when there is no actual difference.
• Lower significance levels indicate that you require stronger
evidence before you will reject the null hypothesis.

8. • Use significance levels during hypothesis testing to help you
determine which hypothesis the data support.
• Compare your p-value to your significance level.
• If the p-value is less than your significance level, you can reject
the null hypothesis and conclude that the effect is statistically
significant.
• In other words, the evidence in your sample is strong enough
to be able to reject the null hypothesis at the population level.

9. P-value
• A p-value is the probability that you would obtain the effect
observed in your sample, or larger, if the null hypothesis is true
for the populations.
• P-values are calculated based on your sample data and under
the assumption that the null hypothesis is true.
• Lower p-values indicate greater evidence against the null
hypothesis.

10. P-value (cont..)
• Use p-values during hypothesis testing to help you determine
which hypothesis the data support.
• Compare your p-value to your significance level. If the p-value
is less than your significance level, you can reject the null
hypothesis and conclude that the effect is statistically
significant.
• In other words, the evidence in your sample is strong enough
to be able to reject the null hypothesis at the population level.

11. Sampling Errors
• Hypothesis tests use sample data to make inferences about the
properties of a population. You gain tremendous benefits by
working with random samples because it is usually impossible
to measure the entire population.
• However, there are trade-offs when you use samples. The
samples we use are typically a small percentage of the entire
population. Consequently, they occasionally misrepresent the
population severely enough to cause hypothesis tests to make
errors.

12. Type I Error
• In a hypothesis test, a type I error occurs when you reject a null
hypothesis that is actually true.
• In other words, a statistically significant test result suggests
that a population effect exists but, in reality, it does not exist.
The difference you observed in the sample is the product of
random sample error.

13. Type I (cont..)
• The probability of committing a type I error equals the
significance level you set for your hypothesis test.
• A significance level of 0.05 indicates that you are willing to
accept a 5% chance that you are wrong when you reject the
null hypothesis.
• To lower this risk, you must use a lower value for alpha.
• However, if you use a lower value for alpha, you are less likely
to detect a true difference if one really exists.

14. What causes Type I errors?
• It comes down to sample error. Your random sample has
overestimated the effect by chance.
• It was the luck of the draw. This type of error doesn’t indicate
that the researchers did anything wrong. The experimental
design, data collection, data validity, and statistical analysis can
all be correct, and yet this type of error still occurs.

15. What causes Type II errors?
• Whereas Type I errors are caused by one thing, sample error,
there are a host of possible reasons for Type II errors—small
effect sizes, small sample sizes, and high data variability.
• Furthermore, unlike Type I errors, you can’t set the Type II
error rate for your analysis.
• Instead, the best that you can do is estimate it before you
begin your study by approximating properties of the
alternative hypothesis that you’re studying.
• When you do this type of estimation, it’s called power analysis.

16. 4 possible outcomes
Test Rejects Null Test Fails to Reject Null
Null is True Type I Error False Positive Correct decision No effect
Null is False Correct decision Effect exists Type II error False negative