This document provides an overview of the basics of significance testing. It discusses key concepts like hypotheses (the null and alternative hypotheses), test statistics, p-values, and significance levels. The goal of significance testing is to quantify the evidence against a null hypothesis using collected data. It provides an example of testing whether a population on average experienced weight gain, with the null hypothesis being no average weight gain and the alternative being that there was average weight gain. The document outlines the steps of stating hypotheses, calculating a test statistic, determining the p-value, and interpreting based on the significance level.

1. Basics of Significance Testing
4TESTS OF SIGNIFICANCE: THE BASICS

2. SIGNIFICANCE TESTING
• Also called “hypothesis testing”
• Objective: to test a claim about parameter μ
(population mean)
• Procedure:
A.State hypotheses H0 and Ha
B.Calculate test statistic
C.Convert test statistic to P-value and interpret
D.Consider significance level (optional)
Basics of Significance Testing 2

3. HYPOTHESES
• H0 (null hypothesis) claims “no difference”
• Ha (alternative hypothesis) contradicts the
null
• Example: We test whether a population
gained weight on average…
H0: no average weight gain in population
Ha: H0 is wrong (i.e., “weight gain”)
• Next  collect data  quantify the extent
to which the data provides evidence
against H0
Basics of Hypothesis Testing 3

4. ONE-SAMPLE TEST OF MEAN
• To test a single mean, the null hypothesis is
H0: μ = μ0, where μ0 represents the “null
value” (null value comes from the research
question, not from data!)
• The alternative hypothesis can take these
forms:
Ha: μ > μ0 (one-sided to right) or
Ha: μ < μ0 (one-side to left) or
Ha: μ ≠ μ0 (two-sided)
• For the weight gain illustrative example:
H0: μ = 0
Ha: μ > 0 (one-sided) or Ha: μ ≠ μ0 (two-sided)
Note: μ0 = 0 in this example
Basics of Significance Testing 4

5. P-VALUE
• The P value or calculated probability is the estimated
probability of rejecting the null hypothesis (H0) of a study
question when that hypothesis is true. The p-value is a
number between 0 and 1
• Smaller-and-smaller P-values → stronger-and-stronger
evidence against H0
• Conventions for interpretation
• Small p-value (typically ≤ 0.05) indicates strong evidence against
the null hypothesis, so you reject the null hypothesis.
• A large p-value (> 0.05) indicates weak evidence against the null
hypothesis, so you fail to reject the null hypothesis.
• p-values very close to the cutoff (0.05) are considered to be
marginal (could go either way).
Basics of Significance Testing 5

6. SIGNIFICANCE LEVEL
• α ≡ threshold for “significance”
• We set α
• For example, if we choose α = 0.05, we
require evidence so strong that it would
occur no more than 5% of the time when
H0 is true
• Decision rule
P ≤ α  statistically significant evidence
P > α  nonsignificant evidence
• For example, if we set α = 0.01, a P-value of
0.0006 is considered significant
Basics of Significance Testing 6

7. 6/5/2014 Basics of Significance Testing 7
QUESTIONS?