This document provides an overview of analysis of variance (ANOVA). It defines ANOVA as comparing means between more than two groups and partitioning total variation into treatment and error variations. The document outlines assumptions of ANOVA, how to set up one-way ANOVA models, calculate test statistics like F-ratio, and interpret ANOVA outputs. Examples are provided to demonstrate one-way ANOVA calculations and interpreting results using SPSS. Post-hoc tests for multiple comparisons are also introduced.

1. Analysis of Variance
[ANOVA]
Emiru Merdassa (MSc, Assistant Professor)
24 February 2023 1

2. Learning Objectives
By the end of this session, students will be able to:
1. Understand and identify appropriate occasion to employ analysis of variance
(ANOVA)
2. State assumptions of the ANOVA
3. Run one-way and two-way ANOVA (using SPSS)
4. Interpret the F-statistic and the output from one-way and Two ANOVA
5. Understand multiple comparisons (post hoc comparisons) in ANOVA
24 February 2023 2

3. Introduction
• The term "Analysis of Variance" was introduced by Prof. R. A. Fisher in 1920.
• The basic purpose of the analysis of variance is to test the homogeneity of
several means.
• ANOVA consists in estimation of the amount of variation due to each of the
independent factors (causes) separately and then comparing these estimates
due to assignable factors (causes) with the estimate due to chance factor (error).
• ANOVA stands for analysis of variance
24 February 2023 3

4. Prof. R. A. Fisher (1890-1962)
• He was active as a mathematician, statistician,
biologist, geneticist, and academic.
• Fisher laid the foundation of statistical inference,
invented experimental design, randomization, ANOVA,
...
• Fisher published a number of important texts; in
particular Statistical Methods for Research Workers in
1925
24 February 2023 4

5. Definition: ANOVA
• It compares means between more than 2 groups
• A factor refers to a categorical quantity under examination in an experiment as a
possible cause of variation in the response variable.
• Levels -refer to the categories, measurements, or strata of a factor of interest in
the experiment.
24 February 2023 5

6. ANOVA Basic Idea
• Compares two types of variation to test equality of means.
• If treatment variation is significantly greater than random variation then
means are Not equal.
• Variation measures are obtained by ‘Partitioning’ total variation.
24 February 2023 6

7. One-Way ANOVA Partitions
Sum of Squares Within
Sum of Squares Error (SSE)
Within Groups Variation
Sum of Squares Among
Sum of Squares Between
Sum of Squares Treatment (SST)
Among Groups Variation
Variation due to
treatment
Variation due to
random sampling
Total variation
24 February 2023 7

8. 𝐒𝐒 𝐓𝐨𝐭𝐚𝐥 = x11 − lj
x 2
+ x21 − lj
x 2
+ ⋯ + xij − lj
x
2
Total variation
Treatment Variation[see the unbroken line arrows]
𝐒𝐒𝐓 = 𝐧𝟏 lj
𝐱𝟏 − lj
𝐱 𝟐
+ 𝐧𝟐 lj
𝐱𝟐 − lj
𝐱 𝟐
+ ⋯ + 𝐧𝐩 lj
𝐱𝐩 − lj
𝐱
𝟐
Random (Error) Variation[see the broken line arrows]
𝐒𝐒𝐄 = 𝑥11 − lj
𝑥1
2
+ 𝑥21 − lj
𝑥1
2
+ ⋯ + 𝑥𝑝𝑗 − lj
𝑥𝑝
2
24 February 2023 8

9. Cont’d
Group 2
Group 3
Group 1
24 February 2023 9

10. The one-way ANOVA model
❖One-way ANOVA requires calculation of the following:
–Between groups sum of squares:𝐒𝐒𝐁 = σi=1
k
ni ത
yi − ത
y 2
–Within-groups sum of squares: SSW= σij=1
n
yij − ത
yj
2
–Total Sum of Squares: TSS = SSB + SSW
24 February 2023 10

11. Cont’d
–Between-groups degrees of freedom = k-1
–Within-groups degrees of freedom = n-k, where n = n1 + n2 + n3 + … + nk
–The total degrees of freedom is = n-1
• The F-statistic is used to test the hypothesis is Fcalc =
MSB
MSW
• Fcalc ~ Fdist. with (k - 1; n -k) degrees of freedom.
• If Fcalc > F1−𝛼(k-1; n-k) or p-value < 𝛼, then reject Ho.
24 February 2023 11

12. One-Way ANOVA Summary Table
Source of
Variation
Degrees of
Freedom Sum of Squares
Mean Square
(Variance)
F
Treatment k - 1 SST MST = SST/(k - 1) MST
MSE
Error n - k SSE MSE = SSE/(n - k)
Total n - 1 SS(Total) = SST+SSE
24 February 2023 12

13. One-Way ANOVA F-Test Critical Value
13

If means are equal, F = MST / MSE
 1. Only reject large F
Fa k n k
( , )
− −
1
0
Reject H 0
Do Not
Reject H 0
F
© 1984-1994 T/Maker Co.
24 February 2023

14. Variables used
●Tests the Equality of 2 or More (p) Population Means
●Variables
–One categorical Independent Variable
–One Continuous Dependent Variable
24 February 2023 14

15. Assumptions
0The outcome is normally distributed.
0Population variance is assumed constant among the groups.
0Independent random samples among the groups.
24 February 2023 15

16. Example: One way ANOVA
• For one factor with k groups.
• Suppose we have 16 subjects available to participate in an experiment in
which we wish to compare four drugs.
• We number the subjects from 01 through 16.
• We then go to a table of random numbers and select 16 consecutive,
unduplicated numbers between 01 and 16.
24 February 2023 16

17. Cont’d
Figure.1 Allocation of subjects to treatments, completely randomized design.
24 February 2023 17

18. Table 1: Table of Sample Values for the one way ANOVA
24 February 2023 18

19. The ANOVA Procedure
– Step #1. Set up hypotheses and determine level of significance
– H0: μ1 = μ2 = μ3 = μ4
– H1: Means are not all equal
α = 0.05
– Step #2. Select the appropriate test statistic. The test statistic is the F statistic for ANOVA,
F=MSB/MSE.
– Step #3. Set up decision rule. In order to determine the critical value of F we need degrees of
freedom, df1=k-1 and df2 = n-k.
– Step #4. Compute the test statistic
– Step #5. Conclusion.
24 February 2023 19

20. Example
Treatment 1 Treatment 2 Treatment 3 Treatment 4
60 50 48 47
67 52 49 67
42 43 50 54
67 67 55 67
56 67 56 68
62 59 61 65
64 67 61 65
59 64 60 56
72 63 59 60
71 65 64 65
24 February 2023 20

21. 1. State the Hypotheses
Null Hypothesis
 Ho : None of the groups will differ on the mean
 Ho = μ1 = μ2 = μ3
Alternative Hypothesis (Nondirectional)
 HA : At least one of the groups will have a different mean.
 HA : μ1 ≠ μ2 or μ1 ≠ μ3 or μ2 ≠ μ3
oNote: ANOVA models can only tell us if one or more of the group means differ from the
others. If the means of any two groups are different from each other, the null hypothesis
can be rejected.
24 February 2023 21

22. 2. Select appropriate test statistics
Test Statistic
• F = MST / MSE=V.R.
• MST Is Mean Square for Treatment
• MSE Is Mean Square for Error
Degrees of Freedom
✓1 = k -1
✓2 = n - k
• k = # Populations, Groups, or Levels
• n = Total Sample Size
=
𝐒𝐒𝐓/ 𝐤 − 𝟏
𝐒𝐒𝐄/ 𝐧 − 𝐤
❖ Select appropriate test statistics: for One-Way ANOVA: F-Test Test Statistic
24 February 2023 22

23. Define the α–Level and Find the Critical Value
●α‐Level
– Statistical significance will be defined as p < 0.05 (the most common)
●Critical Value
– Using the table of critical values of the F-table, we find that the critical value that
defines the rejection region (Fk-1, n-k)
– If the computed F-ratio (F-statistic) is greater than F-critical, we will reject the null
hypothesis
24 February 2023 23

24. Find the Critical Value
●To find the critical value from the F-table, we need the degrees of freedom for both
between the groups and within the groups.
●The difference between groups = number of groups − 1
In this case, it is 4 − 1=3
●The difference within groups = total n − number of groups
In this case, it is 40 − 4 = 36
●In the F-table, we use the value at F3,36, which is as close as we can find in this
table(2.87).
24 February 2023 24

25. 4. Compute F- statistics
Treatment 1 Treatment 2 Treatment 3 Treatment 4
60 50 48 47
67 52 49 67
42 43 50 54
67 67 55 67
56 67 56 68
62 59 61 65
64 67 61 65
59 64 60 56
72 63 59 60
71 65 64 65
Calculate the sum of squares between groups:
Mean for group 1 = 62.0
Mean for group 2 = 59.7
Mean for group 3 = 56.3
Mean for group 4 = 61.4
Grand mean = 59.85
 SSB = [(62-59.85)2 + (59.7-59.85)2 + (56.3-59.85)2 + (61.4-59.85)2 ] x n per group
= 19.65x10 = 196.5
24 February 2023 25

26. Cont’d
26
Treatment 1 Treatment 2 Treatment 3 Treatment 4
60 50 48 47
67 52 49 67
42 43 50 54
67 67 55 67
56 67 56 68
62 59 61 65
64 67 61 65
59 64 60 56
72 63 59 60
71 65 64 65
✓ calculate the sum of squares within groups:
(60-62) 2+(67-62) 2+ (42-62) 2+ (67-62) 2+ (56-62) 2+
(62-62) 2+ (64-62) 2+ (59-62) 2+ (72-62) 2+ (71-62) 2+
(50-59.7) 2+ (52-59.7) 2+ (43-59.7) 2+67-59.7) 2+ (67-
59.7) 2+ (69-59.7) 2…+….(sum of 40 squared
deviations) = 2060.6
24 February 2023

27. Fill in the ANOVA table
3 196.5 65.5
1.14 .344
36 2060.6 57.2
Source of
variation
d.f. Sum of squares Mean Sum of
Squares
F-statistic p-value
Between
Within
Total 39 2257.1
Interpretation of ANOVA:
• How much of the variance in height is explained by treatment group?
• R2 = “Coefficient of Determination” = SSB/TSS = 196.5/2275.1= 9%
24 February 2023 27

28. Cont’d
●Determine overall significance using F-test
●If statistically significant, need to run post hoc (multiple comparison)
tests, a topic for next lecture
●Since the computed F-statistic of 1.144 is less than the critical value of
2.87, we conclude that there is no difference between the means(not
statistically significant)
24 February 2023 28

29. Conclusion
●A brief example after overall F-test:
– Type of treatment is not significantly associated with the response.
●No need of post hoc tests as the F-test was not significant
24 February 2023 29

30. Coefficient of Determination
30
R2
=
SSB
SSB + SSE
=
SSB
SST
 The amount of variation in the outcome variable (dependent variable) that is
explained by the predictor (independent variable).
24 February 2023

31. SPSS: Check the Assumptions
– The measures constitute an independent random sample
– The factor, type of treatment has at least three levels (it has four)
– The dependent variable, response, is normally distributed (using histogram)
– The assumption of homogeneity of variance is not violated (per the Levene’s
test in SPSS)
– Note: If the data were not normally distributed or the assumption of
homogeneity of variance was not met, we could use the Kruskal-Wallis H-
test instead (to be dealt under nonparametric statistics).
24 February 2023 31

32. Analysis ➔ Descriptive statistics ➔ Explore ➔ Plots
Under plots
Click for
Normality plots with tests
24 February 2023 Slide 32 of 68

33. Normal Q-Q plot, tells us that if the data is normally distributed, then the red dots should lie on the straight diagonal
line
Output
Test of Normality
Kolmogorov- Smirnov and Shapiro-wilk are statistics that differentiate normally from non-normally distributed, If
significant, then it tells us that the data is not normally distributed .
If Significant, it is not
normally distributed
24 February 2023

34. Histogram
24 February 2023 34

35. Homogeneity of Variance
–Levene’s test is not significant (p=0.684, which is p > 0.05), so
homogeneity of variance assumption is satisfied.
24 February 2023 35

36. Cont’d
24 February 2023 36

37. SPSS ANOVA Table output
24 February 2023 37

38. Multiple Comparisons
• ANOVA only tells us if there is an effect
• That is, are the means of the groups not all equal?
• ANOVA does not tell us which means are different from which other means
• Multiple comparisons are used to determine which means are probably different from
which other means
• The α inflation can occur when the same (without adjustment) significant level is applied
to the statistical analysis to one and other families simultaneously
• Inflated α = 1 − (1 − α)𝑛, n = number of hypotheses tested.
38
24 February 2023

39. Post-hoc Tests
I. Equal Variances Assumed
oTukey, Bonferroni, Sidak, Scheffe, LSD, Duncan Hochberg's GT2.,
Gabriel., Waller-Duncan, Dunnett.
II. Violation of the assumption of equivalence of variance
oTamhane's T2, Dunnett's T3, Games-Howell, Dunnett's C..
24 February 2023 39

40. Post-hoc test
I. The Bonferroni Approach
– A conservative way to circumvent the problem of distorted significance levels when
performing several tests involves reducing the significance level used for each individual
test sufficiently to fix the overall significance level (i.e., the probability of incorrectly
rejecting at least one of the null hypotheses being tested) at some desired level (say, 𝛼).
If we perform n such tests, the maximum possible value for this overall significance
level is 𝐧 × 𝜶
– One disadvantage of the Bonferroni method is that the true overall significance level
may be considerably lower than 𝛼, and, in extreme situations, it may be so low that
none of the individual tests will be rejected.
24 February 2023 40

41. Cont’d
II. Tukey–Kramer Method
• It is applicable when pairwise comparisons of population means are of
primary interest
III. Scheffé’s Method
• It is generally recommended when comparisons other than simple pairwise
differences between means are of interest
24 February 2023 41

42. Choosing a multiple-comparison technique
• All of these methods have been developed to control the overall type I error rate at no
more than 5%.
• If all multiple comparison procedures give similar results we can be confident of our
conclusion. However, if different procedures give different results, our conclusion may
need to be more judgmental, relying on the limitations of application. For example,
• Bonferroni method should be used when small numbers of comparisons are performed.
Bonferroni's method does not require equal sample sizes.
24 February 2023 42

43. Choosing a multiple-comparison technique
• Turkey’s method should be used when sample sizes are equal.
• Scheffé’s method should be used when sample sizes are markedly
different, when all possible comparisons are performed. The confidence
intervals produced by Scheffe’s method will generally be wider than the
Tukey or Bonferroni
• Dunnett method: The Dunnett test is used by researchers interested in
testing two or more experimental groups against a single control group.
24 February 2023 43

44. TWO-WAY ANALYSIS OF VARIANCE
• Two-way ANOVA is a type of study design with one numerical outcome variable and
two categorical explanatory variables.
• Example – In a completely randomised design we may wish to compare outcome by
age, gender or disease severity. Subjects are grouped by one such factor and then
randomly assigned one treatment.
• Technical term for such a group is block and the study design is also called
randomised block design.
24 February 2023 44

45. Example: Two way ANOVA
• There are situations where it may be of interest to compare means of a
continuous outcome across two or more factors.
• For example, a clinical trial is designed to compare three different treatments for
joint pain in patients with osteoarthritis. Investigators are also hypothesize that
there are differences in the outcome by sex.
This is an example of a two-way ANOVA where the factors are treatment
(with three levels) and sex (with two levels).
24 February 2023 45

46. Cont’d
• Investigators can assess whether there are differences in means due to the
treatments, due to the sex of the participant, or due to the combination or
interaction of treatment and sex.
• If there are differences in the means of the outcome by treatment, we say there
is a main effect of treatment. If there are differences in the means of the
outcome by sex, we say there is a main effect of sex. If there are differences in
the means of the outcome among treatments but these vary by sex, we say
there is an interaction effect.
24 February 2023 46

47. Table of sample value for the randomized complete block design
24 February 2023 47

48. Way ANOVA Summary Table
48
Source of
Variation
Degrees of
Freedom
Sum of
Squares
Mean
Square F
Bl
(blocks)
k - 1
SSBl MSBl MSBl
MSE
Tr
(treatments)
n - 1
SSTr MSTr
MSTr
MSE
(k-1)(n-1)
Error
SSE MSE
Total kn - 1 SS(Total)
24 February 2023

49. Example
Consider the clinical trial outlined above, in which three competing treatments for joint
pain are compared in terms of their mean time to pain relief in patients with
osteoarthritis. Because investigators hypothesize that there may be a difference in time
to pain relief in men versus women, they randomly assign 15 participating men to one
of the three competing treatments and randomly assign 15 participating women to one
of the three competing treatments. Participating men and women do not know to
which treatment they are assigned. They are instructed to take the assigned medication
when they experience joint pain and to record the time, in minutes, until the pain
subsides.
24 February 2023 49

50. Table 2: Time to Pain Relief by Treatment and Sex
Treatment Male Female
A 12 21
15 19
16 18
17 24
14 25
B 14 21
17 20
19 23
20 27
17 25
C 25 37
27 34
29 36
24 26
22 29
24 February 2023 50

51. Sum of Squares: Two-Way ANOVA
0 Total variability (total sums of squares, SST) in a two-way ANOVA can
be broken down into four components:
SST = SSW + SSA + SSB + SSAB
0 Each between component (SSA, SSB, and SSAB) is contrasted to the
within component (SSW), resulting in three separate Fs
0 SSW is within group SS
0 F-ratios computed based on mean squares
24 February 2023 51

52. SPSS: Two way ANOVA
24 February 2023 52

53. Cont’d
24 February 2023 53

54. SPSS output
24 February 2023 54

55. Summary
oAn ANOVA model can only tell us if one or more of the means are statistically
(significantly) different from the others
oPost-hoc tests are necessary to tell which means are different
oThese tests are conducted only if the overall ANOVA model is statistically
significant.
24 February 2023 55

56. Thank you !
24 February 2023 56