Repeated measures ANOVA, with one-within and one-between factors

1. Two-way repeated measures ANOVA with one-between and one-within subjects factor
SPSS demo
Mike Crowson, Ph.D.
March 2020
Link to video presentation: https://youtu.be/KZHHAuTj8GU

2. In a previous presentation (video: https://youtu.be/TH7FVKevAcQ, Powerpoint: https://drive.google.com/open?id=1aRgrc4-
nh3tie-pECjRnC0UkrtOmljpX), I discussed and demonstrated the use of repeated measures analysis of variance when you
measure an outcome variable repeatedly within a single group. It is oftentimes the case that repeated measures analysis is
applied to data involving repeated measurements when you are working with more than a single group. This approach, for
instance, may be used in cases where the researcher hypothesizes that the variation in means associated with repeated
measurements on the outcome varies across groups. This effectively translates into a hypothesis concerning an interaction
between the repeated factor and a grouping variable.
One might adopt this approach when testing whether differences in means observed over time are the same or different
across levels of a grouping variable. If differences are found, it is possible to examine whether the mean differences reflect
different trends over time. This can be particularly handy if one of the groups being compared is a control condition and
other conditions are experimental in nature.
This approach could also easily be used when testing whether individuals react the same or differently across levels of a
repeated factor (for example, different stimuli for which a person is exposed) and a grouping variable.

3. For the current example (the data below is partial), our between-subjects’ factor is “tx.group” coded 1=control,
2=treatment A, 3=treatment B. We are going to test whether there are significant mean differences in anxiety scores over
three measurement occasions, as well as whether there are group differences in terms of how the means vary over time.
The full dataset can be
downloaded here:
https://drive.google.com/op
en?id=1090t8zuEI8eSD8QW
vDXNWYMna3gE516H

4. Because we are introducing a between-groups
factor, we now include “Homogeneity tests”.

5. The selection of ‘Compare main effects’ and ‘Bonferroni’ as the confidence interval adjustment will yield
Bonferroni-adjusted pairwise comparisons across level levels of the repeated factor. [It will not yield
comparisons within groups, however. That would require simple effects tests, which I will address later.]

6. Plotting the mean anxiety scores by time and by time
and treatment group.

7. If we are interested in testing whether there are pairwise
difference in subjects’ average (i.e., averaged across time)
anxiety score, we can select Tukey’s post hoc tests.

8. Descriptive statistics for each group at each time point.

9. Here, we have the multivariate test
results for time (the within-subjects
factor) and the time X group
interaction.
Box’s test is a test of the assumption of equality variance-covariance matrices of difference scores
between groups (Weinfurt, 2000). This is an assumption of the multivariate tests below. If Box’s test is
significant, then you have evidence of a violation. [Note: Box’s test is sensitive to multivariate
nonnormality, thereby increasing rejection rate.]
Nevertheless, the multivariate test results are fairly robust when you have equal or nearly equal n’s in
your groups (e.g., largest n/smallest n < 1.5). If the ratio of the largest to smallest group size is large, then
the results of the multivariate tests can be biased. When the larger group is associated with smaller
variability, the multivariate test becomes too liberal. When the larger group is associated with the greater
variability, the test becomes too conservative (Pituch & Stevens, 2016).

10. The main effect of time is
statistically significant, Wilks’
lambda=.173, F(2,65)=155.687,
p<.001. This effect, however, is
qualified by a significant time X
group interaction, Wilks’ lambda =
.440, F(4,130)=16.480, p<.001.
A significant Box’s test result indicates a violation of homogeneity of covariance matrices,
whereas a non-significant result is consistent with the MANOVA assumption.
Here, we see that p<.001, where there is evidence of the violation. Nevertheless, the ratio of the
largest n to smallest n is 26/19 = 1.368 (which is less than the 1.5 threshold suggested by Pituch
& Stevens, 2016).
The interaction is indicating that the variation in the means on anxiety over the repeated measurement occasions itself
varies as a function of treatment group membership.

11. The sphericity assumption is required for all
univariate main effects tests and interaction tests
(O’Brien & Kaiser, 1985). Given Mauchly’s test is
impacted by non-normality and by sample size, it is
not highly recommended when evaluating whether
the sphericity condition has been met.
A Greenhouse-Geisser epsilon (ε) value < .75,
suggests using the Greenhouse-Geisser adjustment
with the univariate test of mean differences (see
table of “Tests of within-subjects effects”), whereas a
value falling between .75 and 1 suggests the use of
the Huynh-Feldt adjustment with the univariate
tests. [ε=1 is consistent with sphericity]. The
sphericity assumed test can be used if you determine
sphericity is not violated.
[FYI, the Lower-Bound test is generally overly
conservative and is not typically used]

12. All three test results yield the same conclusions
with respect to the main and interaction effects.
The main effect of time on anxiety scores is
statistically significant, sphericity assumed
F(2,132)=120.752, p<.001.
This effect was qualified by a significant time X
group interaction effect, sphericity assumed
F(4,132)=20.658, p<.001.

13. Although the test of the linear component of the trend is significant (p<.001), the higher-order quadratic component was also
significant [F(1,66)=19.373, p<.001]. This suggests that across groups, the mean level of anxiety exhibited a quadratic trend
over the three measurement occasions. This is further suggested by examining the profile plot of the means.
Assessment of trending over time (irrespective of group membership)

14. Testing for differential trending across groups
We see here that although the test of the interaction between the linear component of the trend and treatment group is
significant, the interaction between treatment group and the higher-order quadratic component was also significant
[F(2,66)=23.903, p<.001]. Moreover, looking at the profile plot of means, we see that the curvature of the lines is less
pronounced for the Control group and Treatment A. However, the line for Treatment B appears more substantially curved.
Since these trends are not parallel, it is no surprise the test of the time X tx.group interaction was significant.

15. Interpretation: The main effect of treatment group on
the average anxiety score across time is statistically
significant, F(2, 66)=28.949, p<.001.
The Levene’s test results involve tests of differences in variances at
each time point, an assumption of the univariate ANOVA (see Tests of
Between-subjects effects). It turns out that the standard Levene’s tests
(and robust tests, based on median, etc.) are significant for Time 1 and
Time 3. Nevertheless, a violation of this assumption is less of an issue
with roughly equivalent sample sizes (where largest n / smallest n <
1.5).
The Tests of Between-subjects Effects is a test of the main effect of the
grouping variable on scores on the repeated measure averaged over
time. The result presented here is simply a test of group differences on
the average of anxiety scores (i.e., those scores averaged over time for
each person).

16. Bonferroni-adjusted paired t-tests. Here we see all pairwise differences
on anxiety are statistically significant (p’s≤.016).

17. These are pairwise comparisons on the average
anxiety score (averaged over time) for each group.
All pairwise differences were significant (as all p’s were
≤ .04).
These are Bonferroni adjusted pairwise
comparisons.

18. These comparisons are based on Tukey’s
tests.

19. Group means on anxiety at each measurement
occasion.

20. Simple effects test as a follow-up to a significant interaction
If you find evidence of a significant interaction (as we found in our analysis) between the between-subjects and within-
subjects factors, you may wish to describe the nature of the interaction using simple-effects tests. An easy approach is to
click on Paste after specifying your options…
When you do this, a syntax editor will open up. You can obtain simple
effects tests with very minor modifications to the syntax.

21. By adding the Compare command
(along with Bonferroni adjustment),
your output (after highlighting
everything and pressing the green
button) will include…

22. Multivariate tests of mean differences in anxiety scores over time by group (see left).
You also obtain pairwise tests of mean differences in anxiety
between time points within each group (see table to the right).

23. References
Lomax, R.G., & Hahs-Vaughn, D.L. (2012). An introduction to statistical concepts (3rd ed). New York: Routledge.
O’Brien, R. G., & Kaiser, M. K. (1985). MANOVA method for analyzing repeated measures designs: An extensive
primer. Psychological Bulletin, 97, 316-333.
Pituch, K.A., & Stevens, J.P. (2016). Applied multivariate statistics for the social sciences (6th ed). New York:
Routledge.
Weinfurt, K.P. (2000). Repeated measures analysis: ANOVA, MANOVA, and HLM. In L.G. Grimm & P.R. Yarnold
(Eds.), Reading and understanding more multivariate statistics (pp. 317-361). Washington, DC: American
Psychological Association.