This document provides an overview of various statistical analyses that can be used in psychological research, including t-tests, analysis of variance (ANOVA), repeated measures ANOVA, factorial ANOVA, mixed ANOVA, analysis of covariance (ANCOVA), and multivariate analysis of variance (MANOVA). Examples of research questions that could be addressed with each type of analysis are provided. Key assumptions, procedures, and interpretations for each analysis are discussed at a high level.

1. Lecture 10 Survey Research & Design in Psychology James Neill, 2011 ANOVA II

2. Overview One-sample t -test

3. Independent samples t -test

4. Paired samples t -test

5. One-way ANOVA

6. One-way repeated measures ANOVA

7. Factorial ANOVA

8. Mixed design ANOVA

9. ANCOVA

10. MANOVA

11. Howell (2010): Ch11 Simple Analysis of Variance

12. Ch12 Multiple Comparisons Among Treatment Means

13. Ch13 Factorial Analysis of Variance

14. Ch14 Repeated-Measures Designs

15. Ch16 Analyses of Variance and Covariance as General Linear Models Readings See also: Inferential statistics decision-making tree

16. Difference between a set value and a variable -> one-sample t -test

17. Difference between two independent groups -> independent samples t -test = BETWEEN-SUBJECTS

18. Difference between two related measures (e.g., repeated over time or two related measures at one time) -> paired samples t -test = WITHIN-SUBJECTS Previous lecture

19. Are the differences in a sample generalisable to a population?

20. Introduction to ANOVA (Analysis of Variance) Extension of a t -test to assess differences in the central tendency ( M ) of several groups or variables.

21. Overall variance is partitioned into between-group and within-group variance

22. Levels of measurement: Single DV: metric,

23. 1 or more IVs: categorical

24. Example ANOVA research questions Are there differences in the degree of religious commitment between countries ( UK, USA, and Australia) ? 1-way ANOVA

25. 1-way repeated measures ANOVA

26. Factorial ANOVA

27. Mixed ANOVA

28. ANCOVA

29. MANOVA

30. Example ANOVA research questions Do university students have different levels of satisfaction for educational, social, and campus-related domains ? 1-way ANOVA

31. 1-way repeated measures ANOVA

32. Factorial ANOVA

33. Mixed ANOVA

34. ANCOVA

35. MANOVA

36. Example ANOVA research questions Are there differences in the degree of religious commitment between countries (UK, USA, and Australia) and gender (male and female)? 1-way ANOVA

37. 1-way repeated measures ANOVA

38. Factorial ANOVA

39. Mixed ANOVA

40. ANCOVA

41. MANOVA

42. Example ANOVA research questions Does couples' relationship satisfaction differ between males and females and before and after having children ? 1-way ANOVA

43. 1-way repeated measures ANOVA

44. Factorial ANOVA

45. Mixed ANOVA

46. ANCOVA

47. MANOVA

48. Example ANOVA research questions Are there differences in university student satisfaction between males and females ( gender ) after controlling for level of academic performance ? 1-way ANOVA

49. 1-way repeated measures ANOVA

50. Factorial ANOVA

51. Mixed ANOVA

52. ANCOVA

53. MANOVA

54. Example ANOVA research questions Does personality (5 factors) change over time (10 years old, 20 years old, and 50 years old)? 1-way ANOVA

55. 1-way repeated measures ANOVA

56. Factorial ANOVA

57. Mixed ANOVA

58. ANCOVA

59. MANOVA

60. Introduction to ANOVA Inferential : What is the likelihood that the observed differences could have been due to chance?

61. Follow-up tests : Which of the M s differ?

62. Effect size : How large are the observed differences?

63. F test ANOVA partitions the sums of squares (variance from the mean) into: Explained variance (between groups)

64. Unexplained variance (within groups) – or error variance F = ratio between explained & unexplained variance

65. p = probability that the observed mean differences between groups could be attributable to chance

66. F is the ratio of between-group : within-group variance

67. ANOVA F -tests are a "gateway". If F is significant, then... Follow-up tests interpret (main and interaction) effects and

68. consider whether to conduct follow-up tests planned comparisons

69. post-hoc contrasts .

70. One-way ANOVA

71. Assumptions – One-way ANOVA Dependent variable (DV) must be: LOM : Measured at interval or ratio level

72. Normality : Normally distributed for all IV groups (robust to violations of this assumption if N s are large and approximately equal e.g., >15 cases per group)

73. Variance : Equal variance across for all IV groups (homogeneity of variance)

74. Independence : Participants' data should be independent of others' data

75. One-way ANOVA: Are there differences in satisfaction levels between students who get different grades ?

76. Recoding needed to achieve min. 15 per group.

77. These groups could be combined.

78. The recoded data has more similar group sizes and is appropriate for ANOVA.

79. SD s are similar (homogeneity of variance). M s suggest that higher grade groups are more satisfied.

80. Levene's test indicates homogeneity of variance.

81. Follow-up tests should then be conducted because the effect of Grade is statistically significant ( p < .05).

82. One-way ANOVA: Does locus of control differ between three age groups? Age 20-25 year-olds

83. 40-45 year olds

84. 60-65 year-olds Locus of Control Lower = internal

85. Higher = external

86. 60-65 year-olds appear to be more internal, but the overlapping confidence intervals indicate that this may not be statistically significant.

87. The SD s vary between groups (the third group has almost double the SD of the younger group). Levene's test is significant (variances are not homogenous).

88. There is a significant effect for Age ( F (2, 59) = 4.18, p = .02). In other words, the three age groups are unlikely to be drawn from a population with the same central tendency for LOC.

89. Which age groups differ in their mean locus of control scores? (Post hoc tests). Conclude: Gps 0 differs from 2; 1 differs from 2

90. Follow-up (pairwise) tests Post hoc : Compares every possible combination

91. Planned : Compares specific combinations (Do one or the other; not both)

92. Post hoc Control for Type I error rate

93. Scheffe, Bonferroni, Tukey’s HSD, or Student-Newman-Keuls

94. Keeps experiment-wise error rate to a fixed limit

95. Planned Need hypothesis before you start

96. Specify contrast coefficients to weight the comparisons (e.g., 1 st two vs. last one)

97. Tests each contrast at critical 

98. Assumptions - Repeated measures ANOVA Repeated measures designs have the additional assumption of Sphericity: Variance of the population difference scores for any two conditions should be the same as the variance of the population difference scores for any other two conditions

99. Test using Mauchly's test of sphericity (If Mauchly’s W Statistic is p < .05 then assumption of sphericity is violated.)

100. Assumptions - Repeated measures ANOVA Sphericity is commonly violated, however the multivariate test (provided by default in PASW output) does not require the assumption of sphericity and may be used as an alternative.

101. The obtained F ratio must then be evaluated against new degrees of freedom calculated from the Greenhouse-Geisser , or Huynh-Feld, Epsilon values.

102. Example: Repeated measures ANOVA Does LOC vary over time ? Baseline

103. 6 months

104. 12 months

105. Mean LOC scores (with 95% C.I.s) across 3 measurement occasions Not much variation between means.

106. Descriptive statistics Not much variation between means.

107. Mauchly's test of sphericity Mauchly's test is not significant, therefore sphericity can be assumed.

108. Tests of within-subject effects Conclude: Observed differences in means could have occurred by chance ( F (2, 118) = 2.79, p = .06) if critical alpha = .05

109. 1-way repeated measures ANOVA Do satisfaction levels vary between Education, Teaching, Social and Campus aspects of university life?

112. Tests of within-subject effects

113. Factorial ANOVA (2-way): Are there differences in satisfaction levels between gender and age ?

114. Factorial ANOVA Levels of measurement 2 or more between-subjects categorical/ordinal IVs

115. 1 interval/ratio DV e.g., Does Educational Satisfaction vary according to Age (2) and Gender (2)? 2 x 2 Factorial ANOVA

116. Factorial designs test Main Effects and Interactions . For a 2-way design: Main effect of IV1

117. Main effect of IV2

118. Interaction between IV1 and IV2 If significant effects are found and

119. there are more than 2 levels of an IV are involved then follow-up tests are required. Factorial ANOVA

125. Factorial ANOVA (2-way): Are there differences in LOC between gender and age ?

126. Example: Factorial ANOVA Main effect 1 : - Do LOC scores differ by Age? Main effect 2 : - Do LOC scores differ by Gender? Interaction : - Is the relationship between Age and LOC moderated by Gender? (Does any relationship between Age and LOC vary as a function of Gender?)

127. Example: Factorial ANOVA In this example, there are: Two main effects (Age and Gender)

128. One interaction effect (Age x Gender) IVs Age recoded into 2 groups (2)

129. Gender dichotomous (2) DV Locus of Control (LOC)

130. Plot of LOC by Age and Gender

131. Age x gender interaction

132. Error-bar graph for Age main effect Age main effect

133. Descriptives for Age main effect

134. Error-bar graph for Gender main effect Gender main effect

135. Descriptives for Gender main effect

136. Descriptives for LOC by Age and Gender

137. Tests of between-subjects effects

138. IV1 = Separate lines for morning and evening exercise.

139. IV2 = Light and heavy exercise

140. DV = Av. hours of sleep per night Interactions

141. Interactions

142. Interactions

143. Mixed design ANOVA (SPANOVA) It is common to have independent groups (e.g., males and females) and to then perform some repeated measures on each group (e.g., word recall under three different character spacing conditions (Narrow, Medium, Wide)).

144. Since such experiments have mixtures of between subjects and within-subject factors they are said to be of mixed design

145. IV1 is between-subjects (e.g., Gender)

146. IV2 is within-subjects (e.g., Social Satisfaction and Campus Satisfaction)

147. Of interest are: Main effect of IV1

148. Main effect of IV2

149. Interaction b/w IV1 and IV2 If significant effects are found and more than 2 levels of an IV are involved, then specific contrasts are required, either: A priori (planned) contrasts

150. Post-hoc contrasts Mixed design ANOVA (SPANOVA)

151. An experiment has two IVs: Between-subjects = Gender (Male or Female) - varies between subjects

152. Within-subjects = Spacing (Narrow, Medium, Wide)

153. Gender - varies within-subjects Mixed design ANOVA (SPANOVA)

154. Mixed design ANOVA: Design If A is Gender and B is Spacing the Reading experiment is of the type A X (B)

155. Brackets signify a mixed design with repeated measures on Factor B

156. Mixed design ANOVA: Assumptions Normality

157. Homogeneity of variance

158. Sphericity

159. Homogeneity of inter-correlations

160. Homogeneity of intercorrelations The pattern of inter-correlations among the various levels of repeated measure factor(s) should be consistent from level to level of the Between-subject Factor(s)

161. The assumption is tested using Box’s M statistic

162. Homogeneity is present when the M statistic is NOT significant at p > .001.

163. Mixed design ANOVA: Example Do satisfaction levels vary between gender for education and teaching ?

165. Tests of within-subjects contrasts

166. Tests of between-subjects effects

168. What is ANCOVA? Analysis of Cov ar iance

169. Extension of ANOVA, using ‘regression’ principles

170. Assesses effect of one variable (IV) on

171. another variable (DV)

172. after controlling for a third variable (CV)

173. A covariate IV is added to an ANOVA (can be dichotomous or metric)

174. Effect of the covariate on the DV is removed (or partialled out) (akin to Hierarchical MLR)

175. Of interest are: Main effects of IVs and interaction terms

176. Contribution of CV (akin to Step 1 in HMLR) e.g., GPA is used as a CV, when analysing whether there is a difference in Educational Satisfaction between Males and Females. ANCOVA (Analysis of Covariance)

177. Why use ANCOVA? Reduces variance associated with covariate (CV) from the DV error (unexplained variance) term

178. Increases power of F -test

179. May not be able to achieve experimental control over a variable (e.g., randomisation), but can measure it and statistically control for its effect.

180. Why use ANCOVA? Adjusts group means to what they would have been if all P s had scored identically on the CV.

181. The differences between P s on the CV are removed, allowing focus on remaining variation in the DV due to the IV.

182. Make sure hypothesis (hypotheses) is/are clear.

183. Assumptions of ANCOVA As per ANOVA

184. Normality

185. Homogeneity of Variance (use Levene’s test)

186. Assumptions of ANCOVA Independence of observations

187. Independence of IV and CV.

188. Multicollinearity - if more than one CV, they should not be highly correlated - eliminate highly correlated CVs.

189. Reliability of CVs - not measured with error - only use reliable CVs.

190. Assumptions of ANCOVA Check for linearity between CV & DV - check via scatterplot and correlation.

191. If the CV is not correlated with the DV there is no point in using it.

192. Assumptions of ANCOVA Homogeneity of regression Assumes slopes of regression lines between CV & DV are equal for each level of IV, if not, don’t proceed with ANCOVA

193. Check via scatterplot with lines of best fit.

194. Assumptions of ANCOVA

195. ANCOVA example 1: Does education satisfaction differ between people with different levels of coping (‘Not coping’, ‘Just coping’ and ‘Coping well’) with average grade as a covariate?

200. ANCOVA Example 2: Does teaching method affect academic achievement after controlling for motivation ? IV = teaching method

201. DV = academic achievement

202. CV = motivation

203. Experimental design - assume students randomly allocated to different teaching methods.

204. ANCOVA example Academic Achievement (DV) Teaching Method (IV) Motivation (CV)

205. ANCOVA example Academic Achievement Teaching Method Motivation

206. ANCOVA example 2 A one-way ANOVA shows a non-significant effect for teaching method (IV) on academic achievement (DV)

207. An ANCOVA is used to adjust for differences in motivation

208. F has gone from 1 to 5 and is significant because the error term (unexplained variance) was reduced by including motivation as a CV. ANCOVA example 2

209. ANCOVA & hierarchical MLR ANCOVA is similar to hierarchical regression – assesses impact of IV on DV while controlling for 3 rd variable.

210. ANCOVA more commonly used if IV is categorical.

211. Summary of ANCOVA Use ANCOVA in survey research when you can’t randomly allocate participants to conditions e.g., quasi-experiment, or control for extraneous variables.

212. ANCOVA allows us to statistically control for one or more covariates.

213. Summary of ANCOVA Decide which variable(s) are IV, DV & CV.

214. Check assumptions: normality

215. homogeneity of variance (Levene’s test)

216. Linearity between CV & DV (scatterplot)

217. homogeneity of regression (scatterplot – compares slopes of regression lines) Results – does IV effect DV after controlling for the effect of the CV?

218. MANOVA Multivariate Analysis of Variance

219. MANOVA ANOVAs test whether mean differences among groups on a single DV are likely to have occurred by chance.

220. MANOVA tests whether mean differences among groups on a combination of DV’s are likely to have occurred by chance.

221. Generalises ANOVA to situations with multiple related DVs.

222. One-way repeated measures ANOVA and mixed ANOVA are subsets of MANOVA.

223. If multivariate F is significant, then examine univariate F s MANOVA

224. Consider whether a mixed ANOVA could be used instead. Mixed ANOVA provides an interaction term.

225. MANOVA provides separate ANOVAs as well as a multivariate ANOVA. MANOVA

226. MANOVA Example design 1 IV = Gender

227. DVs = Satisfaction (Social, Campus, and Education) Use MANOVA to ask whether university satisfaction varies by gender overall (as a function of the three satisfaction factors) and also separately for each of the three satisfaction factors.

228. MANOVA Example design 2 IV = types of treatment (Systematic Desensitisation, Autogenic Training, Waiting List – Control)

229. DVs = types of anxiety (Test Anxiety, Social Anxiety, Public Speaking Anxiety) MANOVA is used to ask whether anxiety varies across the the different types of treatment (overall) and for each of the three anxiety measures.

230. By measuring several DV’s instead of only one the researcher improves the chance of discovering what it is that changes as a result of different treatments and their interactions.

231. e.g., Desensitisation may have an advantage over relaxation training or control, but only on test anxiety. The effect is missing if anxiety is not one of your DV’s. MANOVA advantages over ANOVA

232. When there are several DV’s there is protection against inflated Type 1 error due to multiple tests of likely correlated DV’s.

233. When responses to two or more DV’s are considered in combination, group differences become apparent. MANOVA advantages over ANOVA

234. Low power can mean a non-significant multivariate effect but one or more significant Univariate F ’s

235. When cell sizes > 30 assumptions of normality and equal variances are of little concern.

236. Equal cell sizes preferred (but not essential)

237. Ratios of smallest to largest cell sizes of 1:1.5+ may cause problems. MANOVA

238. MANOVA is sensitive to violations of univariate and multivariate normality . Test each group or level of the IV using the split file option.

239. Multivariate outliers which affect normality can normally be identified using Mahalanobis distance in the Regression sub-menu. MANOVA assumptions

240. Linearity : Linear relationships among all pairs of DV’s must be assumed. Within cell scatterplots must be conducted to test this assumption.

241. Homogeneity of regression : It is assumed that the relationships between covariates and DV’s in one group is the same as other groups. Necessary if stepdown analyses required. MANOVA assumptions

242. Homogeneity of variance : Covariance Matrices similar to assumption of homogeneity of variance for individual DV’s.

243. Box’s M test is used for this assumption and should be non-significant p >.001.

244. Multicollinearity and singularity : When correlations among DV’s are high, problems of multicollinearity exist. MANOVA assumptions

245. Wilks’ lambda ( λ ) Several multivariate statistics are available to test significance of main effects and interactions.

246. Wilks’ lambda is one such statistic.

247. Three effect sizes are relevant to ANOVA: Eta-square (  2 ) provides an overall test of size of effect

248. Partial eta-square (  p 2 ) provides an estimate of the effects for each IV.

249. Cohen’s d : Standardised differences between two means. Effect sizes

250. Effect Size: Eta-squared (  2 ) Analagous to R 2 from regression

251. = SS between / SS total = SS B / SS T

252. = prop. of variance in Y explained by X

253. = Non-linear correlation coefficient

254. = prop. of variance in Y explained by X

255. Ranges between 0 and 1

256. Effect Size: Eta-squared (  2 ) Interpret as for r 2 or R 2

257. Cohen's rule of thumb for interpreting  2 : .01 is small

258. .06 medium

259. .14 large

260.  2 = SS between /SS total = 395.433 / 3092.983 = 0.128 Eta-squared is expressed as a percentage: 12.8% of the total variance in control is explained by differences in Age

261. Effect Size: Eta-squared (  2 ) The eta-squared column in PASW F -table output is actually partial eta-squared (  p 2 ) .

262.  2 is not provided by PASW – calculate separately.

263. R 2 at the bottom of PASW F -tables is the linear effect as per MLR – however, if an IV has 3 or more non-interval levels, this won’t equate with  2 .

264. Results - Writing up ANOVA Establish clear hypotheses – one for each main or interaction effect

265. Test the assumptions , esp. LOM, normality, univariate and multivariate outliers, homogeneity of variance, N

266. Present the descriptive statistics (usually in a table)

267. Consider presenting a figure to illustrate the data (bar, error-bar or line graph)

268. Results - Writing up ANOVA F and sig. test results, noting direction of any effects

269. Conduct planned or post-hoc testing as appropriate

270. Report and interpret effect sizes

271. State whether or not results support hypothesis (hypotheses)

272. Summary Hypothesise each main effect and interaction effect.

273. F is an omnibus “gateway” test; may require follow-up tests.

274. Conduct follow-up tests where sig. Main effects have three ore more levels.

275. Summary Choose from 1-way ANOVA, 1-way repeated measure ANOVA, Factor ANOVA, Mixed-design ANOVA, ANCOVA and MANOVA

276. Repeated measure designs include the assumption of sphericity

277. Summary Report on the size of effects potentially using: Eta-square (  2 ) as the omnibus ES

278. Partial eta-square (  p 2 )for each IV

279. Standardised mean differences for the differences between each pair of means

280. Open Office Impress This presentation was made using Open Office Impress.

281. Free and open source software.

282. http://www.openoffice.org/product/impress.html