INFERENTIAL STATISTICS
© LOUIS COHEN, LAWRENCE
MANION & KEITH MORRISON
STRUCTURE OF THE CHAPTER
• Measures of difference between groups
• The t-test (a difference test for parametric data)
• An...
STRUCTURE OF THE CHAPTER
• The Kruskal-Wallis and Friedman tests (difference
tests for non-parametric data)
• Regression a...
MEASURES OF DIFFERENCE
BETWEEN GROUPS
• Are there differences between two or more
groups of sub-samples, e.g.:
– Is there ...
MEASURES OF DIFFERENCE
BETWEEN GROUPS
• The t-test (for two groups): parametric data
• Analysis of Variance (ANOVA) (for t...
t-TEST
• Devised by William Gossett in 1908;
• Used when we have 2 conditions; the t-test
assesses whether there is a stat...
t-TEST FOR PARAMETRIC DATA
• t-tests (parametric, interval and ratio data)
– To find if there are differences between two
...
t-TEST FOR PARAMETRIC DATA
Formula for computing the t-test
Sample one mean – sample two mean
t = 
Stand...
Formula for calculating t














+







−+
+
−
=
∑ ∑
2121
2
2
2
1
21
11
2 NNNN
dd
MM
t
M ...
t-TEST FOR INDEPENDENT SAMPLES
The t-test computes a ratio between a measure of
the between-groups variance and the within...
INDEPENDENT AND RELATED
SAMPLES IN A t-TEST: EXAMPLES
1. Independent sample (two groups):
• A group of scientists wants to...
INDEPENDENT AND RELATED
SAMPLES IN A t-Test: AN EXAMPLE
24 people were involved in an experiment to
determine whether back...
AN EXAMPLE OF A t-TEST
Participants were asked to memorize a list of 20
words in two minutes.
Half of the sample performs ...
NOISE NO NOISE
5 15
10 9
6 16
6 15
7 16
3 18
6 17
9 13
5 11
10 12
11 13
9 11
Σ = 87 Σ = 166
= 7.3 = 13.8
SD = 2.5 SD = 2.8...
t-TEST FOR INDEPENDENT SAMPLES
Group statistics
In which condition
are you? N Mean Std. Deviation
Std. Error
Mean
How many...
t-TEST FOR INDEPENDENT SAMPLES (SPSS)
Independent Samples Test
Levene’s Test
for Equality of
Variances t-test for Equality...
REPORTING FINDINGS FROM THE
EXAMPLE
Participants in the NOISE condition recalled fewer
words (t (22) = 7.25, SD = 2.49) th...
Group Statistics
Which group are you N Mean
Std.
Deviation
Std. Error
Mean
Mathematics post-test
score
Control group 166 8...
Read the line ‘Levene’s Test for Equality of Variances’. If the
probability value is statistically significant then your v...
PAIRED SAMPLE t-TEST (SAME GROUP
UNDER TWO CONDITIONS) WITH SPSS
Paired Samples Statistics
Mean N
Std.
Deviation
Std. Erro...
PAIRED SAMPLE t-TEST (SAME GROUP
UNDER TWO CONDITIONS) WITH SPSS
This shows that there is no association between the
score...
PAIRED SAMPLE t-TEST (SAME GROUP UNDER
TWO CONDITIONS) WITH SPSS
This shows that :
1.The difference between the mean of ea...
RESULT
It can be seen from the paired t-test that the
hypothesis is not supported (t (251) = 20.186;
ρ=.000).
DEGREES OF FREEDOM
The number of individual scores that can vary
without changing the sample mean.
The number of scores on...
DEGREES OF FREEDOM (WITH SPSS)
Which group are you * Who are you Crosstabulation
Chinese or non-Chinese
TotalChinese Non-C...
ANALYSIS OF VARIANCE (ANOVA)
• Analysis of variance
– Parametric, interval and ratio data
– To see if there are any statis...
ANOVA AND MANOVA
• One way analysis of variance (one categorical
independent variable and one continuous
dependent variabl...
( )
( )
( )groupsN
groups
mean
df
d
varianceWithin
df
NXd
ancevariBetween
variancegroups-within
variancegroups-between
ati...
Between-groups and within-groups variance:
Variation between the groups (9 to 22.2);
Variation within the first group (no ...
ANOVA
1. First, ANOVA calculates the mean for each of
the three groups;
2. Then it calculates the grand mean (the three
me...
F RATIO
When we conduct our experiment, we hope that the
between-groups variance is very much larger than the
within-group...
Descriptives
Records of students' progress
7 3.29 .76 .29 2.59 3.98 2 4
5 3.80 1.30 .58 2.18 5.42 2 5
4 3.25 .96 .48 1.73 ...
RESULTS
An F ratio of .420 has been given, with a
probability of ρ=.742. This tells us that there
is no statistically sign...
EFFECT SIZE: PARTIAL ETA SQUARED
erroreffect
effect
partial
SSSS
SS
squaredetaPartial
+
=)( 2
η
SSeffect = The sums of the...
EFFECT SIZE: PARTIAL ETA SQUARED FOR
INDEPENDENT SAMPLES IN SPSS
Analyze → General Linear Model → Univariate →
Options → E...
Between-Subjects Factors
Value Label N
Which group are you 1 Control group 166
2 Experimental Group One 166
3 Experimental...
EFFECT SIZE: PARTIAL ETA
SQUARED IN SPSS
Tests of Between-Subjects Effects
Dependent Variable:Mathematics post-test score
...
THE POST HOC TUKEY TEST
• The null hypothesis for the F-test ANOVA is
always that the samples come from populations
with t...
POST HOC TESTS
• To determine which samples are statistically
significantly different; after having performed the
F-test a...
THE POST HOC TUKEY TEST
Tukey’s Honestly Significant Difference (HSD)
Test is used to test the hypothesis that all
possibl...
THE SCHEFFE TEST
The Scheffe test is very similar to the Tukey
hsd test, but it is more stringent that the Tukey
test in r...
FINDING PARTIAL ETA SQUARED IN SPSS
Multivariate Testsb
Effect Value F
Hypothesis
df Error df Sig.
Partial Eta
Squared
sco...
USING TUKEY TO LOCATE
DIFFERENCE IN SPSS
Multiple Comparisons
MEASURE_1
Tukey HSD
(I) Which
group are you (J) Which group ...
USING TUKEY TO LOCATE DIFFERENCE IN SPSS
MEASURE_1
Tukey HSDa,,b,,c
Which group are you N
Subset
1 2
Control group 166 7.8...
CHI-SQUARE
• A measure of a relationship or an association
developed by Karl Pearson in 1900;
• Measures the association b...
TYPES OF CHI-SQUARE
• One-variable Chi-Square (goodness-of-fit test)
– used when we have one variable;
• Chi-Square test f...
FORMULA FOR CHI-SQUARE
χ2
=
Where:
O = observed frequencies
E = expected frequencies
∑ = the sum of
∑
−
E
EO 2
)(
ONE-VARIABLE CHI-SQUARE OR
GOODNESS-OF-FIT TEST
• Enables us to discover whether a set of obtained
frequencies differs fro...
EXAMPLE: PREFERENCE FOR
CHOCOLATE BARS
A sample of 120 people were asked which of
four chocolate bars they preferred;
• We...
ONE-VARIABLE CHI-SQUARE OR
GOODNESS-OF-FIT TEST
If all brands of chocolate are equally popular, the
observed frequencies w...
ONE-VARIABLE CHI-SQUARE/GOODNESS-
OF-FIT TEST
Observed
N
Expected
N
Residual
(Difference between
observed and expected
fre...
CHI-SQUARE TEST FOR
INDEPENDENCE (BIVARIATE): 2 X 2
Enables us to discover whether there is a
relationship or association ...
A WORKED EXAMPLE
Imagine that we have asked 110 students the
following:
A. Do you smoke and drink?
B. Do you smoke but do ...
CHI-SQUARE TEST FOR
INDEPENDENCE: 2 X 2 (WITH SPSS)
Do you drink? * Do you smoke? Crosstabulation
Do you smoke?
Yes No Tot...
CHI-SQUARE TEST FOR
INDEPENDENCE: 2 X 2 (WITH SPSS)
Value df Asymp. Sig. Exact Sig.
(2-sided)
Exact Sig.
(1-sided)
Pearson...
RESULTS
A 2 x 2 Chi-square was carried out to discover
whether there was a significant relationship
between smoking and dr...
MANN-WHITNEY U-TEST FOR
INDEPENDENT SAMPLES
• Mann-Whitney (non-parametric, nominal and
ordinal data) for two groups under...
MANN-WHITNEY U-TEST (SPSS)
Ranks
22 43.52 957.50
64 43.49 2783.50
86
form
Primary 3
Primary 4
Total
the contents
are inter...
Test Statisticsa
703.500
2783.500
-.006
.996
Mann-Whitney U
Wilcoxon W
Z
Asymp. Sig.
(2-tailed)
the
contents
are
interesti...
THE WILCOXON TEST FOR RELATED
SAMPLES
This is the non-parametric equivalent of the t-test
for related samples.
For paired ...
THE WILCOXON TEST FOR RELATED SAMPLES (SPSS)
Test Statisticsb
How good at mathematics do you
think you are? - How much do ...
KRUSKAL-WALLIS TEST FOR
INDEPENDENT SAMPLES
• Kruskal-Wallis (non-parametric, nominal and
ordinal data) for three or more ...
Ranks
7 6.57
5 10.70
5 10.70
17
Age
20-29
30-39
40-49
Total
own made-up tests
N Mean Rank
KRUSKAL-WALLIS TEST (SPSS)
Test Statisticsa,b
4.319
2
.115
Chi-Square
df
Asymp. Sig.
own made-up
tests
Kruskal Wallis Testa.
Grouping Variable: Ageb....
THE FRIEDMAN TEST FOR 3 OR MORE
RELATED GROUPS
This is the non-parametric equivalent of ANOVA for
related samples.
For thr...
Upcoming SlideShare
Loading in …5
×

Chapter36a

937 views

Published on

Research Methods in Education 6th Edition

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
937
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
36
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Chapter36a

  1. 1. INFERENTIAL STATISTICS © LOUIS COHEN, LAWRENCE MANION & KEITH MORRISON
  2. 2. STRUCTURE OF THE CHAPTER • Measures of difference between groups • The t-test (a difference test for parametric data) • Analysis of variance (a difference test for parametric data) • The chi-square test (a difference test and a test of goodness of fit for non-parametric data) • Degrees of freedom (a statistic that is used in calculating statistical significance in considering difference tests) • The Mann-Whitney and Wilcoxon tests (difference tests for non-parametric data)
  3. 3. STRUCTURE OF THE CHAPTER • The Kruskal-Wallis and Friedman tests (difference tests for non-parametric data) • Regression analysis (prediction tests for parametric data) • Simple linear regression (predicting the value of one variable from the known value of another variable) • Multiple regression (calculating the different weightings of independent variables on a dependent variable) • Standardized scores (used in calculating regressions and comparing sets of data with different means and standard deviations)
  4. 4. MEASURES OF DIFFERENCE BETWEEN GROUPS • Are there differences between two or more groups of sub-samples, e.g.: – Is there a significant difference between the amount of homework done by boys and girls? – Is there a significant difference between test scores from four similarly mixed-ability classes studying the same syllabus? – Does school A differ significantly from school B in the stress level of its sixth form students?
  5. 5. MEASURES OF DIFFERENCE BETWEEN GROUPS • The t-test (for two groups): parametric data • Analysis of Variance (ANOVA) (for three or more groups: parametric data • The chi-square test: for categorical data • The Mann-Whitney and Wilcoxon tests (for two groups): non-parametric data • The Kruskal-Wallis and the Friedman tests (for three or more groups): non-parametric data
  6. 6. t-TEST • Devised by William Gossett in 1908; • Used when we have 2 conditions; the t-test assesses whether there is a statistically significant difference between the means of the two conditions; • The independent t-test is used when the participants perform in only one of two conditions; • The related or paired t-test is used when the participants perform in both conditions.
  7. 7. t-TEST FOR PARAMETRIC DATA • t-tests (parametric, interval and ratio data) – To find if there are differences between two groups – Decide whether they are are independent or related samples Independent sample: two different groups on one occasion Related sample: one group on two occasions
  8. 8. t-TEST FOR PARAMETRIC DATA Formula for computing the t-test Sample one mean – sample two mean t =  Standard error of the difference in means
  9. 9. Formula for calculating t               +        −+ + − = ∑ ∑ 2121 2 2 2 1 21 11 2 NNNN dd MM t M = Mean d = difference between the means N = Number of cases
  10. 10. t-TEST FOR INDEPENDENT SAMPLES The t-test computes a ratio between a measure of the between-groups variance and the within group variance. The larger the variance between the groups (columns), compared with the variance within the groups (rows), the larger the t-value.
  11. 11. INDEPENDENT AND RELATED SAMPLES IN A t-TEST: EXAMPLES 1. Independent sample (two groups): • A group of scientists wants to study the effects of a new drug for insomnia. They have applied this drug to a random group of people (control group) and to a group of people suffering from insomnia (experimental group); 1. Related sample (same group in two conditions): • A group of therapists wants to study whether there is any difference in doing relaxation techniques on the beach or in an apartment. A group of people is asked to first do relaxation on the beach and later in an apartment;
  12. 12. INDEPENDENT AND RELATED SAMPLES IN A t-Test: AN EXAMPLE 24 people were involved in an experiment to determine whether background noise affects short-term memory (recall of words); – If half of the sample were allocated to the NOISE condition and the other half to the NO NOISE condition (independent sample) – we use independent t-test; – If everyone in the sample has performed at both conditions (related sample) – we use paired or related t-test.
  13. 13. AN EXAMPLE OF A t-TEST Participants were asked to memorize a list of 20 words in two minutes. Half of the sample performs in a noisy environment and the other half in a quiet environment; Independent variable - two types of environment: Quiet environment (NO NOISE condition) Noisy environment (NOISE condition) Dependent variable – the number of words each participant can recall.
  14. 14. NOISE NO NOISE 5 15 10 9 6 16 6 15 7 16 3 18 6 17 9 13 5 11 10 12 11 13 9 11 Σ = 87 Σ = 166 = 7.3 = 13.8 SD = 2.5 SD = 2.8 XX NOTE: participants vary within conditions: in the NOISE condition, the scores range from 3 to 11, and in the NO NOISE condition. They range from 9 to 18; The participants differ between the conditions too: the scores of the NO NOISE condition, in general, are higher than those in the NOISE condition – the means confirm it; Are the differences between the means of our groups large enough for us to conclude that the differences are due to our independent variable: NOISE/NO NOISE manipulation?
  15. 15. t-TEST FOR INDEPENDENT SAMPLES Group statistics In which condition are you? N Mean Std. Deviation Std. Error Mean How many words can you recall? NOISE 12 7.2500 .71906 NO NOISE 12 13.8333 .79614 This shows: the name of 2 conditions; the number of cases in each condition; the mean of each condition; the standard deviation and standard error of the mean, of the two conditions.
  16. 16. t-TEST FOR INDEPENDENT SAMPLES (SPSS) Independent Samples Test Levene’s Test for Equality of Variances t-test for Equality of Means F Sig t df Sig. (2-tailed) Mean Differences Std. Error Differences 95% Confidence Interval of the Difference Lower Upper How many words can you recall? Equal variances assumed .177 .676 -6.137 22 .000 -6.5833 1.07279 -8.808 -4.359 Equal variances not assumed -6.137 21.78 .000 -6.5833 1.07279 -8.809 -4.357 The Levene test is for ‘homogeneity of variance’, and the t-test here indicates whether you should use the upper or lower row. Mean Difference means the difference between the means of the two groups.
  17. 17. REPORTING FINDINGS FROM THE EXAMPLE Participants in the NOISE condition recalled fewer words (t (22) = 7.25, SD = 2.49) than in the NO NOISE condition (t (22) = 13.83, SD = 2.76). The mean difference between conditions was 6.58; the 95% confidence interval for the estimated population mean difference is between 4.36 and 8.81. An independent t-test revealed that, if the null hypothesis is true, such a result would be highly unlikely to have arisen (t (22) = 6.14; p<0.001). It is therefore concluded that listening to noise affects short-term memory, at least in respect of word recall.
  18. 18. Group Statistics Which group are you N Mean Std. Deviation Std. Error Mean Mathematics post-test score Control group 166 8.69 1.220 .095 Experimental Group One 166 9.45 .891 .069 t-TEST FOR INDEPENDENT SAMPLES WITH SPSS
  19. 19. Read the line ‘Levene’s Test for Equality of Variances’. If the probability value is statistically significant then your variances are unequal; otherwise they are regarded as equal. If the Levene’s probability value is not statistically significant then you need the row ‘equal variances assumed’; if the Levene’s probability value is statistically significant then you need the row ‘equal variances not assumed’. Look to the column ‘Sig. (2-tailed)’ and the appropriate row, and see if the results are statistically significant. Independent Samples Test Levene's Test for Equality of Variances t-test for Equality of Means 95% Confidence Interval of the Difference F Sig. t df Sig. (2- tailed) Mean Difference Std. Error Difference Lower Upper Mathematics post-test score Equal variances assumed 28.856 .000 -6.523 330 .000 -.765 .117 -.996 -.534 Equal variances not assumed -6.523 302.064 .000 -.765 .117 -.996 -.534
  20. 20. PAIRED SAMPLE t-TEST (SAME GROUP UNDER TWO CONDITIONS) WITH SPSS Paired Samples Statistics Mean N Std. Deviation Std. Error Mean Pair 1 Mathematics pre-test score 6.95 252 1.066 .067 Mathematics post-test score 8.94 252 1.169 .074 This indicates: 1.The two conditions; 2.The mean of each condition; 3.The number of cases in each condition; 4.The standard deviation and standard error of the mean, for the two conditions.
  21. 21. PAIRED SAMPLE t-TEST (SAME GROUP UNDER TWO CONDITIONS) WITH SPSS This shows that there is no association between the scores on the pre-test and the scores on the post test for the group in question (r = .02 and ρ = .749). Paired Samples Correlations N Correlation Sig. Pair 1 Mathematics pre-test score & Mathematics post-test score 252 .020 .749
  22. 22. PAIRED SAMPLE t-TEST (SAME GROUP UNDER TWO CONDITIONS) WITH SPSS This shows that : 1.The difference between the mean of each condition (6.95 and 8.94) is 1.992. 2.The confidence intervals shows that we are 95% certain that the population difference lies somewhere between -2.186 and -1.798. 3.There is a statistically significant difference between the two sets of scores. Paired Samples Test Paired Differences t df Sig. (2-tailed)Mean Std. Deviation Std. Error Mean 95% Confidence Interval of the Difference Lower Upper Pair 1 Mathematics pre-test score - Mathematics post-test score -1.992 1.567 .099 -2.186 -1.798 -20.186 251 .000
  23. 23. RESULT It can be seen from the paired t-test that the hypothesis is not supported (t (251) = 20.186; ρ=.000).
  24. 24. DEGREES OF FREEDOM The number of individual scores that can vary without changing the sample mean. The number of scores one needs to know before one can calculate the others. E.g.: If you are asked to choose 2 numbers that must add up to 100, and the first is 89, then the other has to be 11; there is 1 degree of freedom (89 + x = 100). If you are asked to choose 3 numbers that must add to 100, and the first of these is 20, then you have 2 degrees of freedom (20 + x + y = 100).
  25. 25. DEGREES OF FREEDOM (WITH SPSS) Which group are you * Who are you Crosstabulation Chinese or non-Chinese TotalChinese Non-Chinese Which group are you Control group 156 10 166 94.0% 6.0% 100.0% Experimental Group One 166 0 166 100.0% .0% 100.0% Experimental Group Two 143 25 168 85.1% 14.9% 100.0% Total 465 35 500 93.0% 7.0% 100.0% Degrees of freedom = 2 (1 degree of freedom in each of 2 rows, which fixes what must be in the third row)
  26. 26. ANALYSIS OF VARIANCE (ANOVA) • Analysis of variance – Parametric, interval and ratio data – To see if there are any statistically significant differences between the means of two or more groups; – It calculates the grand mean (i.e. the mean of the means of each condition) and sees how different each of the individual means is from the grand mean. – Premised on the same assumptions as t-tests (random sampling, a normal distribution of scores, independent variable(s) is/are categorical (e.g. teachers, students,) and one is a continuous variable (e.g. marks on a test).
  27. 27. ANOVA AND MANOVA • One way analysis of variance (one categorical independent variable and one continuous dependent variable) • Two-way analysis of variance (two categorical independent variables and one continuous dependent variable) • Multiple analysis of variance (MANOVA) (one categorical independent variable and two or more continuous variables) • Post-hoc tests (e.g. Tukey hsd test, Sheffe test) to locate where differences between means lie (in which group(s))
  28. 28. ( ) ( ) ( )groupsN groups mean df d varianceWithin df NXd ancevariBetween variancegroups-within variancegroups-between atiorF − − ∑ ∑ = = = 2 1 2 FORMULA FOR ANOVA
  29. 29. Between-groups and within-groups variance: Variation between the groups (9 to 22.2); Variation within the first group (no variation since all participants scored the same); Variation within the second group (from 15 to 16); Variation within the third group (from 17 to 26). A1 A2 A3 9 15 21 9 15 25 9 16 17 9 15 22 9 16 26 = 9 = 15.4 = 22.2X X X
  30. 30. ANOVA 1. First, ANOVA calculates the mean for each of the three groups; 2. Then it calculates the grand mean (the three means added then divided by three); 3. For each group separately, the total deviation of each individual’ s score from the mean of the group is calculated (within-groups variation); 4. Then the deviation of each group mean from the grand mean is calculated (between-groups variation).
  31. 31. F RATIO When we conduct our experiment, we hope that the between-groups variance is very much larger than the within-groups variance, in order to get a bigger F ratio; This shows us that one (or more) of the individual group means is significantly different from the grand mean; However, it does not tell us which means are statistically significantly different. cevarianroupsg-within variancegroups-between ratioF =
  32. 32. Descriptives Records of students' progress 7 3.29 .76 .29 2.59 3.98 2 4 5 3.80 1.30 .58 2.18 5.42 2 5 4 3.25 .96 .48 1.73 4.77 2 4 1 4.00 . . . . 4 4 17 3.47 .94 .23 2.99 3.96 2 5 20-29 30-39 40-49 50+ Total N Mean Std. Deviation Std. Error Lower Bound Upper Bound 95% Confidence Interval for Mean Minimum Maximum ANOVA Records of students' progress 1.257 3 .419 .420 .742 12.979 13 .998 14.235 16 Between Groups Within Groups Total Sum of Squares df Mean Square F Sig. Between-groups variation Within-groups variation F (3,13) = .420, ρ=.742
  33. 33. RESULTS An F ratio of .420 has been given, with a probability of ρ=.742. This tells us that there is no statistically significant difference between any of the groups.
  34. 34. EFFECT SIZE: PARTIAL ETA SQUARED erroreffect effect partial SSSS SS squaredetaPartial + =)( 2 η SSeffect = The sums of the squares for whatever effect is of interest; SSerror = the sums of the squares for whatever error term is associated with that effect.
  35. 35. EFFECT SIZE: PARTIAL ETA SQUARED FOR INDEPENDENT SAMPLES IN SPSS Analyze → General Linear Model → Univariate → Options → Estimates of effect size
  36. 36. Between-Subjects Factors Value Label N Which group are you 1 Control group 166 2 Experimental Group One 166 3 Experimental Group Two 168 EFFECT SIZE: PARTIAL ETA SQUARED IN SPSS
  37. 37. EFFECT SIZE: PARTIAL ETA SQUARED IN SPSS Tests of Between-Subjects Effects Dependent Variable:Mathematics post-test score Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Corrected Model 48.583a 2 24.291 23.168 .000 .085 Intercept 41113.093 1 41113.093 39211.30 1 .000 .987 group 48.583 2 24.291 23.168 .000 .085 Error 521.105 497 1.049 Total 41684.000 500 Corrected Total 569.688 499 a. R Squared = .085 (Adjusted R Squared = .082)
  38. 38. THE POST HOC TUKEY TEST • The null hypothesis for the F-test ANOVA is always that the samples come from populations with the same Mean (i.e., no statistically significant differences): H0 = μ1 = μ2 = μ3 = … • If the p-value is so low that we reject the null hypothesis, we have decided that, at least one of these populations has a mean that is not equal to the others; • The F-test itself only tells us that there are differences at least between one pair of means, not where these differences lie.
  39. 39. POST HOC TESTS • To determine which samples are statistically significantly different; after having performed the F-test and rejected the null hypothesis, we turn to post hoc comparisons; • The purpose of a post hoc analysis is to find out exactly where those differences are; • Post hoc tests allow us to make multiple pair wise comparisons and determine which pairs are statistically significantly different from each other and which are not.
  40. 40. THE POST HOC TUKEY TEST Tukey’s Honestly Significant Difference (HSD) Test is used to test the hypothesis that all possible pairs of means are equal; Tukey’s HSD test compares the mean differences between each pair of means to a critical value. If the mean difference from a pair of means exceeds the critical value, we conclude that there is a significant difference between these pairs.
  41. 41. THE SCHEFFE TEST The Scheffe test is very similar to the Tukey hsd test, but it is more stringent that the Tukey test in respect of reducing the risk of a Type I error, though this comes with some loss of power – one may be less likely to find a difference between groups in the Sheffe test.
  42. 42. FINDING PARTIAL ETA SQUARED IN SPSS Multivariate Testsb Effect Value F Hypothesis df Error df Sig. Partial Eta Squared scores Pillai's Trace .675 1033.477a 1.000 497.000 .000 .675 Wilks' Lambda .325 1033.477a 1.000 497.000 .000 .675 Hotelling's Trace 2.079 1033.477a 1.000 497.000 .000 .675 Roy's Largest Root 2.079 1033.477a 1.000 497.000 .000 .675 scores * group Pillai's Trace .040 10.366a 2.000 497.000 .000 .040 Wilks' Lambda .960 10.366a 2.000 497.000 .000 .040 Hotelling's Trace .042 10.366a 2.000 497.000 .000 .040 Roy's Largest Root .042 10.366a 2.000 497.000 .000 .040 a. Exact statistic b. Design: Intercept + group Within Subjects Design: scores
  43. 43. USING TUKEY TO LOCATE DIFFERENCE IN SPSS Multiple Comparisons MEASURE_1 Tukey HSD (I) Which group are you (J) Which group are you Mean Difference (I-J) Std. Error Sig. 95% Confidence Interval Lower Bound Upper Bound Control group Experimental Group One -.42* .084 .000 -.61 -.22 Experimental Group Two -.15 .084 .173 -.35 .05 Experimental Group One Control group .42* .084 .000 .22 .61 Experimental Group Two .27* .084 .005 .07 .46 Experimental Group Two Control group .15 .084 .173 -.05 .35 Experimental Group One -.27* .084 .005 -.46 -.07 Based on observed means. The error term is Mean Square(Error) = .588. *. The mean difference is significant at the .05 level.
  44. 44. USING TUKEY TO LOCATE DIFFERENCE IN SPSS MEASURE_1 Tukey HSDa,,b,,c Which group are you N Subset 1 2 Control group 166 7.86 Experimental Group Two 168 8.01 Experimental Group One 166 8.27 Sig. .174 1.000 Means for groups in homogeneous subsets are displayed. Based on observed means. The error term is Mean Square(Error) = .588. a. Uses Harmonic Mean Sample Size = 166.661. b. The group sizes are unequal. The harmonic mean of the group sizes is used. Type I error levels are not guaranteed. c. Alpha = .05.
  45. 45. CHI-SQUARE • A measure of a relationship or an association developed by Karl Pearson in 1900; • Measures the association between two categorical variables; • Compares the observed frequencies with the expected frequencies; • Determines whether two variables are independent; • Allows us to find out whether various sub- groups are homogeneous.
  46. 46. TYPES OF CHI-SQUARE • One-variable Chi-Square (goodness-of-fit test) – used when we have one variable; • Chi-Square test for independence: 2 x 2 – used when we are looking for an association between two variables, with two levels, e.g. the association between (drinking alcohol/does not drink alcohol) and (smoke/does not smoke); • Chi-Square test for independence: r x c – used when we are looking for an association between two variables, where one has more than two levels (heavy smoker, moderate smoker, does not smoke) and (heavy drinker, moderate drinker, does not drink).
  47. 47. FORMULA FOR CHI-SQUARE χ2 = Where: O = observed frequencies E = expected frequencies ∑ = the sum of ∑ − E EO 2 )(
  48. 48. ONE-VARIABLE CHI-SQUARE OR GOODNESS-OF-FIT TEST • Enables us to discover whether a set of obtained frequencies differs from an expected set of frequencies; • One variable only; • The numbers that we find in the various categories are called the observed frequencies; • The numbers that we expect to find in the categories, if the null hypothesis is true, are the expected frequencies; • Chi-Square compares the observed and the expected frequencies.
  49. 49. EXAMPLE: PREFERENCE FOR CHOCOLATE BARS A sample of 120 people were asked which of four chocolate bars they preferred; • We want to find out whether some brands (or one brand) are preferred over others – Research Hypothesis; • If some brands are not preferred over others, then all brands should be equally represented – Null Hypothesis; • If the Null Hypothesis is true, then we expect 30 (120/4) people in each category
  50. 50. ONE-VARIABLE CHI-SQUARE OR GOODNESS-OF-FIT TEST If all brands of chocolate are equally popular, the observed frequencies will not differ much from the expected frequencies; If, however, the observed frequencies differ a lot from the expected frequencies, then it is likely that all brands are not equally popular; Frequencies Chocolate A Chocolate B Chocolate C Chocolate D Observed 20 70 10 20 Expected 30 30 30 30
  51. 51. ONE-VARIABLE CHI-SQUARE/GOODNESS- OF-FIT TEST Observed N Expected N Residual (Difference between observed and expected frequencies) Brand A 20 30 -10.0 Brand B 70 30 40.0 Brand C 10 30 -20.0 Brand D 20 30 -10.0 Total 120 120 Chocolate Chi-square df Asymp. Sig 73.333 3 .000 A chi-square value of 73.3, df = 3 was found to have an associated probability level of 0.000. A statistically significant difference was found between the observed and the expected frequencies, i.e. all brands of chocolate are not equally popular. More people prefer chocolate B (70) than the other bars of chocolate.
  52. 52. CHI-SQUARE TEST FOR INDEPENDENCE (BIVARIATE): 2 X 2 Enables us to discover whether there is a relationship or association between two categorical variables of 2 levels; If there is no association between the two variables, then we conclude that the variables are independent of each other.
  53. 53. A WORKED EXAMPLE Imagine that we have asked 110 students the following: A. Do you smoke and drink? B. Do you smoke but do not drink? C. Do you not smoke but drink? D. Do you abstain from both? Each student can only fall into one group, and thus we have 4 groups (they must be mutually exclusive);
  54. 54. CHI-SQUARE TEST FOR INDEPENDENCE: 2 X 2 (WITH SPSS) Do you drink? * Do you smoke? Crosstabulation Do you smoke? Yes No Total Do you drink? Yes Count 50 15 65 Expected Count 41.4 23.6 65.0 No Count 20 25 45 Expected Count 28.6 16.4 45.0 Total Count 70 40 110 Expected Count 70.0 40.0 110.0 totalOverall totalcolumnxtotalrow cellaofvalueExpected =
  55. 55. CHI-SQUARE TEST FOR INDEPENDENCE: 2 X 2 (WITH SPSS) Value df Asymp. Sig. Exact Sig. (2-sided) Exact Sig. (1-sided) Pearson Chi-Square 12.12 1 .000 Continuity Correction 10.759 1 .001 Likelihood Ratio 12.153 1 .001 Fisher’s Exact Test .001 .001 Linear-by-Linear Association 12.011 1 .001 N of Valid Cases 110 Chi-Square = 12.12 df (degrees of freedom) = (columns -1) x (rows -1) = (2-1) x (2-1) = 1
  56. 56. RESULTS A 2 x 2 Chi-square was carried out to discover whether there was a significant relationship between smoking and drinking. The Chi- square value of 12.12 has an associated probability value of p<0.001, df = 1, showing that such an association is extremely unlikely to have arisen as a result of sampling error. It can therefore be concluded that there is a significant association between smoking and drinking.
  57. 57. MANN-WHITNEY U-TEST FOR INDEPENDENT SAMPLES • Mann-Whitney (non-parametric, nominal and ordinal data) for two groups under one condition – Difference between two independent groups (independent samples), based on ranks • This is the non-parametric equivalent of the t-test for independent samples. • Find the significant differences and then run a crosstabs to look at where the differences lie. • Note where there are NO statistically significant differences as well as where there are statistically significant differences
  58. 58. MANN-WHITNEY U-TEST (SPSS) Ranks 22 43.52 957.50 64 43.49 2783.50 86 form Primary 3 Primary 4 Total the contents are interesting N Mean Rank Sum of Ranks
  59. 59. Test Statisticsa 703.500 2783.500 -.006 .996 Mann-Whitney U Wilcoxon W Z Asymp. Sig. (2-tailed) the contents are interesting Grouping Variable: forma. MANN-WHITNEY U-TEST (SPSS)
  60. 60. THE WILCOXON TEST FOR RELATED SAMPLES This is the non-parametric equivalent of the t-test for related samples. For paired (related) samples in a non-parametric test, e.g. the same group under two conditions. For example, here is the result for one group of females who have rated (a) their own ability in mathematics and (b) their enjoyment of mathematics, both variables using a 5-point scale (‘not at all’ to ‘a very great deal’).
  61. 61. THE WILCOXON TEST FOR RELATED SAMPLES (SPSS) Test Statisticsb How good at mathematics do you think you are? - How much do you enjoy mathematics? Z -2.631a Asymp. Sig. (2-tailed) .009 a. Based on positive ranks. b. Wilcoxon Signed Ranks Test Ranks N Mean Rank Sum of Ranks How good at mathematics do you think you are? - How much do you enjoy mathematics? Negative Ranks 11 94.08 11101.00 Positive Ranks 73 99.11 7235.00 Ties 57 Total 248
  62. 62. KRUSKAL-WALLIS TEST FOR INDEPENDENT SAMPLES • Kruskal-Wallis (non-parametric, nominal and ordinal data) for three or more independent groups under one condition – Difference between more than two independent groups (independent samples), based on ranks • This is the non-parametric equivalent of ANOVA for independent samples. • Find the statistically significant differences and then run a crosstabs to look at where the differences lie. • Note where there are NO statistically significant differences as well as where there are statistically significant differences.
  63. 63. Ranks 7 6.57 5 10.70 5 10.70 17 Age 20-29 30-39 40-49 Total own made-up tests N Mean Rank KRUSKAL-WALLIS TEST (SPSS)
  64. 64. Test Statisticsa,b 4.319 2 .115 Chi-Square df Asymp. Sig. own made-up tests Kruskal Wallis Testa. Grouping Variable: Ageb. KRUSKAL-WALLIS TEST (SPSS)
  65. 65. THE FRIEDMAN TEST FOR 3 OR MORE RELATED GROUPS This is the non-parametric equivalent of ANOVA for related samples. For three or more related samples in a non-parametric test, e.g. the same groups under two conditions. For example, the result for 4 groups of students, grouped according to their IQ (Group 1= IQ up to 90; Group 2 = IQ from 91-110; Group 3 = IQ from 111-125; Group 4 = IQ over 125) who have rated (a) their own ability in mathematics and (b) their enjoyment of mathematics, both variables using a 5-point scale (‘not at all’ to ‘a very great deal’).

×