Chapter 5 - Data Analysis and Research B. Interpreting, Analyzing, and Reporting the Results from Data William Allan Kritsonis, PhD INTRODUCTION The purpose of this chapter is to interpret, analyze, and report theresults from data. This chapter will introduce the methods and examples ofthe paired samples t-test, independent samples t-test, one-way ANOVA, andBivariate-Pearson-Correlation. Steps to the tables and External Links foronline tutorial are provided for each test. Software package SPSS 10.0 wasused to analyze the data. THE T-TEST The t-test provides the probability that the null hypothesis is truewhen examining the difference between the means of two groups. Normallywe use this test when data sets are small. There are two different t-tests ♦ The “paired samples t-test,” and ♦ The “independent samples t-test.” THE PAIRED SAMPLES T-TEST When do we use it? We assume that the confidence interval is at 95% allthe time.
1. When there is a natural relationship between the subjects from whom the two sets of scores are obtained. Example 1: Looking at differences between pre- and post-tests of one group, you would choose the paired samples t-test, the scores of both data sets came from the same persons. Example 2: A teacher diagnostically tests her students at the beginning of the year. After intensive instruction, the test is repeated at the end of the semester. She is interested in knowing if the students have made significant gains. THE INDEPENDENT SAMPLES T-TEST (We use this t-test more often)When do we use it?1. When there is no natural relationship between subjects whose scores are being contrasted. Comparing scores obtained from two different groups of people, you would use this t-test.2. The data descriptions are normally distributed and of the groups are homogeneous. Example 1: Two groups of students are identified: an experimental and a control group. Both groups are pretested (Both groups are posttested), an intervention is used with the experimental group and is withheld from the control group. Both groups are posttested.
Example 2: TLI scores are collected on students who have attended school using block scheduling and students who have attended schools with traditional scheduling. We want to know if the TLI scores are significantly different according to the schedule experienced by the students. (We reject the null when p<0.05; we fail to reject the null when p>0.05). ONE-WAY ANALYSIS OF VARIANCE (ANOVA)About the one-way ANOVA,1. The probability that the null hypothesis is true when examining the mean differences among three or more groups. This procedure is equal to the test, except that it handles more than two groups.2. The assumptions for one-way ANOVA are the same as for the t-test: normal distributions and homogeneity of variances. We have to run Levene’s test for homogeneity. There are two situations: (1) use Bonferroni when the data fail to reject the null and (2) use Tamhane when the data reject the null.3. A probability value p < 0.05 indicates that a significant difference exists among the various means, but it does not indicate which means are significantly different and which are chance differences.
Example 1: If the GPA averages were significantly different according toundergraduate majors. We would input all of the GPAs into a variable (thiswould be the dependent variable), and in a second variable (an independentvariable often called a “factor”) assigning a “1” if the GPA belonged to anEnglish major, a “2” for History majors, a “3” for Psychology majors, etc..(Don’t use “0” for anything because it does not work sometimes.)Example 2: TAAS scores are collected to describe scores of students. Threedifferent methods of teaching were used after the students had been dividedinto three equal groups. The socioeconomic level of each student wasidentified. The hypothesis was used: “was there a significant difference inTAAS scores according to method used.”Example 3: Professors who are primarily university administrators, regulartenured professors, and regular non-tenured professors are rated by studentsaccording to enthusiasm displayed in the classes they teach. The nullhypothesis is, “there is no significant difference in the degree of enthusiasmdisplayed among the three groups of professors.”
♦ The standard way to report the one-way ANOVA:The null hypothesis is that there will be no significant differences in______________________________________________________________. To test this hypothesis, the one-way ANOVA from SPSS (10.0) wasused. The null hypothesis is accepted/ not accepted F=(n-1, N-n), p = ____<or > than 0.05.We reject the null when p< 0.05; we fail to reject the null when p> 0.05.Example 4: It was hypothesized that students who excel in fine arts are alsothe best students in the academic subjects. A measure of fine artsachievement and a measure of academic achievement were collected. Therelationship of the two measures was analyzed statistically. (Bivariate-Pearson-Correlation)Example 5: There will not be a significant relationship between the percentof students passing all TAAS tests and the size of the school districts.(Bivariate-Pearson-Correlation)♦ The correlation coefficient is between –1 and 1. The closer to the positive/negative 1 the stronger the relationship. The closer to the 0 the weaker the relationship.
♦ The standard way to report the Bivariate-Pearson-Correlation: The null hypothesis is that there is no significant relationship between the ________________________________________________________. To test this hypothesis, the Bivariate-Person-Correlation from SPSS (10.0) is used. The null hypothesis is accepted/not accepted. (We need to interpret some more details about the data.) Following question was the review question for cohort VII in 2001.
JOINT UNIVERSITY DOCTORAL PROGRAM REVIEW FOR THE COMPREHENSIVE EXAMINATION Review question for Cohort VII A Single Factor ANOVA To compare the effectiveness of three different methods ofteaching reading, 26 children of equal reading aptitude were dividedinto three groups. Each group was instructed for a give period of timeusing one of the three methods. After completing the instructionperiod, all students were tested. The test results are shown in thefollowing table. Is the evidence sufficient to reject the hypothesis thatall three instruction-methods are equally effective? Use α = 0.05. Method I Method II Method IIITest scores: 45 45 44 51 44 50 48 46 45 50 44 55 46 41 51 48 43 51 45 46 45 48 49 47 47 44
To do the following:1) Test the Normality Assumption.2) Test the Equality of Variance Assumption.3) Run the ANOVA test and produce the ANOVA Table.4) Run Post-hoc comparisons.SPSS Data Entry: (check on scale)readscr teachmth 45 1 51 1 48 1 50 1 46 1 48 1 45 1 48 1 47 1 45 2 44 2 46 2 44 2 41 2 43 2 46 2 49 2 44 2 44 3 50 3 45 3 55 3 51 3 51 3 45 3 47 3 Run the Analysis1) Check Normality.
Steps to the tables:1. Analyze → Descriptive Statistics → Explore → Dependent : readscr Factor List: teachmth Go to and check: Statistics → Descriptives Go to and check: Plots → Box plots ♦ Factor Levels to get that ♦ Normality plots with testsExploreTEACHMTH Case Processing Summary Cases Valid Missing Total TEACHMTH N Percent N Percent N Percent READSCR 1.00 9 100.0% 0 .0% 9 100.0% 2.00 9 100.0% 0 .0% 9 100.0% 3.00 8 100.0% 0 .0% 8 100.0% Tests of Normality a Kolmogorov-Smirnov Shapiro-Wilk TEACHMTH Statistic df Sig. Statistic df Sig. READSCR 1.00 .193 9 .200* .933 9 .490 2.00 .173 9 .200* .950 9 .667 3.00 .193 8 .200* .919 8 .437 *. This is a lower bound of the true significance. a. Lilliefors Significance CorrectionTest these two assumptions for each of the three groups:
(a) Normality (b) Homogeneity (equality) of variance Write a short paragraph in which you describe the results.Analyzing the data: (a) The assumption of Normality was analyzed using two tests of significance: the Kolmogorov-Smirnov test and the Shapiro-Wilk test. The Kolmogorov-Smirnov test showed a probability coefficient of 0.2 for each group since this value is greater than 0.05, the Kolmogorov-Smirnov test did not reject the null hypothesis that the scores for each group is normally distributed. The Shapiro-Wilk test showed a probability coefficient of 0.49 for method 1, .667 for method 2, and 0.437 for method 3. In all three cases the coefficient is greater than 0.05. Therefore, the Shapiro- Wilk test did not reject the null hypothesis that the scores for each group are normally. The results for both the Kolmogorov-Smirnov test and Shapiro-Wilk test provide support for the assumption of normality. (b) The assumption of homogeneity of variance was tested using the Levene test. The results for the Levene test showed a probability coefficient of 0.042. Since this value is less than 0.05, the null
hypothesis is rejected. The assumption of homogeneity of variance is not supported. Post-hoc comparisons among the groups could be tested with either theBonferroni or Tamhane test, depending on whether or not the homogeneityof variance assumption was rejected. The Bonferroni test is appropriate ifthe homogeneity of variance assumption is supported and the Tamhane testis appropriate when it is not supported. Since the Levene test showed thatthe homogeneity of variance assumption was not supported, the Tamhanetest was used to test differences between the means of the three groups.
Descriptives TEACHMTH Statistic Std. ErrorREADSCR 1.00 Mean 47.5556 .6894 95% Confidence Lower Bound 45.9657 Interval for Mean Upper Bound 49.1454 5% Trimmed Mean 47.5062 Median 48.0000 Variance 4.278 Std. Deviation 2.0683 Minimum 45.00 Maximum 51.00 Range 6.00 Interquartile Range 3.5000 Skewness .335 .717 Kurtosis -.651 1.400 2.00 Mean 44.6667 .7454 95% Confidence Lower Bound 42.9479 Interval for Mean Upper Bound 46.3855 5% Trimmed Mean 44.6296 Median 44.0000 Variance 5.000 Std. Deviation 2.2361 Minimum 41.00 Maximum 49.00 Range 8.00 Interquartile Range 2.5000 Skewness .450 .717 Kurtosis 1.300 1.400 3.00 Mean 48.5000 1.3628 95% Confidence Lower Bound 45.2776 Interval for Mean Upper Bound 51.7224 5% Trimmed Mean 48.3889 Median 48.5000 Variance 14.857 Std. Deviation 3.8545 Minimum 44.00 Maximum 55.00 Range 11.00 Interquartile Range 6.0000 Skewness .429 .752 Kurtosis -.887 1.481
To analyzing, interpreting and reporting the results from data: Method I has 9 scores ranging from 45 as the lowest score to thehighest score of 51. The mean of the distribution is 47.56, the median is 48,and the standard deviation is 2.07. The skew and Kurtosis coefficients are0.34 and –0.65, respectively. Method I can be considered as a normaldistribution. Method II has 9 scores ranging from 41 as the lowest score to thehighest score of 49. The mean of the distribution is 44.67, the median is 44,and the standard deviation is 2.24. The skew and Kurtosis coefficients are0.45 and 1.3, respectively. Method II can be considered as a normaldistribution. Method III has 8 scores ranging from 44 as the lowest score to thehighest score of 55. The mean of the distribution is 48.5, the median is 48.5,and the standard deviation is 3.85. The skew and Kurtosis coefficients are0.43 and –0.887, respectively. Method III can be considered as a normaldistribution.
2) Finish the Analysis: run the ANOVA test/table, and Post-hoc comparisons.Steps to the tables:1) Analyze → Compare means → one way ANOVA Dependent: readscr Factor: teachmth Go to: Post-hoc, check: ♦ Bonferroni ♦ Tamhane T2 Go to: Options, check: ♦ Statistics ♦ Descriptives ♦ Homogeneity of VarianceOneway Descriptives READSCR 95% Confidence Interval for Mean Std. Lower Upper N Mean Deviation Std. Error Bound Bound Minimum Maximum 1.00 9 47.5556 2.0683 .6894 45.9657 49.1454 45.00 51.00 2.00 9 44.6667 2.2361 .7454 42.9479 46.3855 41.00 49.00 3.00 8 48.5000 3.8545 1.3628 45.2776 51.7224 44.00 55.00 Total 26 46.8462 3.1457 .6169 45.5756 48.1167 41.00 55.00
Test of Homogeneity of Variances READSCR Levene Statistic df1 df2 Sig. 3.641 2 23 .042 ANOVA READSCR Sum of Mean Squares df Square F Sig. Between Groups 69.162 2 34.581 4.463 .023 Within Groups 178.222 23 7.749 Total 247.385 25 The null hypothesis (H0) is that there is no significant difference in theeffectiveness of three different methods of teaching reading. To test thishypothesis, the one-way ANOVA from SPSS (10.0) was used. The nullhypothesis is rejected F(2/23) = 4.463, p = 0.023<0.05.
Post Hoc Tests Multiple Comparisons Dependent Variable: READSCR 95% Confidence Mean Interval Difference Lower Upper (I) TEACHMTH (J) TEACHMTH (I-J) Std. Error Sig. Bound Bound Bonferroni 1.00 2.00 2.8889 1.3122 .114 -.4993 6.2771 3.00 -.9444 1.3526 1.000 -4.4369 2.5480 2.00 1.00 -2.8889 1.3122 .114 -6.2771 .4993 3.00 -3.8333* 1.3526 .028 -7.3258 -.3408 3.00 1.00 .9444 1.3526 1.000 -2.5480 4.4369 2.00 3.8333* 1.3526 .028 .3408 7.3258 Tamhane 1.00 2.00 2.8889* 1.3122 .035 .1815 5.5963 3.00 -.9444 1.3526 .909 -5.2769 3.3880 2.00 1.00 -2.8889* 1.3122 .035 -5.5963 -.1815 3.00 -3.8333 1.3526 .091 -8.2019 .5352 3.00 1.00 .9444 1.3526 .909 -3.3880 5.2769 2.00 3.8333 1.3526 .091 -.5352 8.2019 *. The mean difference is significant at the .05 level. Post-hoc comparisons among the groups could be tested with eitherthe Bonferroni or Tamhane test, depending on whether or not thehomogeneity of variance assumption was rejected. The Bonferroni test isappropriate if the homogeneity of variance assumption is supported and theTamhane test is appropriate when it is not supported. Since the Levene testshowed that the homogeneity of variance assumption was not supported, theTamhane test was used to test differences between the means of the threegroups.The Tamhane test indicated the following:
♦ There was a statistically significant difference between the mean of Method I and Method II (sig.=0.035). The mean for Method I was 2.889 higher than the mean for Method II.♦ There were no statistically significant differences between Methods I and III or Methods II and III. Links for SPSS 10.0 TutorialStatistical Package for the Social SciencesIt covers a broad range of statistical procedures that allow you to summarizedata (e.g., compute means and standard deviations), determine whether thereare significant differences between groups (e.g., t-tests, analysis ofvariance), examine relationships among variables (e.g., correlation, multipleregression), and graph results (e.g., bar charts, line graphs). Written by GilEinstein and Ken AbernethyAnalysis of Variance (ANOVA) ProceduresJ. Cooper Cutting SPSS for Windows: Brief How-Tos Introduction SPSS basics GraphsDescriptive Statistics Crosstabulation and Chi-square Reliability Regression ProceduresT-tests ANOVAs Nonparametric tests Factor Analysis Cluster Analysis Discriminalilt.ilstu.eduUsing SPSS 10.0 for Windows (for statistical analysis)This link introduces you how to collect, interpret, and report the data by using WindowsExcel program. ANOVA, Independent T-test, Levene’s test, and other statistical methodsfor analysis are included.SPSS Tutorial for Research and AnalysisThis link will take you to the Confidence Interval, One sample T-test, Independent T-test,Mann Whitney U test, Paired T-Test, and Wilcoxon Signed Rank Test and Sign Test andinterpreting the output data.Computing a T-Test for Between-Subjects DesignsIn this link, we will describe how to analyze the results of between-subjects designs.It is important to distinguish between these two types of designs because they requiredifferent versions of the t-test.