Poster presentation of a quantitative study comparing the TOEFL iBT to the PBT presented at the doctoral forum of TESOL 2011 in New Orleans, Louisiana and the Graduate Research Forum of UCF in Orlando, Florida
Presenter BiosFirst-year TESOL Doctoral students atthe University of Central Florida.Alison Youngblood has taught ESP,EAP, and K-12 in Japan and South Korea
Theoretical FrameworkThe iBT speaking section is met with unfavorable attitudes by test- takers from across the globe (Stricker&Atali, 2010)Researchers also observed this negative attitude while teaching in an EAP
Our SampleBased on ETS’s 2009 Score ReportLooked at the average composite score and average section score on the iBT and PBT listed by countrySample included scores from countries where both iBT and PBT were reported by ETS
MethodologyResearchers believe it is logical to assume that composite scores include individual test takers’ multiple attemptsStatistical procedures used:1. Dependent t-test 2. Repeated measures
Research Question 1Is there a significant difference in the mean composite scores for iBT and PBT?
ResultsThe test was statistically significant t(143)=-2.65, p<.05. There is a significant difference in mean composite iBT scores (M=551.79, sd=25.87) and mean composite PBT scores (M=545.67, sd=38.05). The 95% confidence interval for the mean
DiscussionOur results indicate that students on average actually score higher on the iBTThis challenges student perceptions that the iBT is ‘harder’However, the converted mean composite of 551.79 on the iBT is
Even though the test showed significance, our effect size was calculated by eta squared and found to be .046. This means only 5% of the variance in composite scores can be attributed to test versionDifficult to make exact comparisons based on composite scores because of differences in
Research Question 2Do students score significantly lower on the speaking section of the iBT in comparison to the other three sections?
ResultsThere is a statistically significant difference in section score ( p<.05) on the iBT sections (F3, 429=95.21). 40% of the variance in score can be accounted for by section.
DiscussionThe speaking section had the highest mean composite score (m=21, sd=1.99).The reading section had the lowest mean composite score(m=18.80, sd= 3.28)The is the opposite of student
LimitationsSmall sample sizeUse of mean composite scores instead of individual scoresUnable to determine the number of individuals who took the test multiple times versus the individuals who only attempted
Further ResearchStudies exist that compare the CBT and PBT, but unable to find studies that compare the PBT to iBTDirect research comparing the iBT to PBT could be helpfulObtain individual scores
References Educational Testing Service (2010). Test and score data summary for TOEFL internet-based and paper- based tests. January 2009-December 2009 Test Data. Retrieved from http://www.ets.org/Media/Research/pdf/71943_web.p df Educational Testing Service (2005). Score Comparison Tables.TOEFL Internet-based Test. Retrieved from http://www.ets.org/Media/Tests/TOEFL/pdf/TOEFL_i BT_Score_Comparison_Tables.pdf Stricker, L.J., &Attali, Y. (2010). Test taker’s attitudes