The document summarizes earlier studies that examined relationships between writing and speaking skills for EFL learners. It presented findings from two previous studies that analyzed TOEFL writing and speaking samples. Both studies found high correlations between holistic and analytic scoring rubrics, and among scores from each rubric. The present study aims to explore relationships between writing and speaking as productive skills, using the same participants and scoring rubrics as the earlier studies.

1. Relationships between EFL
Writing and Speaking Skills
- The Case of 16 Japanese EFL
Students’ Performances
on the TOEFL CBT/iBT -
MA Thesis Oral Examination
Shuta Miyazaki
2012-2-14

2. Background of the Study
• Key words: EAP, TOEFL, Productive Skills,
Holistic Assessment, Analytic Assessment
• Purpose: To find out…
– Whether EFL Writing and Speaking skills are
correlated
– If so, what kind of relationships?
• An experimental study on Japanese EFL
learners’ performance on TOEFL

3. Earlier Studies (1)
• Lee, Gentile, and Kantor (2008)
• Holistic and analytic scoring of 930 samples
for TOEFL CBT’s Writing section.
• Holistic rubrics – ETS (1998)
• Analytic rubrics – Developed for this study by
a panel of applied linguists
• Pearson Correlation

5. P rom pt 1
O veral
l D evel ent
opm O rgani on
zati Vocabul
ary Sentence G ram m ar M echani
cs
O veral
l C orrel on C oeffi ent
ati ci 1 0.
88 0.
85 0.
9 0.
87 0.
83 0.
72
D evel ent C orrel on C oeffi ent
opm ati ci 0.
88 1 0.
84 0.
85 0.
81 0.
77 0.
66
O rgani on C orrel on C oeffi ent
zati ati ci 0.
85 0.
84 1 0.
81 0.
78 0.
74 0.
66
Vocabul
ary C orrel on C oeffi ent
ati ci 0.
9 0.
85 0.
81 1 0.
87 0.
83 0.
70
Sentence C orrel on C oeffi ent
ati ci 0.
87 0.
81 0.
78 0.
87 1 0.
88 0.
72
G ram m ar C orrel on C oeffi ent
ati ci 0.
83 0.
77 0.
74 0.
83 0.
88 1 0.
73
M echani
cs C orrel on C oeffi ent
ati ci 0.
72 0.
66 0.
66 0.
70 0.
72 0.
73 1
P rom pt 2
O veral
l D evel ent
opm O rgani on
zati Vocabul
ary Sentence G ram m ar M echani
cs
O veral
l C orrel on C oeffi ent
ati ci 1 0.
88 0.
83 0.
88 0.
87 0.
85 0.
75
D evel ent C orrel on C oeffi ent
opm ati ci 0.
88 1 0.
81 0.
82 0.
81 0.
77 0.
73
O rgani on C orrel on C oeffi ent
zati ati ci 0.
83 0.
81 1 0.
78 0.
77 0.
71 0.
71
Vocabul
ary C orrel on C oeffi ent
ati ci 0.
88 0.
82 0.
78 1 0.
83 0.
81 0.
72
Sentence C orrel on C oeffi ent
ati ci 0.
87 0.
81 0.
77 0.
83 1 0.
89 0.
72
G ram m ar C orrel on C oeffi ent
ati ci 0.
85 0.
77 0.
71 0.
81 0.
89 1 0.
73
M echani
cs C orrel on C oeffi ent
ati ci 0.
75 0.
73 0.
71 0.
72 0.
72 0.
73 1

6. Earlier Studies (2)
• Xi and Mollaun (2006)
• Holistic and analytic scoring of 280 samples
for TOEFL iBT’s Speaking section.
• Holistic rubrics – ETS (2004)
• Analytic rubrics – developed for this study by
a panel of language teaching and testing
specialists
• Pearson Correlation

7. P rom pt 1
Topi
c
O veral
l D elvery
i Language U se
D evel ent
opm
O veral
l C orrel on C oeffi ent
ati ci 1 0.
85 0.
87 0.
84
D elvery
i C orrel on C oeffi ent
ati ci 0.
85 1 0.
84 0.
84
Language U se C orrel on C oeffi ent
ati ci 0.
87 0.
84 1 0.
84
Topi
c
C orrel on C oeffi ent
ati ci 0.
84 0.
84 0.
84 1
D evel ent
opm
P rom pt 2
Topi
c
O veral
l D elvery
i Language U se
D evel ent
opm
O veral
l C orrel on C oeffi ent
ati ci 1 0.
72 0.
72 0.
75
D elvery
i C orrel on C oeffi ent
ati ci 0.
72 1 0.
79 0.
78
Language U se C orrel on C oeffi ent
ati ci 0.
72 0.
79 1 0.
81
Topi
c
C orrel on C oeffi ent
ati ci 0.
75 0.
78 0.
81 1
D evel ent
opm

8. Findings of Earlier Research
• Dependability of analytic scoring rubrics
• Large-scale EAP (ESL, EFL) writing study
– High correlations
• Between holistic scores and analytic scores
• Among each analytic score
• Large-scale EAP (ESL, EFL) speaking study
– High correlations
• Between holistic scores and analytic scores
• Among each analytic score

9. So, How about
Relationship between
Writing and Speaking??

10. Present Study
• Writing and Speaking as two aspects of
productive skills – interface??
• Same scoring rubrics as earlier studies (both
holistic and analytic)
• Same prompts (writing test only)
• Case study of Japanese EFL learners

11. Participants
O nset of
M other Ti e spent
m G rade
I # Sex
D Age M aj
or l
earning (Pl
ace) Score(TO EFL) Score(TO EI )
C
tongue abroad (STEP)
Englsh
i
G erm an/Inter
1 M 21 national Japanese 10 1 m onth N ew Zeal
and 800
Rel ons
ati
2 M 21 Phiosophy
l Japanese 13 0 755
Englsh
i
3 M 18 Japanese 12 0 445
Literature
4 M 20 Econom i cs Japanese 12 0
Englsh
i
5 F 19 Japanese 12 0
Literature
6 M 22 Econom i cs Japanese 13 0 735
Englsh
i
7 F 19 Japanese 13 0 2nd
Literature
Englsh
i
8 F 19 Japanese 12 0
Literature
9 F 20 B i ogy
ol Japanese 10 3 years U ni States
ted
Englsh/I
i nter
10 F 19 national Japanese 10 0 Pre-1st
Rel ons
ati
French
11 F 20 Japanese 8 0 Pre-1st
Literature
Englsh/I
i nter
12 F 19 national Japanese 12 0 800 Pre-1st
Rel ons
ati
13 M 20 Law Japanese 13 0 760
Englsh/Li
i ngu
14 M 20 Japanese 10 1 year Sw eden 580(PB T) Pre-1st
i cs
sti
15 F 21 Portuguese Japanese 12 0
16 F 22 Soci ogy
ol Japanese 6 0 650

12. Writing Prompts
Prom pt ID Prom pt Topi
c Sam pl Si
e ze
D o you agree or di
sagree w i the folow i topi
th l ng c?
I i m ore i portant for students to study hi
ts m story and lterature
i
A than i i for them to study sci
ts ence and m athem atics. 9
U se speci c reasons and exam pl to support your opi on.
fi es ni
Som e young chidren spend a great am ount of ti e practi ng sports.
l m ci
B Discuss the advantages and di
fi
sadvantages of thi
es
s.
U se speci c reasons and exam pl to support your opi on.
ni
7

13. Speaking Prompts
Prom pt ID Prom pt Topi
c Sam pl Si
e ze
W here i your hom e country w oul you m ost lke to go on vacati
n d i on
A and w hy?
I ude detais and exam pl i your expl
ncl l es n anation.
8
Som e students lke to study i the m orni
i n ng.
O thers feel they study best i the eveni
n ng.
B W hi ti e do you thi i better for you to study and w hy?
ch m nk s 7
I ude detais and exam pl i your expl
ncl l es n anati
on.

14. Procedure
• Test administration – 30 mins. Writing & 45
secs. Speaking (after 15 secs. preparation)
• Rater training and scoring – 1 NS, 2 NNS,
ESL/EFL teachers
• Culculations with SPSS
– Inter-rater reliability
– Non-parametric analysis (Spearmann’s rho)
among writing and scoring scores

15. Results
• Inter-rater reliability
Rater A Rater B Rater C
Spearm an' rho Rater A
s C orrel on C oeffi ent
ati ci 1 0.
387 .
702**
Si (2-taied)
g. l . 0.
138 0.
002
Rater B C orrel on C oeffi ent
ati ci 0.
387 1 .
675**
Si (2-taied)
g. l 0.
138 . 0.
004
Rater C C orrel on C oeffi ent
ati ci .
702** .
675** 1
Si (2-taied)
g. l 0.
002 0.
004 .

16. Results (Con’d)
• Writing
W ri n g P ro m pt A
ti
O ver l
al D evel
opm ent Organi
zat on
i V ocabul y
ar S entence Gram m ar M echani
cs
1 3.
67 4.
00 2.
67 3.
00 2.
33 4.
00 4.
00
2 3.
00 3.
00 3.
33 3.
33 2.
33 3.
67 3.
67
3 2.
00 2.
67 3.
00 3.
00 2.
33 3.
00 3.
00
4 2.
00 1.
67 1.
67 1.
67 2.
67 1.
00 1.
33
7 2.
67 2.
67 3.
33 2.
67 2.
33 3.
67 3.
67
11 3.
67 4.
00 3.
33 3.
33 3.
00 3.
67 3.
00
12 4.
00 4.
00 4.
00 3.
33 3.
33 3.
67 4.
33
13 3.
33 3.
67 3.
67 3.
00 3.
33 3.
67 3.
67
16 2.
67 3.
00 2.
67 2.
67 2.
67 3.
33 2.
33
M ean 3.
00 3.
19 3.
07 2.
89 2.
70 3.
30 3.
22
SD 0.
73 0.
80 0.
68 0.
53 0.
42 0.
90 0.
93
W ri n g P ro m pt B
ti
O ver l
al D evel
opm ent O rgani i
zat on V ocabul y
ar S entence G r m ar
am M echani
cs
5 4.
00 4.
33 4.
67 3.
67 4.
00 4.
00 4.
00
6 2.
33 2.
67 3.
00 3.
00 2.
67 3.
00 3.
00
8 2.
33 2.
33 3.
00 2.
67 2.
33 3.
33 3.
33
9 2.
33 2.
33 2.
67 2.
00 2.
33 2.
67 3.
33
10 3.
00 3.
33 3.
67 2.
67 2.
67 3.
67 3.
67
14 3.
00 3.
33 3.
33 3.
00 3.
00 3.
67 3.
67
15 3.
33 3.
00 3.
33 2.
67 2.
67 3.
67 3.
67
M ean 2.
90 3.
05 3.
38 2.
81 2.
81 3.
43 3.
52
SD 0.
63 0.
71 0.
65 0.
50 0.
57 0.
46 0.
33

17. Results (Con’d)
• Writing (Con’d) n=16
O veral D evel ent O rgani on Vocabul
l opm zati ary Sentence G ram m ar M echani cs
O veral
l C orrel on C oeffi ent
ati ci 1 .
926** .
646** .
626** .697** .710** .
611*
Si (2-taied)
g. l 0 0.007 0.009 0.
003 0.002 0.012
D evel ent
opm C orrel on C oeffi ent
ati ci .926** 1 .
624** .
729** .743** .644** 0.488
Si (2-taied)
g. l 0 0.
01 0.001 0.
001 0.007 0.055
O rgani on
zati C orrel on C oeffi ent
ati ci .646** .
624** 1 .
649** .835** 0.487 .
581*
Si (2-taied)
g. l 0.007 0.01 0.007 0 0.056 0.018
Vocabul
ary C orrel on C oeffi ent
ati ci .626** .
729** .
649** 1 .626** .
507* 0.33
Si (2-taied)
g. l 0.009 0.
001 0.007 0.
009 0.045 0.212
Sentence C orrel on C oeffi ent
ati ci .697** .
743** .
835** .
626** 1 0.423 0.329
Si (2-taied)
g. l 0.003 0.
001 0 0.009 0.103 0.213
G ram m ar C orrel on C oeffi ent
ati ci .710** .
644** 0.487 .
507* 0.
423 1 0.379
Si (2-taied)
g. l 0.002 0.
007 0.056 0.045 0.
103 0.148
M echani
cs C orrel on C oeffi ent
ati ci .
611* 0.
488 .
581* 0.
33 0.
329 0.379 1
Si (2-taied)
g. l 0.012 0.
055 0.018 0.212 0.
213 0.148

18. Results (Con’d)
• Speaking n=15
S peaking P rom pt A S peaking P rom pt B
Language Topi
c Language Topi
c
O veral D elvery
l i O veral
l D elvery
i
U se D evel ent
opm U se D evel ent
opm
1 4.00 4.
00 3.33 3.
67
2 3.
00 2.
67 3.
00 2.
67
5 3.00 3.
00 3.00 2.
67
3 2.
33 2.
67 2.
33 2.
00
6 2.67 3.
00 2.00 2.
67
8 2.00 2.
33 2.00 2.
00
7 2.
00 2.
33 2.
33 2.
00
12 2.67 3.
00 2.33 2.
33 9 1.
00 1.
67 1.
00 1.
00
13 2.00 2.
33 2.33 2.
00 10 3.
33 3.
67 3.
00 3.
33
14 3.00 3.
33 3.00 2.
67 11 2.
67 3.
00 2.
33 2.
33
16 2.67 2.
67 2.67 2.
00 15 2.
00 2.
33 2.
00 1.
67
M ean 2.75 2.
96 2.58 2.
50 M ean 2.
33 2.
62 2.
29 2.
14
SD 0.64 0.
55 0.50 0.
56 SD 0.
77 0.
62 0.
68 0.
74

19. Results (Con’d)
• Speaking (Con’d)
Language Topi
c
O veral D elvery
l i
U se D evel ent
opm
C orrel on C oeffi ent
ati ci 1 .
919** .
876** .
940**
O veral
l
Si (2-taied)
g. l . 0 0 0
C orrel on C oeffi ent
ati ci .
919** 1 .
715** .
907**
D elvery
i
Si (2-taied)
g. l 0 . 0.
003 0
C orrel on C oeffi ent
ati ci .
876** .
715** 1 .
765**
Language U se
Si (2-taied)
g. l 0 0.
003 . 0.
001
Topi
c C orrel on C oeffi ent
ati ci .
940** .
907** .
765** 1
D evel ent Si (2-taied)
opm g. l 0 0 0.
001 .

20. Results (Con’d)
• Writing and Speaking n=15
S peaki
ng
Language T opi
c
O ver l
al D elver
i y
U se D evelopm ent
O ver l
al C or el i C oef i ent
r at on f ci 0.
441 0.
437 0.
463 0.
393
Si (
g. 2-t l
aied) 0.
10 0.
10 0.
08 0.
15
D evel
opm ent C or el i C oef i ent
r at on f ci .
622* .
642** .
640* .
559*
Si (
g. 2-t l
aied) 0.
01 0.
01 0.
01 0.
03
Organi i
zat on C or el i C oef i ent
r at on f ci 0.
196 0.
194 0.
243 0.
245
Si (
g. 2-t l
aied) 0.
48 0.
49 0.
38 0.
38
V ocabul y
ar C or el i C oef i ent
r at on f ci 0.
513 0.
492 0.
443 .
536*
W r t ng
ii
Si (
g. 2-t l
aied) 0.
05 0.
06 0.
10 0.
04
S entence C or el i C oef i ent
r at on f ci 0.
208 0.
319 0.
163 0.
191
Si (
g. 2-t l
aied) 0.
46 0.
25 0.
56 0.
50
G r m ar
am C or el i C oef i ent
r at on f ci .
576* 0.
493 .
698** .
554*
Si (
g. 2-t l
aied) 0.
03 0.
06 0.
00 0.
03
M echani
cs C or el i C oef i ent
r at on f ci 0.
360 0.
304 0.
448 0.
396
Si (
g. 2-t l
aied) 0.
19 0.
27 0.
09 0.
14

21. Discussion
• Weak to moderate correlation among W&S
holistic and analytic scores.
• Writing and speaking abilities could be
correlated, although the correlation
coefficients were not always statistically
significant.
• Strengths of relationships among W&S sub-
skills seem to be different.

22. Limitations
• Small-scale (not big enough for parametric
tests)
• Relatively high-proficiency participants
• Participants were not quite prepared to take
the tests.

23. Pedagogical implications
• W&S skill teaching
– is ought to be implemented in a systematic
manner to build each sub-skill of both aspects of
ESL/EFL productive skills
– Could be well combined, which could help the
learners to build both abilities.