Kim, J. (2014, September). Exploring the Effect of the Self-Directed English Learning on the English Speaking Test Scores of Korean College Students. Paper presented at the meeting of KAMALL Annual Conference 2014, Seoul, Korea. [Abstract] Most Korean adult learners of English desire to achieve a high level of English speaking proficiency because they value communicative competence in their various work places. To obtain this goal, Self-Directed English Learning (SDEL) supported through multimedia has great potential to help English learners manage their learning process. This presentation explored the effect of the capability of Korean college students to utilize SDEL on their English speaking proficiency. Both the English speaking test and the SDEL Questionnaire were administered by means of computer and mobile technologies. At the beginning of the spring of 2014, 90 students responded to the online SDEL Questionnaire at a university in Daejeon, Korea. They also took the computer based English Speaking and Writing Test (ESWT). The pertinent information of these participants is as follows: 37 males and 53 females, ages ranging from 20 to 30 years old, all possessing diverse English levels, and all of whom were TESOL majors. The questionnaire was developed by means of the Google Docs survey. The ten features of self-directedness are: (1) interpersonal ability, (2) self-esteem, (3) self-confidence, (4) anxiety with English, (5) goals, (6) motivation, (7) self-directedness, (8) information process ability, (9) self-understanding, and (10) overall level. They were measured using seven scales. Most students filled out the online questionnaire with their smartphones. In addition, the learners were required to practice their English speaking using two multimedia English programs. These were DynEd and Reading Assistant. DynEd is a conversation program or application, while Reading Assistant is an online read aloud program. The students were required to study English with the speaking programs for up to 200 hours as part of their graduation requirement as stipulated by their department. The seven scoring criteria of the ESWT include (1) task completion, (2) coherence, (3) pronunciation, (4) fluency, (5) language use, (6) grammar, and (7) overall scores. They were rated using five scales including 0.5 units. Two raters rated the speech samples after receiving appropriate rater training. [....]

1. Exploring the Effect of the Self-Directed
English Learning on the English Speaking Test
Scores of Korean College Students
Jungtae Kim, Ph.D.
Department of TESOL,
Pai Chai University

2. Backgrounds
• Criticism of English courses at Korean
colleges
– Ambiguous goals: What to teach?
– Mass instruction: more than 35 ss
– Teacher centered
– Mismatch between students’ needs and
English education at colleges
– TOEIC L/R?
– Requiring English speaking scores from
companies

3. 1. The need of communicative competence
Students’
needs
Social
needs
Commu
nicative
compete
nce

4. 2. What do they really want?
• Students’ needs
– Achieving high speaking ability
– Practical English > academic English
– TOEIC L/R?
• Social needs
– English speaking skills?
– OPIc, TOEFIC Speaking
> Writing skills?
> Reading skills?

5. 3. Colleges’ Reaction to the needs
• TOEIC L/R
• Conversations courses
• Teaching strategies for high scores of
commercials tests
• Adopting TOEIC Speaking test, OPIc
• Mass instruction
• Passive students

6. 4. Student Interview & Survey
• High TOEIC score
• Conversation skills
• Fluency
• Good pronunciation
• Reading (TOEIC Listening > Reading
scores)
• Feedback

7. 5. Students’ SDL capability
Many students are
• Low motivation
• Low self-directedness
• Low self-esteem
• Low self-confidence
• High English anxiety
• Test wiseness
=> Self-Directed English Learning

8. Theoretical Foundations
• Self-directed language learning relies on the
student to have acquired learner autonomy.
• Little, D. (2014)
Autonomous learners understand the purpose of
their learning program, explicitly accept responsibility
for their learning, share in the setting of learning
goals, take initiatives in planning and executing
learning activities, and regularly review their learning
and evaluate its effectiveness (cf. Holec 1981, Little
1991)

9. • Lee and Son (2007)
- A difficult situation of learning English in
Korea
- Limited English learning opportunities:
limited time to learn English in the
classroom
- The concept of self-directed learning is
quite necessary in addressing common
challenges uniquely associated with EFL
learning.

10. • Dam & Legenhausen 1996, Legenhausen
1999.
Autonomisation = strategy training
• Autonomy in language education covers
Learning strategies, self-regulation,
motivation, individual differences,
sociocultural approaches, teacher
development

11. • In sum, mostly language learning
strategies and attitudes to language
learning were researched.
• The relationship between SDEL capability
and language ability improvement –
elementary level, a little research on
college level
• A gap between self-directed language
learning and L2 speaking ability

12. Research Question
To what extent are students’ SDEL
capability correlated with their English
speaking scores?
• SDEL capability vs. English speaking
scores

13. PCU SDEL Model
Learning
Coaching Assessment

14. 1. English Consulting System
Assessment -
Diagnosis
- Online survey
- L/S+R/W
- Rater Training
Placement-
Learning
-Different levels
-Self-directed
learning
-Computer-
Assisted Learning
Self-directed
learning
Coaching
-Training coaches
-Management of
learning
– Weekly group
meeting
-

15. 2. Self-directed English Learning System
SDEL
• Online self-directed English
learning survey
ESWT
• English Speaking & Writing
Tests

16. DynEd
Real life Conversation
Improving speaking
proficiency
Reading Assistant
30min/day, 4days/week
-> 25 hrs/semester
Graduation Requirement
Reading aloud
Improving fluency +
reading speed

19. Methodology

20. 1. Participants
• 90 Korean English learners at PCU,
Daejeon
• 37 male, 53 female
• 20-30 years old
• Diverse English levels
• TESOL or double major

21. 2. SDEL Survey
• Developed by Y. W. Cho based on
– Griffiths, C. (Ed.). (2008). Lessons from good language learners.
Cambridge University Press.
– Dornyei, Z., & Ushioda, E. (Eds.). (2009). Motivation, language
identity and the L2 self. Multilingual Matters.
• The survey consists of 10 features:
(1) interpersonal ability, (2) self-esteem, (3) self-confidence,
(4) resisting to anxiety with English, (5) goals, (6) motivation,
(7) self-directedness, (8) information process ability,
(9) self-understanding, and (10) overall level.
• Reliability: Cronbach’s alpha =.76

22. 3. ESWT
• ESWT(English Speaking & Writing Test)
• Providing English speaking/writing scores
per semester
• Preparing for TOEIC Speaking or OPIc
• Monitoring score improvement over
years

23. 4. Test administration
• Spring, 2014
• Mass admin: 60 Ss per time
• Computer-Based Test

24. 5. Introduction to ESWT
• Speaking Part (4 items)
– Self introduction
– Picture story telling with 6 pictures of
students’ real life
– Describing a chart
– Giving opinions
– Preparation time: 30 sec
– Response time: 1 min
– 2 chances for answering

25. • Writing Part (1 item)
– Giving opinions
– 15 min for preparation and response

35. 6. Rating
• Two raters, Analytic+Holistic Scoring
• Rater training – 3 time, 6 hours per
semester
• 10 scales: 0, 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5
• Scoring Rubric based on ACTFL Scoring
Guidelines

36. ESWT Standard Setting
ESWT 토익스피킹 OPIc 토익L/R
5 8 AH 950-990
4.5 7 AM 915-945
4 7 AL 880-910
3.5 6 IH 780-875
3 6 IM 675-775
2.5 5 IL 535-670
2 3,4 NH 345-530
1.5 2 NM 0-345
1 1 NL 0-345
0 0 0 0

37. Scoring Constructs
Feature Descriptors
Task
Completion
The speaker’s ability to complete tasks by selecting language functions to reas
onably address the task and to select the language needed to carry out require
d functions in academic settings.
Fluency The speaker’s natural flowing speech including hesitation, repetition,
re-structuring, inappropriate words and sentences in academic settings.
Accuracy The speaker’s appropriateness of pronunciation, grammar, and language use i
n academic settings.
Coherence The speaker’s clear and logical organization of his/her utterances in academic
settings

38. Level Descriptors
Score Descriptors
5 The speaker’s communicative competence is almost always effective in terms of fluency, functional compete
nce, accuracy, and coherence of their speech in academic settings. They perform given tasks very competently
. It is near-native speaker’s English proficiency.
4 The speaker’s communicative competence is generally effective in terms of fluency, functional competence,
accuracy, and coherence of their speech in academic settings. They perform given tasks competently.
3 The speaker’s communicative competence is somewhat effective in terms of fluency, functional competence,
accuracy, and coherence of their speech in academic settings. They perform given tasks somewhat competentl
y.
2 The speaker’s communicative competence is NOT generally effective in terms of fluency, functional compet
ence, accuracy, and coherence of their speech in academic settings. They do NOT perform given tasks general
ly competently.
1 The speaker’s communicative competence is extremely poor in terms of fluency, functional competence, acc
uracy, and coherence of their speech in academic settings. They can NOT perform given tasks at all.
0 No Response

39. Experimental Rating Procedure
• Top-down rating
1. Task completion
2. Holistic Score (allow 1 scale difference)
3. Coherence
4. Fluency & Pronunciation
5. Language use
6. Grammar

40. Results

41. 1. ESWT Scores Stat
Variable Mean S.D. Min Max Range
S_level 2.50 0.93 1.00 5.00 4.00
Task 2.94 1.10 1.00 5.00 4.00
Coher 2.38 1.09 1.00 5.00 4.00
Fluency 2.41 1.01 1.00 5.00 4.00
Pronun 2.60 1.03 1.00 5.00 4.00
LangUse 2.37 0.95 1.00 5.00 4.00
Grammar 2.35 0.88 1.00 4.75 3.75

42. 2. SDEL Survey Stat
Variable Mean S.D. Minimum Maximum Range
InterP 5.59 1.11 1.50 7.00 5.50
SelfES 5.15 0.98 3.29 7.00 3.71
SelfConf 5.26 0.91 3.29 7.00 3.71
EnAnxi 4.45 1.17 1.86 7.00 5.14
Goals 4.93 1.07 3.00 7.00 4.00
Motiv 5.74 0.97 3.67 7.00 3.33
SelfDir 4.57 1.19 2.63 7.00 4.37
InforPro 4.92 0.95 2.25 7.00 4.75
SelfUnder 5.72 1.04 2.33 7.00 4.67
SDLevel 5 0.83 3.57 6.59 3.02

43. Crosstab = ESWT Level vs. SDEL Level
ESWT_level
1 1.5 2 2.5 3 3.5 4 4.5 5 Total
SDEL
Level
4 6 7 10 1 4 0 0 0 0 28
5 4 3 9 4 5 4 1 1 1 32
6 1 2 3 4 9 1 3 1 2 26
7 0 1 0 1 2 0 0 0 0 4
Total 11 13 22 10 20 5 4 2 3 90
ESWT – beginner level (46), intermediate (35), advanced (9)
Beginner ESWT+Mid SDEL=10,
Beginner ESWT+Level 5 SDEL=9
Intermed ESWT+Level 6 SDEL=9

44. Spearman Correlation Coefficients
Variable ESWT_level SDEL_level
ESWT_level 1.00
SDEL_level 0.45 1.00
Mean 2.40 5.08
S.D. 0.93 0.83

45. Spearman Correlation Coefficients – ESWT
Variable S_level Task Coher Fluency Pronun LangUse Grammar
S_level 1.00
Task 0.94 1.00
Coher 0.96 0.94 1.00
Fluency 0.96 0.91 0.96 1.00
Pronun 0.88 0.78 0.84 0.85 1.00
LangUse 0.96 0.91 0.91 0.92 0.86 1.00
Grammar 0.94 0.89 0.90 0.91 0.83 0.97 1.00
Mean 2.40 2.94 2.38 2.41 2.60 2.37 2.35
S.D. 0.93 1.10 1.09 1.01 1.03 0.95 0.88

46. Spearman Correlation Coefficients – SDEL Survey
Variab InterP SelfES SelfC EnAnxi Goals Motiv SelfDir InforPr SelfUn SDLev
InterP 1.00
SelfES 0.682 1.000
SelfConf 0.510 0.723 1.000
EnAnxi 0.349 0.646 0.575 1.000
Goals 0.360 0.558 0.622 0.616 1.000
Motiv 0.350 0.539 0.580 0.490 0.627 1.000
SelfDir 0.339 0.591 0.675 0.576 0.737 0.643 1.000
InforPr 0.473 0.636 0.728 0.661 0.737 0.631 0.689 1.000
SelfUn 0.358 0.553 0.548 0.414 0.482 0.550 0.546 0.546 1.000
SDLeve 0.637 0.841 0.841 0.778 0.827 0.757 0.829 0.867 0.624 1.000
Mean 5.592 5.154 5.265 4.451 4.925 5.735 4.573 4.922 5.722 5.076
S.D. 1.111 0.984 0.911 1.167 1.073 0.966 1.187 0.947 1.043 0.829

47. Spearman Correlation Coefficients
– ESWT vs. SDEL Survey
Variable S_level Task Coher Fluency Pronun LangUse Grammar
InterP 0.24 0.26 0.27 0.22 0.30 0.24 0.16
SelfES 0.33 0.34 0.37 0.35 0.34 0.33 0.30
SelfConf 0.44 0.43 0.46 0.45 0.41 0.46 0.44
EnAnxi 0.31 0.30 0.34 0.32 0.26 0.30 0.30
Goals 0.38 0.35 0.40 0.39 0.35 0.37 0.35
Motiv 0.38 0.34 0.37 0.39 0.37 0.35 0.32
SelfDir 0.44 0.42 0.45 0.45 0.40 0.45 0.42
InforPro 0.38 0.38 0.40 0.37 0.35 0.39 0.36
SelfUnder 0.28 0.28 0.29 0.25 0.31 0.27 0.25
SDLevel 0.45 0.44 0.48 0.46 0.44 0.45 0.42
Mean 2.40 2.94 2.38 2.41 2.60 2.37 2.35
S.D. 0.93 1.10 1.09 1.01 1.03 0.95 0.88

48. Spearman Correlation Coefficients
- SDEL criteria vs. ESWT Level
• The correlation coefficients between the SDEL
criteria and the overall ESWT score: mostly low
- 0.22 to 0.45.
• The lowest correlation coefficient was 0.22
between motivation and the overall ESWT
score
• The highest coefficient was 0.45 between self-directedness
and the overall ESWT score.

49. Spearman Correlation Coefficients
- ESWT criteria vs. SDEL Level
• The correlation coefficients between the overall
level of the SDEL and the scoring criteria of
the ESWT - 0.38 to 0.47
• The lowest correlation coefficient was 0.38
between the overall level of the SDEL and the
grammar score
• The highest coefficient was 0.47 between the
overall level of the SDEL and the fluency score

50. • A post-hoc test with High and Low
groups - low and moderate correlations
• the students’ SDEL capability might not
affect their English speaking scores.

51. Case 1=High SDEL w/ Low ESWT Level
Grade 3, Female, Married to American
S_level Task Coher Fluency Pronun LangUse Gramma
2.8 4.25 3.25 3.2 2.4 3 3.5
SDLevel InterP SelfES SelfCon EnAnxi Goals Motiv SelfDir InforPr Self_Un
6.52 4 6.14 7 7 7 7 7 7 7
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
S_level Task Coher Fluency Pronun LangUse Grammar
- Lack of interaction w/ classmates
- Low Inter personal ability
- Slow speaker
- Fossilization on her pronunciation
- High Task completion
- Unique pattern

52. Case 2=Low SDEL w/ Low ESWT
Grade 1, Female, Low motivation
S_level Task Coher Fluency Pronun LangUse Grammar
1 1 1 1.2 1.2 1.5 1.75
SDLevel InterP SelfES SelfConf EnAnxi Goals Motiv SelfDir InforPro SelfUnd
4 5 4 4 3 5 5 5 5 6
7
6
5
4
3
2
1
0
- Mid SDEL w/ low ability to
overcome English anxiety
- Mid self-esteem & self-confidence
- Very beginner level in L2 speaking
- Not expect high achievement in
the future in English speaking due
to low SDEL

53. Case 2=High SDEL w/ Mid ESWT
Grade 4, Female, high motivation
S_level Task Coher Fluency Pronun LangUse Grammar
3 3.5 3 3 4 3 2.5
SDLevel InterP SelfES SelfConf EnAnxi Goals Motiv SelfDir InforPro SelfUnd
6 7 6 7 6 6 7 5 6 7
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
S_level Task Coher Fluency Pronun LangUse Grammar
- High SDEL w/ relatively low
ability of self-directedness
- High GPA
- Expected advanced level, but
intermedaite speaking scores
- Low accuracy
- Maybe nervous on taking
speaking test

54. Discussion
• Low reliability from the SDEL Survey with
different students – Cronbach alpha: 0.76,
but moderate correlation among the
features of the SDEL Survey.
• High SDEL ability does NOT guarantee
high English speaking scores.
• High interest in English learning does
NOT guarantee high English speaking
scores.

55. • Dominant low level ESWT scores w/
various SDEL levels
• Troubles to make connections between
low level ESWT scores w/ mid SDEL
levels
• Caring low level SDEL levels along with
improving English speaking ability
• Without maintaining high SDEL capability,
NO English speaking improvement!

56. Thank you for listening.
Contact: kim.jungtae@gmail.com