ISBN2286-8909
Volume26Number1June2013
Thailand TESOL Organization
under the Patronage of Her Royal Highness Princess Galyani
Vadhana Krom Luang Naradhiwas Rajan...
THAILAND TESOL EXECUTIVE COMMITTEE
March 2011 – May 2013
Advisory Board Arunee Wiriyachitra Independent Consultant
Suntana...
SIG Chairs
ESP Panna Chaturongakul Thammasat University
Research Kanjana Chattrakul Suan Dusit Rajabhat
University
CALL Su...
THAITESOL
JOURNAL
Volume 26 Number 1
Published by Thailand TESOL Organization under the Patronage of Her Royal Highness
Pr...
EDITORS
Editor
Pannathon Saengarun
Suranaree University of Technology
Thailand
Managing Editor
Maneepen Apibalsri
Thailand...
Nattaya Puekpong
Suranaree University of Technology
Thailand
Nopporn Sarobol
Thammasart University
Thailand
Panna Chaturon...
Editorial
Welcome to the first issue of the Thai TESOL Journal.
Until now, this journal was known as the Thai TESOL
Bullet...
knowledge to be found both in the pages that follow and in subsequent issues of
the journal. We are grateful for these con...
Join us at
34th
Annual Thailand TESOL
International Conference
January 17-18, 2014
The Empress Hotel, Chiang Mai, Thailand...
THE AUTHORS
Mary Sarawit has been involved in English language teaching in Thailand since
her role as a US Peace Corps Vol...
AN ANALYSIS OF READING MATERIALS USING COH-METRIX
IN CONJUNCTION WITH STUDENTS’ READING
COMPREHENSION SCORES
Mary Sarawit
...
THAITESOL JOURNAL Vol.26, No.1 June 20132
reading comprehension scores and lexical diversity of content words in the
passa...
An Analysis of Reading Materials Using Coh-Metrix in Conjunction
with Students’ Reading Comprehension Scores
MARY SARAWIT
...
THAITESOL JOURNAL Vol.26, No.1 June 20134
Rose level and study which of the features I used related to the student‟s readi...
An Analysis of Reading Materials Using Coh-Metrix in Conjunction
with Students’ Reading Comprehension Scores
MARY SARAWIT
...
THAITESOL JOURNAL Vol.26, No.1 June 20136
Lexical diversity. This is the diversity of words in the passage as
measured by ...
An Analysis of Reading Materials Using Coh-Metrix in Conjunction
with Students’ Reading Comprehension Scores
MARY SARAWIT
...
THAITESOL JOURNAL Vol.26, No.1 June 20138
readings are appropriate for adults and culturally diverse. “SRA has not
confine...
An Analysis of Reading Materials Using Coh-Metrix in Conjunction
with Students’ Reading Comprehension Scores
MARY SARAWIT
...
THAITESOL JOURNAL Vol.26, No.1 June 201310
Table 1 displays the findings for the 5 lexical characteristics of the 11
Rose ...
An Analysis of Reading Materials Using Coh-Metrix in Conjunction
with Students’ Reading Comprehension Scores
MARY SARAWIT
...
THAITESOL JOURNAL Vol.26, No.1 June 201312
39.852%. The mean number of modifiers per noun phrase was only 0.819 with a
coe...
An Analysis of Reading Materials Using Coh-Metrix in Conjunction
with Students’ Reading Comprehension Scores
MARY SARAWIT
...
THAITESOL JOURNAL Vol.26, No.1 June 201314
over frequency level itself was 2,517.574 for all 11 readings from a high of
3,...
An Analysis of Reading Materials Using Coh-Metrix in Conjunction
with Students’ Reading Comprehension Scores
MARY SARAWIT
...
THAITESOL JOURNAL Vol.26, No.1 June 201316
the readings more difficult because the connective supplies the temporal meanin...
An Analysis of Reading Materials Using Coh-Metrix in Conjunction
with Students’ Reading Comprehension Scores
MARY SARAWIT
...
THAITESOL JOURNAL Vol.26, No.1 June 201318
analyze the texts. This study reconfirms the relationship between connectives, ...
An Analysis of Reading Materials Using Coh-Metrix in Conjunction
with Students’ Reading Comprehension Scores
MARY SARAWIT
...
THAITESOL JOURNAL Vol.26, No.1 June 201320
References
Beck, I. L., McKeown, M. G. & Kueau, L. (2002). Bringing Words to Li...
An Analysis of Reading Materials Using Coh-Metrix in Conjunction
with Students’ Reading Comprehension Scores
MARY SARAWIT
...
THAITESOL JOURNAL Vol.26, No.1 June 201322
Appendix A
Data from the Coh-Metrix Analysis of the 11 Rose Level of the SRA 3B...
An Analysis of Reading Materials Using Coh-Metrix in Conjunction
with Students’ Reading Comprehension Scores
MARY SARAWIT
...
THAITESOL JOURNAL Vol.26, No.1 June 201324
Appendix C
Data from the Coh-Metrix Analysis of the 11 Rose Level of the SRA 3B...
EFFECTS OF ONLINE TASK-BASED INTERACTIVE LISTENING
INSTRUCTION FOR EFL LEARNERS
Xingbin Tian
Suranaree University of Techn...
THAITESOL JOURNAL Vol.26, No.1 June 201326
Introduction
The importance of listening for language acquisition has been
emph...
Effects of Online Task-Based Interactive Listening Instruction for EFL Learners
XINGBIN TIAN, SUKSAN SUPPASETSEREE
27
buil...
THAITESOL JOURNAL Vol.26, No.1 June 201328
groupwere higher than those of the control classes. The students‟ attitude towa...
Effects of Online Task-Based Interactive Listening Instruction for EFL Learners
XINGBIN TIAN, SUKSAN SUPPASETSEREE
29
Figu...
THAITESOL JOURNAL Vol.26, No.1 June 201330
1) Are there any significant differences in listening comprehension
between the...
Effects of Online Task-Based Interactive Listening Instruction for EFL Learners
XINGBIN TIAN, SUKSAN SUPPASETSEREE
31
Test...
THAITESOL JOURNAL Vol.26, No.1 June 201332
interviews were recorded and transcribed for data analysis. Chinese language, t...
Effects of Online Task-Based Interactive Listening Instruction for EFL Learners
XINGBIN TIAN, SUKSAN SUPPASETSEREE
33
As c...
THAITESOL JOURNAL Vol.26, No.1 June 201334
the students in the experimental class, instructed with OTIL, had made more
dem...
Effects of Online Task-Based Interactive Listening Instruction for EFL Learners
XINGBIN TIAN, SUKSAN SUPPASETSEREE
35
Tabl...
THAITESOL JOURNAL Vol.26, No.1 June 201336
Statements N SD
10. Online task-based interactive listening can provide
easy ac...
Effects of Online Task-Based Interactive Listening Instruction for EFL Learners
XINGBIN TIAN, SUKSAN SUPPASETSEREE
37
Resu...
THAITESOL JOURNAL Vol.26, No.1 June 201338
The present study was conducted to meet the current trends in the
teaching of E...
Effects of Online Task-Based Interactive Listening Instruction for EFL Learners
XINGBIN TIAN, SUKSAN SUPPASETSEREE
39
Four...
THAITESOL JOURNAL Vol.26, No.1 June 201340
forum interactions. Except for the topics, a Moodle library was constructed to
...
Effects of Online Task-Based Interactive Listening Instruction for EFL Learners
XINGBIN TIAN, SUKSAN SUPPASETSEREE
41
stud...
THAITESOL JOURNAL Vol.26, No.1 June 201342
satisfied with the BOLA packages which could promote autonomous learning
effect...
Effects of Online Task-Based Interactive Listening Instruction for EFL Learners
XINGBIN TIAN, SUKSAN SUPPASETSEREE
43
Refe...
THAITESOL JOURNAL Vol.26, No.1 June 201344
MOE. (2007). College English Curriculum Requirements. Beijing: Foreign
Language...
Effects of Online Task-Based Interactive Listening Instruction for EFL Learners
XINGBIN TIAN, SUKSAN SUPPASETSEREE
45
Yao,...
EXPLORING BELIEFS OF PRE-SERVICE TEACHERS TOWARD
ENGLISH AS AN INTERNATIONAL LANGUAGE
Flora Debora Floris
Petra Christian ...
Exploring Beliefs of Pre-Service Teachers toward English as an International Language
FLORA DEBORA FLORIS
47
Introduction
...
THAITESOL JOURNAL Vol.26, No.1 June 201348
mother tongue that they have learned. As there is an increase in the number of
...
Exploring Beliefs of Pre-Service Teachers toward English as an International Language
FLORA DEBORA FLORIS
49
the EIL princ...
THAITESOL JOURNAL Vol.26, No.1 June 201350
traditional EFL classrooms since they were in elementary level. As the concept
...
Exploring Beliefs of Pre-Service Teachers toward English as an International Language
FLORA DEBORA FLORIS
51
practices, an...
THAITESOL JOURNAL Vol.26, No.1 June 201352
motivation constructs and to pre-service teachers‟ academic performances and
ac...
Exploring Beliefs of Pre-Service Teachers toward English as an International Language
FLORA DEBORA FLORIS
53
Having read a...
THAITESOL JOURNAL Vol.26, No.1 June 201354
Classroom discussions and reflective papers of the students were used to collec...
Exploring Beliefs of Pre-Service Teachers toward English as an International Language
FLORA DEBORA FLORIS
55
English thoug...
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Thai tesol journal vol.26 no.1 june 2013
Upcoming SlideShare
Loading in...5
×

Thai tesol journal vol.26 no.1 june 2013

415

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
415
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Thai tesol journal vol.26 no.1 june 2013"

  1. 1. ISBN2286-8909 Volume26Number1June2013
  2. 2. Thailand TESOL Organization under the Patronage of Her Royal Highness Princess Galyani Vadhana Krom Luang Naradhiwas Rajanagarindra THAILAND TESOL ORGANIZATION is a professional non-profit and non- political organization under the patronage of H.R.H. Princess Galyani Vadhana Krom Luang Naradhiwas Rajanagarindra. Its purposes are to strengthen English Language education at all levels, to undertake research in the teaching of English to speakers of other languages, to offer scholarships, to disseminate information, and to cooperate in appropriate ways with other groups having similar concerns. THAILAND TESOL ORGANIZATION is an affiliate of TESOL, Inc. located in the United States of America; the International Association of Teachers of English as a Foreign Language (IATEFL), located in Great Britain; JALT, located in Japan; Korea TESOL; STETS, located in Singapore; ETAROC, located in Taiwan; and TESL Canada. THAITESOL JOURNAL (ISSN 2286-8909) is published 2 times a year, in January and May by Thailand TESOL Organization under the Patronage of H.R.H. Princess Galyani Vadhana Krom Luang Naradhiwas Rajanagarindra. It is intended to provide a forum for research and the exchange of information including opinion on theory and practice in the teaching of English to speakers of other languages. PHOTOCOPYING Material in THAITESOL JOURNAL may be photocopied for educational purposes. Under no circumstances may any part of this bulletin be photocopied for commercial purposes. Printed in Nakhon Ratchasima, Thailand
  3. 3. THAILAND TESOL EXECUTIVE COMMITTEE March 2011 – May 2013 Advisory Board Arunee Wiriyachitra Independent Consultant Suntana Sutadarat Ramkhamhaeng University Naraporn Chan-Ocha Distance Learning Foundation Sonthida Keyuravong King Mongkut University Thonburi Suchada Nimmannit Chulalongkorn University Chaleosri Pibulchol Distance Learning Foundation Maneepen Apibalsri Rungsit University Damon Anderson RELO, US Embassy Samatha Grainger British Council Scott Evans Australian Embassy Paul Humphries AUA Language Center Wattanaporn Rangabtook Ministry of Education Past President Akara Akaranithi Chulalongkorn University Immediate Past President Ubon Sanpatchayapong Rangsit University President Nopporn Sarobol Thammasat University First Vice President Unchalee Sermsongswad Chiang Mai University Second Vice President Anamai Damnet Kasetsart University Secretary Edward B. Geerson Thammasat University Assistant secretary Nipaporn Chalermnirundorn Rangsit University Treasurer Kittitouch Soontornwipast Thammasat University Webmaster Sarapol Chirasawadi Suan Dusit Rajabhat University Public Relations Monthon Kanokpermpoon Thammasat University Wacharapong Kributr Thai Airways International Publication Andrew Lian Suranaree University of Technology Pannathon Sangarun Suranaree University of Technology Membership Coordinator Jiraporn Kakaew Kasetsart University Member-at-large Chantharat Hongboontri Mahidol University Chirasiri Kasemsin Mahidol University Paneeta Nitayaphorn Thai Airways International
  4. 4. SIG Chairs ESP Panna Chaturongakul Thammasat University Research Kanjana Chattrakul Suan Dusit Rajabhat University CALL Supanit Kulsiri Srinakarinwirot University STED Chanarong Rachbuanoy Udon Thani Educational Service Area Office 4 Literature Rajeev Ramnath Assumption University IELLT Anamai Damnet Kasetsart University ICT Sarapol Chirasawadi Suan Dusit Rajabhat University SAL Nopporn Sarobol Thammasat University Regional Affiliates Northern Affiliate Wichian Sunitham Chiang Mai University Northeastern Affiliate Uthaivan Danvivath Khon Kaen University Krauwan Thongwundee Roi-Et Wittayalai School Southern Affiliate Pragasit Sitthitikul Thammasat University Pareedar Pakawan Naradhiwas University Western Affiliate Singhanat Nomnian Mahidol University
  5. 5. THAITESOL JOURNAL Volume 26 Number 1 Published by Thailand TESOL Organization under the Patronage of Her Royal Highness Princess Galyani Vadhana Krom Luang Naradhiwas Rajanagarindra EDITORS Editor Pannathon Saengarun Suranaree University of Technology Thailand Managing Editor Maneepen Apibalsri Rungsit University Thailand CONTENTS ARTICLES An Analysis of Reading Materials Using Coh-Metrix in Conjunction with Students’ Reading Comprehension Scores MARY SARAWIT Effects of Online Task-Based Interactive Listening Instruction for EFL Learners XINGBIN TIAN SUKSAN SUPPASETSEREE Exploring Beliefs of Pre-Service Teachers toward English as an International Language FLORA DEBORA FLORIS 1 25 46
  6. 6. EDITORS Editor Pannathon Saengarun Suranaree University of Technology Thailand Managing Editor Maneepen Apibalsri Thailand EDITORIAL REVIEW BOARD Akara Akaranithi Chulalongkorn University Thailand Anamai Damnat Kasetsart University Thailand Andrew Lian Suranaree University of Technology Thailand Apisak Pupipat Thammasart University Thailand Arunee Wiriyachitra Independent Consultant Thailand Chaleosri Pibulchol Distance Learning Foundation Thailand Chantharat Hongboontri Mahidol University Thailand Chirasiri Kasemsin Mahidol University Thailand Debra Hoven Athabasca University Canada Harald Kraus Dhurakitbandit University Thailand Kanjana Chattrakul Suan Dusit Rajabhat University Thailand Kittitouch Soontornwipast Thammasat University Thailand Leslie Burke Barratt Indiana State University U.S.A Maneepen Apibalsri Rungsit University Thailand
  7. 7. Nattaya Puekpong Suranaree University of Technology Thailand Nopporn Sarobol Thammasart University Thailand Panna Chaturongakul Thammasat University Thailand Pannathon Sangarun Suranaree University of Technology Thailand Pragasit Sitthitikul Thammasart University Thailand Rajeev Ramnah Assumption University Thailand Ruja Polsawad Rangsit University Thailand Singhanat Nomnian Mahidol University Thailand Somsak Boonsathorn Maeh Fah Luang University Thailand Sonthida Keyurawong King Mongkut University Thailand Suchada Nimmannit Chulalongkorn University Thailand Supanit Kulsiri Srinakarinwirot University Thailand Supatra Thongkalaya Srinakarinwirot University Thailand Unchalee Sermsongswad Chiang Mai University Thailand Ubon Sanpatchayapong Rangsit University Thailand Uthaivan Danvivath Khon Kaen University Thailand Wichian Sunitham Chiang Mai University Thailand Assistant Editor Rajeev Ramnah Assumption University Thailand
  8. 8. Editorial Welcome to the first issue of the Thai TESOL Journal. Until now, this journal was known as the Thai TESOL Bulletin. However, we felt it was important to rename it so as to avoid confusion with other forms of publication and to reflect clearly its status as a peer-reviewed, refereed, journal of international standing. The learning (and therefore teaching) of English is at a critically important point in its history, especially in Asian contexts. In particular, the inauguration of the AEC (ASEAN Economic Community) in 2015 as part of ASEAN’s development plans will require the use of English as the common language and will place increasing pressure on English language learning resources (as well as on resources for the learning and teaching of other languages). This is especially true in light of the possible development of a Regional Economic Partnership which would include Australia and New Zealand. These developments, which is are approximately two years away, will significantly strain the already overloaded language-learning and teaching systems in the region. As a result, it is almost inevitable that the demand for English language-learning support, in both formal and informal settings, will outstrip the resources currently available. Dealing with this situation will require substantial increases in the efficiency and effectiveness of English language learning and teaching in order to meet regional needs. Thailand, as a member nation of the AEC and as an Asian country where English is not an official language, has a critically important role to play both within its borders and beyond. It is therefore very pleasing to be able to report that the Thai TESOL Journal extends the tradition of quality established by the Thai TESOL Bulletin and continues to receive a flow of high quality submissions from both Thai and International scholars. These contributions will help refine our theoretical and practical understandings of the language learning process thus helping to alleviate some of the problems which lie ahead. We hope that you will both enjoy and benefit professionally from the new and relevant contributions to
  9. 9. knowledge to be found both in the pages that follow and in subsequent issues of the journal. We are grateful for these contributions and very much look forward to receiving yours in the near future. Pannathon Sangarun Editor
  10. 10. Join us at 34th Annual Thailand TESOL International Conference January 17-18, 2014 The Empress Hotel, Chiang Mai, Thailand www.thaitesol.org
  11. 11. THE AUTHORS Mary Sarawit has been involved in English language teaching in Thailand since her role as a US Peace Corps Volunteer in Thailand from 1967 to 1969. After completing her studies for a Ph.D. in Linguistics at the University of Michigan, she returned to teach in Phitsanulok, Thailand. She was Assistant to the Dean of the Humanities Faculty, Deputy Director of the Naresuan University International College, and is presently Distinguished Specialist in Language and Advisor for International Student Activities in the Office of the President at Naresuan University. Xingbin Tian received his Bachelor’s Degree in English from Guizhou University, China in 1982, and Master’s Degree in history from Shandong University, China in 2008. He has been teaching English at Tongren University, China since 1982. His research interests include technology enhanced language learning, instructional systems design and language learning strategies. Suksan Suppasetseree is currently a lecturer in English and the supervisor of the Foreign Languages Resource Unit (FLRU), the self-access language learning center, at Suranaree University of Technology, Nakhon Ratchasima, Thailand. He is now a MA and PhD supervisor in his institute. His research interests include technology enhanced language learning, e-learning and instructional systems design. Flora Debora Floris completed her Master’s Degree in English Language Teaching at the Institute for English Language Education, Assumption University Thailand as a scholar of the United Board for Christian Higher Education in Asia (UBCHEA). She is currently a teacher in the English Department of Petra Christian University, Surabaya where she teaches English for Education Business subjects. Her areas of interest include issues of English as an International Language and teachers' professional development.
  12. 12. AN ANALYSIS OF READING MATERIALS USING COH-METRIX IN CONJUNCTION WITH STUDENTS’ READING COMPREHENSION SCORES Mary Sarawit Naresuan University, Phitsanulok, Thailand marys@nu.ac.th Abstract The purposes of the study were (1) to analyze the SRA Lab 3B Rose Level using Coh-Metrix indices at a lexical level (reading ease, reading grade level, frequency of content words, concreteness of words, and lexical diversity), syntactic level (words before the main verb, pronoun ratio to noun phrases, personal pronoun incidence, number of modifiers before the noun phrase, and words in a sentence), and semantic level (additive, temporal, causal, logical connectives, and latent semantic analysis) and (2) to identify relationships between the students‟ comprehension scores on the 11 Rose readings and the 15 characteristics of the readings. The subjects were Naresuan University International College students. They read all 11 Rose readings and completed the questions that follow the readings. This study reconfirms the relationship between connectives and reading ease. The higher incidence of causal and logical connectives related to a higher Flesch ease score. Also, a higher Flesch grade correlated with a lower word frequency level. The Flesch grade level correlated with the number of modifiers before the noun phrases for which the number of words before the verb also correlated. Correlations between the students‟ comprehension scores and the 15 text indices showed correlations with temporal and causal connectives. This study also found relationships between student
  13. 13. THAITESOL JOURNAL Vol.26, No.1 June 20132 reading comprehension scores and lexical diversity of content words in the passages. Keywords: SRA Reading Lab 3B, Coh-Metrix, reading comprehension, reluctant reader Introduction In a previous study on students‟ attitudes towards reading (Sarawit, 2009), I found that my students fit the definition of the reluctant reader: reading when necessary but not as a hobby or leisure time activity even though they consider reading in English enjoyable and useful. Teachers of English take great efforts to get their students to read more efficiently by building their students‟ vocabulary and reading skills, such as having students identify main ideas and details, identify the tone of a passage and the author‟s opinions, and make inferences. We use scaffolding, such as graded readers to build student confidence in reading. While graded readers do seem to offer a bridge to full texts, they often miss their mark. A lower level reading is found to be more difficult than a higher graded reading. In part, the background knowledge of the reader comes into play. Recent research has also shown that traditional readability scales that use number of syllables, words, and sentences only reveal surface components of language and that a better measure of readability needs to include semantic and syntactic features of a text. Researchers at the University of Memphis have over the past decade developed new tools to better assess readability that go beyond the surface structure of a text to include deeper language structures (McNamara et al., 2010).
  14. 14. An Analysis of Reading Materials Using Coh-Metrix in Conjunction with Students’ Reading Comprehension Scores MARY SARAWIT 3 Comatrix as a Reading Analysis Program Coh-Metrix is an online computerized analysis program that produces 60 indices which describe not only the traditional syllable, word, and sentence count readability scales but also give deeper linguistic and discourse representations of the text (Schmitt, 2010, 215). Coh-Metrix was developed and is maintained by the Department of Psychology at the University of Memphis (McNamara et al., 2002). The program has been used extensively over the years to validate the use of syntactic and semantic components to analyze readability and text comprehension. A study by McNamara et al. (2011) found that five components related well with text difficulty: narrativity, syntactic simplicity, word concreteness, referential cohesion, and deep cohesion. I have for many years used the Science Research Associates, otherwise known as the SRA Reading Lab as a supplement to the two required freshman English courses. Students expand their vocabulary and develop their reading skills. They are required to read one story a week. This is only about two A4 pages and accompanied by 7 to 10 reading comprehension questions and vocabulary development exercises. At the end of the freshman year, I encourage students to continue the reading program. However, after freshman year few students come to me to continue the program. The students who do continue are the best students in the group. I have noticed that their progression from one level (each of which is color coded) to another is often not linear. That is, they move from one color to another, but often some of the readings in the new color are extremely difficult for them while others at the same color level are quite easy for them. Having become familiar with Coh-Metrix and its capacity to analyze not only surface level features of a text but deeper semantic and syntactic features, I decided to conduct an analysis of the SRA readings at the
  15. 15. THAITESOL JOURNAL Vol.26, No.1 June 20134 Rose level and study which of the features I used related to the student‟s reading comprehension scores. Identifying what makes a passage more difficult can help the teacher to help the student develop the skills the student needs to be a successful reader, on the assumption that a successful reader will read more which ultimately may lead to him/her becoming a lifelong reader who will constantly expand his/her vocabulary and viewpoint. With this in mind, I decided to use Coh-Metrix as follows: Research objectives The purposes of the study were 1. To conduct a text analysis of Reading Level 5, or „Rose‟ level, Science Research Associates (SRA) reading materials using Coh-Metrix. 2. To study the relationship between the features of the texts and the students‟ reading comprehension scores for the SRA reading materials. Research Questions Two questions were addressed in the study. 1. What does a text analysis of SRA reading materials reveal as regards reading ease, frequency of content words, concreteness of words, words before the main verb, pronoun ratio to noun phrases, personal pronoun incidence, number of modifiers before the noun phrase, words in a sentence, connectives, and LSA (Latent Semantic Analysis) cohesion? 2. Which factors in the text analysis are significantly related to the students‟ reading comprehension scores? Importance of the Study Teachers are constantly trying to get their students to read more. The more students read, the better readers they will become. By identifying textual
  16. 16. An Analysis of Reading Materials Using Coh-Metrix in Conjunction with Students’ Reading Comprehension Scores MARY SARAWIT 5 features which students find difficult, and subsequently aiming to improve comprehension skills by focusing on these difficulties, it is hoped that students will improve their vocabulary and reading skills. As such, this study aims to identify a number of text features, from overall basic ease of reading to deeper characteristics of the text. More importantly, the study aims to identify which features of the text are related to the students‟ comprehension scores. Definition of Terms Text. The texts were readings from the SRA Reading Laboratory 3B. There were 11 readings at the Rose level. Coh-Metrix. This is an on-line program created and maintained by the University of Memphis that calculates the readability and coherence of a text using a wide range of measures. Reading ease. Reading ease refers to the difficulty of a text. In this study two measures were used to determine level of difficulty: Flesch Reading Ease and Flesch-Kincaid Grade Level. Both these measures rely on the number of syllables per word and the number of words per sentence. Flesch Reading Ease formula. The Flesch Reading Ease formula is presented as a number from 0-100 with a higher score indicating an easier reading. The Flesch-Kincaid Grade Level is represented as a grade level from 0- 12. Word frequency. Word frequency is the mean raw frequency of Nouns, Verbs, Adjectives, and Adverbs in the text based on the COBUILD word corpus. Concreteness of words. This is a measure of how concrete the words in the passage are. Values vary from 100-700 with a higher score indicating a more concrete word.
  17. 17. THAITESOL JOURNAL Vol.26, No.1 June 20136 Lexical diversity. This is the diversity of words in the passage as measured by type-token ratio (TTR). A higher score reflects a greater diversity of words (content words) in a passage. Words before the main verb. This is a measure of the mean score of the number of words before the main verb in the main clause. Pronoun ratio. This is the proportion of pronouns to noun phrases. Personal pronoun incidence. This is the number of personal pronouns in the reading. Modifiers before the noun phrase. This is the mean number of modifiers per noun phrase. Number of words per sentence. This is the average number of words per sentence in the passage. Connectives. These are words that signal specific relationships. Four types of connectives will be analyzed: Additive (e.g., also, moreover, but, however), Causal (e.g., because, so, consequently, although, nevertheless), Logical (e.g., or, actually, if), and Temporal (e.g., after, before, when, until). The incidence of the four types of connectives in the texts is presented. Latent Semantic Analysis of all sentence combinations. LSA is a statistical technique “for representing world knowledge” (Coh-Metrix, 2005, p.16). The mean LSA cosines are presented for all sentence combinations in the text. This measures “how conceptually similar each sentence is to every other sentence in the text” (Coh-Metrix, 2005, p.16). Students. The students were students at the Naresuan University International College. Comprehension scores. These were the scores in terms of percentage correct on the comprehension tests at the end of the SRA readings.
  18. 18. An Analysis of Reading Materials Using Coh-Metrix in Conjunction with Students’ Reading Comprehension Scores MARY SARAWIT 7 Method Participants Naresuan University International College freshman students who had received an A or B+ in Foundations of English and second year English for Business majors were asked to read the SRA readings and complete the comprehension questions at the end of the readings. The researcher chose these students because the students at the International College use English as the medium of instruction for all their courses, and thus it was hoped that they would be motivated to improve their reading abilities. In addition, the researcher teaches at the college and was able to easily follow-up on the students‟ progress. Task From previous research (Sarawit, 2009), the researcher knew that students enjoyed the SRA readings. The SRA Reading Laboratory 3B was used and the students were asked to read one reading a week from the Rose level. There are 11 readings. The researcher corrected the comprehension questions and kept a reading chart for each student. For the purposes of this research, students were asked to complete all 11 readings in Rose. Material Eleven readings from the SRA Reading Laboratory 3B , the Rose level, were used. The SRA reading labs are an integral part of remedial and self-access programs worldwide. The iconic color coded reading labs first appeared in schools in the United States in the late 1950s. Today, as part of McGraw-Hill publishing, they are used worldwide in reading development programs. The 3B SRA Reading Lab has readings that cover general topics in the sciences and social sciences covering 9 levels from Rose (grade 5) to Purple (grade 12). The
  19. 19. THAITESOL JOURNAL Vol.26, No.1 June 20138 readings are appropriate for adults and culturally diverse. “SRA has not confined itself to an exclusively Western view of culture. SRA has ranged more widely over non-Western sources, drawing on the traditions of Asia, Eastern Europe, Africa, and Central America” (Parker, 2000, p.6). Data Analysis In using Coh-Metrix, “a minimum of 10 cases of data for each predictor is considered to be accurate” (Crossley et al., 2008, p.482). The researcher typed one A4 page from each reading and entered it into the Coh-Metrix program which analyzed the text. To successfully calculate the Flesch Reading Ease and Flesch-Kincaid Grade Level scores a passage should have more than 200 words (Coh-Metrix). The 11 readings for the Rose level which were entered into the Coh-Metrix program ranged from 442-655 words with a mean of 530 words per passage. Indicators for reading ease included both the Flesch Reading Ease and the Felsch-Kincaid Grade Level. In addition, the other following Coh-Metrix indicators were used for text analysis: frequency of content words, concreteness of words, lexical diversity using type-token ratio, words before the main verb, ratio of pronouns to noun phrases, incidence of personal pronouns, number of modifiers per noun phrase, number of words per sentence, connectives, i.e., additive connectors, causal connectors, logical connectors, and temporal connectors as well as conceptual similarity of sentences in the passage. Frequency, concreteness of words, and lexical diversity focus on the integral part played by vocabulary in reading; words before the main verb, pronoun ratio, incidence of personal pronouns, number of modifiers before the noun phrase, and number of words per sentence relate to syntactic properties of the sentence while connectives and conceptual similarity of all the sentences in the text focus on the
  20. 20. An Analysis of Reading Materials Using Coh-Metrix in Conjunction with Students’ Reading Comprehension Scores MARY SARAWIT 9 semantic relationships relating to cohesion in the readings. These 15 indicators were chosen from the 62 indicators calculated by the on-line Coh-Metrix program for this study. The data for the various indicators were recorded for all eleven Rose level SRA readings and then tallied. Mean, standard deviation, coefficient of variability, and analysis of correlations were calculated for each of the 15 characteristics of the 11 texts under study. The descriptive data can be seen in Appendix A. An analysis of correlations was also conducted to find out which, if any, of the indicators related to the students‟ comprehension scores. Results Part 1 An analysis of the SRA 3B Laboratory (Rose) was conducted using Coh- Metrix, a computational on-line program for analyzing text. Fifteen indices were used in this study: 5 for studying lexical characteristics of the texts, 5 for studying syntactic characteristics, and 5 for studying semantic characteristics. The lexical characteristics involved the Flesch Reading Ease and the Flesch-Kincaid Grade Level both of which use the number of syllables in a word and the number of words in a sentence for calculations. Word frequency, word concreteness, and lexical diversity were also investigated. Coh-Metrix uses frequencies taken from the 1991 version of the COBUILD 17.9 million-word corpus. Word concreteness is measured using the MRC Psycholinguistics Database (Coh-Metrix citing Coltheart, 1981). Lexical diversity was calculated using the type (word) divided by token (instances). Since the passages were of the same approximate length, TTR was considered an appropriate measure of lexical diversity.
  21. 21. THAITESOL JOURNAL Vol.26, No.1 June 201310 Table 1 displays the findings for the 5 lexical characteristics of the 11 Rose SRA readings. Details for the 11 readings can be found in the Appendix A. Table 1 Lexical indices Flesch Reading Ease Flesch-Kincaid Grade Level Word Frequency Word Concreteness Lexical Diversity x SD CV 78.334 5.554 7.090 5.480 0.712 12.993 2517.574 862.207 34.248 407.099 26.122 6.417 0.705 0.050 7.092 The Flesch Reading Ease mean score was 78.334 on a scale of 0-100 with the coefficient of variance only 7.090%; however, the Flesch-Kincaid Grade Level of a 5.48 mean for the 11 readings while close to the SRA Rose grade of 5 showed a coefficient of variance of 12.003% with a Flesch-Kincaid Grade Level of 6.503 for Rose reading 7 and 4.148 for Rose reading 11. The frequency of the content words showed a mean of 2,517.574 where the word with the lowest frequency would have a value of 0 and the word with highest frequency would have a value of 1,000,000. The mean word frequency for the 11 Rose readings had a coefficient of variance of 34.248%. The mean value for concreteness of the words in the readings was 407.099 with a rather low coefficient of variance of 6.417%. The higher the value the more concrete the word tends to be on a scale of 100 to 700. The average lexical diversity score was 0.705 with a range of 0.638 to 0.775 where the coefficient of variance was only 7.092 %. The closer the score gets to 1, the more unique the words in the passage are, and hence the more difficult for the reader to process.
  22. 22. An Analysis of Reading Materials Using Coh-Metrix in Conjunction with Students’ Reading Comprehension Scores MARY SARAWIT 11 The indices used to analyze the syntactic characteristics of the 11 Rose SRA readings included the number of words before the verb. The greater the distance from the subject to the main verb, the more difficult the sentence is to comprehend. Pronoun ratio to noun phrases and the number of personal pronouns were also considered since pronouns require the reader to identify the noun referred to. The number of modifiers per noun phrase was also studied along with the number of words per sentence which tend to indicate more complex sentences or as Crossley et al. (2007, p.199) state: “longer sentences are a rough estimate of the number of propositions contained.” Table 2 displays the findings for the 5 syntactic characteristics of the 11 Rose SRA readings. Details for the 11 readings can be found in the Appendix B. Table 2 Syntactic indices Words before Verb Pronoun Ratio to Noun Phrases No. of Personal Pronouns No. of Modifiers per Noun Phrase No. of Words per Sentence x SD CV 3.599 1.063 29.536 0.301 0.113 37.541 87.723 34.959 39.852 0.819 0.154 18.803 12.515 1.785 14.263 The mean for words before the main verb was 3.599 words with a coefficient of variance of 29.536%. The mean ratio of pronouns to noun phrases was 0.301 with a coefficient of variance of 37.541%. The mean number of personal pronouns in the selection was 87.723 with a coefficient of variance of
  23. 23. THAITESOL JOURNAL Vol.26, No.1 June 201312 39.852%. The mean number of modifiers per noun phrase was only 0.819 with a coefficient of variance of 18.803 %; and the mean number of words per sentence was 12.515 with a coefficient of variance of 13.263%. The indices used to analyze the semantic characteristics of the 11 Rose SRA readings included the incidence of four types of connectives: additive, causal, logical, and temporal. The use of connectives is related to cohesion of the text as they indicate the semantic relationships in the sentences and paragraphs. The fifth semantic characteristic of the text used was the value for the conceptual similarity among the sentences in the passage using Latent Semantic Analysis presented as cosine scores where the higher the score the more similar the meanings (Dumais, 2005, p.188). Here again, conceptual similarity adds to the cohesion of the passage. Table 3 displays the findings for the 5 semantic characteristics of the 11 Rose SRA readings. Details for the 11 readings can be found in the Appendix C. Table 3 Semantic indices Additive Connectives Causal Connectives Logical Connectives Temporal Connectives LSA Sentence Combinations x SD CV 36.429 9.435 25.900 21.512 7.551 35.101 29.640 8.902 30.034 13.726 4.253 30.985 0.184 0.066 35.870 The mean incidence of connectors in the 11 Rose SRA readings from highest to lowest incidence is as follows: Additive ( x 36.429), Logical ( x 29.640), Causal ( x 21.512), and Temporal ( x 13.726). The coefficient of
  24. 24. An Analysis of Reading Materials Using Coh-Metrix in Conjunction with Students’ Reading Comprehension Scores MARY SARAWIT 13 variance was at least 25% for all 4 connectors: 25.900%, 30.034%, 35.101%, and 30.985%, respectively. The mean cosine for the LSA of conceptual similarity for all sentence combinations in the passages was 0.184 for which the coefficient of variance was 35.870%. Values close to 1 indicate very similar words in the sentences while values close to 0 indicate very dissimilar words which should hinder the reader. An analysis of correlations between the 15 indices showed 12 significant correlations. There were significant negative correlations between the Flesch Reading Ease level and the Flesch-Kincaid Grade level (-.805 at the 0.01 level of significance). This is in line with the fact that the Flesch Reading Ease level means the higher the score the easier the reading while the Flesch-Kincaid Grade Level relates to school grade 0 through 12. Both formulas are based on the average sentence length and the average number of syllables per word. The Flesch Reading Ease level also related positively with the causal and logical connectors (.822 and .848 respectively at the 0.01 level of significance). The higher occurrence of the causal connectors (because, so, although) and logical connectives (and, or, if) should make reading easier since the semantic relations in the sentence are explicitly presented. As McNamara et al. (2011, p.8) note in their discussion of deep cohesion: “…connectives help the reader to form a more coherent and deeper understanding of the causal events, processes, and actions in the text.” The Flesch-Kincaid Grade Level was correlated with the word frequency level (-.709 at the 0.05 level of significance). The higher the grade level, the lower the word frequency. The Rose level is set at grade 5; however, as analyzed by the Flesch-Kincaid formula used by Coh-Metrix the 11 readings varied from a high of 6.503 to a low of 4.158 with a mean of 5.4804. The all
  25. 25. THAITESOL JOURNAL Vol.26, No.1 June 201314 over frequency level itself was 2,517.574 for all 11 readings from a high of 3,696.116 to a low of 1,316.016. It is generally agreed that automatic decoding of words is a strong predictor of reading performance, and as Crossley et al. (2008, 483) note “Texts which assist such decoding (e.g., by containing a greater proportion of high frequency words) can thus be regarded as easier to process.” The Flesch-Kincaid Grade Level also correlated with the number of modifiers before the noun phrases (.646 at the 0.05 level of significance). The more modifiers before the noun phrases makes it more difficult for students to parse the sentence often moving the main verb farther away from the beginning of the sentence. For example, instead of Birds flew overhead we might have Three large black and white elegant duck-billed birds flew overhead. Word frequency was negatively related to word concreteness (-.881 at the 0.01 level of significance). This is unexpected. The more concrete words would tend to be of higher frequency. The small pool of 11 readings may askew the results here. For example, one story was about cowboys and included many words related to the equipment they used. Another story was about the Venus fly trap and included specific words for parts of the flower. While these words were concrete, they would not be extremely high in frequency. Words before the verb correlated with both the number of modifiers before the noun phrases and the number of words in the sentence (.689 and .729 respectively at the 0.05 level of significance). More elaboration of the noun phrase in the subject position would result in more words before the verb which also means longer sentences or a greater number of words per sentence. The pronoun ratio to nouns was, as expected, related to the number of personal pronouns (.989 at the 0.01 level of significance) and negatively related to the number of modifiers before the noun phrases (-.653 at the 0.05 level of
  26. 26. An Analysis of Reading Materials Using Coh-Metrix in Conjunction with Students’ Reading Comprehension Scores MARY SARAWIT 15 significance). Sentences with long modifiers before the noun phrases might tend to exclude pronoun usage in favor of the nouns. Likewise, the total number of personal pronouns was negatively related to the number of modifiers before the noun phrases (.-698 at the 0.05 level of significance). Lastly, there was a significant correlation between the incidence of causal and logical connectives (.780 at the 0.01 level of significance). Both types of connectors often occur in the same types of discourse; such as cause and result, reasons, and other opinion paragraphs. Part 2 The second objective of this research was to examine any relationships between the 15 features of the eleven Rose level texts and the students‟ reading comprehension scores. The twenty one students read all eleven Rose SRA readings and completed the comprehension questions that followed. The average comprehension score on the 11 readings was a low of 49.660 percent (median of 42.857) for reading 3 and a high of 77.550 percent (median of 71.429) on reading 7. With regard to the students, the highest mean score for all eleven readings was 81.818 percent (median of 85.714) and the lowest mean score was 48.052 percent (median of 42.857). The minimum individual score was 0 for readings 3 and 9, and the maximum individual score was 100 percent for all but readings 3 and 5. Correlation coefficient analysis between reading indicators and reading comprehension scores using median, mean, standard deviation, minimum, and maximum as representative of the reading score revealed four significant correlations. The median reading score correlated significantly but negatively with the incidence of temporal connectives (-.710 at the 0.05 level of significance). This is unexpected. While lack of connectives should have made
  27. 27. THAITESOL JOURNAL Vol.26, No.1 June 201316 the readings more difficult because the connective supplies the temporal meaning, for these reluctant readers the additional clauses may have made the sentences longer and hence more difficult. The standard deviation in the students‟ comprehension scores was also significantly related to temporal connectives (-.611 at the 0.05 level of significance). The incidence of temporal connectives resulted in a narrowing of the deviation in the students‟ comprehension scores because the addition of the temporal clauses might confuse the better readers, thereby lowering their scores. This may be related to results found in a study by O‟Reilly and McNamara where they investigated text cohesion and students with high-knowledge and students with low-knowledge. They found that less skilled high-knowledge readers benefited from low-cohesion texts while skilled high-knowledge readers benefited from high cohesion texts. They concluded that there is “..a more complex view of when and for whom textual cohesion affects comprehension” (O‟Reilly et al., 2002, p.121). The standard deviation of the students‟ comprehension scores was related (.623 at the 0.05 level of significance) to lexical diversity. The increased lexical diversity of the passages resulted in further separating the high and low scores on the comprehension tests. This is in line with the importance of vocabulary recognition as a factor in the successful reader. Finally, the maximum students‟ comprehension scores was negatively related to the incidence of causal connectives (-.710 at the 0.05 level of significance). While the presence of the causal connectives (because, so, consequently, nevertheless, although) should help the reader with the intended relationships in the sentence, these connectives are difficult for intermediate level Thai learners of English. The causal connectives like although and
  28. 28. An Analysis of Reading Materials Using Coh-Metrix in Conjunction with Students’ Reading Comprehension Scores MARY SARAWIT 17 nevertheless which have a contrastive meaning are especially difficult for Thai learners of English. So, although the students overall had the highest comprehension scores, the presence of the causal connectives may not have aided their reading comprehension as much as they should have. Hence, the scores were not as high as they should have been. Discussion This study using Coh-Metrix showed that even for readings at the same graded level there can be large variations for the lexical, structural, and semantic content of the texts. The SRA Rose Level or grade 5 showed variations of more than 15% for word frequency, number of words before the verb, pronoun ratio to noun phrases, number of personal pronouns, number of modifiers per noun phrase, connectors, and (LSA) conceptual similarity of sentences. An analysis of the eleven Rose readings found that Rose 7 was the most difficult with a Flesch Reading Ease score of 67.586 (the closer to 100 the easier the reading) and a Flesch-Kincaid Grade Level 6.503 (the closer to grade 12 the more difficult the reading). However, for the students in this study it was the reading for which they had the highest score x 77.550% (median of 71.429%). The students‟ lowest comprehension score was for reading 3 ( x 49.660%, median 42.857%) for which the Flesch Reading Ease score was 83.678 with a Flesch- Kincaid Reading Grade of 5.372. There are obviously factors other than number of syllables and words in the sentences that are affecting the students‟ reading ability on these readings. While the small sample, both in terms of student population and readings, did not allow for higher level statistical analysis; a simple correlation study revealed a number of interesting relationships between the 15 indices used to
  29. 29. THAITESOL JOURNAL Vol.26, No.1 June 201318 analyze the texts. This study reconfirms the relationship between connectives, a measure of semantic cohesion, and reading ease. The higher incidence of causal and logical connectives related significantly with a higher Flesch Reading Ease score. With regard to the Flesch-Kincaid Grade Level, the higher the grade level was the lower the frequency of the words in the passage. This study also offers support for looking beyond lexical factors to syntactic factors when determining grade level. The Flesch-Kincaid Grade Level correlated significantly with the number of modifiers before the noun phrases for which the number of words before the verb also correlated significantly. This is despite the fact that the number of words per sentence did not correlate significantly for this set of 11 Rose readings. In addition, correlations between the students‟ comprehension scores and the 15 text indices showed significant correlations between temporal and causal connectives and student comprehension scores. These are both semantic cohesion features and should be looked at when considering reading difficulty levels. This study also found relationships between student reading comprehension scores and lexical diversity of content words in the passages. Conclusion This study reconfirms the relationship between connectives and reading ease. It also found relationships between student reading comprehension scores and lexical diversity of content words in the passages. Hence, it is recommended that a larger number of reading passages at a similar grade be studied with a larger number of students to test for the role of lexical diversity and connectives on student reading comprehension. It might be useful to separate
  30. 30. An Analysis of Reading Materials Using Coh-Metrix in Conjunction with Students’ Reading Comprehension Scores MARY SARAWIT 19 the causal connectives into the positive connectives (so, because, consequently) and negative connectives (although, nevertheless). Specifically for these reluctant readers, the results of this study point to a need to help the students better understand the temporal and causal connectives in reading passages. The incidence of both these connectives tended to make the passages more difficult for these readers. Coh-Metrix is a tool that can help us better understand students‟ problems in reading. A larger group of subjects and more readings at more levels of difficulty may help to better identify factors that affect the reading comprehension of Thai university students. Teachers need tools to help support and encourage the reluctant reader. It is hoped that this small exploratory study will encourage other researchers to use Coh-Metrix to study the deeper aspects of reading and their effects on the reader.
  31. 31. THAITESOL JOURNAL Vol.26, No.1 June 201320 References Beck, I. L., McKeown, M. G. & Kueau, L. (2002). Bringing Words to Life: Robust Vocabulary Instruction. New York: Guilford Press. Coltheart, M. (1981). The MRC Psycholinguistic Database Quarterly. Journal of Experimental Psychology, 33(A), 497-505. Crossley, S. A., Greenfield, I. McNamara, Vaud D. S. (2008). Assessing text readability using cognitively based indices. TESOL Quarterly, (43)3, 475-192. Crossley, S. A., McCarthy, P. M., Duffy, D. F. & McNamara, D. S. (2007). Toward a new readability: A mixed model approach. Retrieved from http://csjarchive.rpi.edu/Proceeding/2007/docs/p197.pdf. McNamara, D. S., Louwerse, M. M., Cai, Z. & Graesser, A. (2005). Cohmethrix version 1. 4. Retrieved from http://cohmetrix, menuphis.edu. Dumais, S. T. (2004). Latent semantic analysis. Annual Review of Information Sciences and Technology, 38(1), 188-230. Graesser, A. C., McNamara, D. S. & Kulkowich, J. M. (2011). Coh-Metrix: Providing multilevel analysis of text characteristics, Educational Researcher, 40(5), 223-234. Graesser, A. C., McNamara, D. S., Louwerse, M. M. & Cai, Z. (2004). Analysis of text on cohesion and language. Behavior Research Methods. Instruments & Computers, 36(2), 193-202. Hall, C., Lewis, G. A., McCarthy, P., Lee, D. S. & McNamara, D. S. Language in law: Using Coh-Metrix to access differences between American and English/Welsh language varieties. Retrieved from http://citeseerx.ist.psu.edu
  32. 32. An Analysis of Reading Materials Using Coh-Metrix in Conjunction with Students’ Reading Comprehension Scores MARY SARAWIT 21 Louwerse, M. M., McCarthy, P. M., McNamara, D. S. & Graesser, A. C. Variation in language and cohesion across written and spoken registers. Retrieved from http://citeseerx.ist.pso.edu/viewdoc/downloaddoi= 10.1.1.123.1183& rep=rep1&tupe=pdf McNamara, D. S., Graesser, A. C., Zhiqiang, C. & Kulikowitch, J. M. (2011). AERA 2011. Coh-Metrix easability components: Aligning text difficulty with theories of text comprehension. Retrieved from http://129.219.222.66 pdf/AERA_Coh-Metrix.pdf McNamara, D. S., Louwerse, M. M., McCarthy, P. M. & Graesser, A. C. Coh-Metrix: Capturing linguistic features of cohesion. Discourse Processes, 47, 292-330. McNamara, D. S., Louwerse, M. M., McCarthy, P. M. & Graesser, A. C. (2002). Coh-Metrix: Automated cohesion and coherence scores to predict text readability and facilitate comprehension. Institute for Intelligent Systems, University of Memphis, Memphis: Tenessee. O‟Reily, T. & McNamara, D. S. (2007). Reversing the reverse cohesion effect: Good texts can be better for strategic high-knowledge readers. Discourse Processes, 43(2), 121-152. Parker, D. H. (2000). Teacher’s Handbook SRA Reading Laboratory 3b. Washington, Ohio: SRA/McGraw Hill. Sarawit, M. (2009). Freshman Students’ Attitudes towards Reading: A Case Study at the Naresuan University International College. Available at www.sarawit. com/ msarawit appeared in 2012 in the Humanities Journal of Rajabhat University, Phitsanulok. Schmitt, N. (2010). Researching Vocabulary: A Vocabulary Research Manual. Palgrave Macmillan.
  33. 33. THAITESOL JOURNAL Vol.26, No.1 June 201322 Appendix A Data from the Coh-Metrix Analysis of the 11 Rose Level of the SRA 3B Lab: Lexical Indices Rose # Flesch Reading Ease Flesch- Kincaid Grade Level Word Frequency Word Concreteness Lexical Diversity 1 2 3 4 5 6 7 8 9 10 11 x SD CV 78.935 79.321 83.678 75.387 85.561 77.907 67.586 76.974 79.500 71.386 85.437 78.334 5.554 7.090 4.813 5.141 5.372 6.135 4.932 5.850 6.503 6.271 5.238 5.871 4.158 5.480 0.712 12.993 3003.453 4213.075 2425.300 1962.864 1994.321 2055.969 2216.330 1316.016 2740.976 1968.889 3796.116 2517.574 862.207 34.248 369.857 360.067 408.505 431.698 433.719 421.894 403.382 433.287 421.072 414.669 379.939 407.099 26.122 6.417 0.775 0.734 0.638 0.647 0.664 0.658 0.743 0.742 0.732 0.757 0.664 0.705 0.050 7.092
  34. 34. An Analysis of Reading Materials Using Coh-Metrix in Conjunction with Students’ Reading Comprehension Scores MARY SARAWIT 23 Appendix B Data from the Coh-Metrix Analysis of the 11 Rose Level of the SRA 3B Lab: Syntactic Indices Rose # Words before Verb Pronoun Ratio to Noun Phrases No. of Personal Pronouns No. of Modifiers per Noun Phrase No. of Words per Sentence 1 2 3 4 5 6 7 8 9 10 11 x SD CV 2.720 3.190 4.326 3.243 3.521 3.472 3.458 6.486 3.587 2.915 2.667 3.599 1.063 29.536 0.475 0.326 0.495 0.313 0.221 0.342 0.157 0.157 0.216 0.342 0.270 0.301 0.113 37.541 145.349 86.345 141.985 91.089 65.253 99.800 48.450 37.879 58.099 106.996 83.710 87.723 34.959 39.852 0.734 0.780 0.729 0.918 0.844 0.678 0.987 1.118 0.922 0.704 0.599 0.819 0.154 18.803 10.320 11.857 15.233 13.649 12.833 13.917 10.750 15.086 12.348 10.340 11.333 12.515 1.785 14.263
  35. 35. THAITESOL JOURNAL Vol.26, No.1 June 201324 Appendix C Data from the Coh-Metrix Analysis of the 11 Rose Level of the SRA 3B Lab: Semantic Indices Rose # Additive Connectives Causal Connectives Logical Connectives Temporal Connectives LSA Sentence Combinations 1 2 3 4 5 6 7 8 9 10 11 x SD CV 27.132 54.217 36.341 39.277 27.597 51.896 27.132 34.091 40.493 30.864 31.674 36.429 9.435 25.900 21.318 26.104 29.008 23.762 35.714 21.956 7.752 18.939 19.366 12.346 20.362 21.512 7.551 35.101 27.132 32.092 38.167 25.743 40.584 25.948 13.566 35.985 38.732 16.461 31.629 29.640 8.902 30.985 7.752 20.080 18.381 11.881 9.740 15.968 15.504 17.045 10.563 8.230 15.837 13.726 4.253 30.985 0.102 0.125 0.210 0.125 0.189 0.339 0.218 0.215 0.192 0.130 0.173 0.184 0.066 35.870
  36. 36. EFFECTS OF ONLINE TASK-BASED INTERACTIVE LISTENING INSTRUCTION FOR EFL LEARNERS Xingbin Tian Suranaree University of Technology, Thailand xingbintian@yahoo.com Suksan Suppasetseree Suranaree University of TechnologyThailand. suksan@sut.ac.th Abstract In order to meet the current trends in English listening teaching in China, the researcher constructed online task-based interactive listening (OTIL) lessons to enhance non-English major undergraduate students‟ English listening ability. The study aims at investigating the effects of using OTIL and students‟ attitudes toward OTIL. Two intact classes, 92 students in total, were selected as the sample for the study. The research instruments consisted of OTIL lessons, tests, a questionnaire and interview. The results showed that a significant difference was found in the post-test score between the experimental and control classes (P=0.000, P≤0.05). Meanwhile, the students had positive attitudes toward OTIL because the system was convenient, flexible, interactive and well-designed. The findings seem to indicate that OTIL might serve to develop English listening teaching and to improve students‟ listening ability for EFL learners. Keywords: online task-based interactive listening, English listening teaching, EFL learners, China
  37. 37. THAITESOL JOURNAL Vol.26, No.1 June 201326 Introduction The importance of listening for language acquisition has been emphasized by a number of researchers (Brown, 2006; Feyten, 1991; Krashen, 1982; Rost, 2002).From this point of view, listening teaching is the core of ESL/EFL study. Rost(2002) defined listening teaching as “approaches to curriculum design and methods of instruction that aim to assist learners in improving their listening”(p. 103). In China, English listening teaching is also an important part of college English teaching. However, the listening ability of most undergraduate students‟ of English has been unsatisfactory(Cheng, 2009b; Gan, Humphreys, & Hamp-Lyons, 2004; Tan, 2003; Yan, 2006). For instance, Lin (2002) pointed out after ten years‟ English learning, students were still “deaf and dumb” (when students can read, but cannot speak English fluently, or understand what people say) (p. 8). The problem above occurs at Tongren University (TU), China as well. After two years of learning English listening in the university, the listening competence of the most students remains weak (Tian, 2010; Wei, 2006). According to Wei‟s study (2006), there was lack of interaction between students in college English listening teaching at TU. In other words, classroom instruction was still teacher-centered, and the active role of students could not be exerted completely. Furthermore, the teaching process of college English listening teaching via multimedia at TU was very unitary: applying the CD-ROM attached to the listening textbook and then doing the exercises associated with the listening textbook (Tian, 2010). In class, teachers stood by the computer and operated the mouse. They did not pay much attention to the communication with the students. Students fixed their attention only on the screen and played a quite passive role in such a learning environment. Although language labs at TU were
  38. 38. Effects of Online Task-Based Interactive Listening Instruction for EFL Learners XINGBIN TIAN, SUKSAN SUPPASETSEREE 27 built to enable teachers and students to teach and learn English with an E- learning environment, no teacher created his/her websites for listening instruction. Nowadays, the importance of English listening in tertiary education has been realized. In 2007, the Ministry of Education of China(MOE) issued a new English curriculum for non-English majors, namely the College English Curriculum Requirements (CECR), which emphasized “developing students‟ ability to use English in an all-round way, especially in listening and speaking” (MOE, 2007, p. 18).For this reason, the CECR (MOE, 2007) claimed that universities across China should satisfy 3 requirements as follows: 1) exploring and establishing a web-based listening teaching model; 2) delivering listening courses via the Internet or Intranet; 3) shifting teacher-centered pattern to learner-centered teaching. In order to meet the requirement of the CECR, a great number of studies on EFL listening instruction in China (Cai & Li, 2008; Cheng, 2009b; Wei, 2012; Yao, 2010) have been performed on how to promote learners‟ listening ability. For example, Cai and Li (2008) studied asynchronous listening instruction in the EFL context with pedagogical tasks, showing that the average listening comprehension score of the experimental group was much higher than the score of the control group. Meanwhile, Cheng (2009a) conducted an experiment to introduce learner-centered teaching theory into the listening class, and investigated its effectiveness in improving learners‟ listening skills. The results showed that the achievement of the experimental group was higher than that of the control group. After the experiment, the subjects stated that ILT aroused their learning interest and motivation. Yao (2010) carried out research on a task-based approach to College English listening teaching via the Internet. The results showed the achievements of the students in the experimental
  39. 39. THAITESOL JOURNAL Vol.26, No.1 June 201328 groupwere higher than those of the control classes. The students‟ attitude toward the task-based approach in college English listening teaching via the Internet was positive. More recently, Wei (2012) conducted research on college English listening teaching via Moodle. After the experiment, the experimental class had made more progress than the control class. The research found that English listening teaching via Moodle provided students with more interaction. The feedback from the Moodle platform was very useful to motivate students to learn more. All in all, the findings of previous studies indicated that listening instruction via a network with interactive tasks and activities could enhance learners‟ listening ability. In other words, the teaching of listening skills through a technology-supported task-based approach was beneficial to the integration of language skills. This was because students were able to make comprehensive use of their language ability and communicative competence. However, for the most part, tasks used in the previous research above were pedagogical tasks. Little research used real-world tasks in English listening teaching. According to Nunan(2004), “the point of departure for task-based language teaching is real- world or target tasks” (p. 19). Under real-world tasks, learners can also undertake many pedagogical tasks which represent a bridge to real world tasks. Real-world tasks do not have a single correct answer. They can often be solved in several ways and allow learners to perform the task in its natural setting (see Figure 1).
  40. 40. Effects of Online Task-Based Interactive Listening Instruction for EFL Learners XINGBIN TIAN, SUKSAN SUPPASETSEREE 29 Figure 1. A Framework for TBA (Nunan, 2004, p. 25) The present study aimed at 1) investigating the effects of using online task-based interactive listening (OTIL) with real-world tasks for EFL learners, and 2) determining students‟ attitudes toward OTIL. Online task-based interactive listening (OTIL) is defined as English listening lessons used to deliver learning online through real-world tasks. The term “online” in the present study refers to Moodle, short for Modular Object-Oriented Dynamic Learning Environment, which is used as the online learning platform for English listening at TU. OTIL aims to create a highly interactive E-learning environment for English listening, during which listeners have maximum interaction with peers, contents and teachers. The term “EFL learners” in the study refers to the non- English major undergraduate students at TU who have registered to study college English. The study was carried out on the basis of two research questions as follows. Real-world/Target Tasks Pedagogical Tasks Enabling Skills Rehearsal Tasks Activation Tasks Language Exercises Communication Activities
  41. 41. THAITESOL JOURNAL Vol.26, No.1 June 201330 1) Are there any significant differences in listening comprehension between the experimental and control classes? 2) What are the students‟ attitudes toward online task-based interactive listening (OTIL) instruction? Methodology Population and Samples The population for the study consisted of1075 second-year non-English major undergraduate students at TU. There were 26 intact classes within 20 majors. Purposive sampling was adopted in the present study. The college English final examination in the second semester of academic year 2012 was used to determine the sample. According to the results, two intact classes, 92 students in total, were selected as the sample for the study because 1) the number of students in each class was similar and 2) the students‟ English proficiency level was similar. The two intact classes were a Politics Class (N=46, =64.78, SD=11.105) and a Chinese Class (N=46, =64.72, SD=8.471). The Chinese Class was chosen as the experimental class and the Politics Class as the control class. Research Instruments Four research instruments were employed in the present study, including OTIL, tests, a questionnaire and an interview. Online task-based interactive listening (OTIL). The term “online task- based interactive listening” (OTIL)refers to the English listening lessons constructed by the researcher after a needs analysis. Real-world tasks were designed for OTIL to provide an environment to promote students‟ English listening ability.
  42. 42. Effects of Online Task-Based Interactive Listening Instruction for EFL Learners XINGBIN TIAN, SUKSAN SUPPASETSEREE 31 Tests. Two English listening tests were employed for the pre-test and post-test. The tests were adapted from the listening sub-tests of the standardized CET 4 (College English Test, Band 4). These were designed to evaluate the overall English proficiency of undergraduates in Chinese universities. The total score of pre-test and post-test was 100 points. A questionnaire. A questionnaire designed by the researcher was used to determine the students‟ attitudes toward OTIL. It consisted of two parts, including students‟ general information and the students‟ opinions toward OTIL. The format of the second part was a five-point Likert scale. That is, values on the scale were labeled from 1 to 5, namely strongly agree, agree, uncertain, disagree, and strongly disagree. Interview. Data from one source only is sometimes not sufficient to draw good conclusions from findings. A triangulation process should be applied by using more instruments, such as interviews, to collect more data. Thus, a semi- structured interview was employed in the study. The interview, designed by the researcher, consisted of six guided questions. Data Collection and Analysis The data collection consisted of two phases: 1) collecting the scores of the pre-test and post-test which were given to the experimental and control classes before and after the experiment, and 2) obtaining information from questionnaires(for the experimental class only). After the students had responded to the questionnaire, fifteen interviewees were selected from the experimental class. They came from three different English proficiency levels: high, medium and low. Stratified sampling was used to select samples. Discrimination between levels was performed on the basis of z-scores from the post-test (-1.00≤Z≤1.00). Five interviewees were selected from each level for a small-group interview. All
  43. 43. THAITESOL JOURNAL Vol.26, No.1 June 201332 interviews were recorded and transcribed for data analysis. Chinese language, the respondents‟ native language, was employed for both questionnaire and interview for better understanding and convenience. Quantitative and qualitative analyses were conducted in the study. The software Statistical Product and Service Solutions (SPSS) package was used to conduct descriptive and statistical analyses. Descriptive analysis involved the mean score and standard deviation, while statistical analysis included t-test and analysis of covariance (ANCOVA). Records of the interviews were transcribed verbatim. To interpret the data, content analyses were conducted using open and axial coding. Results and Discussion Students’ English Listening Achievement for the Experimental and Control Groups Tests were used to evaluate students‟ English listening ability and to compare students‟ English listening ability before and after the treatment as shown in Table 1. Table 1 Results of Students’ English Listening Achievement Tests N Experimental Group Control Group SD SD Pre-test 46 57.30 8.897 57.17 8.160 Post-test 46 75.57 8.702 66.35 9.374
  44. 44. Effects of Online Task-Based Interactive Listening Instruction for EFL Learners XINGBIN TIAN, SUKSAN SUPPASETSEREE 33 As can be seen, no significant differences were found between the two classes‟ pre-test (P=0.942, P≤0.05). However, after the intervention, students of both classes improved their listening ability as shown in Table 2. The results showed that a significant difference between the pair scores (pre-test and post- test scores) of each class was found(P=0.000, P≤0.05) by using a paired-samples T-test. Table 2 Results of Paired Samples T-test for Experimental and Control Class Classes Tests Paired Differences T Sig. (2- tailed)SD 95% Confidence Interval of the Difference Lower Upper Experimental Class Pretest - Posttest -18.261 4.855 -19.703 -16.819 -25.508 .000 Control Class Pretest - Posttest -9.174 4.312 -10.454 -7.893 -14.430 .000 In order to evaluate the effects of using OTIL, analysis of covariance was used to increase statistical power (Tabachnick & Fidell, 2007), namely by removing extraneous variability (students‟ prior English listening ability) that derived from pre-existing individual differences. The findings revealed that a significant difference was found in the post-test score between the experimental and control classes (P=0.000, P≤0.05) as shown in Table 3. This indicated that
  45. 45. THAITESOL JOURNAL Vol.26, No.1 June 201334 the students in the experimental class, instructed with OTIL, had made more demonstrable progress than those of the control class. Therefore, the OTIL approach delivered via Moodle appeared to help EFL learners develop English listening more effectively. Table 3 Comparison of Post-test Scores for the Experimental and Control Classes Source Type III Sum of Squares df Mean Square F Sig. Corrected Model 7463.978a 2 3731.989 179.360 .000 Intercept 669.914 1 669.914 32.196 .000 pretest 5509.891 1 5509.891 264.806 .000 Classes 1903.611 1 1903.611 91.488 .000 Error 1851.848 89 20.807 Total 472520.000 92 Corrected Total 9315.826 91 Students’ Attitudes to Online Task-based Interactive Listening (OTIL) Results of the Questionnaires To determine students‟ attitudes toward OTIL, a frequency analysis was applied to analyze the questionnaires. The results of the analysis were presented in Table 4.
  46. 46. Effects of Online Task-Based Interactive Listening Instruction for EFL Learners XINGBIN TIAN, SUKSAN SUPPASETSEREE 35 Table 4 Students’ Attitudes toward OTIL Statements N SD 1. Online task-based interactive listening can make English learning enjoyable. 46 4.07 .574 2. Online task-based interactive listening can meet my learning objectives. 46 3.93 .490 3. Online task-based interactive listening can enhance my listening comprehension. 46 4.09 .626 4. Online task-based interactive listening provides me opportunities to practice listening skills. 46 4.37 .610 5. Online task-based interactive listening can enhance student-student interaction. 46 4.07 .533 6. Online task-based interactive listening can facilitate student-teacher interaction. 46 3.96 .631 7. Online task-based interactive listening is convenient for reviewing the lessons. 46 4.15 .666 8. Online task-based interactive listening can promote learning. 46 3.96 .729 9. Online task-based interactive listening provides abundant materials to learn. 46 4.37 .645
  47. 47. THAITESOL JOURNAL Vol.26, No.1 June 201336 Statements N SD 10. Online task-based interactive listening can provide easy access to useful feedback from the teacher. 46 4.09 .725 11. Materials in online task-based interactive listening are interesting. 46 4.15 .595 12. Materials in online task-based interactive listening are suitable for my English proficiency level. 46 3.76 .524 13. The activities in online task-based interactive listening are interactive. 46 3.91 .626 14. The activities in online task-based interactive listening can help me improve listening ability effectively. 46 4.24 .603 15. The interactive modules, such as forum, chat room and wiki in online task-based interactive listening are very useful for group discussion. 46 4.11 .605 Total 46 4.08 .166 It should be noted that the highest mean score was 4.37, while the lowest was 3.76. The first three highest frequency statements were: 1) OTIL provides students opportunities to practice listening skills ( =4.37, SD=0.610); 2) OTIL provides abundant materials to learn ( =4.37, SD=0.645); and 3) The activities in OTIL can help students improve listening ability effectively ( =4.24, SD=0.603). The total mean score of the questionnaire was 4.08 (SD=0.166), indicating that the students had positive attitudes toward OTIL.
  48. 48. Effects of Online Task-Based Interactive Listening Instruction for EFL Learners XINGBIN TIAN, SUKSAN SUPPASETSEREE 37 Result of Semi-structured Interview All interviewees had positive attitudes toward OTIL. They claimed that they preferred learning English listening through OTIL for two reasons. First, the tasks in OTIL met learning objectives of the course. Second, OTIL could help them improve their listening comprehension. After reading through the whole transcripts, two major categories were identified from the reported statements. Each major category covered several sub-categories as shown in Table 5. Table 5 Categories of the Semi-structured Interview Categories Sub-categories Satisfaction with OTIL Abundant materials with different media Real-world tasks Well-designed listening stages Learning styles Convenience Flexibility Interactivity Favorite items in OTIL Videos Audios Self-test Comprehension exercises
  49. 49. THAITESOL JOURNAL Vol.26, No.1 June 201338 The present study was conducted to meet the current trends in the teaching of English listening skills in China for EFL learners, by investigating the effects of using OTIL and students‟ attitudes toward OTIL.The results showed that the post-test scores of the experimental class were significantly higher than those of the control class (P=0.000, P≤0.05). Meanwhile the students had positive attitudes toward OTIL. The results in response to the research questions are discussed below. Listening Achievement Although the students of both the experimental and control classes made great progress after the intervention, a significant differences between the post- test scores of the two classes (P=0.000, P≤0.05) wasfound. Five main reasons may account for the experimental class‟ listening improvement. First, OTIL was more learner-centered than teacher-centered. The goal of OTIL was for students to take on more of the responsibility of learning English listening and become more actively involved in the learning process. Students learned English listening though OTIL by doing things for themselves. They became active participants not just receivers of knowledge. Second, the tasks in OTIL provided meaning and motivation for learning. Ur (1984) argued that listening activities are most effective if they are constructed around a task. Some types of tasks in OTIL might be found in real life or approached real life tasks. This raised students‟ interest. Third, OTIL via Moodle offered a lot of useful and interactive modules such as Wikis, Forums, Workshops, Chats and Glossaries to enhance interactivity. Lee (2001) pointed out that the Internet has tremendous potential as a tool for teaching EFL. Based on a constructivist approach to learning, OTIL emphasized learning through a group‟s collaborative construction of knowledge. Students could interact with each other synchronously and asynchronously.
  50. 50. Effects of Online Task-Based Interactive Listening Instruction for EFL Learners XINGBIN TIAN, SUKSAN SUPPASETSEREE 39 Fourth, OTIL motivated students to learn English by providing immediate and effective feedback. Siragusa (2000, cited in Herridge-Group, 2004) claimed that students may be more motivated to succeed in a learning program if regular and effective feedback is provided. Finally, students could use various authentic listening resources of authentic listening drawn from different media in OTIL to practice their listening in a cultural context. Students’ Attitudes The results of the questionnaire and interview revealed that the students had positive attitudes toward OTIL. First, students strongly agreed that OTIL provided them opportunities to practice listening skills ( =4.37, SD=0.610). This might be due to the fact that OTIL was convenient, flexible, and interactive. This finding agreed with He‟s (2002)observation. He pointed out that excellent online learning provided the learners with services and opportunities, including opportunities to learn some topics by different means through different media, opportunities to experience their own knowledge, opportunities to receive feedback according to the perceptions of the learners, opportunities to consult specialists, opportunities to share and develop understanding with other learners, and opportunities to participate equally during rich and colorful discussions and collaborations. The findings of the interview also demonstrated this. Second, the students strongly agreed that OTIL provided abundant materials for learning ( =4.37, SD=0.645).The materials including images, audios, videos and text files were rich, authentic and humorous. In each topic, the lesson was broken up into five sections: warm-up, listening practice, real- world task, self-test and further development. Listening material was supported with glossaries, pictures, examples of listening skill training, audios, videos, comprehension exercises, discussion questions, real-world tasks, self-tests, and
  51. 51. THAITESOL JOURNAL Vol.26, No.1 June 201340 forum interactions. Except for the topics, a Moodle library was constructed to provide different types of materials for students to train and practice in or out of class. Third, the students also strongly agreed that the activities in OTIL helped students improve listening ability effectively ( =4.24, SD=0.603). The activities in OTIL included pair work, group discussions, listening skill practice, glossary brainstorm, vocabulary learning, prediction, forum, true or false, matching, multiple-choice, gap-filling, reporting, interview and testing. Activities in pre- listening served as preparation for listening, using prior knowledge to predict the context. While-listening activities related directly to engagement with listening materials. Students were required to complete them during or immediately after listening or watching. In the post-listening phase, analysis and practice activities were designed to shift students‟ attentions from meaning to forms contextualized through the task (Willis, 1996). The findings met a „learner-centered‟ , „learning by doing‟ and „learning in interaction‟ language teaching approach (Brown, 2001; Foster, 1999; Richards & Rodgers, 2001). Finally, according to the results of the interview, students liked OTIL because it provided abundant authentic, interesting and useful materials. The materials not only raised their interest, but also helped them gain a great deal of knowledge. Among the materials in OTIL, students preferred videos more than other types. They enjoyed videos because the contents were rich, colorful, humorous and authentic. The most important aspect of this was that the videos helped them understand the contents and learn cultures of foreign countries. The students also liked the online learning style which made them learn English effectively, improve their English listening, and raised their interest. OTIL provided several listening stages to promote their listening ability. Moreover, the
  52. 52. Effects of Online Task-Based Interactive Listening Instruction for EFL Learners XINGBIN TIAN, SUKSAN SUPPASETSEREE 41 students liked OTIL due to convenience, flexibility and interactivity. Students could visit OTIL anywhere, anytime, listen to audios or watch videos as much as possible. They found it easy to assess their listening level by using OTIL and find the meanings of new words through the Internet. When they met difficulties, they could discuss and solve the problems by chatting directly with the teacher and classmates via the Chat Room. They could also use the Forum to communicate and share with others. The findings of the study agree with the study of Fan, Wang, He and Hu (2010) who developed Autonomous College English Listening Learning via Moodle in China. The results showed that the post-test scores of the experimental group who received tutoring via Moodle ( =82.30, SD=8.879) were higher than those of the control group who were taught with traditional college English listening teaching ( =76.86, SD=9.136). The students of the experimental group had positive attitudes toward the Moodle platform. Furthermore, Yang (2011) conducted a comparison between college English listening teaching via Moodle and traditional English listening teaching in China. The findings showed that the students of the experimental group ( =67.15) had a higher average post-test score than those of the control group ( =57.26). A significant difference was found between the experimental and control groups. 83.8% students reported that they were interested in learning English listening through Moodle. In addition, Dennis (2012) constructed Blended Online Learning Approach (BOLA) packages for teaching English for Careers in Technology in Thailand. The results showed that the students of the experimental group obtained higher mean scores ( =12.1) after the treatment. According to the results of the questionnaires and interview, students were very
  53. 53. THAITESOL JOURNAL Vol.26, No.1 June 201342 satisfied with the BOLA packages which could promote autonomous learning effectively. In short, OTIL was an effective and suitable English listening website for promoting student listening ability. It provided students with many kinds of materials which were authentic and interesting. Real-world tasks in OTIL were designed to meet students‟ learning goals and help them improve listening comprehension. Additionally, OTIL is convenient, flexible and interactive, thus supporting and motivating students to learn English listening without the restrictions of time and place. Conclusion The present study was conducted to meet the current trends in English listening for EFL learners. From the results of the study, it can be found that OTIL contributes significantly to the successful development of EFL listening skills, and that real-world tasks are very important to engage students in the whole learning process. The study provides some insights into how English listening with network support might be effectively used to promote EFL learners. Furthermore, the study might serve to contribute to finding solutions for the development of college English listening teaching programs for improving students‟ listening ability in China. However, because of the limitation of the sample size, it is difficult to generalize the results of the study beyond the current sample and more research is needed before the system can be implemented as a part of China‟s reform in college English teaching.
  54. 54. Effects of Online Task-Based Interactive Listening Instruction for EFL Learners XINGBIN TIAN, SUKSAN SUPPASETSEREE 43 References Brown, H. D. (2001). Teaching by principles: An interactive approach to language pedagogy. Beijing: Foreign Language Teaching and Research Press. Brown, S. (2006). Teaching listening. New York: Cambridge University Press. Cai, Y. & Li, M. (2008). A study on asynchronous listening instruction in the EFL context. CELEA Journal, 31(3), 48-54. Cheng, J. (2009a). An analysis of effectiveness of learner-centered listening teaching format. Journal of Xi'an International Studies University, 17(4), 85-88. Cheng, J. (2009b). The development and new trends of English listening teaching. Foreign Language World, 1, 51-56. Fan, Z., Wang, H., He, Y. & Hu, S. (2010). Research on college listening teaching via Moodle. China Educational Technology, 11, 106-109. Feyten, C. M. (1991). The power of listening ability: an overlooked dimension in language acquisition. Modern Language Journal, 75(2), 173-180. Foster, P. (1999). Task-based learning and pedagogy. ELT Journal, 1, 69-70. Gan, Z., Humphreys, G. & Hamp-Lyons, L. (2004). Understanding successful and unsuccessful EFL students in Chinese universities. The Modern Language Journal, 88(2), 229-244. He, K. (2002). E-learning and teaching innovation in universities. China Educational Technology, 2, 8-12. Krashen, S. D. (1982). Principles and Practices of Second Language Acquisition. New York: Pergamon. Lin, L. (2002). English education in present-day China. ABD, 33(2), 8-9.
  55. 55. THAITESOL JOURNAL Vol.26, No.1 June 201344 MOE. (2007). College English Curriculum Requirements. Beijing: Foreign Language Teaching and Research Press. Nunan, D. (2004). Task-based language teaching. Cambridge: Cambridge University Press. Richards, J. C. & Rodgers, T. S. (2001). Approaches and methods in language teaching (2nd ed.). Cambridge: Cambridge University Press. Rost, M. (2002). Teaching and researching listening. Harlow, Essex: Longman. Tabachnick, B. G. & Fidell, L. S. (2007). Using Multivariate Statistics (5th ed.). Boston: Pearson Education, Inc. Tan, S. (2003). Strategy training for English language learners from PRC. In G. L. Lee, L. Ho, J. E. L. Meyer, C. Varaprasad & C. Young (Eds.), Teaching English to students from China (pp. 158-173). Singapore: Singapore University Press. Tian, X. (2010). A study on need analysis of College English listening learning in E-learning environment. Journal of Tongren University, 12(4). Wei, Y. (2006). On the difficulties and countermeasure of the College English teaching on listening and saying. Journal of Tongren Teachers College, 8(3), 62-66. Wei, Y. (2012). Study on Moodle-based college English listening teaching. Journal of Heilongjiang College of Education, 31(7), 167-169. Willis, J. (1996). A framework for task-based learning. London: Longman. Yan, M. (2006). An investigation of the factors affecting Chinese students' listening comprehension of English. Sino-US English Teaching, 3(3), 27-30. Yang, H. (2011). College English listening teaching based on Moodle platform. Journal of Tongren University, 13(3), 100-102.
  56. 56. Effects of Online Task-Based Interactive Listening Instruction for EFL Learners XINGBIN TIAN, SUKSAN SUPPASETSEREE 45 Yao, Y. (2010). An organic integration of multimedia network and task-based approach in College English listening teaching. Journal of Yichun College, 32(10), 166-167.
  57. 57. EXPLORING BELIEFS OF PRE-SERVICE TEACHERS TOWARD ENGLISH AS AN INTERNATIONAL LANGUAGE Flora Debora Floris Petra Christian University Jl. Siwalankerto 131-136 - Surabaya 60236, Indonesia debora@petra.ac.id Abstract Research has repeatedly shown that pre-service teachers have well well- established beliefs about teaching which have been formed on the basis of their own learning experiences. These beliefs strongly influence their professional development and how pre-service teachers interpret and acquire information from their teacher education courses. This paper presents the results of a small research conducted on 11 pre-service teachers‟ beliefs toward English as an international language (EIL) before and after joining World Englishes class in order to detect any changes in the nature of these beliefs. 4 major areas of EIL namely (a) the role of English as an international language, (b) the best variety, (c) the use of the students‟ mother tongue in English classrooms, and (d) the best teacher of English were highlighted in this study. Keywords: pre-service teacher, teacher education, beliefs, English as an international language
  58. 58. Exploring Beliefs of Pre-Service Teachers toward English as an International Language FLORA DEBORA FLORIS 47 Introduction The fact that English is now the major international language in intercultural communications, business, science, technology, and other areas is indisputable. It is also the most widely learned second or foreign language in many countries; and it makes the number of second and foreign language speakers far exceed the number of the first language speakers of English. In 2003, for example, the number of English speakers according to Crystal (2003) reached approximately 1,500 million only 20% of which are the native speakers of the language. In 2008, this number had been revised upwards again in the direction of 2 billion. In fact, within 25 years, the number of speakers of English had moved from a fifth to quarter to a third of the world‟s population (Crystal, 2008). In 2070, Graddol (1999) claimed that the projected number of English speakers would be nearly 10 billion. This situation has resulted in remarkable demand in English language teachers, which has also led to an increase in the number of non-native English language teachers. In Indonesia, for example, approximately 3,047 non-native English language teachers were employed at formal schools in 2011 and 3,442 teachers in 2012. The number keeps increasing due to the growing number of Indonesian students who study English at formal schooling. In 2013, for example, Indonesia needs approximately 3,733 new English teachers and 4,130 new teachers in 2014 (Yahya, 2011). Matsuda and Matsuda (2001) state the fact that the majority of English teachers in the world are not native speakers of English. Some of these teachers learned English as children; others learned it as adults. For some, English is their third or fourth language; for others, it is the only language other than their
  59. 59. THAITESOL JOURNAL Vol.26, No.1 June 201348 mother tongue that they have learned. As there is an increase in the number of non-native English teachers, accordingly there is also an increase in the number of non-native English language speakers as pre-service teachers joining Department of English Language Teaching at universities all around the world. This is because for non-native English teachers, pre-service education is often seen as the start of teacher preparation program or in other words the first step in the professional development of teachers. Wilke (2004) says that such education often exposes pre-service teachers to new perspectives as well as prepares them in knowledge and skills. In many universities, programs for pre-service EFL teachers still tend to focus on the inner circle. This might be because people had some long- established assumptions such as (a) students need to learn the English of native speakers, (b) the native speakers should serve as the model and standard, (c) American or British culture should be taught, and (d) communicative language teaching is the best way to teach the language (Brown, 2012). However considering the fact that English now is considered as the international language and that non-native speakers outnumber native speakers, work should be done to create pre-service teacher programs that focus spirit of English as an International Language (EIL). Pre-service teachers need to be informed that a language program should incorporate the promotion of intercultural competence, an awareness of other varieties of English, multilingualism in the classroom, instructional materials that include both local and international cultures and the adoption of socially and culturally sensitive teaching methodology (McKay, 2012). The English Education Business Program of the English Department of Petra Christian University Surabaya also updated its curriculum to incorporate
  60. 60. Exploring Beliefs of Pre-Service Teachers toward English as an International Language FLORA DEBORA FLORIS 49 the EIL principles by offering some new subjects such as World Englishes, Education Policy, Current Issues in Global Education and Intercultural Teaching & Learning and by “slipping-in” the EIL principles in other subjects such as Spoken English in which pre-service students are introduced to varieties of English or in Language Teaching & Learning in which they are asked to critically evaluate various teaching approaches. New subjects such as World Englishes whose scope is not limited to the inner circle are offered cause such courses are likely to result in a “world view . . . [that is] more consistent with the sociolinguistic realities of the spread of English as an international language” (Brown and Peterson, 1997, p. 44). The principles of EIL are also integrated in other courses so pre-service students will be aware of the current landscape of English. By introducing the EIL issues as the content of the course or peripherally, it is hoped that pre-service students will develop a favorable attitude towards EIL. The implementation of the revised curriculum started in the 2nd semester of 2011/2012 (February – August 2012). I was assigned to facilitate World Englishes course which aimed to enable the students to consider past, present, and future varieties of the English languages. Attention was given to the historical, political, and sociocultural issues associated with the globalization of Englishes and on the ideological underpinnings of debates about nativization, standardization, identity, and ownership. Students were required to attend 14 consecutive meetings. There were 11 pre-service students of Batch 2010 registered for this course. As none of the topics discussed are covered in the first two years of their university study, these students had never come across the relevant concepts before. In addition, these 11 pre-service teachers had been immersed in the
  61. 61. THAITESOL JOURNAL Vol.26, No.1 June 201350 traditional EFL classrooms since they were in elementary level. As the concept of EIL was new for these students, I was interested to observe the prior beliefs of these pre-service teachers and wished to ascertain to what extent the exposure to the relevant concepts influenced their prior beliefs. Therefore the present study was guided by the following research questions: 1. What sorts of beliefs do pre-service teachers have at the beginning of the course about (1) the role of English as the international language, (2) the best English variety, (3) the use of students‟ mother tongue in the EFL classrooms, and (4) the best English language teacher. 2. Do these pre-service students‟ initial beliefs change at the end of semester? 3. How do these pre-service students welcome the EIL principles in their future pedagogical practices? Attention was given to (1) the role of English as the international language, (2) the best English variety, (3) the use of students‟ mother tongue in the EFL classrooms, and (4) the best English language teacher because these areas are closely related to the native-speaker paradigm (see Hassall, 1996 and Matsuda, 2003) and are widely addressed in Indonesian context nowadays. Another reason why I was interested to observe these students‟ beliefs was because the literature portrays that beliefs play a central role on guiding teachers‟ instructional behavior. Teachers‟ beliefs can powerfully shape what teachers do (Pajares, 1992 and Borg, 2011). Beliefs form “the bedrock of teachers' intentions, perceptions, and interpretations of a given classroom situation and the range of actions the teacher considers in responding to it" (Chapman, 2002, p. 180). Johnson (1994) also states that (1) teachers‟ beliefs influence perception and judgment, (2) teachers‟ beliefs are reflected in classroom
  62. 62. Exploring Beliefs of Pre-Service Teachers toward English as an International Language FLORA DEBORA FLORIS 51 practices, and (3) teachers‟ beliefs should be understood with a view to improving teaching practices and teacher education programs. In the context of language teacher education, beliefs are seen to be a key element. Borg (2011) states that teacher education is more likely to have an impact when it is based on an understanding of the beliefs these pre-service teachers hold. Kagan (1992, p. 85) has suggested that beliefs “may be the clearest measure of a teacher‟s professional growth”. Therefore most of the research available on the impact of teacher education on language teachers‟ beliefs has been conducted in pre-service contexts. Studies highlight that pre-service students carry with them some strong ideas and beliefs. These beliefs are formed during the “apprenticeship of observation” which refers to the years these students have spent sitting in the student desk prior to entering a teacher education program (Lortie, 2002). In other words, pre-service teachers‟ beliefs are formed through many years of exposure to educational practices and can be traced back to early experiences, from primary education up to tertiary level. Furthermore, pre-service teachers use their previous educational experiences to interpret the input provided in their teacher education program (Kagan, 1992). The majority of pre-service student-teachers who start their education program, for example, view teaching as telling or lecturing-that is, directly transmitting information to a passive learner (Torff, 2003).This is because they are exposed to such lecturing style during their school years. Scholars have found that pre-service teachers‟ prior beliefs brought to a teacher education program significantly impact what and how pre-service teachers learn. Prior beliefs can function as filters for processing experiences and knowledge (Borko and Putnam, 1996). Prior beliefs are also related to
  63. 63. THAITESOL JOURNAL Vol.26, No.1 June 201352 motivation constructs and to pre-service teachers‟ academic performances and achievement (Pajares, 2003). Prior beliefs may hinder their ability to see the relevance of their teacher education program as well because the existing beliefs may be incompatible with their new learning in their teacher education program (Borko and Putnam, 1996). However a strong sense of confidence built on positive prior learning will expend a higher level of energy to work through the difficulty (Pajares, 2003). To change prior beliefs of pre-service teachers on language education program is a challenge. Borg (2011) has in fact observed that some studies such as Borg (2005), Peacock (2001), and Urmston (2003) report no changes in the pre- and post-course beliefs of pre-service teachers. In contrast, other studies such as Clarke (2008), Mattheoudakis (2007) and Busch (2010) provide evidence of change in student teachers‟ beliefs during language teacher education (cited in Borg, 2011). A research done by Minor et.al. (2001) also showed that at the end of the semester of their observation, the beliefs of 84 pre-service teachers observed had moved to be more in line with the instructor that is to have a more progressive orientation. Does instruction impact pre-service teachers‟ beliefs? A study by Liu and Fisher (2006) reported that belief change could be promoted in variable ways (as cited in Borg, 2011). Pre-service teachers involved in Minor et.al. (2001) „s study, for example, were required to complete a variety of assignment including a written critique of an article from a refereed education journal, an individual presentation, a group presentation, reflections of reading assignments, active participation in class activities, exams, and the development of a professional portfolio. At the end of the semester, their prior beliefs changed.
  64. 64. Exploring Beliefs of Pre-Service Teachers toward English as an International Language FLORA DEBORA FLORIS 53 Having read all of the review of literature and considered the nature of the World Englishes course taught at my university and 11 pre-service teachers registered for this course, I became interested to observe the beliefs that these pre-service teachers had at the beginning of the course, to see whether there were any changes on their initial beliefs, and to find out how these 11 student teachers perceive the EIL principles in their future pedagogical practices. The focus would be on (1) the role of English as the international language, (2) the best English variety, (3) the use of students‟ mother tongue in the EFL classrooms, and (4) the best English language teacher. Method Participants The participants of this study were 11 student teachers registered for the World Englishes course of English Education Businness program of Petra Christian University Surabaya. All of them were Indonesians in their early twenties and in their 5th semester. There were 10 female students and 1 male student. Procedure This study was conducted in the 2nd semester of 2011/2012 (February – August 2012). The course itself was conducted for 14 meetings in which each meeting lasted for 2 hours. Each meeting was conducted once per week. All the names mentioned in this paper were pseudonyms. The research mainly adopted a qualitative approach. Dörnyei (2007) addressed several reasons for choosing this type of research namely: the research uses small sample size of participant, the data analysis is done with words, and it is concerned with subjective opinions.
  65. 65. THAITESOL JOURNAL Vol.26, No.1 June 201354 Classroom discussions and reflective papers of the students were used to collect data. In order to answer the first research question which aims to explore the prior beliefs, at the beginning of the semester, the pre-service teachers were required to write what they know about English as an international language, the best English variety, the use of students‟ mother tongue in their language classrooms, and the best English language teacher. They were asked to reflect upon their personal teaching and learning experiences and beliefs and explicitly provide examples. These reflective papers were not meant to be research papers and the students were not expected to do any additional reading. The paper was written in English and the participants submitted their reflective papers at the beginning of the second meeting. For answering the other research questions, I employed the online and offline classroom discussions and students‟ final reflective papers. For the purpose of online classroom discussions, I set up a closed Facebook group. All students were members of this online group and some participated in the thread of discussions quite actively. Ongoing throughout the class meetings, the pre-service teachers and I always had (offline) classroom discussions in which I asked my students some questions related to the issues of EIL so that I knew what their before-course-perceptions were. Then I asked each participant of this study to share their initial beliefs with each other, and together overview and examine the present EIL situation. During the discussions, I took notes on some interesting points of views that my students expressed. I also asked the participants to note down if listening to the classroom discussion had sparked some ideas and they felt their prior beliefs needed revision. All online and offline discussions were mainly conducted in
  66. 66. Exploring Beliefs of Pre-Service Teachers toward English as an International Language FLORA DEBORA FLORIS 55 English though some Indonesians occurred occasionally. At the end of the course, the students were once again asked to revisit their personal beliefs and write their reflections towards the 4 major issues highlighted in their World Englishes class. They also got an additional task, i.e. to think about how they would apply their EIL beliefs in their future classrooms. Data Collection and Analysis Data gathered from the participants‟ reflective papers and classroom discussions were then analyzed through content analysis as proposed by Cresswell (2011). The first step that I took was to explore or to obtain a general sense of the data. Then I started the process of coding the data according to the categories relevant to the research questions. After that I tried to interconnect the categories and report findings to the research questions. By doing these, I could identify the teachers‟ beliefs related to (1) the role of English as the international language, (2) the best English variety, (3) the use of students‟ mother tongue in the EFL classrooms, and (4) the best English language teacher. I could also find out the participants‟ opinions on how to implement EIL pedagogy in their classrooms. Findings and Discussion The findings of this study would be presented in accordance with the research questions. Thus I would start with the findings on pre-service teachers‟ initial beliefs related to the following areas: (1) the role of EIL, (2) the best English variety, (3) the use of students‟ mother tongue in the EFL classrooms, and (4) the best English language teacher. For each major area, there would also be some discussions on the participants‟ (un)shifting beliefs and proposed pedagogical implementations. All the names mentioned below were pseudonyms.

×