The document discusses key concepts for valid and reliable testing of reading and listening skills for language learners. It outlines important considerations for test design including using authentic texts, clear instructions, and ensuring each skill is adequately represented. It also describes stages of reading and listening like prediction, scanning, and interpretation. Different task types are explained as being more or less controlled. Validity and reliability are important to ensure tests accurately measure the intended skills.

1. Prof. Victor Ojeda - 2011 UNIVERSIDAD PEDAGÓGICA EXPERIMENTAL LIBERTADOR INSTITUTO PEDAGÓGICO DE CARACAS DEPARTAMENTO DE IDIOMAS MODERNOS PROGRAMA DE INGLÉS CÁTEDRA DE LINGÚÍSTICA Testing receptive skills: reading & listening

2. Backwash (washback) Validity construct content face internal external Reliability internal external Practicality Impact Norm-referenced vs. criterion-referenced Some important terms Testing reading and listening

3. In order for tests to be valid, these must have some communicative intend in the way they elicit language performance. Even the most discrete-point items can be given some communicative intend if they are presented in a context or related to one another instead of being presented as a list of separated, isolated and unnatural sentences or list of words. In tasks, it is important too to provide the test taker with a purpose and a context. Some important terms Testing reading and listening

4. In both listening and reading, a key element in authenticity is that oral and written texts are taken from real sources (newspaper, radio shows, advertising, television, etc.). If the texts are not genuine, they should at least display natural or at least quasi realistic language and not some artificial modified form unlikely to be heard anywhere in the real world. Some important terms Testing reading and listening

5. The instructions of each part of the test must be simple, clear and straightforward. The wording must not be complicated so that the test-taker is unable to complete a task/item not because of lack of ability, but because of not understanding what he/she is supposed to do. On the other hand, instructions should not give away the right answer in advance because of over explanation or exemplification. It is also crucial that the test be carefully reviewed and checked by more than one teacher before it is administered. A colleague might see flaws in the test that the test-writer might have not noticed. This does not only help ascertain validity but also reliability. Typos and other mistakes in instructions or stems might affect students' performance. Some important terms Testing reading and listening

6. Deciding the appropriate weight might depend on the difficulty of the task/items of each part or on the overall importance of the objectives assessed in each part. As a thumb-rule, each skill should have at least a part to represent it and each of these parts should have equal weighting. However, if within the specific context of instruction there has been an overt emphasis on a skill over the others, it is fair that part has a major weight than the others. For each part there should be enough items or a task or set of tasks that are representative of the skill/component. It is also recommended that there is some relationship among the parts. For instance, the reading and the listening parts might be based on two texts (an oral and a written one) about the same topic. The writing component might imply the test-taker to write about that very topic. Some important terms Testing reading and listening

7. Hughes (1989, p. 1) states simply that "the effect of testing on teaching and learning is known as backwash" (and this term, as he uses it, is synonymous to washback) . What is the backwash effect? Testing reading and listening

8. The ultimate goal language is determine if students have developed their reading skills to cope with the same kind of texts that are encountered by native speakers of the target language. Assessing reading Testing reading and listening

9. Authentic reasons for reading Testing reading and listening Tracy is looking for a number in a telephone directory. Rachel is reading the instructions on how to send a mail. Paul is writing his university dissertation and is looking for a quote he needs. Nicole is reading the directions of how to get to her holiday destination. Maria is flicking through a magazine while she waits for the dentist. Steven is looking through the ads because he wants to buy a car. Belinda is reading Twilight . Richard is in a bookshop, looking for something to read during his flight.

10. Reading stages Testing reading and listening Pre-reading While reading Post reading Prediction Skimming text exploration prior knowledge

11. Reading stages Testing reading and listening Pre-reading While reading Post reading Predicting Scanning Sequencing Locating misplaced information Identifying patterns of organization locating markers Finding examples Identifying definitions Interpreting main ideas Summarizing

12. Reading stages Testing reading and listening Pre-reading While reading Post reading asking questions filling in tables, graphs, charts matching locating main idea matching referents to references. Inferences Other view points Concluding a text Matching implicit and explicit information Changing genre Evaluation Distinguishing facts from opinions Note bias Identify author’s view point Literary analysis Summarizing (text or cloze) Text mapping

13. Assessing listening Testing reading and listening The main objective is to check if students are prepared to function successfully in real-life listening situations (interviews, instructions, loudspeaker announcements, conversation, watching television, etc.)

14. Answer these questions Testing reading and listening 1. Which would be harder, listening to the news on the radio or watching it on television? Why? 2. Which would be harder, talking to a friend on the telephone or face to face? Why? 3. Which would be harder, talking to a friend or to a group of friends? Why? 4. How is watching a movie different from watching 10 minutes of daytime television? 5. How is listening to a lecture different from listening to music on a CD? 6. How is listening to a lecture different from taking part in a small discussion group?

15. Listening stages Testing reading and listening Pre-listening While listening Post-listening Setting the context Activating current (previous) knowledge Acquiring knowledge - back-wash effect Activating vocabulary / language Predicting content Pre-learning vocabulary Quotes Pictures, maps, graphs, diagrams Grammar or vocabulary review (in a context) Predicting Band, author, artist, topic information with tasks Jokes, riddles, puzzles, cartoons Semantic maps

16. Listening stages Testing reading and listening Questions: direct, short and long answers Table completion (grid) Diagrams, maps, pictures (labeling and tagging] True, false, not mentioned and wrong answers corrected Listing Multiple choice (a,b,c,d) Underlying the correct option Sequencing (words, sentences, texts, pictures) Text completion (missing ideas or titles) Cloze task C-test Gap-fill (with options) Problem solving (guessing, predicting) Image interpretation (sound off) Paraphrasing, rephrasing or finding original won Checking (ticking) Pre-listening While listening Post-listening

17. Listening stages cont… Testing reading and listening Interpretation (by his look, we can say he is...) Word attack (meaning from context) Identify topic or main idea (text, paragraph) Linking (references) Matching (columns, words, meanings, pieces of sentences) Discrepancies (identifying, correcting) « Matching words with definitions Recognize and identify Following instructions (directions) Sound discrimination (segmentals and supra segmentals) Distinguishing between formal and informal registers Pre-listening While listening Post-listening

18. Characteristics of the tasks Testing reading and listening Controlled or close Cued or semi controlled Free or open Warm-up Setting Organizational Content explanation Role-play demonstration Dialog/Narrative presentation Dialog/Narrative recitation Reading aloud Checking Question-answer, display Drill Translation Dictation Copying Identification Recognition

19. Characteristics of the tasks Testing reading and listening Controlled or close Cued or semi controlled Free or open Brainstorming Story-telling Question-answer, referential Cued narrative Dialog Information transfer Information exchange Wrap-up Narration/exposition Preparation

20. Characteristics of the tasks Testing reading and listening Controlled or close Cued or semi controlled Free or open Role-play Games Report Problem solving Drama Simulation Interview Discussion Composition

21. Many procedures and techniques employed in assessing reading can be and have been adapted to assess listening as well. However, the cognitive demand has to be taken into account. Other tasks that assess listening involve note-taking and dictation. Other techniques that aim at testing or assessing speaking also include a listening component (such as role-plays, open dialogues and interviews). Scoring scales for such tasks should include a scale or parameter for assessing comprehension since it is vital for the learner to produce a correct response. Characteristics of the tasks Testing reading and listening

22. What makes a listening difficult Testing reading and listening Clustering Redundancy Reduced forms Performance Colloquial language Rate Stress, rhythm and intonation Interaction not being able to go back too fast noise accents conversations are harder meaning of unknown words

23. References Testing reading and listening Brown, H.D. (2001). Teaching by principles. An Interactive approach to language pedagogy. London: Longman Harmer, J. (2001). The practice of english langauge teaching. Harlow: Longman. Harris, D. (1969). Testing english as a second language. New York: McGraw-Hill Richards, J. and Renandya, W. (2002). Methodology in Language teaching. Cambridge: Cambridge University Press. Peachy, Nik, 2002. A framework for planning a listening skills lesson. Available at: http://www.teachingenglish.org.uk/think/articles/a-framework-planning-a-listening- skills-lesson. NA. , 2002 Pre-listening activities. Available at: http://www.teachingenglish.org.uk/think/articles/pre-listening-activities Mikulecky, B. and Linda Jeffries (2004). More Reading Power. London: Longman Brown, D. (2001). Teaching by principles. An interactive approach to language pedagogy. New York: Longman. Nunan, D. (1999). Second language learning and teaching. Boston: Heinle and Heinle Publishers. Omaggio, A. (1986). Teaching language in context. Boston: Heinle and Heinle Publishers. Richards, J. Platt, J. and Platt, H. (1992) Dictionary of language teaching and applied linguistics. Essex: Longman. Ur, P. (1996) A course in language teaching, practice and theory. Cambridge: Cambridge University Press.