1. There are challenges in testing productive language skills like writing and speaking, such as eliciting representative samples of ability and obtaining reliable scores. 2. These problems can be addressed by setting appropriate tasks, restricting responses, selecting test formats, and developing reliable scoring criteria like analytic scales that separate different aspects of ability. 3. When developing tests, it is important to consider the purpose of the test and whether a holistic or analytic approach would be most appropriate and useful.

1. Testing Writing and Speaking Jonathan Magdalena

2. What are we trying to MEASURE ? The student’s ability to interact in the target language in written or spoken form. This involves comprehension as well as production . (Hughes, 1991)

3. Why is it so difficult to test productive skills ? Difficult to elicit behavior in early stages Tasks should elicit a representative sample of the student’s ability Formal formats tend to elicit unreal behavior Teachers tend to score by impressions Scoring criteria is often inconsistent among testers

4. We can mend these problems by… Setting appropriate tasks Testing only writing or speaking abilities, and nothing else Restricting the student’s response Selecting appropriate test formats Obtaining reliable scores

5. Setting appropriate tasks means… Considering questions like these: What is the communicative function I want to elicit from the student? (expressing… giving… describing…) What is the form of interaction expected? (letter, postcard, email, voice chat, interview, oral presentation) What is the context for the interaction? (formal, informal, real, unreal, school context, the street, home, abroad…)

6. Obtaining a representative sample means… Setting a flexible context Considering a variety of tasks Asking for authentic and meaningful answers Setting the criteria for production (1 paragraph, the following aspects…) Giving the student enough time to complete the task.

7. Restricting the student’s response means… Defining tasks in detail (telling exactly what you expect from the students) Establishing boundaries (controlling ideas that will stop students from going too far astray) Specifying the minimum requirements for production Provide information or cues in the form of notes, examples or pictures

8. Testing the skill only means… Avoiding the assessment of… Opinions IQ Creativity or imagination Previous/general knowledge Other skills (reading, listening…) E.g. Read and explain this quote:“Bigotry is the sin which most harms the sinner”

9. Test formats for Speaking Oral questionnaire Free interviews Peer-interaction/role-plays Picture-cued tasks (Describing a picture, structuring a story, giving directions, giving instructions…) Response to recordings Paraphrasing a story, a situation… Prepared monologue * Reading aloud *

10. Test formats for Writing Essay questions Guided writing (a letter, postcard, email, note, form, advertisement…) Picture-cued tasks (Describing a picture, structuring a story, giving directions, giving instructions…) Response to written/spoken stimuli Paraphrasing a story, a situation… Prepared written production * Dictation*

11. Obtaining reliable scores means… Having a well-structured criteria based on bands and descriptors 5 Excellent writer 4 Good writer 3 Average writer 2 Substandard writer 1 Poor writer

12. However, one question arises… What makes a student an ' excellent ' writer? Is it comprehensibility ? Is it grammatical accuracy ? Is it spelling ? Is it the way they organize the text?

13. One solution is… Either produce overall descriptions of writing and spoken ability… Or concentrate on different aspects of writing and speaking and then separate these aspects out into individual scales. Holistic scales : those containing different features of ability Analytic scales : Those which separate out different aspects of ability

14. Holistic Scales 5 Constructs grammatically correct sentences and phrases and shows full mastery of appropriate vocabulary. Text is organized coherently. Correct spelling at all times. Excellent content and presentation. Message wholly relevant 4 In general, grammatically correct sentences and phrases but some errors which do not affect understanding. Does not use. appropriate vocabulary at all times. Some difficulties with organization of text. Some errors in spelling. Good content and presentation. Message mostly relevant. 3 Grammatical errors and use of vocabulary affects understanding as does organization of text. Many errors in spelling. Satisfactory content and presentation. Message not always relevant. 2 Text understood with difficulty due to inaccurate grammar and inappropriate use of vocabulary. Poor content and presentation. Message generally lacks relevancy. . 1 Impossible to understand text due to frequency of grammatical errors and incoherence of organization. Poor spelling Message irrelevant.

15. Why using Holistic Scales? They are relatively easy to construct Scoring takes less time than with the analytic method The integration of sub-skills tends concentrates on the overall effect of the piece of writing Each work can be assessed more than once More bands can be added depending on the context and the purpose of the tester

16. Analytic Scales Grammar 6. Few (if any) noticeable error of grammar or word order. 5. Some errors of grammar and word order which do not interfere with comprehension. 4. Errors of grammar or word order fairly frequent; occasional re-reading necessary for full comprehension. 3. Errors of grammar or word order frequent; efforts of interpretation sometimes required on reader’s part 2. Errors of grammar and word order very frequent; reader often has to rely on own interpretation. 1. Errors of grammar or word order so severe as to make comprehension virtually impossible. Score : Gramm____ Voc____ Mech____ Form____ TOTAL ____

17. Why using Analytic Scales? They consider each student’s particular differences in terms of sub-skills Testers need to consider the different aspects of performance individually The sole fact that the scorer needs to give a number of individual marks makes the scoring more reliable Each aspect can have a different weight depending on the purpose of the tester Some sub-skills might not fit together in a single band

18. Holistic or Analytic Scales? What is the purpose of your test? Are you looking for diagnostic information? Do you want to assess overall performance? Is it for a small and homogeneous group? Is it for a large heterogeneous group? Will it be carried by yourself? Will it be applied in several different places?