Successfully reported this slideshow.

Testing Productive Skills


Published on

Published in: Education
  • it was great
    i really used it
    Are you sure you want to  Yes  No
    Your message goes here

Testing Productive Skills

  1. 1. Testing Writing and Speaking Jonathan Magdalena
  2. 2. What are we trying to MEASURE ? <ul><li>The student’s ability to interact </li></ul><ul><li>in the target language </li></ul><ul><li>in written or spoken form. </li></ul><ul><li>This involves </li></ul><ul><li>comprehension as well as </li></ul><ul><li>production . </li></ul><ul><li>(Hughes, 1991) </li></ul>
  3. 3. Why is it so difficult to test productive skills ? <ul><li>Difficult to elicit behavior in early stages </li></ul><ul><li>Tasks should elicit a representative sample of the student’s ability </li></ul><ul><li>Formal formats tend to elicit unreal behavior </li></ul><ul><li>Teachers tend to score by impressions </li></ul><ul><li>Scoring criteria is often inconsistent among testers </li></ul>
  4. 4. We can mend these problems by… <ul><li>Setting appropriate tasks </li></ul><ul><li>Testing only writing or speaking abilities, and nothing else </li></ul><ul><li>Restricting the student’s response </li></ul><ul><li>Selecting appropriate test formats </li></ul><ul><li>Obtaining reliable scores </li></ul>
  5. 5. Setting appropriate tasks means… <ul><li>Considering questions like these: </li></ul><ul><li>What is the communicative function I want to elicit from the student? (expressing… giving… describing…) </li></ul><ul><li>What is the form of interaction expected? (letter, postcard, email, voice chat, interview, oral presentation) </li></ul><ul><li>What is the context for the interaction? (formal, informal, real, unreal, school context, the street, home, abroad…) </li></ul>
  6. 6. Obtaining a representative sample means… <ul><li>Setting a flexible context </li></ul><ul><li>Considering a variety of tasks </li></ul><ul><li>Asking for authentic and meaningful answers </li></ul><ul><li>Setting the criteria for production (1 paragraph, the following aspects…) </li></ul><ul><li>Giving the student enough time to complete the task. </li></ul>
  7. 7. Restricting the student’s response means… <ul><li>Defining tasks in detail (telling exactly what you expect from the students) </li></ul><ul><li>Establishing boundaries (controlling ideas that will stop students from going too far astray) </li></ul><ul><li>Specifying the minimum requirements for production </li></ul><ul><li>Provide information or cues in the form of notes, examples or pictures </li></ul>
  8. 8. Testing the skill only means… <ul><li>Avoiding the assessment of… </li></ul><ul><li>Opinions </li></ul><ul><li>IQ </li></ul><ul><li>Creativity or imagination </li></ul><ul><li>Previous/general knowledge </li></ul><ul><li>Other skills (reading, listening…) </li></ul><ul><li>E.g. Read and explain this quote:“Bigotry is the sin which most harms the sinner” </li></ul>
  9. 9. Test formats for Speaking <ul><li>Oral questionnaire </li></ul><ul><li>Free interviews </li></ul><ul><li>Peer-interaction/role-plays </li></ul><ul><li>Picture-cued tasks (Describing a picture, structuring a story, giving directions, giving instructions…) </li></ul><ul><li>Response to recordings </li></ul><ul><li>Paraphrasing a story, a situation… </li></ul><ul><li>Prepared monologue * </li></ul><ul><li>Reading aloud * </li></ul>
  10. 10. Test formats for Writing <ul><li>Essay questions </li></ul><ul><li>Guided writing (a letter, postcard, email, note, form, advertisement…) </li></ul><ul><li>Picture-cued tasks (Describing a picture, structuring a story, giving directions, giving instructions…) </li></ul><ul><li>Response to written/spoken stimuli </li></ul><ul><li>Paraphrasing a story, a situation… </li></ul><ul><li>Prepared written production * </li></ul><ul><li>Dictation* </li></ul>
  11. 11. Obtaining reliable scores means… <ul><li>Having a well-structured criteria based on bands and descriptors </li></ul><ul><li>5 Excellent writer </li></ul><ul><li>4 Good writer </li></ul><ul><li>3 Average writer </li></ul><ul><li>2 Substandard writer </li></ul><ul><li>1 Poor writer </li></ul>
  12. 12. However, one question arises… <ul><li>What makes a student an ' excellent ' writer? </li></ul><ul><li>Is it comprehensibility ? </li></ul><ul><li>Is it grammatical accuracy ? </li></ul><ul><li>Is it spelling ? </li></ul><ul><li>Is it the way they organize the text? </li></ul>
  13. 13. One solution is… <ul><li>Either produce overall descriptions of writing and spoken ability… </li></ul><ul><li>Or concentrate on different aspects of writing and speaking and then separate these aspects out into individual scales. </li></ul><ul><li>Holistic scales : those containing different features of ability </li></ul><ul><li>Analytic scales : Those which separate out different aspects of ability </li></ul>
  14. 14. Holistic Scales <ul><li>5 Constructs grammatically correct sentences and phrases and shows full mastery of appropriate vocabulary. Text is organized coherently. Correct spelling at all times. Excellent content and presentation. Message wholly relevant </li></ul><ul><li>4 In general, grammatically correct sentences and phrases but some errors which do not affect understanding. Does not use. appropriate vocabulary at all times. Some difficulties with organization of text. Some errors in spelling. Good content and presentation. Message mostly relevant. </li></ul><ul><li>3 Grammatical errors and use of vocabulary affects understanding as does organization of text. Many errors in spelling. Satisfactory content and presentation. Message not always relevant. </li></ul><ul><li>2 Text understood with difficulty due to inaccurate grammar and inappropriate use of vocabulary. Poor content and presentation. Message generally lacks relevancy. . </li></ul><ul><li>1 Impossible to understand text due to frequency of grammatical errors and incoherence of organization. Poor spelling Message irrelevant. </li></ul>
  15. 15. Why using Holistic Scales? <ul><li>They are relatively easy to construct </li></ul><ul><li>Scoring takes less time than with the analytic method </li></ul><ul><li>The integration of sub-skills tends concentrates on the overall effect of the piece of writing </li></ul><ul><li>Each work can be assessed more than once </li></ul><ul><li>More bands can be added depending on the context and the purpose of the tester </li></ul>
  16. 16. Analytic Scales <ul><li>Grammar </li></ul><ul><li>6. Few (if any) noticeable error of grammar or word order. </li></ul><ul><li>5. Some errors of grammar and word order which do not interfere with comprehension. </li></ul><ul><li>4. Errors of grammar or word order fairly frequent; occasional re-reading necessary for full comprehension. </li></ul><ul><li>3. Errors of grammar or word order frequent; efforts of interpretation sometimes required on reader’s part </li></ul><ul><li>2. Errors of grammar and word order very frequent; reader often has to rely on own interpretation. </li></ul><ul><li>1. Errors of grammar or word order so severe as to make comprehension virtually impossible. </li></ul><ul><li>Score : Gramm____ Voc____ Mech____ Form____ TOTAL ____ </li></ul>
  17. 17. Why using Analytic Scales? <ul><li>They consider each student’s particular differences in terms of sub-skills </li></ul><ul><li>Testers need to consider the different aspects of performance individually </li></ul><ul><li>The sole fact that the scorer needs to give a number of individual marks makes the scoring more reliable </li></ul><ul><li>Each aspect can have a different weight depending on the purpose of the tester </li></ul><ul><li>Some sub-skills might not fit together in a single band </li></ul>
  18. 18. Holistic or Analytic Scales? <ul><li>What is the purpose of your test? </li></ul><ul><li>Are you looking for diagnostic information? </li></ul><ul><li>Do you want to assess overall performance? </li></ul><ul><li>Is it for a small and homogeneous group? </li></ul><ul><li>Is it for a large heterogeneous group? </li></ul><ul><li>Will it be carried by yourself? </li></ul><ul><li>Will it be applied in several different places? </li></ul>