Downloaded 159 times
Uploaded byMaury Martinez
Testing writing
This document discusses issues related to testing writing ability. It emphasizes the importance of ensuring tests are representative, valid, and reliably scored. Representative tasks should sample all possible content domains. Multiple tasks increase validity by providing more opportunities to demonstrate skills. Scoring should be either holistic, considering overall impression, or analytic, separating different skill aspects. Proper scorer training and calibration are needed to reliably rate performance. The goals are to accurately measure writing as an independent skill while maintaining practicality and meaningfulness.
More Related Content
Similar to Testing writing
More from Maury Martinez
Testing writing
- 1. TESTING WRITING CarolinaSalazar Testing and Evaluation
- 2. THE TESTING PROBLEM REPRESENTATIVEVALIDLY SAMPLED VALIDLY AND RELIABLY SCORED REPRESENTATIVE
- 3.
- 4. SPECIFICATION OF ALLTHE POSSIBLE CONTENT Operations Types of text Adressees Length of texts Topics Dialect Style Expressing, directing, describing… Form, letter, fax, note… Boss, friends, family… 1 page, 150 words… Not specialist, relevant… Standard Formal, informal…
- 5. REPRESENTATIVE SAMPLE OFTHE SPECIFIED CONTENT Try to include a representative sample of tasks MORE TASKS MORE VALIDITY
- 6.
- 7. CCSE (CAMBRIDGE CERTIFICATESIN COMMUNICATIVE SKILLS IN ENGLISH) FIRST STAGE: READING TOPIC: Working in a Summer Camp for Children in America
- 8. TASK 1 Write aletter to American Summer Camps. Find out about Dates Hours of work Type of accomodation Ask for the application form
- 9. TASK 2 Fill inthe application form.
- 10. TASK 3 You arenow working in the ASCFC. You write a postcard to an English-speaking friend On your postcard tell your friend: Where you are Why you are there Two things you like about the summer camp
- 11. TASK 4 You havearranged to go out tonight with two other Helpers at the ASCFC in Florida. You have to change your plans suddenly, and cannot meet them. Leave them a note. Apologize and explain why you cannot meet them Suggest a different day to go out
- 12.
- 13. THE TESTING PROBLEM REPRESENTATIVEVALIDLY SAMPLED VALIDLY AND RELIABLY SCORED VALIDLY SAMPLED
- 14. VALID SAMPLE OFWRITING ABILITY
- 15. SET AS MANYSEPARATE TASKS AS FEASIBLE People’s performance Consistent Offer them more opportunities to start Desirable Practical
- 16. TEST ONLY WRITING,NOTHING ELSE NO YES
- 17. TEST ONLY WRITING,NOTHING ELSE READING ILLUSTRATIONS SERIES OF PICTURES GRAPHICS – CHARTS NOTES
- 18. THE TESTING PROBLEM REPRESENTATIVEVALIDLY SAMPLED VALIDLY AND RELIABLY SCORED VALIDLY AND RELIABLY SCORED
- 19. VALID AND RELIABLESCORING
- 20. ENSURE VALID ANDRELIABLE SCORING Set tasks which can be reliably scored Set as many tasks as posible Restrict candidates Give no choice of tasks Ensure long enough samples Create appropriate scales for scoring
- 21. HOLISTIC SCORING “Impressionistic” Overall impression Purposeof the test PRO Very rapid CON Scorer reliability? http://www.ets.org/toefl/pbt/scores/writing_score_guide/
- 22. ANALYTIC SCORING http://www.ode.state.or.us/wma/teachlearn/testing/scoring/gui des/2011-12/wriscorguide_eng_no-dates.pdf Separate score– separate aspects PROS Uneven development of subskills Scorers consider details of performance More reliable CONS Time consuming Too specific (The whole is greater than the sum of its parts)
- 23. HOLISTIC OR ANALYTIC? PURPOSECIRCUMSTANCES ACCURACY RESOURCES DECISIONMULTIPLE SCORING
- 24. MORE CONSIDERATIONS ONSCORING
- 25. SOME CONSIDERATIONS Calibrate thescale Collect samples of performance Cover the range of scales Select and train scorers Native speakers Sensitive to language Experienced Training (3 stages) Follow acceptable scoring procedures Score each task independently Several scorers Identify discrepancies Give feedback Useful – positive Decide the content during callibration
- 26.
- 27. CONCLUSIONS Pay attention tovalidity and reliability Give importance to writing as an independent skill Always be meaningful and enhance real communication
- 28.