Communicative testing written and oral-cccn--wb

2,760 views

Published on

Talk given at the CCCN, Costa Rica, June 24, 2010

Published in: Education, Technology
  • Be the first to comment

Communicative testing written and oral-cccn--wb

  1. 1. Communicative Testing JoAnn Miller Macmillan Publishers [email_address]
  2. 2. Kinds of Testing <ul><li>Placement tests </li></ul><ul><ul><li>student’s suitability to take a specific course </li></ul></ul><ul><ul><li>based on specific textbook </li></ul></ul><ul><li>Proficiency tests </li></ul><ul><ul><li>check students’ progress in general </li></ul></ul><ul><ul><li>TOEFL, First Certificate, etc. </li></ul></ul><ul><li>Achievement tests </li></ul><ul><ul><li>check how much a student has learned </li></ul></ul><ul><ul><li>based on what a student has studied in a specific course </li></ul></ul>
  3. 3. What are your exams going to be like? <ul><li>Many variables: </li></ul><ul><ul><li>Institution </li></ul></ul><ul><ul><li>Students </li></ul></ul><ul><ul><li>Teachers </li></ul></ul><ul><ul><li>Text </li></ul></ul><ul><ul><li>Time </li></ul></ul>
  4. 4. Teachers <ul><li>Level of English </li></ul><ul><li>Lack of mathematical skills </li></ul><ul><li>Time factor </li></ul><ul><li>Ease of grading </li></ul><ul><li>Answer key </li></ul>
  5. 5. Students <ul><li>Younger students </li></ul><ul><ul><li>More images </li></ul></ul><ul><ul><li>Shorter exams </li></ul></ul><ul><ul><li>Humor </li></ul></ul><ul><li>Older students </li></ul><ul><ul><li>Professionalism </li></ul></ul><ul><ul><li>Humor </li></ul></ul>
  6. 6. How many points for each skill? <ul><li>If institution tells you, just follow through </li></ul><ul><li>If not, base exam on the textbook (the common denominator) </li></ul><ul><ul><li>General text analysis </li></ul></ul><ul><ul><li>How much time is spent on each skill </li></ul></ul><ul><ul><li>Count exercises in a few units, determine percent </li></ul></ul><ul><li>Keep institutional goals in mind </li></ul>
  7. 7. Content Validity <ul><li>Assessment should be based on a content-analysis of the text being used </li></ul>
  8. 8. Content Analysis <ul><li>You must test only material students have seen </li></ul><ul><li>The only common denominator is the textbook </li></ul><ul><li>Analysis of percent of time spent on each topic (grammar structure, vocabulary item, function, etc.) </li></ul>
  9. 9. Content Analysis: Information from the contents Functions (10 points): Talking about imitation products Talking about food and food festivals Discussing the movie industry Making a business plan Grammar (5 points): Nouns in groups Indefinite Pronouns Vocabulary (10 points): Food Business language ///// /// ///// ///// ///// //// ///// /// ///// ///// /// 8 10 9 5 32 3 5 8 5 3 8 8 / 32 =__% 25% 31% 28% 16% 25% X 10 = ___pts 2.5 pts 3 pts 3 pts 1.5 pts 37% 2 pts 63% 3 pts 63% 6 pts 37% 4 pts
  10. 11. Communicative testing <ul><li>We teach “communicatively” but we test “traditionally”. </li></ul><ul><li>What IS communicative testing? </li></ul><ul><li>Communicative testing means testing in context . </li></ul>
  11. 12. Grammar? What will you test?
  12. 13. Which version? Why? <ul><li>Circle the correct answer </li></ul><ul><li>1. Do you like __________? </li></ul><ul><li>swimming b. to swum c. swim </li></ul><ul><li>2. Where ________ live? </li></ul><ul><li>does she b. she does c. she </li></ul><ul><li>3. I _________ speak French. </li></ul><ul><li>no speak b. doesn’t c. don’t </li></ul><ul><li>4. What __________? </li></ul><ul><li>a. does he do b. does he c. he does do </li></ul><ul><li>Write the correct forms of the words in parentheses. </li></ul><ul><li>Alice: Where (1)______ you </li></ul><ul><li>_________ (live)? </li></ul><ul><li>Bart: Acapulco. </li></ul><ul><li>Alice: My brother (2)___________ (go) </li></ul><ul><li>there every summer on vacation, </li></ul><ul><li>but he (3)_________(not speak) </li></ul><ul><li>Spanish. </li></ul><ul><li>Bart: Acapulco (4)_________ (attract) </li></ul><ul><li>tourists from all over the world. </li></ul><ul><li>Many people there </li></ul><ul><li>(5)___________(speak) English </li></ul><ul><li>very well. What about you, </li></ul><ul><li>(6)______ you ________(speak) </li></ul><ul><li>Spanish? </li></ul><ul><li>Alice: A little. </li></ul>
  13. 14. Vocabulary? What will you test?
  14. 15. Which version? Why? <ul><li>Match the letters (a to e) with the numbers (1 to 5). </li></ul><ul><li>1. Your mother’s husband is your___. </li></ul><ul><li>2. Your mother’s father is your___. </li></ul><ul><li>3. Your mother’s brother is your___. </li></ul><ul><li>4. Your uncle’s son is your___. </li></ul><ul><li>5. Your father’s sister is your ___. </li></ul><ul><li>uncle </li></ul><ul><li>cousin </li></ul><ul><li>aunt </li></ul><ul><li>father </li></ul><ul><li>grandfather </li></ul><ul><li>Underline the word in each pair that completes the conversation correctly. </li></ul><ul><li>My (1)[ uncle / aunt ] likes (2) [ playing / going to ] movies. He is my father’s (3)[ sister / brother] . He’s (4)[ heavy / average ] and he has (5)[ blue / brown ] hair. His birthday is on October (6)[twelve / twelfth ] . </li></ul>
  15. 16. Functions? What will you test?
  16. 17. What is a function? <ul><li>The communicative purpose of the users of the language. </li></ul><ul><li>How language is used. </li></ul><ul><li>Usually expressed as gerunds: introducing, apologizing, asking directions, requesting </li></ul>
  17. 18. Examples of a Functional Cycle <ul><li>Function: Requesting </li></ul><ul><li>(1) Open the window, please. </li></ul><ul><li>(2) Would you open the window? </li></ul><ul><li>(3) Could you please open the window? </li></ul><ul><li>(4) Would you mind opening the window? </li></ul><ul><li>(5) I was wondering if you would mind opening the window. </li></ul><ul><li>(6) I’d be grateful if you opened the window. </li></ul><ul><li>Each time the difference in register is emphasized. </li></ul>
  18. 19. Where can you find them in your textbook?
  19. 20. How to test? Complete the conversation. <ul><li>Complete the conversation logically. Use the words in parentheses. </li></ul><ul><li>Miriam: Tell me about your new apartment. </li></ul><ul><li>Mary: (1)____________________(big / living room). </li></ul><ul><li>Miriam: (2)___________________(how / bedrooms)? </li></ul><ul><li>Mary: There are two, but (3)________(any furniture) </li></ul><ul><li>in one of them. </li></ul>Or: Miriam: Tell me about your new apartment. Mary: (1)____________________(living room). Or: Miriam: Tell me about your new apartment. Mary: (1)___________________________.
  20. 21. Practice vs Testing In class practice During exam Goals Content Learner activity Teacher activity Class-room climate learning feedback on learning process oriented product oriented open ended close ended ss know material students might not know success-oriented success/failure oriented peer teaching no peer teaching helps performance gives tasks cooperative competitive relaxed tense intrinsic motivation extrinsic motivation
  21. 22. Balance <ul><li>Ideally an exam will balance: </li></ul><ul><ul><li>Accuracy and fluency </li></ul></ul><ul><ul><li>Production and recognition </li></ul></ul><ul><ul><li>Objective and subjective sections </li></ul></ul>
  22. 23. Accuracy and Fluency <ul><li>Fluency </li></ul><ul><ul><li>The ability to produce written and / or spoken language with ease </li></ul></ul><ul><ul><li>Communicate ideas effectively </li></ul></ul><ul><ul><li>The ability to use vocabulary chunks (phrases) to facilitate communication </li></ul></ul><ul><li>Accuracy </li></ul><ul><ul><li>Ability to produce grammatically correct sentences </li></ul></ul>
  23. 24. Production and Recognition <ul><li>Production </li></ul><ul><ul><li>Student writes more than one word </li></ul></ul><ul><ul><li>Can be creative / involves more “mental” work </li></ul></ul><ul><ul><li>More than one answer may be possible </li></ul></ul><ul><li>Recognition </li></ul><ul><ul><li>Student recognizes correct answer </li></ul></ul><ul><ul><li>Not creative </li></ul></ul><ul><ul><li>Only one correct answer </li></ul></ul>
  24. 25. Objective and Subjective Sections <ul><li>Objective </li></ul><ul><ul><li>There is only one answer </li></ul></ul><ul><ul><li>Anyone can correct the exam </li></ul></ul><ul><ul><li>No surprises </li></ul></ul><ul><ul><li>No argument from students </li></ul></ul><ul><li>Subjective </li></ul><ul><ul><li>There is more than one possible answer </li></ul></ul><ul><ul><li>Corrector must be trained and experienced </li></ul></ul><ul><ul><li>There can be surprises </li></ul></ul><ul><ul><li>Students can protest grading </li></ul></ul>
  25. 26. <ul><li>Grammar / Vocabulary / Functions? </li></ul><ul><li>Accuracy or fluency? </li></ul><ul><li>Production or recognition? </li></ul><ul><li>Subjective or objective? </li></ul>
  26. 27. <ul><li>Grammar / Vocabulary / Functions? </li></ul><ul><li>Accuracy or fluency? </li></ul><ul><li>Production or recognition? </li></ul><ul><li>Subjective or objective? </li></ul>
  27. 28. <ul><li>Grammar / Vocabulary / Functions? </li></ul><ul><li>Accuracy or fluency? </li></ul><ul><li>Production or recognition? </li></ul><ul><li>Subjective or objective? </li></ul>
  28. 29. <ul><li>Grammar / Vocabulary / Functions? </li></ul><ul><li>Accuracy or fluency? </li></ul><ul><li>Production or recognition? </li></ul><ul><li>Subjective or objective? </li></ul>
  29. 30. <ul><li>Grammar / Vocabulary / Functions? </li></ul><ul><li>Accuracy or fluency? </li></ul><ul><li>Production or recognition? </li></ul><ul><li>Subjective or objective? </li></ul>
  30. 31. <ul><li>Grammar / Vocabulary / Functions? </li></ul><ul><li>Accuracy or fluency? </li></ul><ul><li>Production or recognition? </li></ul><ul><li>Subjective or objective? </li></ul>
  31. 32. What is an exam section? <ul><li>A certain number of items testing the same skill / aspect </li></ul><ul><li>To be communicative, they should be written as a conversation, note, letter, or some “real” type of discourse </li></ul><ul><li>All items in a section should be worth the same number of points and test a similar skill (all grammar, all vocabulary, all functions, etc.) </li></ul>
  32. 33. Correcting communicative sections
  33. 34. Correcting <ul><li>Grammar, Reading, Vocabulary, Listening </li></ul><ul><li>In general these sections are all right or all wrong . </li></ul><ul><li>We rarely give partial credit. </li></ul><ul><li>These sections test accuracy . </li></ul><ul><li>Communicative sections </li></ul><ul><li>You can give partial credit </li></ul><ul><li>These sections test fluency. </li></ul><ul><li>Ask yourself if the S’s answer communicates what the S wants to say . </li></ul>
  34. 35. Examples of partial credit <ul><li>Correct answer: What’s your name? </li></ul><ul><li>Student writes: What you name? </li></ul><ul><li>Correct answer: If you invited me, I’d go. </li></ul><ul><li>Student writes: If you invite me, I go. </li></ul><ul><li>Correct answer: I went to the movies yesterday. </li></ul><ul><li>Student writes: I go to the movies yesterday. </li></ul><ul><li> I go to the movies. </li></ul>
  35. 36. IV. The clerk knows Cleopatra. Caesar asks the clerk about Cleo. Complete the conversation. Use the words in parentheses. ( 4 points, .5 each) Clerk : Yes, I know her. Julius : (1) _______________________________________ (work) ? Clerk : (2) ________________________________ ( palace downtown). Julius : (3) _______________________________________ ( do) ? Clerk : (4) ___________________________________ ( help people). Julius : (5) ___________________________________ ( close friend)? Clerk : Yes, (6) ____________________________________ (funny). Julius : (7) _______________________________________ ( sports)? Clerk : Yes, (8) ___________________________________ ( tennis ). Actual Student Responses on the worksheet
  36. 37. Example 1 <ul><li>Clerk : Yes, I know her. </li></ul><ul><li>Julius : (1) Where does she works? </li></ul><ul><li>Clerk : (2) She does work palace downtown . </li></ul><ul><li>Julius : (3) Where do she does? </li></ul><ul><li>Clerk : (4) She does help people. </li></ul><ul><li>Julius : (5) Where does she close friend ? </li></ul><ul><li>Clerk : Yes, (6) she does funny. </li></ul><ul><li>Julius : (7) Where does she lift sports ? </li></ul><ul><li>Clerk : Yes, (8) she does play tennis. </li></ul>
  37. 38. Example 2 <ul><li>Clerk : Yes, I know her. </li></ul><ul><li>Julius : (1) You do? </li></ul><ul><li>Clerk : (2) I’m work in the palace downtown . </li></ul><ul><li>Julius : (3) What do you do? </li></ul><ul><li>Clerk : (4) I’m help help people. </li></ul><ul><li>Julius : (5) How you close friend ? </li></ul><ul><li>Clerk : Yes, (6) they are funny </li></ul><ul><li>Julius : (7) ar you play sports ? </li></ul><ul><li>Clerk : Yes, (8) I’m play. </li></ul>
  38. 39. Example 3 <ul><li>  Clerk : Yes, I know her. </li></ul><ul><li>Julius : (1) Where does she work? </li></ul><ul><li>Clerk : (2) She work at the palace downtown . </li></ul><ul><li>Julius : (3) What does she does? </li></ul><ul><li>Clerk : (4) She helps people. </li></ul><ul><li>Julius : (5) Does she have a close friend ? </li></ul><ul><li>Clerk : Yes, (6) she does. She’s very funny.. </li></ul><ul><li>Julius : (7) Does she like sports ? </li></ul><ul><li>Clerk : Yes, (8) she play tennis. </li></ul>
  39. 40. Example 4 <ul><li>  Clerk : Yes, I know her. </li></ul><ul><li>Julius : (1) What does she works? </li></ul><ul><li>Clerk : (2) She works in palace downtown . </li></ul><ul><li>Julius : (3) What does she do? </li></ul><ul><li>Clerk : (4) Work with help people. </li></ul><ul><li>Julius : (5) Are you close friend ? </li></ul><ul><li>Clerk : Yes, (6) she is funny. </li></ul><ul><li>Julius : (7) Are you practice sports ? </li></ul><ul><li>Clerk : Yes, (8) she plays tennis. </li></ul>
  40. 41. Where written and oral meet <ul><li>Functions and their relation to communication </li></ul><ul><li>Accuracy vs. Fluency </li></ul><ul><li>Grading of communicative written sections </li></ul>
  41. 42. Oral Testing of Accuracy and Fluency
  42. 43. Fluency <ul><li>The ability to produce written and / or spoken language with ease </li></ul><ul><li>Speak with a good but not necessarily perfect command of intonation, vocabulary and grammar </li></ul><ul><li>Communicate ideas effectively </li></ul><ul><li>Produce continuous speech without causing comprehension difficulties or a breakdown in communication </li></ul>Longman Dictionary of Language Teaching and Applied Linguistics
  43. 44. Accuracy <ul><li>Ability to produce grammatically correct sentences </li></ul><ul><li>May not include the ability to speak or write fluently. </li></ul>Longman Dictionary of Language Teaching and Applied Linguistics
  44. 45. An example: foreign visitors <ul><li>Tour of factory—fluency; must be prepared for unexpected questions and explanations </li></ul><ul><li>Formal presentation of a product—accuracy; time to plan, rehearse </li></ul>
  45. 46. What is a task? <ul><li>A goal-oriented activity in which learners use language to achieve a real outcome . </li></ul><ul><li>Learners use whatever target language resources they have in order to solve a problem, do a puzzle, play a game, or share and compare experiences. </li></ul>
  46. 47. Fluency in Tasks <ul><li>“ Learners need opportunities to process language for communicative purposes as receivers and producers. </li></ul><ul><li>“ These opportunities should be unfettered by the perceived need to conform to teacher expectations in terms of the production of specific language forms.” </li></ul>Dave Willis, “Accuracy, fluency and conformity” Challenge and Change in Language Teaching, J. Willis and D. Willis, ed. Heinemann, 1996. P. 50
  47. 48. Accuracy in Tasks <ul><li>“ Whenever learners are involved in communication they are concerned with accuracy…making the best use of their language systems… </li></ul><ul><li>“ In spontaneous communication [they] have little time to reflect on the language they produce. </li></ul><ul><li>“ If…they are given time to prepare what they have to produce, there will be a concern for formal accuracy…” </li></ul>Dave Willis, “Accuracy, fluency and conformity” Challenge and Change in Language Teaching, J. Willis and D. Willis, ed. Heinemann, 1996. P. 50
  48. 49. Implications <ul><li>“ Teachers should balance issues of fluency and accuracy depending on the specific needs of learners and the resources of time and materials for instruction.” </li></ul>Miriam Eisenstein Ebsworth, “Accuracy Vs. Fluency: Which Comes First in ESL Instruction?” , ESL Magazine. 1:2, 24-26. March/April 1998.
  49. 50. The teaching situation <ul><li>Private, multi-level (high school, university, post-grad) university in Mexico (17 campuses throughout country) </li></ul><ul><li>Institutional EFL programs </li></ul><ul><li>Total 35,000+ students </li></ul><ul><li>Groups of 25-30 in language lab </li></ul><ul><li>Native and non-native teachers with varying abilities and experience </li></ul>
  50. 51. Grading <ul><li>Objective grading required as much as possible (by government, school and parents) </li></ul><ul><li>Each course: 2 written exams in classroom (functions, structures, vocabulary, reading, writing); 2 exams in lab (listening comprehension and oral production) </li></ul>
  51. 52. Oral Exam <ul><li>Accuracy </li></ul><ul><li>Role play </li></ul><ul><li>5 minutes+ preparation </li></ul><ul><li>Perform for teacher </li></ul><ul><li>1-2 minutes maximum </li></ul><ul><li>Fluency </li></ul><ul><li>Role play with teacher and another student </li></ul><ul><li>No preparation </li></ul><ul><li>Cues only </li></ul>
  52. 53. Oral Exam Organization <ul><li>Students (pairs) given role cards as they enter classroom </li></ul><ul><li>Told to prepare—when finish exam, can leave </li></ul><ul><li>Can use any reference or ask questions </li></ul><ul><li>Come to the front of the class, talk only to teacher </li></ul><ul><li>Perform </li></ul><ul><li>Either stay together or separate for second role-play </li></ul>
  53. 54. 5 point scale <ul><li>Accuracy </li></ul><ul><li>1 ---Little or no language produced. </li></ul><ul><li>2 --Poor vocabulary, serious mistakes in grammar, poor pronunciation. </li></ul><ul><li>3 --Adequate vocabulary, mistakes in grammar, adequate pronunciation. </li></ul><ul><li>4 --Good vocabulary, occasional errors in grammar, good pronunciation. </li></ul><ul><li>5 --Wide vocabulary, very few errors in grammar, very good pronunciation. </li></ul><ul><li>Fluency </li></ul><ul><li>1 ---Little or no communication. </li></ul><ul><li>2 ---Very hesitant and brief utterances, sometimes difficult to understand. </li></ul><ul><li>3 ---Communicates ideas, but hesitantly and briefly </li></ul><ul><li>4 ---Effective communication, but does not elaborate on response. </li></ul><ul><li>5 ---Easy and efficient communication. Elaborates on responses. </li></ul>
  54. 55. Student A: Imagine you area meeting and not in an exam. All of your classmates are at the meeting too. Your partner doesn’t know anyone. Tell him who the people are. at Student B: Imagine you are at a meeting and not in an exam. All of your classmates are at the meeting too. You don’t know anyone. Ask your partner who the people are. Student A: Tell your partner about an accident you or some member of your family had. When he/she tells you, ask some intelligent questions or make relevant comments. Student B: Tell your partner about an accident you or some member of your family had. When he/she tells you, ask some intelligent questions or make relevant comments. Student A: You are making a survey about what people think they will be able to do with telecommunications in twenty years. Ask your partner at least three questions about the topic. Student B: Your partner is making a survey. Answer his/her questions.
  55. 56. <ul><li>II. Roleplay with the teacher. Use to grade fluency. (5 points) (Do not show these questions to the Ss. ) </li></ul><ul><li>Keep students together. Ask them what they plan to do when they finish school. Then ask them to tell you the pros and cons of that job. </li></ul><ul><li>Separate students for a moment. You are going to give one student a message for the other student. For example, ask Student A to tell Student B you are going to meet him/her after class. Then have the student pass on the message. Make the messages a little bit complicated. When you finish, give Student B messages for Student A. </li></ul><ul><li>3. Separate students. Tell student to imagine his girlfriend / her boyfriend is angry. Ask him / her what he / she will do. Then ask a “what if” question: What if he doesn’t believe you? What if he goes out with someone else?, etc. </li></ul>
  56. 57. <ul><li>II. Oral Exam (10 points) </li></ul><ul><li>Role Play 1: 5 4 3 2 1 0 </li></ul><ul><li>Role Play 2: 5 4 3 2 1 0 </li></ul><ul><li>Total Oral points: ___/10 </li></ul>Part I is the listening comprehension exam. It is on the same page.
  57. 58. Practice <ul><li>The best way to understand what oral testing is all about is to experience it yourselves. </li></ul><ul><li>Get into groups of three. </li></ul>
  58. 59. Formal oral testing <ul><li>common and well-developed in standardized tests </li></ul><ul><li>less common and definitely underdeveloped in classroom testing. </li></ul><ul><li>According to Fulcher, it can cost up to £62,867.79 for a school to develop a good oral testing program. </li></ul>Glenn Fulcher in Testing Second Language Speaking (Pearson/Longman, 2003), pp. 159-160.
  59. 60. Problems with oral testing <ul><li>difficulty in investigating validity and reliability due to the nature of the exams themselves. </li></ul><ul><li>In his book, Fulcher examines various task types and rating scales, but is unable to recommend any of them completely due to the problems inherent in oral testing. </li></ul>
  60. 61. Task types <ul><li>Realistic communication. </li></ul><ul><ul><li>The production = relatively unpredictable; similar to what they do in the textbook. </li></ul></ul><ul><li>  Kind of production wanted </li></ul><ul><ul><li>statements only </li></ul></ul><ul><ul><li>question production </li></ul></ul>
  61. 62. Task Organization <ul><li>Individual </li></ul><ul><li>Pairs </li></ul><ul><li>Groups </li></ul>
  62. 63. Individual Task Types <ul><li>talk about a topic given by examiner (statements only) </li></ul><ul><li>talk about an image (statements only) </li></ul><ul><li>interview the examiner or an imaginary person (question production) </li></ul><ul><li>roleplay with examiner </li></ul>
  63. 64. Pair/Group Tasks Types <ul><li>have a conversation-occasionally role play (realistic communication) </li></ul><ul><li>build a conversation around an image-photo/map (realistic communication) </li></ul><ul><li>survey (question-answer) </li></ul><ul><li>problem solving –often role play (realistic communication) </li></ul><ul><li>interview (question-answer) </li></ul>
  64. 65. Practice <ul><li>Different type of presentation </li></ul><ul><li>Script </li></ul><ul><ul><li>For use with less experienced teachers </li></ul></ul><ul><li>Get into groups of four </li></ul>
  65. 66. Now you work… <ul><li>In pairs: </li></ul><ul><li>Choose a unit in the text. </li></ul><ul><li>Write: </li></ul><ul><ul><li>One individual </li></ul></ul><ul><ul><li>One pair </li></ul></ul><ul><ul><li>One group activity </li></ul></ul>
  66. 67. Grading oral exams <ul><li>Evaluating oral production is difficult for teachers </li></ul><ul><ul><li>insecurity in their own abilities </li></ul></ul><ul><ul><li>lack of time available </li></ul></ul><ul><ul><li>lack of understanding of grading instruments. </li></ul></ul><ul><li>Standardized testing </li></ul><ul><ul><li>intensive examiner training </li></ul></ul><ul><ul><li>not possible in most school scenarios. </li></ul></ul>
  67. 68. What can we do? <ul><li>simple grading instruments </li></ul><ul><li>not truly valid or reliable in a statistical sense </li></ul><ul><li>help the classroom teacher or department standardized oral grading. </li></ul>
  68. 69. Instruments <ul><li>Simple Scales </li></ul><ul><li>Formal Rubrics </li></ul><ul><li>A mixture </li></ul>
  69. 70. Simple scales <ul><li>1 --Little or no language produced. Little or no communication. </li></ul><ul><li>2 --Poor vocabulary, serious mistakes in grammar, poor pronunciation. Very hesitant and brief utterances, sometimes difficult to understand. </li></ul><ul><li>3 --Adequate vocabulary, mistakes in grammar, adequate pronunciation. Communicates ideas, but hesitantly and briefly </li></ul><ul><li>4 --Good vocabulary, occasional errors in grammar, good pronunciation. Effective communication, but does not elaborate on response. </li></ul><ul><li>5 --Wide vocabulary, very few errors in grammar, very good pronunciation. Easy and efficient communication. Elaborates on responses. </li></ul>
  70. 71. Problem of standardization <ul><li>examiner’s general knowledge of the production level expected for each course. </li></ul><ul><li>Most experienced teachers have this knowledge and I have always included this warning: </li></ul><ul><li>  </li></ul>
  71. 72. <ul><li>Note: Please take S's level into account. A Course 1 student cannot produce as much language as a Course 3 student. To get 3 points, the student should be able to use structures and vocabulary taught in the course he / she just finished. However, expect errors since the student has not fully acquired the material. To get 5 points, the student may still make a few isolated errors, but will speak much above a typical student at the same level.  </li></ul>
  72. 73. But…remember… <ul><li>Accuracy and Fluency??? </li></ul>
  73. 74. <ul><li>Accuracy </li></ul><ul><li>1 ---Little or no language produced. </li></ul><ul><li>2 --Poor vocabulary, serious mistakes in grammar, poor pronunciation. </li></ul><ul><li>3 --Adequate vocabulary, mistakes in grammar, adequate pronunciation. </li></ul><ul><li>4 --Good vocabulary, occasional errors in grammar, good pronunciation. </li></ul><ul><li>5 --Wide vocabulary, very few errors in grammar, very good pronunciation. </li></ul><ul><li>Fluency </li></ul><ul><li>1 ---Little or no communication. </li></ul><ul><li>2 ---Very hesitant and brief utterances, sometimes difficult to understand. </li></ul><ul><li>3 ---Communicates ideas, but hesitantly and briefly </li></ul><ul><li>4 ---Effective communication, but does not elaborate on response. </li></ul><ul><li>5 ---Easy and efficient communication. Elaborates on responses. </li></ul>
  74. 75. With 2 scales…. <ul><li>You can … </li></ul><ul><ul><li>give two exams…one accuracy and one fluency </li></ul></ul><ul><ul><li>give one exam….but grade both aspects. </li></ul></ul>
  75. 76. Rubrics <ul><li>&quot;a scoring tool that lists the criteria for a piece of work or 'what counts.” </li></ul><ul><li>Generally rubrics specify the level of performance expected for several levels of quality. </li></ul><ul><li>These levels of quality may be written as different ratings (e.g., Excellent, Good, Needs Improvement) or as numerical scores (e.g., 4, 3, 2, 1) which are then added up to form a total score which then is associated with a grade (e.g., A, B, C, etc).  </li></ul>
  76. 77. Combined System <ul><li>Similar to rubrics, but easier to grade </li></ul><ul><li>Example is based on Common European Framework “Can-do” Statements (B2) </li></ul><ul><li>Just check, yes/no. Can the student do it or not. </li></ul>
  77. 78. Thank you very much <ul><li>JoAnn Miller </li></ul><ul><li>[email_address] / [email_address] </li></ul><ul><li>www.efltasks.net </li></ul>

×