Me, write an exam?

831 views
740 views

Published on

Exam course given to the UNED, Costa Rica. June 22, 2010

Published in: Education
1 Comment
0 Likes
Statistics
Notes
  • congratulations for this great job! would you please allow me to download this? or you may send it to my email id: rebat2000@gmail.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total views
831
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide
  • There are testing alternatives [click] that aren’t the traditional “paper and pencil” exams. [click] You can evaluate students’ progress constantly, not punctually (at specific times) [click] Definitions from REF 1: The Reflective Portfolio: Two Case Studies from the United Arab Emirates, Christine Coombe and Lisa Barlow, Forum Online, http://exchanges.state.gov/forum/vols/vol42/no1/p18.htm
  • Reference 1 (REF1) From The Reflective Portfolio: Two Case Studies from the United Arab Emirates, Christine Coombe and Lisa Barlow, Forum Online, http://exchanges.state.gov/forum/vols/vol42/no1/p18.htm
  • Portfolio assessment is very popular in places like the US where classes are small and teachers have office hours and preparation time [click] But they are more subjective, you can’t guarantee that two teachers will be grading the same way and [click] Teachers in Mexico have physical limitations: lots of students, lots of groups, little free time to correct [click] Would they be accepted by the SEP, schools, parents and students? Have participants discuss in pairs and then compare responses.
  • Why do we rely on written tests here in Mexico? [click] It isn’t the only way to test and it probably isn’t the fairest…some students don’t do well on written tests [click] But they are the easiest way to test with the large numbers of students we have here [click] It is more objective and can be designed so that results can be fairly consistent among teachers at the same institution [click] And they are accepted by institutions (official study plans, schools) parents and the students themselves
  • Only show the title of the slide. Ask teachers how many have exam banks at their schools (If some participants don’t know what exam banks are, ask for a definition). Put students into groups, try to get at least one teacher who has worked with an exam bank in each group. Have them discuss the advantages and disadvantages of the exam banks. When they finish, listen to some of their conclusions. Then continue on with the slide. Exam banks have many advantages for large schools. They are a collection of exams [click] that could be written by the teachers or a special committee [click]. They must be based on institutional guidelines. [click] It is possible to have various cycles of the exams.
  • Benefits: Teachers don’t have to write all their own exams. Instead of writing 5, 10 or more exams a semester, they write one or two. It save time, energy and quality. [click] They are useful in larger schools because they help standardize performance assuring all the students who pass from one level to another are ready. The criteria, instructions and grading are uniform. [click] Face validity (an exam looks like the teachers and students expect it to). The format is similar. All exams in the exam bank follow the same rules so there are no surprises. They even look similar. [click] Exams are also about the same length. In reality exams banks are the most time-efficient, reliable ways to test. Each exam can be written with the same criteria and the results between groups is much more reliable. Various cycles can be created so that cheating is minimized and each individual teacher works less since instead of writing an exam for every class they are teaching, they might just write a couple of exams for the exam bank. In pairs: Do they have exam banks? How do they work? If not, would they work?
  • Only show the title of the slide. Ask teachers how many have exam banks at their schools (If some participants don’t know what exam banks are, ask for a definition). Put students into groups, try to get at least one teacher who has worked with an exam bank in each group. Have them discuss the advantages and disadvantages of the exam banks. When they finish, listen to some of their conclusions. Then continue on with the slide. Exam banks have many advantages for large schools. They are a collection of exams [click] that could be written by the teachers or a special committee [click]. They must be based on institutional guidelines. [click] It is possible to have various cycles of the exams.
  • Benefits: Teachers don’t have to write all their own exams. Instead of writing 5, 10 or more exams a semester, they write one or two. It save time, energy and quality. [click] They are useful in larger schools because they help standardize performance assuring all the students who pass from one level to another are ready. The criteria, instructions and grading are uniform. [click] Face validity (an exam looks like the teachers and students expect it to). The format is similar. All exams in the exam bank follow the same rules so there are no surprises. They even look similar. [click] Exams are also about the same length. In reality exams banks are the most time-efficient, reliable ways to test. Each exam can be written with the same criteria and the results between groups is much more reliable. Various cycles can be created so that cheating is minimized and each individual teacher works less since instead of writing an exam for every class they are teaching, they might just write a couple of exams for the exam bank. In pairs: Do they have exam banks? How do they work? If not, would they work?
  • Base the content analysis on the textbook for the reasons we mentioned previously…Slide 13
  • You can’t test students on something they haven’t seen. [click] If you are writing for other teachers, the only thing you have in common is the textbook. If you are only writing for your students and they are absent, the only resource they have is their textbooks[click] You need to make an analysis of how much time is spent on each aspect you want to test.
  • Here is an example of a final content analysis. If you are using an exam bank, all the teachers should have it. The students should also have it. It can help them study. Look at the Grammar section. Notice that one point of the 15 points on that section of the exam is dedicated to the comparative. If the student didn’t know only one point was dedicated to that structure, he would know that it was more important to study BE and the simple present (7 points). This can also help the teachers since no one would spend hours teaching all aspects of the comparative if it is not represented with more than one point on the exam. This would be a positive washback effect.
  • Ask participants what they think “communicative testing” refers to. After hearing some possibilities, tell them: The “Communicative Approach” has been in existence since the late 80s. We now accept the ideas of teaching communicatively: group work, teaching language in context, etc, but we continue testing traditionally [click] Ask participants what “traditional testing” is (isolated sentences, transformation, grammar-based) [click] Communicative testing means testing in context…not isolated sentences
  • HO2-- Let’s look at some examples. What will you test? [click] Grammar? Refer participants to HO2. Have them look at the grammar examples and compare them in pairs. What are the differences ? When they finish giving you some differences go to the next slide.
  • [click] The example on the left is traditional. There is no context. It is OK for simple structures like this one, but what about more complex structures such as If clauses or present perfect/past? Here is an example you can give them: If I __________ (be invited) to your party, I __________ (go). What is the answer? If I am invited to her party, I’ll go (If 1) If I were invited to her party, I’d go (If 2) If I had been invited to her party, I would have gone (If 3) Since there is no context, any of them would be correct. [click] In the other example the grammar is presented in context, a conversation. If this had been about If clauses, it might have been: “ I’m sorry. I didn’t know she was going to have a party. If I ______________ (be invited), I ___________ (go).” Obviously If 3.
  • What will you test? [click] Vocabulary? Refer participants to the HO2. Have them look at the vocabulary examples and compare them in pairs. What are the differences? When they finish giving you some differences go to the next slide.
  • [click] The example on the left is a traditional vocabulary section. It tests if they learned a vocabulary list, but it doesn’t test if they really know how to use the vocabulary. [click] The example on the right tests many different problems students can have when they work with vocabulary: Difficult pairs: (1) (3) (6) Collocation: (2) (4) (5) The context lets you test more.
  • Ask participants how they could test functions…collect some ideas.
  • Go over the definition of functions just in case someone doesn’t know what they are.
  • Go through the examples. Emphasize how the following aspects become more complex as we go down the list: Grammar structures (easy to more difficult) Length of sentences (short to long) Register (from informal to formal)
  • This is the best type of section to test students’ knowledge of functions. (In HO2) [click] Students complete a conversation (or paragraph) with sentences (or even phrases) that communicate the correct function logically. Go over the first example. Show that there isn’t one correct answer. The first one could be: It has / I have / There is a big living room. Elicit possible answers for # 2 and 3. The section can be even more open as the other two option show. Elicit possible answers. Have participants compare this with the grammar and vocabulary sections they have seen. What are the differences? Emphasize that here the purpose is communication and that communication can occur even if the students don’t use complete sentences or make some grammar errors. This will be seen later in the workshop when they learn to correct these sections.
  • Have participants look at the example and suggest what kinds of exam sections they could write combining vocabulary themes and then [click] dividing them…. If there is time, in pairs, have them think of conversations, paragraphs, etc. and describe what they would be like to practice these sections. Then have them go back to their content analyses and decide how point values can be combined or divided to reach the number of points they determined previously for their exam (see What to test and how often, slides 11-15). 15 minutes.
  • In production, the student writes more than one word (remind them of Complete the Conversation for testing functions) The student can be more creative and the teacher has to be more alert because more than one answer might be correct. In recognition (for example, multiple choice or true/false), there is only one correct answer and it isn’t creative. Production means more work. Teachers with large number of students can’t handle a 50-50 balance. A good exam could be 20-80 or 30-70 (production/recognition) and still be fair. TOEFL and other similar exams are all recognition…they never test whether the student can produce language. Years ago this led to a big influx of Asian students into US universities. They had studied grammar and reading, but no speaking or writing. They did great recognizing correct answers on the TOEFL and got very high scores, but when they arrived in the US, authorities realized they couldn’t say two words in English….The TOEFL exam was revised and now includes writing sections and often oral interviews are required to study in the US.
  • Sections can also be objective or subjective. It is good to have some subjective sections, but an exam with sections that are all graded subjectively make it difficult to judge student ability between different teachers…. However, even though they are more difficult to grade, they do give more information about students’ ability. The limitations can be overcome if the graders are trained…. This is a very common problem with oral grading.
  • It’s easy to correct an “A, b, c” section, but how do you correct a “complete the conversation” section so that it really tests communicative ability and competence?
  • Go over the summary
  • These are examples of partial credit. Go over then one-by-one. Ask participants if they would give partial credit or not if they were grading communication… First example: Does it communicate? Would you be able to answer the student’s questions? Probably. If it were a C1 student on the first exam, I’d give credit, but write in the corrections. On later exams, I’d probably not be a generous. Second example: The meaning is different. In the correct answer, you didn’t invite me, in the student’s answer you might do so in the future. Although the grammar is correct, it doesn’t communicate and I wouldn’t give any credit. Third example: Click slowly, discussing as you go. The first one communicates and if it were the first time students had worked with the past I’d accept it and just write in the correction. They use “yesterday” to indicate the tense. [click] This could also be accepted if it were in answer to the question. “what did you do yesterday?”. Correcting these sections, you have to ask “does the answer communicate the idea required by the conversation.” If so, accept it or give partial credit depending on the level of the students.
  • This is the exam section used in the activity (HO7). Go over it and get possible answers before they begin working. Have them work in pairs, grading the students’ work. (15 minutes) Then go over the worksheet, comparing how they graded.
  • Go over this with the participants. It will be the hardest section to write since they have very little experience with communicative exams.
  • Go over with participants. Ask for their opinions. From (REF5) Designing Test Questions, Grayson H. Walker Teaching Resource Center, The University of Tennessee at Chattanooga, http://www.utc.edu/Administration/WalkerTeachingResourceCenter/FacultyDevelopment/Assessment/test-questions.html
  • Go over… From same source as previous slide
  • Get examples so you can be sure they understand what the stem (first part) and options are (A.b.c)
  • REF 3: From Kehoe, Jerard. Writing Multiple-Choice Test Items. ERIC/AE Digest Series EDO-TM-95-3, October 1995. http://www.ericdigests.org/1997-1/test.html Go through the points one-by-one…checking comprehension as you go
  • Continued
  • You might want to discuss how many items they feel they should use. Kehoe recommends 3-4, but some really formal exams use 5 options.
  • These are the different orders possible…. Talk about which seem best and why? Do multiple Choice handout (HO5) in pairs (30 minutes). These are the poor examples from the Burton article. Have participants think about what they think is wrong with them and how they could be improved. Then go over them. Use REF 4 to correct… REF 4: Burton, Steven J. Richard R. Sudweeks, Paul F. Merrill, Bud Wood. How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty, Brigham Young University Testing Services and The Department of Instructional Science. 1991. http://testing.byu.edu/info/handbooks/betteritems.pdf When they finish HO5, have them write a multiple choice section using their content analyses and the section divisions they recently did. When they finish, have them compare and criticize their work. Be careful they are using multiple choice for reasonable sections…(20 minutes)
  • Me, write an exam?

    1. 1. Me? Write an exam? How to write a communicative exam JoAnn Miller, Macmillan Publishers [email_address] www.efltasks.net
    2. 2. <ul><li>Written </li></ul><ul><li>Oral </li></ul><ul><li>Projects </li></ul><ul><li>Portfolio </li></ul>What kinds of evaluations do you do?
    3. 3. Alternative Testing <ul><li>Not paper and pencil </li></ul><ul><li>Constant not punctual </li></ul>Definition: “ the ongoing process involving the student and teacher in making judgments about the student’s progress in language using non-conventional strategies “ techniques that can be used within the context of instruction and can be easily incorporated into the daily activities of the school or classroom.” The Reflective Portfolio: Two Case Studies from the United Arab Emirates, Christine Coombe and Lisa Barlow, Forum Online, http://exchanges.state.gov/forum/vols/vol42/no1/p18.htm
    4. 4. Portfolios <ul><li>a collection of student production over time </li></ul><ul><li>shows the stages in the learning process </li></ul><ul><li>and the stages of the student’s growth. </li></ul>The Reflective Portfolio: Two Case Studies from the United Arab Emirates, Christine Coombe and Lisa Barlow, Forum Online, http://exchanges.state.gov/forum/vols/vol42/no1/p18.htm
    5. 5. Portfolios? <ul><li>more subjective </li></ul><ul><li>acceptance </li></ul><ul><li>physical limits </li></ul>
    6. 6. Why written tests? <ul><li>Not the only way and maybe not the fairest. </li></ul><ul><li>But easiest with large numbers of students </li></ul><ul><li>More objective </li></ul><ul><li>Accepted by institutions, parents, students </li></ul>
    7. 7. Exam Banks <ul><li>A collection of exams for classroom use maintained by the institution itself. </li></ul><ul><li>Written by the teachers themselves or a special committee </li></ul><ul><li>Following institutional guidelines </li></ul><ul><li>Could be various “cycles” covering the same material </li></ul>
    8. 8. The benefits of an exam bank <ul><li>Less work for teachers </li></ul><ul><li>More standardization in large one-campus schools and in multi-campus schools </li></ul><ul><ul><li>Criteria, instructions and grading </li></ul></ul><ul><li>Face validity </li></ul><ul><ul><li>Format </li></ul></ul><ul><li>Uniform length </li></ul>
    9. 9. The basic questions <ul><li>Why test? </li></ul><ul><li>How often? When? </li></ul><ul><li>What? </li></ul><ul><li>How? </li></ul>
    10. 10. <ul><li>Student’s need to self-evaluate </li></ul><ul><li>Institutions require student evaluations </li></ul><ul><li>Students’ need to be evaluated to see if they are ready for the next level </li></ul><ul><li>Teacher’s need to evaluate how they have taught </li></ul>Why do YOU test?
    11. 11. <ul><li>Once a semester…too little </li></ul><ul><li>Once a week…too much </li></ul><ul><li>Depends on students and institution </li></ul><ul><li>Testing takes time from language exposure in a communicative context </li></ul>How often do YOU test? Why? Do you like it?
    12. 12. Why written tests? <ul><li>Not the only way and maybe not the fairest. </li></ul><ul><li>But easiest with large numbers of students </li></ul><ul><li>More objective </li></ul><ul><li>Better? </li></ul><ul><ul><li>Portfolio: Collection of all the student has done. </li></ul></ul><ul><ul><li>But…it’s more subjective </li></ul></ul>
    13. 13. Exam Banks <ul><li>A collection of exams for classroom use maintained by the institution itself. </li></ul><ul><li>Written by the teachers themselves or a special committee </li></ul><ul><li>Following institutional guidelines </li></ul><ul><li>Could be various “cycles” covering the same material </li></ul>
    14. 14. The benefits of an exam bank <ul><li>Less work for teachers </li></ul><ul><li>More standardization in large one-campus schools and in multi-campus schools </li></ul><ul><ul><li>Criteria, instructions and grading </li></ul></ul><ul><li>Face validity </li></ul><ul><ul><li>Format </li></ul></ul><ul><li>Uniform length </li></ul>
    15. 15. First, the situation <ul><li>Only your students or exam bank? </li></ul><ul><li>Criteria? </li></ul><ul><ul><li>Relative weight of grammar, function, vocabulary, reading, listening, writing, etc.? </li></ul></ul><ul><ul><li>Institutional criteria? </li></ul></ul><ul><li>Students. Age? Interests? </li></ul><ul><li>Who will give it? Correct it? </li></ul>What about your situation? How do you decide the relative weight of grammar, function, vocabulary, etc.? Who are your students? Who will give the exam? Correct it?
    16. 16. Students <ul><li>Younger students </li></ul><ul><ul><li>More images </li></ul></ul><ul><ul><li>Shorter exams </li></ul></ul><ul><ul><li>Humor </li></ul></ul><ul><li>Older students </li></ul><ul><ul><li>Professionalism </li></ul></ul><ul><ul><li>Humor </li></ul></ul>
    17. 17. Teachers <ul><li>Level of English </li></ul><ul><li>Lack of mathematical skills </li></ul><ul><li>Time factor </li></ul><ul><li>Ease of grading </li></ul><ul><li>Answer key </li></ul><ul><li>Understand why </li></ul>
    18. 18. <ul><li>Factors: </li></ul><ul><ul><li>Students (younger / shorter) </li></ul></ul><ul><ul><li>Teachers (mathematical skills) </li></ul></ul><ul><ul><li>Total points in course </li></ul></ul><ul><ul><li>Minimum? Maximum? </li></ul></ul>How many points would you like on your exams?
    19. 19. How many points for each skill? <ul><li>If institution tells you, just follow through </li></ul><ul><li>If not, text (the common denominator) </li></ul><ul><ul><li>General text analysis </li></ul></ul><ul><ul><li>How much time is spent on each skill </li></ul></ul><ul><ul><li>Count exercises in a few units, determine percent </li></ul></ul><ul><li>Keep institutional goals in mind </li></ul>
    20. 20. Discussion <ul><li>What are your students like? </li></ul><ul><li>What are your teachers like? </li></ul><ul><li>What is the purpose of your English programs? </li></ul>
    21. 21. Content Validity <ul><li>Assessment should be based on a content-analysis of the text being used </li></ul>
    22. 22. Content Analysis <ul><li>You must test only material students have seen </li></ul><ul><li>The only common denominator is the textbook </li></ul><ul><li>Analysis of percent of time spent on each topic (grammar structure, vocabulary item, function, etc.) </li></ul>
    23. 23. Content Analysis: Information from the contents Functions (10 points): Talking about imitation products Talking about food and food festivals Discussing the movie industry Making a business plan Grammar (5 points): Nouns in groups Indefinite Pronouns Vocabulary (10 points): Food Business language ///// /// ///// ///// ///// //// ///// /// ///// ///// /// 8 10 9 5 32 3 5 8 5 3 8 8 / 32 =__% 25% 31% 28% 16% 25% X 10 = ___pts 2.5 pts 3 pts 3 pts 1.5 pts 37% 2 pts 63% 3 pts 63% 6 pts 37% 4 pts
    24. 25. (Attitude. Macmillan, 2000.)
    25. 26. What information can you get out of this overview?
    26. 27. Content Analysis: Information from overview Skill Sections in Unit   Functions (10 points)                   Talking about similarites and differences                   Talking about prefrences                   Discussing opinions                   Talking about past events                                       Grammar (10 points)                   comparative adjectives                   comparatives and superlatives                   Superlative adjectives                                       Vocabulary (10 points)                   sports                   feelings                  
    27. 28. Categorization of activities Skill Sections in Unit   Functions (10 points)                   Talking about similarities and differences 1 1               Talking about preferences 2 2 2             Discussing opinions 3 3               Talking about past events 4 4 4 4                               Grammar (10 points)                   comparative adjectives 1 1 1 1 w1 w1       comparatives and superlatives 2 2 w2 w2 w3         Superlative adjectives 3 3 w2                                 Vocabulary (10 points)                   sports 1 2 2 w1 w1         feelings 2 2              
    28. 29. Analysis (now you work) <ul><li>Count number of entries in each row (A) </li></ul>Count Functions (10 points)                   Talking about similarities and differences 1 1         2    
    29. 30. <ul><li>Get total number of entries per skill aspect (B) </li></ul><ul><li>    </li></ul>Skill Sections in Unit Count Functions (10 points)                   Talking about similarities and differences 1 1         2     Talking about prefernces 2 2 2       3     Discussing opinions 3 3         2     Talking about past events 4 4 4 4     4                   11    
    30. 31. <ul><li>Divide A by B to get percent of A (C%) </li></ul>Skill Sections in Unit Count % Functions (10 points)                   Talking about similarities and differences 1 1         2 18%   Talking about prefrences 2 2 2       3     Discussing opinions 3 3         2     Talking about past events 4 4 4 4     4                   11                                            
    31. 32. <ul><li>Multiply C% by the number of points for that skill / aspect (D) = the number of points per topic </li></ul>Skill Sections in Unit Count % Points Functions (10 points)                   Talking about similarities and differences 1 1         2 18% 1.8 Talking about prefernces 2 2 2       3     Discussing opinions 3 3         2     Talking about past events 4 4 4 4     4                   11                                            
    32. 33. Skill Sections in Unit Count % Points Functions (10 points)                   Talking about similarities and differences 1 1         2 18% 1.8 Talking about prefernces 2 2 2       3 27% 2.7 Discussing opinions 3 3         2 18% 1.8 Talking about past events 4 4 4 4     4 36% 3.6               11     Grammar (10 points)                   comparative adjectives 1 1 1 1 w1 w1 6 43% 4.3 comparatives and superlatives 2 2 w2 w2 w3   5 36% 3.6 Superlative adjectives 3 3 w2       3 21% 2.1               14     Vocabulary (10 points)                   sports 1 2 2 w1 w1   5 71% 7.1 feelings 2 2         2 29% 2.9               7                        
    33. 34. Communicative testing <ul><li>We teach “communicatively” but we test “traditionally”. </li></ul><ul><li>What IS communicative testing? </li></ul><ul><li>Communicative testing means testing in context . </li></ul>
    34. 35. Grammar? What will you test?
    35. 36. Which version? Why? <ul><li>Circle the correct answer </li></ul><ul><li>1. Do you like __________? </li></ul><ul><li>swimming b. to swum c. swim </li></ul><ul><li>2. Where ________ live? </li></ul><ul><li>does she b. she does c. she </li></ul><ul><li>3. I _________ speak French. </li></ul><ul><li>no speak b. doesn’t c. don’t </li></ul><ul><li>4. What __________? </li></ul><ul><li>a. does he do b. does he c. he does do </li></ul><ul><li>Write the correct forms of the words in parentheses. </li></ul><ul><li>Alice: Where (1)______ you </li></ul><ul><li>_________ (live)? </li></ul><ul><li>Bart: Acapulco. </li></ul><ul><li>Alice: My brother (2)___________ (go) </li></ul><ul><li>there every summer on vacation, </li></ul><ul><li>but he (3)_________(not speak) </li></ul><ul><li>Spanish. </li></ul><ul><li>Bart: Acapulco (4)_________ (attract) </li></ul><ul><li>tourists from all over the world. </li></ul><ul><li>Many people there </li></ul><ul><li>(5)___________(speak) English </li></ul><ul><li>very well. What about you, </li></ul><ul><li>(6)______ you ________(speak) </li></ul><ul><li>Spanish? </li></ul><ul><li>Alice: A little. </li></ul>
    36. 37. Vocabulary? What will you test?
    37. 38. Which version? Why? <ul><li>Match the letters (a to e) with the numbers (1 to 5). </li></ul><ul><li>1. Your mother’s husband is your___. </li></ul><ul><li>2. Your mother’s father is your___. </li></ul><ul><li>3. Your mother’s brother is your___. </li></ul><ul><li>4. Your uncle’s son is your___. </li></ul><ul><li>5. Your father’s sister is your ___. </li></ul><ul><li>uncle </li></ul><ul><li>cousin </li></ul><ul><li>aunt </li></ul><ul><li>father </li></ul><ul><li>grandfather </li></ul><ul><li>Underline the word in each pair that completes the conversation correctly. </li></ul><ul><li>My (1) [ uncle / aunt ] likes (2) [ playing / going to ] movies. He is my father’s ( 3) [ sister / brother] . He’s (4) [ heavy / average ] and he has (5) [ blue / brown ] hair. His birthday is on October (6) [twelve / twelfth ] . </li></ul>
    38. 39. Functions? What will you test?
    39. 40. What is a function? <ul><li>The communicative purpose of the users of the language. </li></ul><ul><li>How language is used. </li></ul><ul><li>Usually expressed as gerunds: introducing, apologizing, asking directions, requesting </li></ul>
    40. 41. Examples of a Functional Cycle <ul><li>Function: Requesting </li></ul><ul><li>(1) Open the window, please. </li></ul><ul><li>(2) Would you open the window? </li></ul><ul><li>(3) Could you please open the window? </li></ul><ul><li>(4) Would you mind opening the window? </li></ul><ul><li>(5) I was wondering if you would mind opening the window. </li></ul><ul><li>(6) I’d be grateful if you opened the window. </li></ul><ul><li>Each time the difference in register is emphasized. </li></ul>
    41. 42. How to test? Complete the conversation. <ul><li>Complete the conversation logically. Use the words in parentheses. </li></ul><ul><li>Miriam: Tell me about your new apartment. </li></ul><ul><li>Mary: (1)____________________(big / living room). </li></ul><ul><li>Miriam: (2)___________________(how / bedrooms)? </li></ul><ul><li>Mary: There are two, but (3)________(any furniture) </li></ul><ul><li>in one of them. </li></ul>Or: Miriam: Tell me about your new apartment. Mary: (1)_______________________(living room). Or: Miriam: Tell me about your new apartment. Mary: (1)________________________________.
    42. 43. What is an exam section? <ul><li>A certain number of items testing the same skill / aspect </li></ul><ul><li>To be communicative, they should be written as a conversation, note, letter, or some “real” type of discourse </li></ul><ul><li>Isolated sentences are difficult to contextualize and don’t represent real communication </li></ul>
    43. 44. Determining sections <ul><li>You can combine point values within the same aspect. </li></ul><ul><li>You can divide point values between two sections </li></ul><ul><li>You shouldn’t combine point values across aspects / skills…the testing methods are different. </li></ul>
    44. 45. Combining point values within the same aspect. <ul><li>Vocabulary (8 points) </li></ul><ul><li>Family relationships 2.5 </li></ul><ul><li>Describing people 0.5 </li></ul><ul><li>say tell ask 1 </li></ul><ul><li>Phrasal verbs 2 </li></ul><ul><li>Every day expressions 2 </li></ul>Dividing point values between two sections <ul><li>Vocabulary (8 points) </li></ul><ul><li>Family relationships 2.5 </li></ul><ul><li>Describing people 0.5 </li></ul><ul><li>say tell ask 1 </li></ul><ul><li>Phrasal verbs 2 </li></ul><ul><li>Every day expressions 2 </li></ul>1.5 1
    45. 46. Accuracy and Fluency <ul><li>Fluency (Communication) </li></ul><ul><ul><li>The ability to produce written and / or spoken language with ease </li></ul></ul><ul><ul><li>Communicate ideas effectively </li></ul></ul><ul><li>Accuracy </li></ul><ul><ul><li>Ability to produce grammatically correct sentences </li></ul></ul>
    46. 47. Production and Recognition <ul><li>Production </li></ul><ul><ul><li>Student writes more than one word </li></ul></ul><ul><ul><li>Can be creative / involves more “mental” work </li></ul></ul><ul><ul><li>More than one answer may be possible </li></ul></ul><ul><li>Recognition </li></ul><ul><ul><li>Student recognizes correct answer </li></ul></ul><ul><ul><li>Not creative </li></ul></ul><ul><ul><li>Only one correct answer </li></ul></ul>
    47. 48. Objective and Subjective Sections <ul><li>Subjective </li></ul><ul><ul><li>There is more than one possible answer </li></ul></ul><ul><ul><li>Corrector must be trained and experienced </li></ul></ul><ul><ul><li>There can be surprises </li></ul></ul><ul><ul><li>Students can protest grading </li></ul></ul><ul><li>Objective </li></ul><ul><ul><li>There is only one answer </li></ul></ul><ul><ul><li>Anyone can correct the exam </li></ul></ul><ul><ul><li>No surprises </li></ul></ul><ul><ul><li>No argument from students </li></ul></ul>
    48. 49. So what does this all mean? Accuracy Accuracy Accuracy Accuracy Accuracy / Fluency Fluency Fluency Prod. /Recog. Prod. /Recog. Prod. /Recog. Recognition Recognition Production Production Subj. / Obj Subj. / Obj Objective Objective Objective Subjective Subjective
    49. 50. Point Values <ul><li>Give more points to… </li></ul><ul><ul><li>Production items </li></ul></ul><ul><li>Give fewer points to… </li></ul><ul><ul><li>Recognition items </li></ul></ul><ul><li>Give partial credit in </li></ul><ul><ul><li>Fluency / Production sections </li></ul></ul><ul><li>Use fractions only if users are matematical </li></ul>
    50. 51. Correcting communicative sections
    51. 52. Correcting <ul><li>Grammar, Reading, Vocabulary, Listening </li></ul><ul><li>In general these sections are all right or all wrong . </li></ul><ul><li>We rarely give partial credit. </li></ul><ul><li>These sections test accuracy . </li></ul><ul><li>Communicative sections </li></ul><ul><li>You can give partial credit </li></ul><ul><li>These sections test fluency. </li></ul><ul><li>Ask yourself if the S’s answer communicates what the S wants to say . </li></ul>
    52. 53. Examples of partial credit <ul><li>Correct answer: What’s your name? </li></ul><ul><li>Student writes: What you name? </li></ul><ul><li>Correct answer: If you invited me, I’d go. </li></ul><ul><li>Student writes: If you invite me, I go. </li></ul><ul><li>Correct answer: I went to the movies yesterday. </li></ul><ul><li>Student writes: I go to the movies yesterday. </li></ul><ul><li> I go to the movies. </li></ul>
    53. 54. IV. The clerk knows Cleopatra. Caesar asks the clerk about Cleo. Complete the conversation. Use the words in parentheses. ( 4 points, .5 each) Clerk : Yes, I know her. Julius : (1) _______________________________________ (work) ? Clerk : (2) ________________________________ ( palace downtown). Julius : (3) _______________________________________ ( do) ? Clerk : (4) ___________________________________ ( help people). Julius : (5) ___________________________________ ( close friend)? Clerk : Yes, (6) ____________________________________ (funny). Julius : (7) _______________________________________ ( sports)? Clerk : Yes, (8) ___________________________________ ( tennis ). Actual Student Responses on the worksheet
    54. 55. Communicative Exam Sections <ul><li>Now you are going to write an exam section. </li></ul><ul><li>Get out </li></ul><ul><ul><li>Your content analysis </li></ul></ul><ul><ul><li>You exam plan </li></ul></ul><ul><ul><li>You textbook </li></ul></ul>
    55. 56. Complete the conversation <ul><li>These sections evaluate a student’s ability to communicate ideas if they are corrected for communication and not for accuracy. </li></ul><ul><li>They can be written with different degrees of cueing. </li></ul>
    56. 57. Testing Reading and Listening <ul><li>Two formats are commonly used: </li></ul><ul><ul><li>T/ F </li></ul></ul><ul><ul><li>Multiple Choice </li></ul></ul>
    57. 58. True / False <ul><li>Advantages : </li></ul><ul><li>Can test large amounts of content </li></ul><ul><ul><li>Students can answer 3-4 questions per minute </li></ul></ul><ul><li>Disadvantages: </li></ul><ul><li>They are easy </li></ul><ul><ul><li>Students have a 50-50 chance of getting the right answer by guessing </li></ul></ul><ul><ul><li>It is difficult to discriminate between students that know the material and students who don't </li></ul></ul><ul><li>Need a large number of items for high reliability </li></ul>Designing Test Questions, Grayson H. Walker Teaching Resource Center, The University of Tennessee at Chattanooga, h ttp://www.utc.edu/Administration/WalkerTeachingResourceCenter/FacultyDevelopment/Assessment/test-questions.html
    58. 59. Tips for Writing Good True/False items <ul><li>Avoid double negatives. </li></ul><ul><li>Avoid long/complex sentences. </li></ul><ul><li>Use specific determinants with caution: </li></ul><ul><ul><li>never, only, all, none, always, could, might, can, may, sometimes, generally, some, few. </li></ul></ul><ul><li>Use only one central idea in each item; don't emphasize the trivial. </li></ul><ul><li>Don't lift items straight from the text. </li></ul><ul><li>Make more false than true (60/40). </li></ul><ul><ul><li>(Students are more likely to answer true.) </li></ul></ul>
    59. 60. How to “save” a T/F section… <ul><li>Add a third option </li></ul><ul><ul><li>“ Not mentioned” </li></ul></ul><ul><li>OR </li></ul><ul><li>Have student correct F answers </li></ul><ul><li>But only if students have practiced this version in the textbook </li></ul>
    60. 61. Multiple Choice <ul><li>Parts of question </li></ul><ul><ul><li>Stem </li></ul></ul><ul><ul><li>Options </li></ul></ul><ul><ul><li>Distractors </li></ul></ul><ul><li>XXXXXXXXXXXXXXXXXX? </li></ul><ul><ul><li>YYYYYYYYY </li></ul></ul><ul><ul><li>ZZZZZZZZ </li></ul></ul><ul><ul><li>AAAAAAAA </li></ul></ul><ul><ul><li>{ </li></ul></ul><ul><ul><li>{ </li></ul></ul>Correct answer
    61. 62. Stems <ul><li>Don’t include words that do not contribute to the basis for choosing among the options. </li></ul><ul><li>The American flag has three colors. One of them is (1) red (2) green (3) black </li></ul><ul><li>vs. </li></ul><ul><li>One of the colors of the American flag is (1) red (2)green (3) black </li></ul><ul><li>Or </li></ul><ul><li>If the pressure of a certain amount of gas is held constant, what will happen if its volume is increased? </li></ul><ul><li>a. The temperature of the gas will decrease. </li></ul><ul><li>*b. The temperature of the gas will increase. </li></ul><ul><li>c. The temperature of the gas will remain the same. </li></ul>Kehoe, Jerard. Writing Multiple-Choice Test Items. ERIC/AE Digest Series EDO-TM-95-3, October 1995. http://www.ericdigests.org/1997-1/test.html
    62. 63. <ul><li>Include as much information in the stem and as little in the options as possible. </li></ul><ul><li>California: </li></ul><ul><li>a. Contains the tallest mountain in the United States </li></ul><ul><li>b. Has an eagle on its state flag. </li></ul><ul><li>c. Is the second largest state in terms of area. </li></ul><ul><li>*d. Was the location of the Gold Rush of 1849. </li></ul>
    63. 64. <ul><li>Avoid irrelevant clues to the correct option. For example, grammatical construction: </li></ul><ul><li>A word used to describe a noun is called an: </li></ul><ul><li>*a. Adjective. </li></ul><ul><li>b. Conjunction. </li></ul><ul><li>c. Pronoun. </li></ul><ul><li>d. Verb. </li></ul>
    64. 65. Options (Kehoe) <ul><li>1. Use three or four options. </li></ul><ul><li>2. Construct distractors that are comparable in length, complexity, grammatical form </li></ul>Which of the following would do the most to promote the application of nuclear discoveries to medicine? a. Trained radioactive therapy specialists. *b. Developing standardized techniques for treatment of patients. c. Do not place restrictions on the use of radioactive substances. d. If the average doctor is trained to apply radioactive treatments.
    65. 66. <ul><li>8. You have just spent ten minutes trying to teach one of your new employees how to change a typewriter ribbon. The employee is still having a great deal of difficulty changing the ribbon, even though you have always found it simple to do. At this point, you should: </li></ul><ul><li>a. Tell the employee to ask an experienced employee working nearby to change the ribbon in the future. </li></ul><ul><li>b. Tell the employee that you never found this difficult, and ask what he or she finds difficult about it. </li></ul><ul><li>*c. Review each of the steps you have already explained, and determine whether the employee understands them. </li></ul><ul><li>d. Tell the employee that you will continue teaching him or her later, because you are becoming irritable. </li></ul>
    66. 67. <ul><li>9. Which of the following is the best indication of high morale in a supervisor’s unit? </li></ul><ul><li>a. The employees are rarely required to work overtime. </li></ul><ul><li>*b. The employees are willing to give first priority to attaining group objectives, subordinating any personal desires they may have. </li></ul><ul><li>c. The supervisor enjoys staying late to plan the next day. </li></ul><ul><li>d. The unit gives expensive birthday presents </li></ul>
    67. 68. Ordering Multiple Choice Items <ul><li>Numerical </li></ul><ul><ul><li>a. 1939 </li></ul></ul><ul><ul><li>b. 1940 </li></ul></ul><ul><ul><li>c. 1941 </li></ul></ul><ul><ul><li>d. 1942 </li></ul></ul>Burton, Steven J. Richard R. Sudweeks, Paul F. Merrill, Bud Wood. How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty , Brigham Young University Testing Services and The Department of Instructional Science. 1991. http://testing.byu.edu/info/handbooks/betteritems.pdf Sequential a. Heating ice from -100°C to 0°C. b. Melting ice at 0°C. c. Heating water from 0°C to 100°C. d. Evaporating water at 100°C. Alphabetical a. C hanging a from .01 to .05. b. D ecreasing the degrees of freedom. c. I ncreasing the spread of the exam scores. d. R educing the size of the treatment effect. After the options are written, vary the location of the answer randomly.
    68. 71. Thank you very much <ul><li>JoAnn Miller </li></ul><ul><li>[email_address] / [email_address] </li></ul><ul><li>www.efltasks.net </li></ul>

    ×