Assessing grammar


Published on

Published in: Education

Assessing grammar

  1. 1. Assessing Grammar Christine Alice Stacey Jenny Ann
  2. 2. Test Evaluation <ul><li>Summary Teaching </li></ul><ul><li>Evaluation of an Existing Test </li></ul>
  3. 3. Five characteristics to measure communicative grammar <ul><li>The test must provide more context than only a single sentence. </li></ul><ul><li>The test taker should understand what the communicative purpose of the task is. </li></ul><ul><li>He or she should also know who the intended audience is. </li></ul><ul><li>He or she must have to focus on meaning and not form to answer correctly. </li></ul><ul><li>Recognize is not sufficient. The test taker must be able “to produce grammatical responses.” </li></ul>By: P.R. Dickins
  4. 4. Therefore… <ul><li>Have the test taker say or write something (5) of discourse length (1) in order to perform some communicative task (2) for a known audience (3), and what is said or written must make sense (4). </li></ul><ul><li>Assessing grammar can only be tested as part of a test of writing or speaking? </li></ul>
  5. 5. Important Principle: What are we trying to Measure? <ul><li>“ English grammar is chiefly a system of syntax that decides the order and patterns in which words are arranged into sentences.” – Close 1982 </li></ul><ul><li>Separated but closely related to assessing writing and assessing speaking. </li></ul><ul><ul><li>Oral performance </li></ul></ul><ul><ul><li>Writing samples </li></ul></ul><ul><ul><ul><li>Accuracy and fluency </li></ul></ul></ul><ul><ul><ul><li>Count the errors and rate their severity </li></ul></ul></ul><ul><li>Focus on Grammar only </li></ul><ul><ul><li>‘ Meaning’ is a separated matter </li></ul></ul><ul><ul><li>Integrated tests </li></ul></ul><ul><ul><li>Discrete point tests </li></ul></ul>
  6. 6. Traditional Paper and Pencil Test <ul><li>NRT: TOEFL </li></ul><ul><li>CRT: Classroom tests </li></ul><ul><li>Types of Questions: </li></ul><ul><ul><li>Complete sentences </li></ul></ul><ul><ul><li>Sentence combining </li></ul></ul><ul><ul><li>Paragraph editing </li></ul></ul><ul><ul><li>Correct topic sentence task </li></ul></ul><ul><ul><li>Multiple Choice </li></ul></ul><ul><ul><li>Fill in the blank </li></ul></ul><ul><ul><li>True/False </li></ul></ul><ul><ul><li>Combination </li></ul></ul>
  7. 7. Alternative Assessment <ul><li>Games </li></ul><ul><li>Class observation </li></ul><ul><li>Online activities </li></ul><ul><li>Portfolio </li></ul>
  8. 8. Games <ul><li>Design to restrict the language need </li></ul><ul><li>Usually these games are defined by their focus on </li></ul><ul><ul><li>The Use of Language </li></ul></ul><ul><ul><li>Limited options for communication </li></ul></ul><ul><ul><li>Accuracy </li></ul></ul>
  9. 9. Split Sentences <ul><li>Write out some sentences, and then cut each sentence in half. Hand out these pieces to the students, and ask them to find the matching half among the other students. </li></ul><ul><li>Students’ abilities in combing sentences </li></ul><ul><li>Grammar knowledge to all forms of the sentences </li></ul>
  10. 10. Examples <ul><li>If you eat that </li></ul><ul><li>If you touch the dog </li></ul><ul><li>If you steal my boyfriend </li></ul><ul><li>If you go out now </li></ul><ul><li>If you don’t leave </li></ul><ul><li>If you don’t book a ticket </li></ul><ul><li>you’ll be sick </li></ul><ul><li>it’ll bite you </li></ul><ul><li>I’ll never speak to you again </li></ul><ul><li>you’ll get soaked </li></ul><ul><li>I’ll call the police </li></ul><ul><li>you’ll be lucky to get a seat </li></ul>
  11. 11. Sentences from pictures <ul><li>Hand out a sheet of pictures. Ask students to come out sentences from the pictures. Or ask students to tell a stories. </li></ul><ul><li>Enough grammar knowledge to make sentences by students. </li></ul><ul><li>Accuracy </li></ul>
  12. 13. Memory Test <ul><li>Give limited time for students to see the picture. Ask them to describe the picture without seeing it. </li></ul><ul><li>Testing students’ ability of making sentences.(enough grammar knowledge) </li></ul><ul><li>Accuracy </li></ul><ul><li>The transformation of students’ sentences </li></ul>
  13. 15. Picture Dictation <ul><li>A student try to describe the picture to other students who haven’t seen it. </li></ul><ul><li>Testing if a student can make sentences which can help him/her communicate well. </li></ul><ul><li>Accuracy </li></ul>
  14. 17. Strip Story <ul><li>Make students into groups and assign them a text from a strip story. Ask them to find the order and resolve the problem from the story. </li></ul><ul><li>See if students can find the time order from the tenses of the sentences. </li></ul><ul><li>Accuracy </li></ul>
  15. 19. Miming an action <ul><li>Ask students to role play in a limited situation. </li></ul><ul><li>Through acting, it can test students’ ability of thinking of lines. (which may contains different tenses.) </li></ul><ul><li>Accuracy </li></ul>
  16. 20. Growing Stories <ul><li>Story-building activities </li></ul><ul><li>Students’ ability of making sentences with different tenses </li></ul>
  17. 22. Questionnaires <ul><li>Turn what have taught in class into a questionnaire. Get students to survey each other. (does not contain fully-written-out questions.) </li></ul><ul><li>It can see if students’ totally understand what have learned in class. Also it can see if students’ can use it well. </li></ul>
  18. 24. Classroom Observation <ul><li>Classroom activities </li></ul><ul><li>Classroom communication: Elicited dialogues </li></ul><ul><li>Written exercise </li></ul>
  19. 25. Online Activities <ul><li>In multimedia teaching times, teachers can use Internet as a way of assessing students’ ability of grammar </li></ul><ul><li>On-line quizzes help students to have more chances of practices </li></ul><ul><li>On-line tests also provide clear explanation after finishing the quizzes </li></ul><ul><li>Students can surf on these grammar website to do practice </li></ul><ul><li>Teachers can ask students to do these exercise and to keep record for themselves. Finally report back for one of the reference of assessment </li></ul>
  20. 26. Online activity : websites <ul><li>http:// </li></ul><ul><li>Ex1. Irregular verbs quiz </li></ul><ul><li>Grammar Assessment http:// / </li></ul><ul><li>Self-study grammar quizzes </li></ul><ul><li>ESL Quizzes Center </li></ul>
  21. 27. Portfolio <ul><li>Definition:a purposeful collection of student work that tells the viewer about the student </li></ul><ul><li>Works should be organized by chronological order and category– teacher can evaluate the child’s achievements </li></ul><ul><li>Works including: </li></ul><ul><li>1. Traditional paper and pencil tests </li></ul><ul><li>2. Classroom observation </li></ul><ul><li>3. Online activities </li></ul><ul><li>4. Written assignments </li></ul>
  22. 28. Written assignments <ul><li>Give students some assignments to practice at home </li></ul><ul><li>Ask students using learned grammar in class to write journal </li></ul><ul><li>Term paper throughout one semester including several drafts– we can see student’s progress in grammar. </li></ul>
  23. 29. Advantages of portfolio <ul><li>Why portfolio is the best way to assess? </li></ul><ul><li>Strengthening students’ learning </li></ul><ul><li>Enhancing the teacher’s role </li></ul><ul><li>Improving processes </li></ul>
  24. 30. Test Evaluation of An Existing Test <ul><li>Section 2 of the Computer-Based Test of TOEFL </li></ul><ul><ul><li>General Background Information </li></ul></ul><ul><ul><li>Title: Section 2 of TOEFL-Structure and Written Expression </li></ul></ul><ul><ul><li>Author: Education Testing Service (ETS) </li></ul></ul>
  25. 31. Test Evaluation of An Existing Test <ul><ul><li>3. Purpose of decision : </li></ul></ul><ul><ul><li>Section 2 measures the ability to recognize language that is appropriate for standard written English. </li></ul></ul>
  26. 32. Test Evaluation of An Existing Test <ul><ul><li>4. Date of Application: </li></ul></ul><ul><ul><li>a. 1962, the formation of a National Council on the TOEFL </li></ul></ul><ul><ul><li>b. July 1998 (Most parts of the world) </li></ul></ul><ul><ul><li>c. October 2000 (Asia, including Taiwan) </li></ul></ul>
  27. 33. Practical Orientation <ul><li>Ease of administration: </li></ul><ul><ul><li>Computers needed </li></ul></ul><ul><ul><li>Convenience </li></ul></ul><ul><li>Price: expensive ($130) </li></ul><ul><li>Quality of items: clarity </li></ul>
  28. 34. Practical Orientation <ul><li>Ease of scoring: </li></ul><ul><ul><li>Photo score reporting </li></ul></ul><ul><ul><li>Computerized scoring (0~30) </li></ul></ul><ul><ul><li>Cumulative scoring </li></ul></ul><ul><li>Ease of interpretation: transmission speed. </li></ul>
  29. 35. Practical Orientation <ul><li>Test construction: </li></ul><ul><ul><li>Computer-adaptive Testing </li></ul></ul><ul><ul><li>Complete an incomplete sentence </li></ul></ul><ul><ul><li>Identify one unacceptable answer form four underlined words. </li></ul></ul><ul><li>Timing: 20-25 questions/ 15-20 minutes (including “pretest questions”) </li></ul><ul><li>“ Description of the Computer-Based TOEFL Test” </li></ul>
  30. 36. Test Evaluation of An Existing Test <ul><li>Model Test </li></ul>
  31. 37. Analysis of TOEFL Grammar Test <ul><li>8 sentence patterns </li></ul><ul><li>Basic Sentence & Compound Sentence </li></ul><ul><li>Adjective Clauses </li></ul><ul><li>Participial Phrases </li></ul><ul><li>Appositive </li></ul><ul><li>Noun Clauses </li></ul><ul><li>Adverbs Clauses </li></ul><ul><li>Prepositional Phrases </li></ul><ul><li>Expletives </li></ul><ul><li>方有毅 托福文法速成 來欣留學英語中心 </li></ul>
  32. 38. Analysis of TOEFL Grammar Test <ul><li>15 Answering Techniques </li></ul><ul><li>Word Form ……… .... 20 % </li></ul><ul><li>Word Choice …… ... . 14 % </li></ul><ul><li>Verbs ……………… .... 11 % </li></ul><ul><li>Parallelism ……… ...... 9 % </li></ul><ul><li>Pronouns …………… ... 7 % </li></ul><ul><li>Singular/Plural Nouns … ... …… ... …… .. 6 % </li></ul><ul><li>Verbal (Infinitives, Gerunds, and Participles) …… .. … ... 6 % </li></ul><ul><li>Prepositions …… .. … .. 6% </li></ul><ul><li>Articles ……………………….5 % </li></ul><ul><li>Comparatives and </li></ul><ul><li>Superlative sentence……4 % </li></ul><ul><li>Conjunctions ……………… 4 % </li></ul><ul><li>Redundancy………………….1 % </li></ul><ul><li>Negative Words ……………2 % </li></ul><ul><li>Word Order ………………….4 % </li></ul><ul><li>Misplaced Modifiers ….…..1 % </li></ul><ul><li>Total ……………………….100 % </li></ul><ul><li>方有毅 托福文法速成 來欣留學英語中心 </li></ul>
  33. 39. Analysis of TOEFL Grammar Test <ul><li>Reliabilities: describes the tendency of a set of scores to be ordered identically on two or more tests, and it can be estimated by a variety of statistical procedures. </li></ul>“ Description of the Computer-Based TOEFL Test” 10.8 0.95 Total Score 2.73 0.88 Reading 4.89 0.88 Structure/Writing 2.76 0.98 Listening SEM Reliability July 1998 – June 1999 Table 1. Reliabilities and Standard Errors of Measurement (SEM)
  34. 40. Analysis of TOEFL Grammar Test <ul><li>Validity </li></ul><ul><li>------pure lead, the lead ore is mined, then melted, and finally refined. </li></ul><ul><li>(A) Obtaining (B) Being Obtained </li></ul><ul><li>(C) To obtain (D) It is obtained </li></ul><ul><li>Louisa May Alcott is mainly noted for her very popular </li></ul><ul><li>A B </li></ul><ul><li>novel, Little Women , whom she published in 1868. </li></ul><ul><li>C D </li></ul><ul><li>Some hangars, buildings used to hold large aircraft, are </li></ul><ul><li>A </li></ul><ul><li>very tall that rain occasionally falls from clouds that form </li></ul><ul><li>B C </li></ul><ul><li>along the ceilings. ( 聶群 精修文法 來欣留學英語中心 ) </li></ul><ul><li>D </li></ul>
  35. 41. Analysis of TOEFL Grammar Test <ul><li>Strength: </li></ul><ul><li>Accurate and objective scores </li></ul><ul><li>Careful monitoring of test </li></ul><ul><li>Commitment to international education </li></ul><ul><li>Expert advisory committees </li></ul><ul><li>Highly skilled test developers </li></ul><ul><li>Longstanding reliability and consistency </li></ul><ul><li>Ongoing test improvements </li></ul><ul><li>Standardized delivery procedures </li></ul><ul><li>“ Description of the Computer-Based TOEFL Test” </li></ul>
  36. 42. Analysis of TOEFL Grammar Test <ul><li>Conclusion </li></ul><ul><li>No other test in the world is as reliable as a standard for measuring a nonnative speaker’s ability to use English at the university level. </li></ul><ul><li>“ Description of the Computer-Based TOEFL Test” </li></ul>
  37. 43. <ul><li>END </li></ul>