Test construction 1

64,554 views

Published on

Published in: Education
38 Comments
107 Likes
Statistics
Notes
No Downloads
Views
Total views
64,554
On SlideShare
0
From Embeds
0
Number of Embeds
676
Actions
Shares
0
Downloads
6,231
Comments
38
Likes
107
Embeds 0
No embeds

No notes for slide

Test construction 1

  1. 1. Are you the kind of teacher who ask the following questions?
  2. 2. The poem “The Raven” ______ <ul><li>was written by Edgar Allan Poe </li></ul><ul><li>was written by Elizabeth Browning </li></ul><ul><li>was written by Omar Khayyan </li></ul><ul><li>was written by Jose Garcia Villa </li></ul>
  3. 3. <ul><li>Is it NOT true that Magellan discovered the Philippines? </li></ul>
  4. 4. When did the People Power Revolution take in the Philippines? <ul><li>February 23, 1986 </li></ul><ul><li>after the Snap Election </li></ul><ul><li>March 1, 1956 </li></ul><ul><li>after Valentines’ Day in 1986 </li></ul>
  5. 5. <ul><li>Who was the author of the book quoted in the footnote of Chapter 1 of the present textbook </li></ul>
  6. 6. If you answered “ YES ” to any of the choices presented then you have a BIG PROBLEM !
  7. 7. YOUR STUDENTS ARE NOT LEARNING!!!!
  8. 8. <ul><li>“ 13% of students who fail in class are caused by faulty test questions” </li></ul><ul><li>WORLDWATCH </li></ul><ul><li>The Philadelphia Trumpet </li></ul><ul><li>August 2005 </li></ul>
  9. 9. <ul><li>It is estimated that 90% of all test questions asked in the US are of “Low level” - knowledge and comprehension (Wilen, W.W., 1992) </li></ul>
  10. 10. <ul><li>“Low level” doesn’t mean easy: </li></ul><ul><ul><ul><li>Write an essay explaining the decline and fall of the Roman Empire incorporating at least five of the seven causes discussed in class from the writings of Gibbon and Toynbee </li></ul></ul></ul><ul><li>“High level” doesn’t mean hard: </li></ul><ul><ul><ul><li>Which movie did you like more, WALL-E or Cars ? Why? </li></ul></ul></ul>
  11. 11. TEST CONSTRUCTION Presented by: Arnel O. Rivera http://www.slideshare.net/ArnelSSI
  12. 12. Outline: <ul><li>Part I </li></ul><ul><ul><li>Principles in Test Construction </li></ul></ul><ul><ul><li>Steps in Preparing Test Questions </li></ul></ul><ul><ul><li>Preparing Multiple Choice Questions </li></ul></ul><ul><ul><li>Preparing True or False Questions </li></ul></ul><ul><li>Part II </li></ul><ul><ul><li>Review of Part I </li></ul></ul><ul><ul><li>Preparing Matching Type Questions </li></ul></ul><ul><ul><li>Preparing Sentence Completion Questions </li></ul></ul><ul><ul><li>Preparing Essay Questions </li></ul></ul><ul><ul><li>Other types of Test Questions </li></ul></ul><ul><ul><li>Wrap-up/Things to Remember </li></ul></ul>
  13. 13. <ul><li>“ The evaluation of pupils’ progress is a major aspect of the teacher’s job.“ </li></ul><ul><li>Evaluating Educational Outcomes </li></ul><ul><li>(Oriondo & Antonio) </li></ul>
  14. 14. Explain the message of the comic strip.
  15. 15. The Purpose of Testing <ul><li>To provide a record for assigning grades. </li></ul><ul><li>To provide a learning experience for students. </li></ul><ul><li>To motivate students to learn. </li></ul><ul><li>To serve as a guide for further study. </li></ul>
  16. 16. The Purpose of Testing <ul><li>To assess how well students are achieving the stated goals of the lesson. </li></ul><ul><li>To provide the instructor with an opportunity to reinforce the stated objectives and highlight what is important for students to remember. </li></ul>
  17. 17. Characteristics of Good Tests <ul><li>Validity – the extent to which the test measures what it intends to measure </li></ul><ul><li>Reliability – the consistency with which a test measures what it is supposed to measure </li></ul><ul><li>Usability – the test can be administered with ease, clarity and uniformity </li></ul>
  18. 18. <ul><li>Scorability – easy to score </li></ul><ul><li>Interpretability – test results can be properly interpreted and is a major basis in making sound educational decisions </li></ul><ul><li>Economical – the test can be reused without compromising the validity and reliability </li></ul>Other Things to Consider
  19. 19. <ul><li>“ To be able to prepare a good test, one has to have a mastery of the subject matter, knowledge of the pupils to be tested, skill in verbal expression and the use of the different test format ” </li></ul><ul><li>Evaluating Educational Outcomes </li></ul><ul><li>(Oriondo & Antonio) </li></ul>
  20. 20. <ul><li>Multiple Choice </li></ul><ul><li>True or False </li></ul><ul><li>Matching Type </li></ul><ul><li>Fill-in the blanks (Sentence Completion) </li></ul><ul><li>Essay </li></ul>5 Most Commonly used Test Format Source: Turn-out of Test Questions in SSI (2003-2007)
  21. 21. General Steps in Test Construction OUTLINE DRAFT ORDER TEST ANALYZE SUBMISSION PRODUCE A T.O.S.
  22. 22. OUTLINE: <ul><ul><li>the unit learning objectives or </li></ul></ul><ul><ul><li>the unit content or major concepts to be covered by the test </li></ul></ul>Back to Main Menu
  23. 23. Table of Specifications (TOS) <ul><li>A two way chart that relates the learning outcomes to the course content </li></ul><ul><li>It enables the teacher to prepare a test containing a representative sample of student behavior in each of the areas tested. </li></ul>
  24. 25. <ul><li>Don’t make it overly detailed. </li></ul><ul><li>It's best to identify major ideas and skills rather than specific details. </li></ul><ul><li>Use a cognitive taxonomy that is most appropriate to your discipline, including non-specific skills like communication skills or graphic skills or computational skills if such are important to your evaluation of the answer. </li></ul>Tips in Preparing the Table of Specifications (TOS)
  25. 26. <ul><li>Weigh the appropriateness of the distribution of checks against the students' level, the importance of the test, the amount of time available. </li></ul><ul><li>MATCH the question level appropriate to the level of thinking skills </li></ul>Tips in Preparing the Table of Specifications (TOS)
  26. 27. Examples of Student Activities and Verbs for Bloom’s Cognitive Levels Table 2.1 in Jacobs & Chase (1992:19) Apply, solve, show, make use of, modify, demonstrate, compute Using a concept or principle to solve a problem Application Explain, predict, interpret, infer, summarize, convert, translate, account for, give example, paraphrase Explaining/interpreting the meaning of material Comprehension Define, list, state, identify, label, name, who?, when?, where?, what? Remembering facts, terms, concepts, definitions, principles Knowledge Words to Use in Item Stem Student Activity Bloom’s Cognitive Level
  27. 28. Examples of Student Activities and Verbs for Bloom’s Cognitive Levels Table 2.1 in Jacobs & Chase (1992:19) Appraise, evaluate, justify, judge, which would be better? Making a judgment based on a pre-established set of criteria Evaluation Design, construct, develop, formulate, imagine, create, change, write a poem or short story Producing something new or original from component parts Synthesis Differentiate, compare/contrast, distinguish ____from ____, how does ____relate to ___, why does ____work Breaking material down into its component parts to see interrelationships/ hierarchy of ideas Analysis Words to Use in Item Stem Student Activity Bloom’s Cognitive Level
  28. 29. Tips in Preparing the Table of Specifications (TOS) <ul><li>The following array shows the most common questions types used at various cognitive levels. </li></ul>Multiple Choice Essay Multiple Choice Short Answer Problems Essay Multiple Choice True/False Matching Type S. Completion Short Answer/RRT Analysis and Evaluation Application Factual Knowledge
  29. 30. Activity: <ul><li>Prepare a short TOS using the selection in your activity sheet. </li></ul>Back to Main Menu
  30. 31. DRAFT the questions covering the content in the outline Back to Main Menu
  31. 32. ORDER the selected questions logically. <ul><li>Place simpler items at the beginning to ease students into the exam. </li></ul><ul><li>Group item types together under common instructions. </li></ul><ul><li>If desirable, order the questions logically from a content standpoint (e.g. chronologically or by conceptual groups, etc.) </li></ul>Back to Main Menu
  32. 33. Test <ul><li>PUT the questions away for one or two days before rereading them or have someone else review them for clarity. </li></ul><ul><li>TEST the questions by actually taking the test. </li></ul>Back to Main Menu
  33. 34. <ul><li>ANALYZE the items to give you an idea whether the questions were well-written or poorly written as well as if there were problems in understanding instruction. </li></ul>Back to Main Menu
  34. 35. General Rules in Writing Test Questions <ul><li>Number test questions continuously. </li></ul><ul><li>Keep your test question in each test group uniform. </li></ul><ul><li>Make your layout presentable. </li></ul><ul><li>Do not put too many test questions in one test group. </li></ul><ul><ul><li>T or F: 10 – 15 questions </li></ul></ul><ul><ul><li>Multiple Choice: max. of 30 questions </li></ul></ul><ul><ul><li>Matching type: 5 questions per test group </li></ul></ul><ul><ul><li>Others: 5 – 10 questions </li></ul></ul>
  35. 36. Some additional guidelines to consider when writing items are described below: <ul><li>Avoid humorous items. Classroom testing is very important and humorous items may cause students to either not take the exam seriously or become confused or anxious. </li></ul><ul><li>Items should measure only the construct of interest, not one’s knowledge of the item context. </li></ul><ul><li>Write items to measure what students know, not what they do not know. (Cohen & Wallack) </li></ul>
  36. 37. Multiple Choice Test
  37. 38. <ul><li>When checking the stems for correctness: </li></ul><ul><li>Ensure that the stem asks a clear question. </li></ul><ul><li>Reading level is appropriate to the students </li></ul><ul><li>The stem is grammatically correct . </li></ul><ul><li>Negatively stated stems are discouraged. </li></ul>What to Look for on Multiple Choice Tests
  38. 39. <ul><li>Example: </li></ul><ul><li>What is the effect of releasing a ball in positive gravity? </li></ul><ul><li>a) It will fall “down.” correct </li></ul><ul><li>b) It will retain its mass. true but unrelated </li></ul><ul><li>c) It will rise. false but related </li></ul><ul><li>d) Its shape will change. false and unrelated </li></ul>What to Look for on Multiple Choice Tests
  39. 40. Multiple Choice Questions <ul><li>Use negatively stated stems sparingly and when using negatives such as NOT , underline or bold the print. </li></ul><ul><li>Use none of the above and all of the above sparingly, and when you do use them, don't always make them the right answer. </li></ul><ul><li>Only one option should be correct or clearly best. </li></ul>
  40. 41. Multiple Choice Questions: <ul><li>All options should be homogenous and nearly equal in length. </li></ul><ul><li>The stem (question) should contain only one main idea. </li></ul><ul><li>Keep all options either singular or plural. </li></ul><ul><li>Have four or five responses per stem (question). </li></ul>
  41. 42. Multiple Choice Questions: <ul><li>When using incomplete statements place the blank space at the end of the stem versus the beginning. </li></ul><ul><li>When possible organize the responses. </li></ul><ul><li>Reduce wordiness. </li></ul><ul><li>When writing distracters, think of incorrect responses that students might make. </li></ul>
  42. 43. Examples <ul><li>Sheldon developed a highly controversial theory of personality based on body type and temperament of the individual. Which of the following is a criticism of Sheldon's work? </li></ul><ul><li>a. He was influenced too much by the Freudian psychoanalysis. b. His rating of physique and temperament were not independent. c. He failed to use empirical approach. d. His research sample was improperly selected. </li></ul>
  43. 44. Examples <ul><li>Better: (Eliminate excessive wording and irrelevant information) </li></ul><ul><li>1. Which of the following is a criticism of Sheldon's theory of personality? </li></ul>
  44. 45. Examples <ul><li>The receptors for the vestibular sense are located </li></ul><ul><li>a. in the fovea. b. in the brain. c. in the middle ear. d. in the inner ear. </li></ul>
  45. 46. Examples <ul><li>Better: (Include in the stem any word(s) that might otherwise be repeated in each option.) </li></ul><ul><li>The receptors for the vestibular senses are located in the _______. </li></ul><ul><li>a. fovea b. brain c. middle ear d. inner ear </li></ul>
  46. 47. Examples <ul><li>Which is not a major technique for studying brain function? </li></ul><ul><li>a. Accident and injury b. Cutting and removing c. Electrical stimulation d. Direct phrenology </li></ul>
  47. 48. Examples <ul><li>Better: (Use negatively stated stems sparingly. When used, underline and/or capitalize the negative word.) </li></ul><ul><li>Which is NOT a major technique for studying brain function? </li></ul>
  48. 49. Examples <ul><li>4. ________________ is the least form of behavior disorder. </li></ul><ul><li>a. Psychosis b. Panic disorder c. Neurasthenia d. Neurosis </li></ul>
  49. 50. Examples <ul><li>Better: (When using incomplete statements avoid beginning with the blank space.) </li></ul><ul><li>The least severe form of behavior disorder is __________________. </li></ul>
  50. 51. Examples <ul><li>The number of photoreceptors in the retina of each human is about </li></ul><ul><li>a. 115 million b. 5 million c. 65 million d. 35 billion </li></ul>
  51. 52. Examples <ul><li>Better: ( When possible, present alternatives in some logical order.) </li></ul><ul><li>The number of photo receptors in the retina of each human is about </li></ul><ul><li>a. 5 million b. 35 million c. 65 million d. 115 million </li></ul>
  52. 53. Examples <ul><li>6. Latane and Darley's smoke-filled room experiment suggested that people are less likely to help in groups than alone, because people </li></ul><ul><li>a. in groups talk to one another. b. who are alone are more attentive. c. in groups do not display pluralistic ignorance. d. in groups allow others to define the situation as a non-emergency </li></ul>
  53. 54. Examples <ul><li>Better: (All alternatives should be approximately equal in length.) </li></ul><ul><li>6. Latane and Darley's smoke-filled room experiment suggested that people are less likely to help in groups than alone, because people in groups </li></ul><ul><li>a. talk to one another </li></ul><ul><li>b. are less attentive than people who are alone </li></ul><ul><li>c. do not display pluralistic ignorance </li></ul><ul><li>d. allow other to define non-emergencies </li></ul>
  54. 55. Activity: <ul><li>Prepare two multiple choice questions based on the selection in your activity sheet. </li></ul>
  55. 56. True or False
  56. 57. <ul><li>Each statement is clearly true or clearly false. </li></ul><ul><li>Trivial details should not make a statement false. </li></ul><ul><li>Statements are written concisely without more elaboration than necessary. </li></ul><ul><li>Statements are NOT quoted exactly from text. </li></ul>What to Look for on True/False Tests
  57. 58. <ul><li>Give emphasis on the use of quantitative terms than qualitative terms. </li></ul><ul><li>Avoid using of specific determiners which usually gives a clue to the answer. </li></ul><ul><ul><li>False = all, always, never, every, none, only </li></ul></ul><ul><ul><li>True = generally, sometimes, usually, maybe, often </li></ul></ul><ul><li>Discourage the use of negative statements. </li></ul><ul><li>Whenever a controversial statement is used, the authority should be quoted. </li></ul><ul><li>Discourage the use of pattern for answers. </li></ul>Tips in Making True/False Tests
  58. 59. Examples: <ul><li>____ 1. Repetition always strengthens the tendency for a response to occur. </li></ul><ul><li>(Using &quot;always&quot; usually means the answer is false.) </li></ul>Find the errors, and/or problems with the following true-false tests.
  59. 60. Examples: <ul><li>_____ 2. The process of extinction is seldom immediate but extends over a number of trials. </li></ul><ul><li>( Words like &quot;seldom&quot; usually indicate a true statement.) </li></ul>
  60. 61. Examples: <ul><li>_____ 3. The mean, median, and mode are measures of central tendency, whereas the standard deviation and range are measures of variability. </li></ul><ul><li>(Express a single idea in each statement.) e.g.“The mean and standard deviation are measures of central tendency.” </li></ul>
  61. 62. Activity: <ul><li>Prepare two true or false questions based on the selection in your activity sheet. </li></ul>

×