Insights and development of tests

  • 276 views
Uploaded on

 

More in: Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
276
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
18
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Insights andInsights and Development ofDevelopment of TestsTests Carlo Magno, PhDCarlo Magno, PhD De La Salle University, ManilaDe La Salle University, Manila
  • 2. Activity 1: Assessment SchemaActivity 1: Assessment Schema Check-upCheck-up Answer the following questions as a group. YourAnswer the following questions as a group. Your answers should reflect your current practices inanswers should reflect your current practices in assessing your students. Write your answers in a manilaassessing your students. Write your answers in a manila paper and post it after.paper and post it after. 1. List down the things that you do when preparing to1. List down the things that you do when preparing to write your test items?write your test items? (procedure)(procedure) 2. What are the things that you consider when writing2. What are the things that you consider when writing your test items?your test items? (concepts)(concepts) 3. What further steps do you do after you have scored3. What further steps do you do after you have scored and recorded the test papers?and recorded the test papers? (procedure)(procedure) 4. What other forms of assessment do you conduct4. What other forms of assessment do you conduct aside from paper and pencil tests?aside from paper and pencil tests?
  • 3. 1. List down the things that you do when1. List down the things that you do when preparing to write your test items?preparing to write your test items? Prepare Table of Specifications (TOS)Prepare Table of Specifications (TOS) Use the Taxonomy of Cognitive skillsUse the Taxonomy of Cognitive skills (Bloom’s taxonomy)(Bloom’s taxonomy) Conduct Item reviewConduct Item review
  • 4. 2. What are the things that you consider2. What are the things that you consider when writing your test items?when writing your test items? Learning objectivesLearning objectives Curriculum/national standardsCurriculum/national standards Needs of studentsNeeds of students Higher order thinking skillsHigher order thinking skills Test lengthTest length Test instructionTest instruction Test layoutTest layout ScoringScoring
  • 5. 3. What further steps do you do after you3. What further steps do you do after you have scored and recorded the test papers?have scored and recorded the test papers? Item analysisItem analysis – Item difficultyItem difficulty – Item discriminationItem discrimination Distracter analysisDistracter analysis Reliability analysisReliability analysis Validity analysisValidity analysis
  • 6. 4. What other forms of assessment do you4. What other forms of assessment do you conduct aside from paper and pencil tests?conduct aside from paper and pencil tests? Alternative forms of assessmentAlternative forms of assessment – Performance-based AssessmentPerformance-based Assessment – Authentic assessmentAuthentic assessment – Portfolio assessmentPortfolio assessment
  • 7. Types of Response FormatTypes of Response Format Selected-response itemsSelected-response items – Binary-choice itemsBinary-choice items – Matching typeMatching type – Multiple choice typeMultiple choice type Constructed-responseConstructed-response – Short-answer itemsShort-answer items – Cloze testCloze test – Essay itemsEssay items
  • 8. Binary TypeBinary Type 1. Avoid the use of “specific determiners”1. Avoid the use of “specific determiners” FAULTY: No picture-no sound in aFAULTY: No picture-no sound in a television set may indicate a bad 5U4G.television set may indicate a bad 5U4G. IMPROVED: A bad 5U4G tube in aIMPROVED: A bad 5U4G tube in a television set will result in no picturetelevision set will result in no picture sound.sound. Why is the item faulty?
  • 9. Binary TypeBinary Type 2. Base true-false items upon statements2. Base true-false items upon statements that are absolutely true or false, withoutthat are absolutely true or false, without qualifications or exceptions.qualifications or exceptions. FAULTY: World War II was fought inFAULTY: World War II was fought in Europe and the Far East.Europe and the Far East. IMPROVED: The primary combatIMPROVED: The primary combat locations in terms of military personnellocations in terms of military personnel during World War II were Europe and theduring World War II were Europe and the Far East.Far East. Why is the item faulty?
  • 10. Binary TypeBinary Type 3. Avoid negative stated items when3. Avoid negative stated items when possible and eliminate all doublepossible and eliminate all double negatives.negatives. FAULTY: It is not frequently observed thatFAULTY: It is not frequently observed that copper turns green as a result ofcopper turns green as a result of oxidation.oxidation. IMPROVED: Copper will turn green uponIMPROVED: Copper will turn green upon oxidizing.oxidizing. Why is the item faulty?
  • 11. Binary TypeBinary Type 4. Use quantitative and precise rather4. Use quantitative and precise rather than qualitative language where possible.than qualitative language where possible. FAULTY: Many people voted for GloriaFAULTY: Many people voted for Gloria Arroyo in the 2003 Presidential election.Arroyo in the 2003 Presidential election. IMPROVED: Gloria Arroyo received moreIMPROVED: Gloria Arroyo received more than 60 percent of the popular votes castthan 60 percent of the popular votes cast in the Presidential election of 2003.in the Presidential election of 2003. Why is the item faulty?
  • 12. Binary TypeBinary Type 5. Avoid stereotypic and textbook statements.5. Avoid stereotypic and textbook statements. FAULTY: From time to time efforts have beenFAULTY: From time to time efforts have been made to explode the notion that there may be amade to explode the notion that there may be a cause-and-effect relationship between arborealcause-and-effect relationship between arboreal life and primate anatomy.life and primate anatomy. IMPROVED: There is a known relationshipIMPROVED: There is a known relationship between primate anatomy and arboreal life.between primate anatomy and arboreal life. Why is the item faulty?
  • 13. Binary TypeBinary Type 6. Avoid making the true items consistently6. Avoid making the true items consistently longer than the false items.longer than the false items. 7. Avoid the use of unfamiliar or esoteric7. Avoid the use of unfamiliar or esoteric language.language. FAULTY: According to some peripateticFAULTY: According to some peripatetic politicos, the raison d’etre for capital punishmentpoliticos, the raison d’etre for capital punishment is retribution.is retribution. IMPROVED: According to some politicians,IMPROVED: According to some politicians, justification for the existence of capitaljustification for the existence of capital punishment can be traced to the Biblicalpunishment can be traced to the Biblical statement, “An eye for an eye.”statement, “An eye for an eye.” Why is the item faulty?
  • 14. Binary TypeBinary Type 8. Avoid complex sentences with many8. Avoid complex sentences with many dependent clauses.dependent clauses. FAULTY: Jane Austen, an AmericanFAULTY: Jane Austen, an American novelist born in 1790, was a prolific writernovelist born in 1790, was a prolific writer and is best known for her novel Pride andand is best known for her novel Pride and Prejudice, which was published in 1820.Prejudice, which was published in 1820. IMPROVED: Jane Austen is best knownIMPROVED: Jane Austen is best known for her novel Pride and prejudice.for her novel Pride and prejudice. Why is the item faulty?
  • 15. Binary TypeBinary Type 9. It is suggested that the crucial elements of9. It is suggested that the crucial elements of an item be placed at the end of the statement.an item be placed at the end of the statement. FAULTY: Oxygen reduction occurs more readilyFAULTY: Oxygen reduction occurs more readily because carbon monoxide combines withbecause carbon monoxide combines with hemoglobin faster than oxygen does.hemoglobin faster than oxygen does. IMPROVED: Carbon monoxide poisoning occursIMPROVED: Carbon monoxide poisoning occurs because carbon monoxide dissolves delicatebecause carbon monoxide dissolves delicate lung tissue.lung tissue. Why is the item faulty?
  • 16. Matching TypeMatching Type Match List A with List B. You will be given oneMatch List A with List B. You will be given one point for each correct match.point for each correct match. List AList A List BList B a. cotton gina. cotton gin a. Eli Whitneya. Eli Whitney b. reaperb. reaper b. Alexander Graham Bellb. Alexander Graham Bell c. wheelc. wheel c. David Brinkleyc. David Brinkley d. TU54G tubed. TU54G tube d. Louisa May Alcottd. Louisa May Alcott e. steamboate. steamboat e. None of thesee. None of these •Directions failed to specify the basis for matching •List are enumerated identically •Responses not listed logically •Lacks homogeneity •Equal number of elements •Use of “None of the above” Why is the item faulty?
  • 17. Matching TypeMatching Type IMPROVED: Famous inventions are listed in the left-handIMPROVED: Famous inventions are listed in the left-hand column and inventors in the right-hand column below.column and inventors in the right-hand column below. Place the letter corresponding to the inventor in the spacePlace the letter corresponding to the inventor in the space next to the invention for which he s famous. Each match isnext to the invention for which he s famous. Each match is worth 1 point, and “None of these” may be the correctworth 1 point, and “None of these” may be the correct answer. Inventors may be used more than once.answer. Inventors may be used more than once. InventionsInventions InventorsInventors __ 1. steamboat__ 1. steamboat a. Alexander Graham-Bella. Alexander Graham-Bell __ 2. cotton skin__ 2. cotton skin b. Robert Fultonb. Robert Fulton __ 3. sewing machine__ 3. sewing machine c. Elias Howec. Elias Howe __ 4. reaper__ 4. reaper d. Cyrus McCormickd. Cyrus McCormick e. Eli Whitneye. Eli Whitney f. None of thesef. None of these
  • 18. Multiple ChoiceMultiple Choice 1. Salvador Dali is1. Salvador Dali is a. a famous Indian.a. a famous Indian. b. important in international law.b. important in international law. c. known for his surrealistic art.c. known for his surrealistic art. d. the author of many avant-garde plays.d. the author of many avant-garde plays. •It is recommended that the stem be a direct question. •The stem should pose a clear, define, explicit, and singular problem. Why is the item faulty?
  • 19. Multiple ChoiceMultiple Choice IMPROVED: With which one of the fineIMPROVED: With which one of the fine arts is Salvador Dali associated?arts is Salvador Dali associated? a. surrealistic paintinga. surrealistic painting b. avant-garde theatreb. avant-garde theatre c. polytonal symphonic musicc. polytonal symphonic music d. impressionistic poetryd. impressionistic poetry
  • 20. Multiple ChoiceMultiple Choice 2. Milk can be pasteurized at home by2. Milk can be pasteurized at home by a. heating it to a temperature of 130a. heating it to a temperature of 130oo b. Heating it to a temperature of 145b. Heating it to a temperature of 145oo c. Heating it to a temperature of 160c. Heating it to a temperature of 160oo d. Heating it to a temperature of 175d. Heating it to a temperature of 175oo •Include in the stem any words that might otherwise be repeated in each response. Why is the item faulty?
  • 21. Multiple ChoiceMultiple Choice IMPROVED: The minimum temperatureIMPROVED: The minimum temperature that can be used to pasteurize milk atthat can be used to pasteurize milk at home is:home is: a. 130a. 130oo b. 145b. 145oo c. 160c. 160oo d. 175d. 175oo
  • 22. Multiple ChoiceMultiple Choice 3. Although the experimental research, particularly that by3. Although the experimental research, particularly that by Hansmocker must be considered equivocal andHansmocker must be considered equivocal and assumptions viewed as too restrictive, most testingassumptions viewed as too restrictive, most testing experts would recommend as the easiest method ofexperts would recommend as the easiest method of significantly improving paper-and-pencil achievementsignificantly improving paper-and-pencil achievement test reliability totest reliability to a. increase the size of the group being tested.a. increase the size of the group being tested. b. increase the differential weighting of items.b. increase the differential weighting of items. c. increase the objective of scoring.c. increase the objective of scoring. d. increase the number of items.d. increase the number of items. e. increase the amount of testing time.e. increase the amount of testing time. Items should be stated simply and understandably, excluding all nonfunctional words from stem and alternatives. Why is the item faulty?
  • 23. Multiple ChoiceMultiple Choice IMPROVED: Assume a 10-item, 10-minuteIMPROVED: Assume a 10-item, 10-minute paper-and-pencil multiple choice achievementpaper-and-pencil multiple choice achievement test has a reliability of .40. The easiest way oftest has a reliability of .40. The easiest way of increasing the reliability to .80 would be toincreasing the reliability to .80 would be to increasedincreased a. group sizea. group size b. scoring objectivityb. scoring objectivity c. differential item scoring weightsc. differential item scoring weights d. the number of itemsd. the number of items e. testing timee. testing time
  • 24. Multiple ChoiceMultiple Choice 4. None of the following cities is a state4. None of the following cities is a state capital exceptcapital except a. Bangora. Bangor b. Los Angelesb. Los Angeles c. Denverc. Denver d. New Havend. New Haven •Avoid negatively stated items Why is the item faulty?
  • 25. Multiple ChoiceMultiple Choice IMPROVED: Which of the following citiesIMPROVED: Which of the following cities is a state capital?is a state capital? a. Bangora. Bangor b. Los Angelesb. Los Angeles c. Denverc. Denver d. New Havend. New Haven
  • 26. Multiple ChoiceMultiple Choice 5. Who wrote Harry Potter and the Goblet5. Who wrote Harry Potter and the Goblet of Fire?of Fire? a. J. K. Rowlinga. J. K. Rowling b. Manny Paquiaob. Manny Paquiao c. Lea Salongac. Lea Salonga d. Mark Twaind. Mark Twain •If possible the alternatives should be presented in some logical, numerical, or systematic order. •Response alternatives should be mutually exclusive. Why is the item faulty?
  • 27. Multiple ChoiceMultiple Choice IMPROVED: Who wrote Penrod?IMPROVED: Who wrote Penrod? a. J. K. Rowlinga. J. K. Rowling b. J. R. R. Tolkienb. J. R. R. Tolkien c. V. Hugoc. V. Hugo d. L. Carrold. L. Carrol
  • 28. Multiple ChoiceMultiple Choice 6. Which of the following statements6. Which of the following statements makes clear the meaning of the wordmakes clear the meaning of the word “electron”?“electron”? a. An electronic toola. An electronic tool b. Neutral particlesb. Neutral particles c. Negative particlesc. Negative particles d. A voting machined. A voting machine e. The nuclei of atomse. The nuclei of atoms •Make all responses plausible and attractive to the less knowledgeable and skillful student. Why is the item faulty?
  • 29. Multiple ChoiceMultiple Choice IMPROVED: Which of the followingIMPROVED: Which of the following phrases is a description of an “electron”?phrases is a description of an “electron”? a. Neutral particlea. Neutral particle b. Negative particleb. Negative particle c. Neutralized protonc. Neutralized proton d. Radiated particled. Radiated particle e. Atom nucleuse. Atom nucleus
  • 30. Multiple ChoiceMultiple Choice 7. What is the area of a right triangle7. What is the area of a right triangle whose sides adjacent to the right anglewhose sides adjacent to the right angle are 4 inches long respectively?are 4 inches long respectively? a. 7a. 7 b. 12b. 12 c. 25c. 25 d. None of the aboved. None of the above •The response alternative “None of the above” should be used with caution, if at all. Why is the item faulty?
  • 31. Multiple ChoiceMultiple Choice IMPROVED: What is the area of a rightIMPROVED: What is the area of a right triangle whose sides adjacent to the righttriangle whose sides adjacent to the right angle are 4 inches and 3 inchesangle are 4 inches and 3 inches respectively?respectively? a. 6 sq. inchesa. 6 sq. inches b. 7 sq. inchesb. 7 sq. inches c. 12 sq. inchesc. 12 sq. inches d. 25 sq. inchesd. 25 sq. inches e. None of the abovee. None of the above
  • 32. Multiple ChoiceMultiple Choice 8. As compared with the American factory8. As compared with the American factory worker in the early part of the 19th century, theworker in the early part of the 19th century, the American factory worker at the close of theAmerican factory worker at the close of the centurycentury a. was working long hoursa. was working long hours b. received greater social security benefitsb. received greater social security benefits c. was to receive lower money wagesc. was to receive lower money wages d. was less likely to belong to a labor union.d. was less likely to belong to a labor union. e. became less likely to have personal contacte. became less likely to have personal contact with employerswith employers Make options grammatically parallel to each other and consistent with the stem. Why is the item faulty?
  • 33. Multiple ChoiceMultiple Choice IMPROVED: As compared with the AmericanIMPROVED: As compared with the American factory worker in the early part of the century,factory worker in the early part of the century, the American factory worker at the close of thethe American factory worker at the close of the centurycentury a. worked longer hours.a. worked longer hours. b. had more social security.b. had more social security. c. received lower money wages.c. received lower money wages. d. was less likely to belong to a labor uniond. was less likely to belong to a labor union e. had less personal contact with his employere. had less personal contact with his employer
  • 34. Multiple ChoiceMultiple Choice 9. The “standard error of estimate’ refer to9. The “standard error of estimate’ refer to a. the objectivity of scoring.a. the objectivity of scoring. b. the percentage of reduced errorb. the percentage of reduced error variance.variance. c. an absolute amount of possible error.c. an absolute amount of possible error. d. the amount of error in estimatingd. the amount of error in estimating criterion scores.criterion scores. Avoid such irrelevant cues as “common elements” and “pat verbal associations.” Why is the item faulty?
  • 35. Multiple ChoiceMultiple Choice IMPROVED: The “standard error ofIMPROVED: The “standard error of estimate” is most directly related to whichestimate” is most directly related to which of the following test characteristic?of the following test characteristic? a. Objectivitya. Objectivity b. Reliabilityb. Reliability c. Validityc. Validity d. Usabilityd. Usability e. Specificitye. Specificity
  • 36. Multiple ChoiceMultiple Choice 10. What name is given to the group of10. What name is given to the group of complex organic compounds that occur incomplex organic compounds that occur in small quantities in natural foods that aresmall quantities in natural foods that are essential to normal nutrition?essential to normal nutrition? a. Caloriea. Calorie b. Mineralsb. Minerals c. Nutrientsc. Nutrients d. Vitaminsd. Vitamins In testing for understanding of a term or concept, it is generally preferable to present the term in the stem and alternative definitions in the options. Why is the item faulty?
  • 37. Multiple ChoiceMultiple Choice IMPROVED: Which of the followingIMPROVED: Which of the following statements is the best description of astatements is the best description of a vitamin?vitamin?
  • 38. Short Answer TypeShort Answer Type 1. Require short, definite, clear-cut, and1. Require short, definite, clear-cut, and explicit answersexplicit answers FAULTY: Earnest Hemingway wrote______FAULTY: Earnest Hemingway wrote______ IMPROVED: The Old Man and the Sea wasIMPROVED: The Old Man and the Sea was written by _______.written by _______. Who wrote The Old man and the Sea?Who wrote The Old man and the Sea? Why is the item faulty?
  • 39. Short Answer TypeShort Answer Type 2. Avoid multimutilated statements2. Avoid multimutilated statements FAULTY: _____ pointed out in ____ theFAULTY: _____ pointed out in ____ the freedom of thought in America wasfreedom of thought in America was seriously hampered by ___, ____, & __.seriously hampered by ___, ____, & __. IMPROVED: That freedom of thought inIMPROVED: That freedom of thought in America was seriously hampered by socialAmerica was seriously hampered by social pressures toward conformity was pointedpressures toward conformity was pointed out in 1830 by ______.out in 1830 by ______. Why is the item faulty?
  • 40. Short Answer TypeShort Answer Type 3. If several equal answers equal credit3. If several equal answers equal credit should be given to each one.should be given to each one. 4. Specify and announce in advance4. Specify and announce in advance whether scoring will take spelling intowhether scoring will take spelling into account.account.
  • 41. Short Answer TypeShort Answer Type 5. In testing for comprehension of terms and5. In testing for comprehension of terms and knowledge of definition, it is often better toknowledge of definition, it is often better to supply the term and require a definition than tosupply the term and require a definition than to provide a definition and require the term.provide a definition and require the term. FAULTY: What is the general measurement termFAULTY: What is the general measurement term describing the consistency with which items in adescribing the consistency with which items in a test measure the same thing?test measure the same thing? IMPROVED: Define “internal consistencyIMPROVED: Define “internal consistency reliability.”reliability.” Why is the item faulty?
  • 42. Short Answer TypeShort Answer Type 6. It is generally recommended that in completion6. It is generally recommended that in completion items the blanks come at the end of theitems the blanks come at the end of the statement.statement. FAULTY: A (an) ________ is the index obtainedFAULTY: A (an) ________ is the index obtained by dividing a mental age score by chronologicalby dividing a mental age score by chronological age and multiplying by 100.age and multiplying by 100. IMPROVED: The index obtained by dividing aIMPROVED: The index obtained by dividing a mental age score by chronological age andmental age score by chronological age and multiplying by 100 is called a (an) ________multiplying by 100 is called a (an) ________ Why is the item faulty?
  • 43. Short Answer TypeShort Answer Type 7. Minimize the use of textbook expressions7. Minimize the use of textbook expressions and stereotyped language.and stereotyped language. FAULTY: The power to declare war isFAULTY: The power to declare war is vested in ______vested in ______ IMPROVED: Which national legislative bodyIMPROVED: Which national legislative body has the authority to declare war?has the authority to declare war? Why is the item faulty?
  • 44. Short Answer TypeShort Answer Type 8. Specify the terms in which the response is to be8. Specify the terms in which the response is to be given.given. FAULTY: Where does the Security Council of theFAULTY: Where does the Security Council of the United Nations hold its meeting?United Nations hold its meeting? IMPROVED: In what city of the United States doesIMPROVED: In what city of the United States does the Security Council of the United Nations holdthe Security Council of the United Nations hold its meeting?its meeting? FAULTY: If a circle has 4-inch diameter, its areaFAULTY: If a circle has 4-inch diameter, its area is_____is_____ IMPROVED: A circle has 4-inch diameter. Its areaIMPROVED: A circle has 4-inch diameter. Its area in square inches correct to two decimal places,in square inches correct to two decimal places, is _____is _____ Why is the item faulty?
  • 45. Short Answer TypeShort Answer Type 9. In general, direct questions are9. In general, direct questions are preferable to incomplete declarativepreferable to incomplete declarative sentences.sentences. FAULTY: Gold was discovered in CaliforniaFAULTY: Gold was discovered in California in the year ___in the year ___ IMPROVED: In what year was goldIMPROVED: In what year was gold discovered in California?discovered in California? Why is the item faulty?
  • 46. Short Answer TypeShort Answer Type 10. Avoid extraneous clues to the correct10. Avoid extraneous clues to the correct answeranswer FAULTY: A fraction whose denominator isFAULTY: A fraction whose denominator is greater than its numerator is a _____greater than its numerator is a _____ IMPROVED: Fractions whose denominatorIMPROVED: Fractions whose denominator are greater than their numerators areare greater than their numerators are called _____called _____ Why is the item faulty?
  • 47. ReferenceReference Magno, C., & Ouano, J. (2010).Magno, C., & Ouano, J. (2010). DesigningDesigning Written Assessment for student learningWritten Assessment for student learning.. Manila: Phoenix.Manila: Phoenix.
  • 48. Activity 4: Insights onActivity 4: Insights on AssessmentAssessment 1.1. After the activities, I realized that my testAfter the activities, I realized that my test …… 2.2. After the activity, now I know that …After the activity, now I know that … 3.3. After the activity, every time I constructAfter the activity, every time I construct test, I will …test, I will …