Item and Distracter Analysis
•Download as PPT, PDF•
52 likes•39,006 views
The document discusses item analysis, which is the process of examining test responses to evaluate the quality of individual test items and the test itself. It aims to improve the effectiveness of items used on future tests. Key aspects covered include item difficulty index, item discrimination, and analyzing items based on how well they measure the effects of instruction. The document provides examples and interpretations for calculating various metrics used in item analysis.
![Report Outline ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Item and Distracter Analysis
Similar to Item and Distracter Analysis (20)
More from Sue Quirante
More from Sue Quirante (20)
Recently uploaded
Recently uploaded (20)
Item and Distracter Analysis
- 3. Item Analysis the effort to improve individual questions after they are used process of examining answers to questions in order to assess the quality of individual test items and test itself
- 4. Item Analysis the effort to improve individual questions after they are used process of examining answers to questions in order to assess the quality of those items and of the test
- 17. Difficulty Index p = total who answered correctly total taking the test *p is the difficulty index
- 18. Difficulty Index p = 22 get the correct answer 25 students take the test *p is the difficulty index p = ?
- 19. Difficulty Index p = 22 get the correct answer 25 students take the test *p is the difficulty index p = 0.88
- 20. Difficulty Index p = 0.88 *p is the difficulty index 88% of the students got it right high difficulty index
- 26. Distribution with Negative Skew picture from http://billkosloskymd.typepad.com
- 30. Distribution with Positive Skew picture from http://www.ken-szulczyk.com/lessons/statistics/asymmetric_distribution_01.png
- 36. Out-of-Level Testing picture from http://www.ken-szulczyk.com/lessons/statistics/asymmetric_distribution_01.png The average score would be only 20% plus a bit more for the guessing factor. The distribution of scores would show a positive skew. The best students would be evident at the upper end of the distribution.
- 37. Out-of-Level Testing picture from http://www.ken-szulczyk.com/lessons/statistics/asymmetric_distribution_01.png The average score would be only 20% plus a bit more for the guessing factor. The distribution of scores would show a positive skew. The best students would be evident at the upper end of the distribution.
- 38. Out-of-Level Testing picture from http://www.ken-szulczyk.com/lessons/statistics/asymmetric_distribution_01.png The average score would be only 20% plus a bit more for the guessing factor. The distribution of scores would show a positive skew. The best students would be evident at the upper end of the distribution.
- 41. Item Analysis with Constructed or Supply-Type Items e.g. essay
- 42. Difficulty Index p = mean score of the class maximum possible score *p is the difficulty index
- 45. Optimum Difficulty* *corrected for guessing 0.75 True-False MCQ 3 alternatives 0.67 MCQ 4 alternatives 0.625 MCQ 5 alternatives 0.60 Essay test 0.50
- 46. Problem When using a rigid grading scale, over half of the students will fail or get a D. picture from http://savingphilippinepupilsandparents.blogspot.com/
- 47. Distracter Analysis a multiple choice item has a low difficulty index ( p < 0.30 ) examine the item’s distracters 1 2
- 48. Distracter Analysis a multiple choice item has a low difficulty index ( p < 0.30 ) examine the item’s distracters 1 2
- 50. Distracter Analysis p = 15/50 or 0.30 Answer Choices Number who selected choice (out of 50) Choice A 2 Choice B 26 Choice C 7 Choice D 15
- 54. Hogan (2007)
- 55. Hogan’s Method N=40 ; Median=35 ; The test contained 50 items . * Indicates correct option. Item Group A B* C D 5 High 0 90 10 0 Low 10 50 30 10 Total 5 70 20 5 in %
- 56. Sample Problem N=40 ; Median=35 ; The test contained 50 items . Item Group A* B C D 5 High 40 60 0 0 Low 60 30 0 10 Total 50 45 0 5
- 65. Gronlund & Linn (1990) For norm-referenced tests:
- 71. Sample Problem * Indicates correct option. Find the p and D . Item Group A B* C D 5 Upper 10 0 10 0 0 Lower 10 2 4 1 3
- 72. Answers p = 0.70 D = 0.90 - 0.50 = 0.40
- 73. Analysis of Criterion-Referenced Mastery Items
- 74. Crucial Questions: To what extent did the test items measure the effects of instruction?
- 75. Item Response Chart + means correct - means incorrect Items 1 2 3 4 5 Pretest (B) Posttest (A) B A B A B A B A B A Jim - + + + - - + - - + Dora - + + + - - + - + + Lois - + + + - - + - - + Diego - + + + - - + - - +
Editor's Notes
- http://free-personality-tests.com/wp-content/uploads/image/test.jpg
- -use before a test is administered helps assure that the test is a reliable and valid measure -use after the test has been administered makes it possible to build a bank of high-quality test items for use during future years -use after allows the teacher to remove errant questions that adversely affect the quality of classroom measures in the current year
- -use before a test is administered helps assure that the test is a reliable and valid measure -use after the test has been administered makes it possible to build a bank of high-quality test items for use during future years -use after allows the teacher to remove errant questions that adversely affect the quality of classroom measures in the current year
- #2knowledge of which incorrect options students select must be a clue to the nature of the misunderstanding, and thus prescriptive of appropriate remediation
- #2knowledge of which incorrect options students select must be a clue to the nature of the misunderstanding, and thus prescriptive of appropriate remediation
- #2knowledge of which incorrect options students select must be a clue to the nature of the misunderstanding, and thus prescriptive of appropriate remediation
- Item Difficulty -easiest -each item should have been correctly answered by a little over half of the class
- p=0.88
- p=0.88
- most scores clustered on high side unusual low scores spread out over the wide area of the distribution
- Most children will score very high on these measures. Children with learning problems will be easily identified by their pattern of lower scores.
- This test would be very difficult for most applicants.
- Most common formula: right – (wrong/n-1) where n is the number of alternatives for an item
- The more difficult the likelier students will guess an answer to the question.
- p=0.88
- p=0.88
- -dependent on teacher’s grading philosophy or goal for testing -if well-constructed test designed w/ items of this difficulty level is administered, student scores should distribute as a normal Gaussian curve; distribution will have a mean score equal to the average difficulty level of test items -if to maximize differences between students, optimal diff. index is 0.50
- problem for teachers who assign grades according to a rigid scale e.g. 60-69% range would be a D for this reason teachers often design test items to have difficulty indexes over 0.70
- A is chosen by only 2; may be too obviously wrong. C is working well. B has drawn far too many students; should be examined to determine what caused this choice. Teacher should consider reteaching the topic and explaining why B is wrong. A and B need to be rewritten for the next edition of the test.
- Valid: students who have higher scores know more about the subject than students with lower scores.
- #2 Split the group at the median, top 50% and bottom 50%. If several students are at the median, allocate them randomly to one or the other group to get balanced groups.
- p=0.70 D=.90-.50=.40
- p=0.50 D=-0.20
- #2 Scores in the middle will not be used #4 Percentage of students who got the item right #5 difference between number of pupils in the upper and lower groups who got the item right
- p=0.70 D=.90-.50=.40
- -use before a test is administered helps assure that the test is a reliable and valid measure -use after the test has been administered makes it possible to build a bank of high-quality test items for use during future years -use after allows the teacher to remove errant questions that adversely affect the quality of classroom measures in the current year
- To answer this a pre-test and post-test must be given, and the results compared. An item by item comparison can be made by means of an item-response chart.