This topic relates to M.A. education. It is a part of the psychological testing.In the presentation, I discussed about the methods of item analysis.

2. What is an item?

3. “a single task or question that usually cannot be broken down
into any smaller units.”
An arithmetical problem may be an item, a manipulative task
may be an item, a mechanical puzzle may be an item.

4. What are the
characteristics of
item?

5. Do you suffer from insomnia? Yes/No
Do you feel shy in talking with members of the opposite sex? Yes/No
An item should be phrased in such a manner that there is no
ambiguity regarding its meaning for both the item writer as
well as the examinees who take the test.

6. Ahmed said that the sun rose in the east. Right/Wrong
He said that honesty is the best policy. Right/Wrong
The item should not be too easy or too difficult.

7. Pakistan came into being on 14th August……… 1946,1947,1948
The capital of Pakistan is………… Lahore, Karachi, Islamabad
It should have discriminating power, that is, it must clearly
distinguish between those who possess a trait and those who do not

8. Write the outcomes of psychological testing on the society?
Do you think our examination system is producing the right manpower?
It should not be concerned with the trivial aspects of the subject-matter,
that is, it must only measure the significant aspects of knowledge or
understanding.

9. The main organ of our respiratory system is……… brain, heart, lung
Define the respiratory system?
As far as possible, it should not encourage guesswork by the
subject.

10. Why to analyze the items of a test paper?
Item analysis demonstrates how effectively a given test item functions
within the total test.

11. What is item analysis?

12. Item analysis is a set of procedures, that is applied to know the indices
for truthfulness(or validly) of items.

13. Item analysis is a technique through which those items which
are valid and suited to the purpose are selected and the rest
are either eliminated or modified to suit the purpose.

14. What are the objectives of item analysis?

15. Difficulty Value
Item analysis indicates which items are difficult, easy, moderately
difficult or moderately easy. In other words, it provide an index of the
difficulty value to each item.

16. Discrimination Value
It also provide indices of the ability of the item to discriminate
between inferior and superior. In other words , item analysis
indicates the discrimination value of each item. This is known
as item validity.

17. Distractor Analysis
It indicates the effectiveness of the distractors in multiple-choice items. Since multiple-
choice items are one of the most powerful and flexible objective items and many of the
standardized tests utilize this form of item. A thorough item analysis is done to indicate
the extent to which the distractors or foils are effective in each item.

18. Function effectiveness
It sometimes also indicates why a particular item in the test
has not functioned effectively and how this might be modified
so that its functional significance can be increased.

19. Power Tests
A power test is one where the examinee is allowed sufficient time for answering all
items of the test. Thus, here emphasis is upon measurement of the ability (or power) of
the examinee and not upon his speed. Ideally, in such a test, items are of different
difficulty values and they are arranged in increasing order of difficulty.

20. Item Difficulty
In item analysis, the first step is to find out the difficulty value of the item
or the index of difficulty of an item.

21. The difficulty value of an item is the proportion or percentage of the
examinees or individuals who answer the item correctly. This proportion
or percentage is known as the index of difficulty of an item.

22. Item answered by 90% of the examinees is very easy
while the item answered by only 3% of examinees is very difficult and
discriminating only genius. Also, an item which is answered correctly
by 100% or 0% examinees has no differentiating significance.

23. 50 x 50 = 2500
When an item is answered correctly by 50% of the examinees and
answered wrongly by 50%. But this does not mean that the tests
composed of items of only 50% difficulty level are always to be preferred
because such tests would simply divide the total examinees into two
groups; those above the 50% difficulty level and those below the 50%
level of difficulty.

24. If a test is composed only to of
items of higher indices of difficulty
or only of lower indices of difficulty,
it will never be a good measuring
instrument.
Any test to be called good measuring
instrument must have some items
of higher indices of difficulty which
should usually be placed at the
beginning of the test, some items,
of moderate indices of difficulty
(ranging from 40% through 60%)
which should preferable appear after
that, and some items of lower indices of
difficulty which should appear last of
all.

25. At the
beginning
of the test can
be placed
those items
which are
passed by
100%,
while at the
end, those
items can
be placed
which are
passed by
no one or zero
percent.

26. Method of Judgment
In this method the difficulty value of an item is determined on the basis of the
judgment of experts. Items are given to a group of experts with the instruction to rank
the item in their increasing order of difficulty. Subsequently, the test constructor takes
a final decision, keeping in view the commonality of ranks assigned to each item by the
different judges or experts. Sometimes, an estimate regarding the index of difficulty is
made by the test constructor himself on the basis of the total time taken by the
examinees in its solution.

27. Empirical Method
The empirical method, also called the statistical method, is the basic and scientific
method of determining the index of difficulty of an item. There are two common
statistical methods through which the index of the difficulty can be estimated. One
method determines the index on the basis of the response of all the examinees whereas
another method determines the index on the basis of the responses of only a portion of
the examinees. Usually, the correct answer in the test is given a score of 1 and the
incorrect answer a score of 0.

28. Optimal Difficulty Value for a Reliable Test
Generally, the test constructor is faced with the question; What should the optimal
level of difficulty of the items be? The question can be answered by referring to what is
meant by optimal level of difficulty. Obviously, most test constructors mean that optimal
level of difficulty is one which produces maximum item intercorrelations, and thereby
maximum item-total correlation and also a higher reliability of test scores. So, the
optimal level has a direct link with the reliability of the test scores. Difficulty levels
producing higher reliability are the optimal levels.

29. Index of
Discrimination
The determination of the index of discrimination also known as
the item validity index is another important aspect in item analysis.

30. Index of discrimination is that ability of the item on the basis of which
the discrimination or distinction is made between superiors and inferiors
“The degree to which the single item separates the superior from the
inferior individuals in the trait or group of traits being measured.”

31. The discriminatory power or validity of the item may be defined as the
extent to which success and failure on that item indicate the possession
of the trait or achievement being measured”
From the point of view of discriminatory power, all test item can be
divided into items that are either ;
i. positively discriminating ii. negatively discriminating
or iii. non-discriminating

32.  A positively discriminating item may be defined as one in which the
proportion or percentage of correct answers is higher in the upper group
 A negatively discriminating item may be defined as one in which
proportion or percentage of correct answers is lower in the upper group.
 Likewise, a non-discriminating item may be defined as one in which
the percentage or proportion of correct answer is equal or approximately
equal in both the upper and lower groups.

33. There are two common ways of determining the index of discrimination:
1. By applying a test of significance of the difference between two
proportions or percentage.
2. By applying correlational techniques.

34. A Test of Significance of the Difference between Two Proportions or Percentage
In this method of determining the validity index of an item, the
examinees are divided preferably into two equal groups on the basis of
the total score.
(For this purpose we can take top and bottom 40% or 50% and compare
the proportions of passing in each of these two extreme groups. But
common observation has been that when the dichotomy is made near the
median, it has little influence upon the discrimination index)

35. Two Extreme Groups
For the purpose of making two extreme groups, the total scores are
arranged from highest to lowest and then the top 27% and bottom 27%
are selected. Thus the total score is the criterion for making these two
extreme groups.

36. Correlational Techniques
Correlational techniques have also been frequently employed as the measures of the
index of item discrimination. In such situations each item is correlated against the
internal criterion of the total score, that is, each item is validated against the internal
criterion of total score.

37. Item Analysis Worksheet

38. Distractor Analysis
Item analysis especially in multiple-choice items aims at determining the effectiveness
of the distractors. Indices of item difficulty and discriminative power simply indicate
whether an item is doing its intended job, not why or why not. If a particular item is not
functioning well or is found to be defective, we have to examine its possible reasons
and one obvious method is to examine the distractibility of the incorrect options.

39. Speed Test
Some standardized tests emphasize the speed of the response of the examinee. They
provide a limited time within which an examinee is expected to reach the last item in
the test. In other words, they place primary importance upon the speed with which an
examinee can answer the item. Ideally, in a speed test all items should be of uniform
degree of difficulty. Often item analysis of such speed tests provides a special problem
to the test constructor. The index of difficulty and the index of discrimination cannot
be determined in exactly the same way as is done in the case of a power test.

40. Index of Difficulty
In a speed test many examinees never reach the items at the end of the test not because
of their intrinsic difficulty but because of their position in the test. Hence, the usual way
to determine the index of difficulty by dividing the number of correct answers by the
number of the examinees who take the test, would be a misleading index. Hence, in the
speed test the index of difficulty of an item is determined by the number of correct
answers to an item divided by the number of examinees who answer or reach that item
(and not the total number of examinees)

41. Index of Discrimination
The index of discrimination in a speed test cannot be determined through the
correlation between the individual item score. Why? The answer is that in a speed test
the items are arranged in an arbitrary order and not from easier to difficult order as in
most power tests. Besides, the time in the speed test does not allow all examinees to
reach the last item in the test. Consequently, items placed in the beginning of the test
will have a higher difficulty value whereas items placed towards the end of the test will
have a lower difficulty value.

42. In the case of the speed tests where the reliability is dependent upon the reasonable
time limit allowed for the completion of items. Therefore, in the speed test items are
selected according to the principles that will establish a reasonable time limit, which
may yield a most reliable distribution of total scores.

43. The ICC summarizes much of the information conveyed by item analysis. It is a graphic
representation of the probability of giving the correct answer to an item as a function
of the level of the attribute assessed by the test. In fact ICC serves as a foundation of one
of the most important theories, the item response theory for latent trait theory. This
theory was specially developed for the purpose of understanding how individual
differences in attributes of the examinees affect his behavior when confronted with a
specific item.