Methods of Standard
Setting
The Nedelsky Method
munsifsail@gmail.com
Introduction
• In 1954 Nedelsky proposed a method for
settinng cut scores.
• The longitivity of the method is perheps
due to the facts that it is intuitive to and
easily performed by participants.
• Participants using this method inspect,
individually each item in a Multiple-Choice
examination, with particular attention to
the options for each item.
• The Nedelsky proposed an imaginary
borderline that can be set before and after
administering the test requiring
performance standard to be set.
• It was the first widely disseminated
criterion-referenced method for setting cut
scores.
• It was the first to shift the focus from
“relative” performance to “absolute”
levels of performance.
Procedure for the Nedelsky Method
• Panel of experts independently determine the
F-responses for the examinees.
• Panel of experts assign the probabilities to each
multiple-choice test items.
• The borderline candidate (F-D Student) respond to
Multiple-choice items by first eliminating the responses
he/she believes are incorrect and than assessing the
plausibility of the remaining responses. Borderline
candidate should be able to reject incorrect answer
responses.
•To establish the pass points judges
independently identify the answer option(s)
that a borderline candidate would recognize
as implausible.
•The Number of remaining option
determines the probability that the candidate
will answer correctly the item:
1 plaussible response= 100%
2 = 50%
3= 33%
4= 25%
5= 20%
probability of a correct response.
•The average of the probabilities determines
the pass point.
•Nedelsky proposed the use of a constant,
K, for the final adjustment to mean cut
scores.
An Example
• Participants judged that, for a certain five-
option item, borderline examinees would
be expected to rule out two of the options
as incorrect, leaving them to choose from
the remaining three options. The Nedelsky
rating for this item would be 1/3 = 0.33.
Repeating the judgment process for each
item would give a number of Nedelsky
values equal to the number of items in the
test (n). The sum of the n values can be
directly used as a raw score cut score. For
example, a 50-item test consisting entirely of
items with Nedelsky ratings of 0.33 would
yield a recommended passing score of 16.5
(i.e., 50 × 0.33 = 16.5).
Problems with Nedelsky Method
• It is not used in all multiple-choice items, it
would need to have the same number of
options.
• It essentially permits participants only a
very limited number of probabilities that
they can assign to items.
• There are not equal intervals between the
probabilities, because raters tend not to
assign probabilities of 1.00.
• It cannot be used in situations where more
than one cut score is requires on the same
test.
• It seems to place a greater burdon on test
construction.
Advantages of Nedelsky
Method
• It can be implemented with or without
examinee performance data on the items
or test form on which standards are based.
• A Nedelsky cut scores can be derived
before or after operational administartion
of the test requiring performance
standards to be set.

The nedelsky method

  • 1.
    Methods of Standard Setting TheNedelsky Method munsifsail@gmail.com
  • 2.
    Introduction • In 1954Nedelsky proposed a method for settinng cut scores. • The longitivity of the method is perheps due to the facts that it is intuitive to and easily performed by participants. • Participants using this method inspect, individually each item in a Multiple-Choice examination, with particular attention to the options for each item.
  • 3.
    • The Nedelskyproposed an imaginary borderline that can be set before and after administering the test requiring performance standard to be set. • It was the first widely disseminated criterion-referenced method for setting cut scores. • It was the first to shift the focus from “relative” performance to “absolute” levels of performance.
  • 4.
    Procedure for theNedelsky Method • Panel of experts independently determine the F-responses for the examinees. • Panel of experts assign the probabilities to each multiple-choice test items. • The borderline candidate (F-D Student) respond to Multiple-choice items by first eliminating the responses he/she believes are incorrect and than assessing the plausibility of the remaining responses. Borderline candidate should be able to reject incorrect answer responses.
  • 5.
    •To establish thepass points judges independently identify the answer option(s) that a borderline candidate would recognize as implausible. •The Number of remaining option determines the probability that the candidate will answer correctly the item:
  • 6.
    1 plaussible response=100% 2 = 50% 3= 33% 4= 25% 5= 20% probability of a correct response. •The average of the probabilities determines the pass point. •Nedelsky proposed the use of a constant, K, for the final adjustment to mean cut scores.
  • 7.
    An Example • Participantsjudged that, for a certain five- option item, borderline examinees would be expected to rule out two of the options as incorrect, leaving them to choose from the remaining three options. The Nedelsky rating for this item would be 1/3 = 0.33.
  • 8.
    Repeating the judgmentprocess for each item would give a number of Nedelsky values equal to the number of items in the test (n). The sum of the n values can be directly used as a raw score cut score. For example, a 50-item test consisting entirely of items with Nedelsky ratings of 0.33 would yield a recommended passing score of 16.5 (i.e., 50 × 0.33 = 16.5).
  • 9.
    Problems with NedelskyMethod • It is not used in all multiple-choice items, it would need to have the same number of options. • It essentially permits participants only a very limited number of probabilities that they can assign to items. • There are not equal intervals between the probabilities, because raters tend not to assign probabilities of 1.00.
  • 10.
    • It cannotbe used in situations where more than one cut score is requires on the same test. • It seems to place a greater burdon on test construction.
  • 11.
    Advantages of Nedelsky Method •It can be implemented with or without examinee performance data on the items or test form on which standards are based. • A Nedelsky cut scores can be derived before or after operational administartion of the test requiring performance standards to be set.