Education is the backbone of development.

1. TEST
DEVELOPMENT
Procedure

2. TEST DEVELOPMENT:
Multiple-Choice Item-Writing Guidelines
Objectives:-to develop skills that are
comprehensive, coherent, and scholarly-
but-practical discussion of all the issues
and sound practices required to
construct tests that have sufficient
validity evidence to support their
intended score interpretations.
Test development practical training, Adama

3. The art and science of test development
Psychometric theory
=
Testing agencies
Use rules and regulations
“on the job training”

4. The reference books for supporting the discussion

5. What tells us about test development?
• Much of the information is based on years of
experience, learning what works and what
does not work, in the actual day-to-day
practice of test development plus The relevant
research literature comes from many diverse
areas of psychometrics and educational
measurement(Millman&Greene,1989).
• Inspire these 12-steps for effective and
efficient test development.

6. • 1. Overall plan 2. Content definition
3. Test specifications 4. Item
Development 5. Test Design and
Assembly 6.Test production 7. Test
Administration 8. Scoring test responses
9. Passing scores 10. Reporting test
results 11. Item Banking and 12. Test
Technical Report Reviews and editor
materialshand book of test
development(VIP).pdf(Page 5)

7. #4 . Item development(I.D)
• Tasks of effective stimuli; formats; validity evidence
related to adherence to evidence-based principles;
training of item writers, reviewers; effective item
editing; CIV(Construct irrelevant variance) owing to
flaws.
• Note that:
Creating effective test items may be more art than
science, although there is a solid scientific basis for
many of the well-established principles of item
writing (Haladyna, Downing, & Rodriguez, 2002).
to measure important content at an appropriate
cognitive level Greater challenges for test
developers

8. Why multiple choice Item(MCI) selected?
The test developer must decide what test item formats
to use for the proposed examination. major source of
validity evidence for the test.
Most large-scale, cognitive achievement testing
programs, the choice of an objectively scorable item
format. It is ninety years of effective use and an
extensive research basis, plus the item format of
choice for most testing programs (Haladyna, 2004).
Workhorse of the testing enterprise
 Strong research evidence demonstrating the high
positive correlation between constructed response
for measuring all levels of knowledge and many other
cognitive and selected-response item scores skills
(Rodriguez, 2003).

9. For complex structure of writing ability
Constructed response, BUT not cost and time effective
for LS-PPExam.
is an extremely versatile test item form; it can be used
to test all levels of the cognitive taxonomy, including
very high-level cognitive processes (Downing, 2002a).
is an extremely efficient format for examinees, but is
often a challenge for the item writer.

10. The principles of writing effective, objectively scored
MCIs are well established and many of these principles
have a solid basis in the research literature.
Knowing the principles are not necessarily doing
effective writing.
Important validity issues
The most essential characteristic of an effective item writer
is content expertise. Writing ability is also a trait closely
associated with the best and most creative item writers.
Writing ability not Born Need train.. Validity evidence
as documented.
The selection and training of item writers(Downing & Haladyna, 1997)

11. What kind of training advised?
• Effective item writing is a unique skill and must be
learned and practiced.
• For new item writers, it is often helpful and
important to provide specific instruction using an
item writer’s guide, paired with a hands-on training
workshop (Haladyna, 2004).
The instruction–practice–feedback–
reinforcement loop is important for the effective
development and maintenance of solid item
writing skills (Jozefowicz et al., 2002).

12. Multiple Choice Item Writing Guidelines
I. Content concerns
1. Every item should reflect specific content and a
single specific mental behavior, as called for in test
specifications (two-way grid, test blueprint).
2. Base each item on important content to learn;
avoid trivial content.
3. Use novel material to test higher level learning.
Also paraphrase textbook language or language
used during instruction when used in a test item to
avoid testing for simply recall.

13. 4. Keep the content of each item independent
from content of other items on the test.
5. Avoid over specific and over general content
when writing MC items.
6. Avoid opinion-based items.
7. Avoid trick items(intention, trivial content, too fine answer
discrimination, stems with window dressing, multiple correct
answers, content presented opposite from instruction, and high
ambiguity).
8. Keep vocabulary simple for the group of
students being tested.
II. Formatting concerns

14. 9. Use the question, completion, and best answer versions of the
conventional MC, the alternate choice, true-false (TF), multiple
true-false (MTF), matching, and the context-dependent item and
item set formats, but AVOID the complex MC (Type K) format.
o MC(K) is a stem followed by choices that are grouped into
sets for test-takers to choose(more diff. than Conventional
MC + need extra time)
10.Format the item vertically instead of horizontally (very easy to
read and ).
III. Style concerns
11. Edit and proof items.
12. Use correct grammar, punctuation, capitalization, and spelling.
13. Minimize the amount of reading in each item.
IV. Writing the stem
14. Ensure that the directions in the stem are very clear.

15. 15. Include the central idea in the stem instead of the
choices(unfocused stems should be avoided).
16. Avoid window dressing (excessive verbiage).
17. Word the stem positively, avoid negatives such as
NOT or EXCEPT. If negative words are used, use the
word cautiously and always ensure that the word
appears capitalized and boldface.
+ Use a Question Format
• Experts encourage multiple-choice items to be
prepared as questions (rather than incomplete
statements).

16. V. Writing the choices/options
18. Develop as many effective choices as you can, but
research suggests three/four are adequate.
19. Make sure that only one of these choices is the right
answer.
20. Vary the location of the right answer according to the
number of choices.. Correct answers are usually the
second and third option((Attali & Bar-Hillel, 2003).)
21. Place choices in logical or numerical order.
22. Keep choices independent; choices should not be
overlapping.
23. Keep choices homogeneous in content and
grammatical structure.

17. 24. Keep the length of choices about equal.
25. None-of-the-above should be used carefully
(better to avoid).
You will never know if students know the correct
answer
26. Avoid All-of-the-above.
27. Phrase choices positively; avoid negatives such
as NOT.

18. 28. Avoid giving clues to the right answer, such as
a. Specific determiners including always, never,
completely, and absolutely.
b. Clang associations, choices identical to or
resembling words in the stem.
c. Grammatical inconsistencies that cue the test-
taker to the correct choice.
d. Conspicuous correct choice.
e. Pairs or triplets of options that clue the test-
taker to the correct choice.

19. 29. Make all distracters plausible.
30. Use typical errors of students to write your
distracters.
31. Use humor if it is compatible with the teacher and
the learning environment(for CRA not LSA).
Some important points
#3 Some techniques of writing MCIs higher
level learning
 Use memory-plus application questions. These
questions require students to recall principles, rules
or facts in a real life context.

20. The key to preparing memory-plus application questions is
to place the concept in a life situation or context that
requires the student to first recall the facts and then apply
or transfer the application of those facts into a situation.
• Require lots of context: Reading selections, Scenarios,
vignettes(small illustrative skech), Tables, charts, graphs.
• Require more testing time: Reading selections, studying
tables and charts and Thinking itself more complex
• Require detail review by others
 Seek support from others who have experience writing
higher-level thinking multiple-choice questions.

21. ITEM REVIEWING and EDITOR
Competent content review and professional editing of
test questions is also an important and essential source of
validity evidence for high-stakes examinations.
Use the above guidelines + reviewing(Reviews and
editor materialsChecklist to Review Multiple Choice
Items1.pdf )and editing check (Reviews and editor
materialsItem Editing checklists.pdf lists however all
focuses mainly on
o written against detailed test specifications
oclarity of the questions and corrects flaws in item content
osensitivity reviews for items (eliminating cultural, ethnic,
religious, or gender issues)

22. TEST DESIGN AND ASSEMBLY
• Assembling a collection of test items (or
performance prompts) into a test or test form is a
critical step in test development.
• It depends on the mode of examination delivery.
 If administered in paper-and-pencil mode, the
test can be assembled manually, by skilled test
developers /computer software.
If multiple “parallel” test forms are to be
assembled simultaneously, human test
developers using advanced computer software
can assemble the tests.

23.  If administered as computer-based, more
specialized computer software will likely be
needed to assemble multiple test forms to
ensure proper formatting of the fixed- length
test form for the computer-delivery software.
If administered as a computer-adaptive test
(CAT), very advanced computer software
(automatic test assembly software) will likely be
required to adequately manage and resolve
multiple test item characteristics simultaneously
to create many equivalent test forms.

24. • Major Issues, tests must be formatted to maximize
the ease of reading and minimize any additional
cognitive burden that is unrelated to the construct
being tested (e.g., minimize CIV). E.g:-Item together
with any visual or graphical stimuli, appear on the
same page (or frame).
• The placement of anchor or common items used in
a common-item equating design is also a formatting
issue with validity implications and appear in a
similar location to eliminate any potential “order
effect” on the common items’ (Kolen & Brennan,
2004).

25. Finally all the process of Test development files must
be properly documented as a validity evidence.