Item Writing in Computer Science

1. ITEM WRITING
GROUP MEMBERS
Magede Thabani : N02127945H
Bushu Wimbisai : N02160383W
Kaliyati David N : N02161135W

2. Definition of terms
Item
In test development for education and certification, an item is the basic building
block of any test or assessment. Items can be questions, tasks, prompts or similar.
They may require test takers to select a response from a set of choices or to
generate their own response based on the information or parameters provided.
Common examples include: multiple-choice questions, true/false questions, short-
answer questions and open-ended essay prompts. In other words an item is a single
question or task that is not often broken down into any smaller units. An item is
part of a test which requires a specified form of response from the test taker.

3. Definition of terms cont.
Item writing
Item writing refers to the process of constructing the items required for a given test
or assessment. The people who write these items are called item writers.
Item pool
It is a reservoir or well from which items will or will not be drawn for the final
version of the test.
Test
A test is a series of tasks used to measure a test-taker’s ability (knowledge, skills).

4. Definition of terms cont.
Task
A task is “what a test-taker has to do during part of a test” (Davies et al,
1999, p.196). A task normally includes instructions, input, and one or more
items.

5. Types of items
Item Construction Scoring
Matching Easy Easy
Completion Easy Difficult
Multiple Choice Difficult Easy
Essay Easy Difficult
Below is a table showing types of items, construction and scoring

6. Item format
Item format refers to variables such as the form, plan, structure, arrangement and
layout of individual test items.
Dichotomous format: Offers two alternatives for each item. E.g. True or false
examination.
Polytomous format: Resembles the dichotomous format except that each item has
more than two alternatives. This means that these are Objective tests in which
there are three or more alternative responses given for each item. Polytomous
format is popular for multiple-choice examinations. Polytomous format is also
called Polychotomous format.

7. Item format cont.
In the field of psychology, a polytomous format refers to a type of response format
used in questionnaires, surveys, and tests, where participants are asked to choose
from more than two response options. This is in contrast to a dichotomous format,
where participants are given only two response options.
Checklists: Subject receives a long list of adjectives and indicates whether each one
is characteristic of himself.
Likert type format: Require that respondent indicate the degree of agreement with a
particular attitudinal question.

8. Multiple choice Item structure
1) Stem
2) Lead in statement
3) Answer options
4) Correct answer and distractors
5) Key

9. Multiple choice Item structure Cont.
The lead in is the question
Distractors are the wrong answers usually 3
Distractors range from not at all likely to less likely, Similar in length and
format balanced e.g. 2 ‘increased’ and 2 ‘decreased’.
The Key is the correct answer and it must be one

10. Writing Effective Multiple choice Items
One has to see if the topic is worthy evaluating.
Does the item test higher cognitive function or just recall.
Ask 90% of questions that test application, analysis, synthesis and
evaluation of knowledge. 10% may test remembering or recall.

11. Prerequisites
A command on the subject matter.
The item writer must have thorough knowledge and complete mastery on
the subject matter. He or she must be fully acquainted with all the facts so
that he or she will be able to write good and appropriate items.
The writer must have complete familiarity of the population in terms of
being aware of the intelligence level or ability level of the persons the item is
meant for.

12. Guidelines to item writing.
Clarity in meaning: Items must be phrased in such a manner that there is
no ambiguity regarding its meaning for both item writers and the test takers.
Clarity in reading: Items should be clear in reading. It should not be
difficult to read.
Doesn’t encourage guesswork: Items should not encourage guesswork.
Limit questions to materials learnt within the syllabus or the parameters of
the curriculum.

13. Guidelines to item writing.
Arrange exam items from simple to complex.
Examine several skills rather than single aspect.
Time allowed for the exam should be adequate to complete all items
comfortably.
Material for the exam should not be too long. For example putting many
items that require calculations and designing in the same test.

14. Guidelines to item writing. cont
 When creating items it is important to have full knowledge of the
cognitive level of the test takers.
 Avoid clues and cues for the test taker. These can sometimes be quite
subtle. For instance, the correct option might begin with a vowel while the
distractors begin with consonants. If the sentence completion ends with
the word “an” this would tip off the examinees.

15. Bad example:
1. An _________ is a large land mammal.
A. elephant
B. whale
C. shrew
D. kangaroo

16. Avoid “NOT” items, “All of the above,” and “A and B only” items, as they can be
confusing.
Bad example:
Which of the following devices is NOT an Input device?
A. Scanner
B. Printer
C. Mouse
D. Barcode reader

17. Guidelines to item writing. cont
Emphasize principles rather than trivial facts.
Bad example:
In the 2010 census, Madison, WI, had a population of _________.
A. 233,209
B. 243,394
C. 227,221
D. 239,874

18. Guidelines to item writing. cont
Remember that distractors can affect difficulty as much as the key. They should be
relevant. Irrelevant distractors make it easier for the test taker to pick the correct
answer.
Use either the best answer or the correct answer format. Avoid the complex
multiple-choice (Type K) format. (e.g., A and D; A and C, All the above; None of
the above; A, B, and C; etc.)
Format the item vertically, not horizontally.

19. Guidelines to item writing. cont
Generate questions that fit into all the levels of difficulty, guided by blooms
taxonomy.
Below is the Blooms taxonomy table showing the types of questions that can
be asked when testing various skills.

21. Guidelines to item writing. cont
Terms that can be used in drafting item.
Explain, State, Describe, Outline, define, Write down, List any, Draw and label,
Illustrate, convert, Identify, analyse, compare, and contrast, design, create.
Below are sample questions and marks that can be allocated:
o Explain the term protocol. (2)
o State any four of the OSI model, in their order (4)
o Define the term network topology. (1)
o Differentiate the following terms:
o Simplex and duplex mode. (2)

22. Mark allocation in item writing. Cont.
allocate marks in proportion with the demands of questions for
example questions requiring just a definition of one term allocate
(1 mark), if the question requires an example then for the
definition and example allocate (2 marks.)
Marks must match the points given in response to a question.
include the mark allocation for each question and parts of a
question, with a more detailed breakdown where necessary
Each mark should reflect equal demand.

23. Mark allocation in item writing. Cont.
• If you look at the sample questions given above. The marks allocated for
each question reflect the demands of the questions

24. Guidelines to item writing. Cont.
Allow time for editing and other types of item revisions.
Use good grammar, punctuation, and spelling consistently.
Minimize examinee reading time in phrasing each item.
Avoid trick items, those which mislead or deceive examinees into answering
incorrectly.

25. Guidelines to item writing. cont
 Base each item on an educational or instructional objective.
 Focus on a single problem
 Keep the vocabulary consistent with the examinees’ level of understanding.
 Avoid cuing one item with another, keep items independent of one another.
 Use the author’s examples as a basis for developing your items.

26. Guidelines to item writing. Cont.
Avoid over-specific knowledge when developing the item.
Avoid textbook, verbatim phrasing when developing the item.
Avoid items based on opinions.
Use multiple-choice to measure higher level thinking.
Test for important or significant material; avoid trivial material.

27. Stem Construction
 State the stem in question form or completion form (note: recent research
findings favour question form over completion).
 19. When using the completion format, don’t leave a blank for completion in the
beginning or middle of the stem of the question (note: recent research findings
favour question form over completion – thus, avoid completion format).
 Ensure that the directions in the stem are clear, and that wording lets the
examinee know exactly what is being asked.

28. Stem Construction
Avoid window dressing (excessive verbiage) in the stem.
Word the stem positively; avoid negative phrasing.
Include the central idea and most of the phrasing in the stem.

29. General Option Development
Use as many options as are feasible; more options are desirable (note:
recent research suggests four to five options).
Place options in logical or numerical order.
Keep options independent; options should not be overlapping.
Keep all options in an item homogeneous in content.
Keep the length of the options fairly consistent.
Avoid, or use sparingly, the phrase “all of the above.”

30. General Option Development cont.
 Avoid, or use sparingly, the phrase “none of the above.”
 Avoid the use of the phrase “I don’t know.”
 Phrase options positively, not negatively.
 Avoid distractors that can clue test-wise examinees; for example, avoid clang
associations, absurd options, formal prompts, or semantic (overly specific or
overly general) clues.
 Avoid giving clues through the use of faulty grammatical construction.
 Avoid specific determiners, such as “never” and “always.”

31. Correct Option Development
Position the correct option so that it appears about the same
number of times each possible position for a set of items.
Make sure there is one and only one correct option.

32. Distractor Development
 Use plausible distractors; avoid illogical distractors
 Incorporate common errors of students in distractors
 Avoid technically phrased distractors
 Use familiar yet incorrect phrases as distractors
 Use true statements that do not correctly answer the item.
 Avoid the use of humour when developing options (note: recent research
suggests benefits for humour properly inserted into exams, yet more research is
necessary).

33. ADDITIONAL ITEM-WRITING
GUIDELINES
Matching Items
 Use only homogenous subject matter.
 Do not have the same number of items and options.
 Arrange the list of responses in a logical order.
 Keep the list of items brief.
 Always place the entire task on the same page.

34. True-False Items
 Avoid statements that are too general
 Do not use negatives or double negatives.
 Do not use long, complex sentences.
 Do not include more than one idea.
 If you are using an opinion, indicate the source.
 True-false statements should be the same length.
 Include the same number of true-false statements.
 Do not use specific qualifiers such as: always, never, all, none, usually,
sometimes, occasionally, may, or could.

35. Oral Examinations
Prepare a list of questions that will be used with all students.
Have at least two people evaluate the student.
Video/audio tape responses.

36. Short-Answer Items
A direct question is generally preferable to an incomplete sentence.
Structure an item so that the required response is concise.
For incomplete-statement types, restrict the number of blanks to one, or
at most, two.

37. Place the blank near the end of an incomplete
sentence or in the margin for a direct question.
Blanks for answers should be equal in length.
Provide sufficient answer space.
Short-Answer Items Cont.

38. END