Evaluation and Accountability

1. READING TEST SPECIFICATIONS
ASSIGNMENT-01: 4.8
GROUP MEMBERS:
BILAL H. YASEEN GS39550
NURUL NAJWA BINTI ABU BAKAR GS36081
HANI SHAKIR GS39584
LECTURER:
PROF. DR. CHAN SWEE HENG
Faculty of Modern Languages and Communication
Universiti Putra Malaysia
November 2014

2. READING TEST SPECIFICATIONS: 4.8
• Introduction:
How can we help students to learn in an efficient, effective,
attractive and accessible way? There is no simple straight ward
answer to this question; depending on the specific situation,
solution X will work best for person Y. It’s generally acknowledged
that we can improve testing consider by making the conditions to
optimal specifications and then use this acknowledge to design new
test events.
We are going to introduce the concepts of test design, and examine
what a test designer must know in order to create high-quality test
designing, and to discuss the nature of this specifications, how it
can be modeled in terms of rules, and how the rules are derived.
We also will use three examples (Designs / Contexts) to introduce
the different modeling concepts and use the term test designer, to
describe those who learning design task to perform.

3. READING TEST SPECIFICATIONS: 4.8
THE PURPOSE OF THE TEST
This is a reading comprehension assessment.
• To measure the testees' performance based on
the curriculum during teaching and learning
sessions in the whole school semester.
• This test is under local control - used to place
learners into classes for the next school semester
• To discover how much the students have
achieved in their L2 learning.

4. READING TEST SPECIFICATIONS: 4.8
THE PURPOSE OF THE TEST Cont.
• This test is low-stakes or summative.
• However, teachers use the information to
make decisions about which classes are more
suitable for the learners to be in for the next
semester of school, according to their levels of
performance.

5. READING TEST SPECIFICATIONS: 4.8
The Test Takers
• Grade 4 ESL students of Baghdad Secondary School, Iraq.
• The assessment is designed to measure the academic
achievement of all test takers with diverse background of
Grade 4.
• Items in the assessment are written in plain language,
without unnecessarily complex syntactical structure
because they are adolescent ESL learners with the kind of
language needs.
• This assessment is carefully designed to provide
accommodations for this type of learner’s level; teenage
ESL students.
• The passage and items are accessible.

6. READING TEST SPECIFICATIONS: 4.8
Context and Modality
• The Modality: Reading skill (Comprehension).
• The context: In this test design the tester chooses THREE efficient contexts
which is:
• Multiple choice item 2) True and False item. 3) Matching item.
The Test design:
Reading comprehension test for measuring the competence and the
performance of the testees. (Achievement test).
• ACHIEVEMENT TEST
Achievement test attempts to measure what an individual has learned-his
or her present level of performance; in this design the testees will be
examined for his/her comprehension in the reading skill.

7. READING TEST SPECIFICATIONS: 4.8
In planning a test, the Designer can proceed the
following :
1. Identify the puspose of the test.
2. Identify the best specification.
3. Select the contents of the test.
4. Consider the form of the test.
5. Write the test item.
6. Consider the layout of the test.
7. Consider the timing of the test.
8. Plan the scoring of the test.

8. READING TEST SPECIFICATIONS: 4.8
1. IDENTIFY THE PURPOSE OF THE TEST
1.To diagnose a student’s strength, weakness
and difficulties.
2.To measure achievement.
3.To measure aptitude and potential.
4.To identify readiness for a program.

9. READING TEST SPECIFICATIONS: 4.8
2. IDENTIFY THE TEST SPECIFICATION
The test specification includes :
1. Which program objectives and student learning
outcomes will be addressed.
2. Which content areas will be addressed.
3. The relative weightings, balance and coverage of items.
4. The total number of items in the test.
5. The number of questions required to address a
particular element of a program or learning outcome
6. The exact item in the test.

10. 3. SELECT THE CONTENTS OF THE TEST
Gronlund and Linn (1990), suggest that an item analysis will need to
consider :
1. The suitability of the format of each item for the (learning)
objective (appropriateness)
2. The ability of each item to enable students to demonstrate their
performance of the (learning) objective (relevance)
3. Clarity of the task for each item
4. The straight forwardness of the task
5. The independence of each item (i.e. where the influence of other
items of the tests minimal and where successful completion of one
another)
6. The adequacy of coverage of each (learning) objective by the items
of the test.

11. READING TEST SPECIFICATIONS: 4.8
4. CONSIDER THE FORM OF THE TEST
The designer (tester) will need to consider
whether the test will be undertaken individually,
or in a group, and what form it will take.
Comprehension test, for example, can be
conducted if the designer feels that reading will
obstruct the true purpose of the test.

12. READING TEST SPECIFICATIONS: 4.8
5. WRITE THE TEST ITEMS
In this test we tackled three contexts:
1. Multiple Choice.
2. True-false Items.
3. Matching.

13. A. CONSTRUCTING MULTIPLE CHOICE ITEMS
1. Design each item to measure a specific objective.
• (Direct MCQ) Example:
Q/ Which of these statements correctly summarizes
how the author of this passage feels about robots?
a. Robots are old.
b. Robots are confusing.
c. Robots are helpful.
d. Robots are dangerous.

14. READING TEST SPECIFICATIONS: 4.8
• (MCQ Indirect question) Example:
Q/ What is the main purpose of paragraph 2?
A. to show how easy it is to make a robot.
B. to tell what a robot is .
C. to describe the things a robot can do.
D. to explain the difference between a robot and a machine.

15. READING TEST SPECIFICATIONS: 4.8
2. State both stem and options as simply and directly
as possible.
We are sometimes tempted to make multiple choice items too
wordy. A good rule is to get directly to the point.
( MCQ cloze) Example:
Q/ Long ago, people ………… robots.
a. played
b. saw
c. imagined
d. broke

16. READING TEST SPECIFICATIONS: 4.8
3. Make certain that intended answer is clearly the only
correct one.
Q/ According to the passage, when was the first real robot made?
a. 1961
b. 1900
c. 2003
d. 2000 years ago

17. READING TEST SPECIFICATIONS: 4.8
4. USE ITEM INDICES TO ACCEPT, DISCARD,
OR REVISE ITEMS
 Item facility (level of difficulty): Is the extent to which
an item is easy or difficult for the proposed group of
test-takers. IF shows how easy or difficult the
particular item that’s proved in the test.
 Item discrimination (discriminating power): Item
discrimination is the extent to which an item
differentiates between high- and low-ability test-takers.
It measures how well the test items are
arranged to identify the differences in the students
competence.

18. 4. USE ITEM INDICES TO ACCEPT, DISCARD,
OR REVISE ITEMS. Cont.
Distractor efficiency: Distractor efficiency is
one more important measure of multiple
choice item’s value in a test. The efficiency of
distractors is the extent to which: -
a. The distractors “lure” a sufficient number of
testees especially lower-ability ones
b. Those responses are somewhat evenly
distributed across all distractors.

19. READING TEST SPECIFICATIONS: 4.8
B. TRUE-FALSE ITEMS
The true-false item typically present a declarative statement
that the students must mark as either true or false. Instructors
generally use true-false items to measure the recall of factual
knowledge such as name, events, dates, definitions, etc. But
this format has the potential to measure higher levels of
cognitive ability, such as comprehension of significant ideas
and their application in solving problems.
Examples:
 A robot is a special kind of machine. (T or F)
 A robot is a human being. (T or F)

20. READING TEST SPECIFICATIONS: 4.8
c. MATCHING
A matching exercise typically consists of a list
of questions or problems to be answered along
with a list of responses. The examinee is
required to make an association between each
question and a response.

21. READING TEST SPECIFICATIONS: 4.8
EXAMPLE
Match the following items in list (A) to their responses
in list (B):
A B
1. Something special is NOT A. 1961
2. Robots may be used to B. Machine
3. When was the first real robot made? C. Normal
4. Who is the first one imagined robots? D. Make things
5. Most robots just look like E. Homer
F. 2000
G. Tired

22. 6. CONSIDER THE LAYOUT
• The Clarity of the Instruction:
 What to do (Objective)
 How long to take (Time)
 How many items to attempt (Quantity)
 What kind of response is required
 How and where to enter the response (Answer placing)
• The Location and Sequence of Items
 The progression from the easy to the more difficult items of the test.
• The Visual Layout of the Page
 Minimize the unnecessary visual material or words.

23. READING TEST SPECIFICATIONS: 4.8
7. CONSIDER THE TIMING OF THE TEST
• The timing refers to two areas:
1. When the test will take place
2. The time allowances to be given to the test and
its component items
8. PLAN THE SCORING OF THE TEST

24. SCORING SYSTEM DESIGN AND
DESCRIPTIONS
Multiple-
Choice Item
True and
False Item
Matching Item
• 4 answer choices (only one
correct answer).
• generic scoring rubric, 1 point
for correct answer or 0 point
for incorrect (dichotomous).
• Only 2 options (1 correct and 1 incorrect
response).
• Scored dichotomously (choose whether the
statement is ‘True’ or ‘False’).
• If they choose the correctly - 1 point
• If they don’t - 0 point
• 5 questions and 7 options can be chosen to
match with the questions.
• only 1 correct response for every item
• 1 point – if they match correctly
• no point - if they match it incorrectly

25. READING TEST SPECIFICATIONS: 4.8
Multiple-Choice Item
SCORE CATEGORY AND DESCRIPTIONS
CORRECT ANSWER (A, B, C, D OR E)
These responses represent one correct choice (1 point)
INCORRECT ANSWER (A, B, C, D OR E)
These responses represent one incorrect choice (0 point)
True and False Item
SCORE CATEGORY AND DESCRIPTIONS
CORRECT ANSWER (TRUE OR FALSE OPTION)
These responses represent one correct choice (1 point)
INCORRECT ANSWER (TRUE OR FALSE OPTION)
These responses represent one incorrect choice (0 point)

26. READING TEST SPECIFICATIONS: 4.8
Matching Item
SCORE CATEGORY AND DESCRIPTIONS
CORRECT MATCH
These responses represent one correct match (1 point)
INCORRECT MATCH
These responses represent one incorrect match (0 point)

27. References
Alderson, J. C. (1988). Testing English for specific purposes – how specific can we get? In Hughes, A. (ed.),
Testing English for University Study. London: Modern English Publications and the British Council, 16–18.
Allman, C. (2005). Building Assessment Initiatives for Schools: An Accommodations Guide for Parents and
Students with Visual Impairments. American Foundation for the Blind. Available online:
http://www.afb.org/Section.asp?SectionID=58&TopicID=264&DocumentID=2762.
Assessment Reform Group (1999). Assessment for Learning: 10 principles. Retrieved from
http://www.qca.org.uk/qca_4336.aspx , 12 May 2009.
Bachman, L. F. (2004). Statistical Analyses for Language Assessment. Cambridge: Cambridge University Press.
Bernstein, J. (1999). Phone Pass TM Testing: Structure and construct. Menlo Park, CA: Ordinate.
Carroll, J.B. (1961). Fundamental considerations in testing for English language proficiency of foreign students.
In Allen, H.B. (Ed.), Teaching English as a Second Language. McGraw Hill, New York, pp. 364-372.
Center for Applied Linguistics. (2007). Foreign Language Assessment Directory. Retrieved 3 July 2009, from
http://www.cal.org/CALWebDB/FLAD
Cohen, A. S. and Wollack, J. A. (2006). Test administration, security, scoring and reporting. In Brennan, R. L.
(ed.), Educational Measurement. 4th edition. New York: American Council on Education/Praeger, 355–386.
Fulcher, G. (2010). Communicative language testing. In de Bot, K. (ed.), The Mouton Handbook of Teaching
English as a Foreign Language. The Hague: Mouton.