MSN

1. © 2013 Springer Publishing Company, LLC.
Chapter 3
Planning for Testing
&Oermann Gaberson
Evaluation and Testing in Nursing Education
4th edition

2. © 2013 Springer Publishing Company, LLC.
Test Construction Steps
• Define test purpose
• Describe population to be
tested
• Determine test length
• Specify difficulty and
discrimination levels
• Determine scoring
procedure
• Select item formats
• Construct test blueprint
• Write test items
• Have test items critiqued
• Arrange items on the test
• Write item-specific
directions
• Write general directions
• Print the test
• Proofread
• Reproduce the test
• Prepare a scoring key
• Prepare students
2

3. © 2013 Springer Publishing Company, LLC.
Purpose and Population
♦ Purpose
– Why are you giving a test?
– What do you intend to measure?
– How will the scores be used?
♦ Population
– Reading level
– English-language literacy
– Previous testing experience
3

4. © 2013 Springer Publishing Company, LLC.
Test Length
♦ Reliability of scores generally increases with
assessment length (number of test items or
tasks)
♦ Include as many items as possible to
adequately sample the content
– May be limited by item formats and available
testing time
– Construct power test rather than speeded test
4

5. © 2013 Springer Publishing Company, LLC.
Difficulty And Discrimination Level
♦ Difficulty
– Depends on the task complexity, ability of the
students, and quality of teaching
– Can only be estimated in advance of the test
– Ideal difficulty level depends on how results will
be interpreted
• Norm-referenced: majority of items should be
moderately difficult
• Criterion-referenced: relatively easy for most test-
takers
5

6. © 2013 Springer Publishing Company, LLC.
Item Formats
♦ All item formats have advantages and
limitations
♦ Selection criteria
– Learning outcomes to be assessed (taxonomy
level)
– Preparation time
– Testing time
– Scoring time
– Learner characteristics
6

7. © 2013 Springer Publishing Company, LLC.
Classification of Item Formats
♦ Selected-response (choice) items
– Test-taker selects correct or best answer from
options provided by the teacher
♦ Constructed-response (supply) items
– Require the learner to supply an answer
– May require limited (short) or extended response
7

8. © 2013 Springer Publishing Company, LLC.
Classification of Item Formats
Selected response
•True-False
•Matching
•Multiple-choice
•Multiple-response
Constructed response
•Short-answer
(fill-in-the-blank)
•Essay
8

9. © 2013 Springer Publishing Company, LLC.
Scoring Procedures
♦ Choice depends somewhat on the item
formats
– Responses to short-answer, numerical calculation,
and essay items usually hand-scored
– Answers to selected-response items (e.g.,
multiple-choice, true-false, and matching) may be
electronically scored on a scannable answer sheet
9

10. © 2013 Springer Publishing Company, LLC.
Test Blueprint
♦ Purpose
– Guides the selection or development of
test items
– Assists in obtaining content-related evidence
of validity
– Prepares learners to take the test
10

11. © 2013 Springer Publishing Company, LLC.
Test Blueprint
♦ Elements
– List of topics or objectives (or both)
– Level of complexity of task to be assessed
(taxonomy)
– Degree of emphasis for each topic or objective
(number or percentage of points)
11

12. © 2013 Springer Publishing Company, LLC.
Writing the Test Items
♦ Each test item should relate to the test
blueprint:
– Content area or objective
– Level of complexity of task (taxonomy)
♦ Select appropriate assessment tasks for the
ability to be tested
12

13. © 2013 Springer Publishing Company, LLC.
General Rules for Writing
Test Items
♦ Every item should measure something important.
♦ Every item should have a correct answer.
♦ Use simple, clear, concise, precise, grammatically
correct language.
♦ Avoid jargon, slang, or unnecessary abbreviations.
♦ Try to use positive terms.
(cont’d)
13

14. © 2013 Springer Publishing Company, LLC.
General Rules for Writing
Test Items (cont’d)
♦ Eliminate irrelevant clues to the correct answer.
♦ Items should not depend on each other for meaning
or for the correct answer.
♦ Eliminate extraneous information .
♦ Critique the items.
♦ Prepare more items than the test blueprint specifies.
14

15. © 2013 Springer Publishing Company, LLC.
Preparing Students for the Test
♦ Teacher-made test typically measures
students’ maximum performance rather than
typical performance
– Teachers should create conditions under which
students will be able to perform at their best
• Requires adequate preparation of students to take
the test
• Begin preparing students when the test is scheduled
15

16. © 2013 Springer Publishing Company, LLC.
Information Needs
♦ Test date and time
– No unannounced (“pop”) tests
♦ Testing conditions
– How much time, access to resources such as textbooks,
number of items, item formats, need for specific tools or
supplies (e.g., calculator, pencil)
♦ Content to be tested
♦ Cognitive level expected
♦ How test results will be used
16

17. © 2013 Springer Publishing Company, LLC.
Information Needs
♦ Use of “cheat sheets”
– Similar to memory aids such as procedure checklists,
pocket guides, reminder sheets
– Students anticipate potential test items and develop a
written cheat sheet that summarizes, prioritizes,
condenses, and organizes content that they think is
important
– Teacher may set limits on length
– Students bring cheat sheets and may use them during the
test; collected with test materials
17

18. © 2013 Springer Publishing Company, LLC.
Test-Taking Skills
♦ Testwiseness
– Ability to use test-taking skills and experience, clues from
poorly written test items to achieve a higher test score
than content mastery would predict
– All students need adequate test-taking skills so that they
are not disadvantaged when their scores are compared
with those of testwise individuals.
18

19. © 2013 Springer Publishing Company, LLC.
Test-Taking Skills
♦ Follow directions accurately.
♦ Read test items carefully.
♦ Record answers to test items accurately.
♦ Avoid physical and mental fatigue.
♦ Use test time wisely.
♦ Bypass difficult items and return to them later.
♦ Make informed guesses rather than omitting answers.
♦ Outline and organize responses to essay items before beginning
to write.
♦ Check answers to test items for clerical errors and change answers
if a better response is indicated.
19

20. © 2013 Springer Publishing Company, LLC.
Test Anxiety
♦ Anxiety is a common response to testing
situations
♦ High levels of anxiety are likely to interfere
with maximum performance
20

21. © 2013 Springer Publishing Company, LLC.
Test Anxiety
♦ Emotional component
– Negative mood and feelings associated with
testing situations
• Nervousness
• Uneasiness
• Fear
• Dread
• Panic
21

22. © 2013 Springer Publishing Company, LLC.
Test Anxiety
♦ Cognitive component
– Concerns before or during a test related to
performance and its consequences
– Worry about possible negative outcomes
• Catastrophic fantasies
• Competitive worry
22

23. © 2013 Springer Publishing Company, LLC.
Cognitive Indications of
Test Anxiety
♦ Impaired ability to concentrate and easy
distractibility during the test
♦ Difficulty recalling information (“going blank”)
♦ Misreading or misunderstanding directions or
test items
♦ Feeling pressured to be perfect
♦ History of poor performance in evaluative situations,
particularly high-stakes tests
23

24. © 2013 Springer Publishing Company, LLC.
Text Anxiety
♦ Negative feelings + negative thoughts often →
behaviors that interfere with students’ ability to
prepare adequately for a test
– Avoidance and procrastination
♦ Treatment to address:
– emotional component of anxiety
– negative thinking or worry aspect
– training to improve general test-taking skills
24