AFACCT 2023 Conference

1. Exam item writing
for educators
Rachael Farrell
EdD, MSN, RN, CNE

2. Purpose

3. By the end
of this
presentation
participants
will
Review basic terminology
related to item writing
Interpret exam statistics
Develop at least one multiple
choice item that reflects
clinical practice

4. Item writing history
• MCQ items most frequent item on
exams
• 50% of items fail to discriminate as
intended
• Faculty learn as they go
• Trial & error
Moran, V., Wade, H., Moore, L., Israel, H., & Bultas, M. (2021).

5. Terminology
Exam analysis
• Reliable
• Valid
Item analysis
Item difficulty
(p value)
Item
discrimination
(point biserial)
MCQ IWF/IWV
Stem Options Key
Distractor

6. Science & math of item writing
Exam analysis
Reliable
Valid
Item analysis

7. p value
• Item difficulty
• Reported as percentage
• Part of exam & item analysis
Billings & Halstead, 2016
p value Item difficulty
81-100% Easy (lower-level Bloom’s)
70-80% Average
51-69% Difficult
<50% Very difficult

8. Point biserial index
• Who is getting item correct
• Range from -1.0 to +1.0
• Negative is bad, positive is
good
• Total-79%, Upper 90%,
Lower 50%, biserial 0.339
A-8, B-0, C-33*, D-1
Billings & Halstead, 2016
Point biserial Discrimination
>0.40 Excellent
0.30-0.39 Good
0.15-0.29 Fair
<0.15 Poor; needs revision

9. Let’s review
p value
Point biserial
<0.15 0.15-0.29 0.30-0.39 >0.40
81-100% Poor
discrimination;
must review
Fair
discrimination;
consider review
Good
consideration;
keep item
Excellent
discrimination;
keep item
70-80% Poor
discrimination;
must review
Fair
discrimination;
consider review
Good
consideration;
keep item
Excellent
discrimination;
keep item
51-69% Poor
discrimination;
must review
Fair
discrimination;
consider review
Good
consideration;
keep item
Excellent
discrimination;
keep item
<50% Poor
discrimination;
must review
Fair
discrimination;
consider review
Good
consideration;
keep item
Excellent
discrimination;
keep item

10. Time to
practice
Option A Option B
Option C
(key)
Option D
p value: 55% 16 2 23 1
Upper: 70%
Lower: 30%
point biserial:
0.243

11. Time to practice-
answers
p value is 55%,
Look at the point
biserial, it has a fair
discrimination.
At least 1 student has
selected each distractor
Option
A
Option
B
Option C
(key)
Option
D
p value:
55%
16 2 23 1
Upper:
70%
Lower:
30%
point
biserial:
0.243

12. Time to
practice
Option A Option B Option C
Option
D(key)
p value:
96%
2 0 0 43
Upper:
91%
Lower:
91%
point
biserial:
- 0.068

13. Time to practice-answers
• p value is 96%
• Look at the point biserial, it has a poor
discrimination.
• All distractors not selected
Option A Option B Option C
Option D
(key)
p value:
96%
2 0 0 43
Upper:
91%
Lower:
91%
point
biserial:
- 0.068

14. Sample question
During transport to surgery the client says to the
nurse, “I don’t know if I want the surgery anymore.”
What should the nurse do?
• Return the client to the room on the acute care unit
• Cancel the surgery and get the client discharged
• Report the incident to the hospital’s ethics committee
• Inform the surgeon about the doubts the client is experiencing

15. 1st revision: change distractors
During transport to surgery the client says to the
nurse, “I don’t know if I want the surgery anymore.”
What should the nurse do?
• Return the client to the room on the acute care unit
• Cancel the surgery and get the client discharged
• Report the incident to the hospital’s ethics committee
• Inform the surgeon about the doubts the client is experiencing

16. 1st revision: change distractors
During transport to surgery the client says to the nurse, “I
don’t know if I want the surgery anymore.” What should the
nurse do?
• Return the client to the room on the acute care unit
• Cancel the surgery and get the client discharged
• Verify whether the client was competent when consent was signed
• Report the incident to the hospital’s ethics committee
• Explain the procedure to the client in laymen terms
• Inform the surgeon about the doubts the client is experiencing

17. 2nd revision: priority action
During transport to surgery the client says to the
nurse, “I don’t know if I want the surgery anymore.”
What is the nurse’s priority action?
• Verify the informed consent was signed
• Return the client to the pre-operative holding area
• Document all conversations with client and healthcare provider
• Inform the surgeon about the doubts the client is experiencing

18. Blooms taxonomy
Lower
levels
found in
texts

19. Practice
item-blooms
low level
A nurse is caring for a client receiving
normal saline at 100mL/hr. Which
manifestation can indicate fluid
volume excess?
A. Decreased blood pressure
B. Weak pulse
C. Increased urine specific gravity
D. Moist crackles

20. Same basic
item-blooms
higher level
A nurse is caring for a client receiving
normal saline at 100mL/hr. The client has
developed moist crackles and bounding
pulses. Which nursing action should the
nurse implement?
A. Weigh client daily
B. Encourage strict bedrest
C. Encourage oral intake of fluids
D. Monitor dietary intake of potassium

21. NCLEX test plan
Familiarize yourself
Familiarize your students
Use it
Love it

22. Exam blue-prints
• Relationship to outcomes/objectives
• Should ID types of questions
• Simple or complex
• Is content tested appropriately
• Bristol, T., & Brett, A. (2015)

23. Item practice
writing
As I move through the
rest of the presentation,
I want you to practice
some of what we’re
discussing. Towards the
end, I hope to give you
an opportunity to share.

24. Cs of item
writing
Bristol, T., & Brett, A. (2015)
CONTENT CLARITY
CLINICALLY
COMPLEX
CRITIQUE

25. Item writing
flaws
• Implausible distractors
• Teaching in stem
• Unnecessary information in
stem
• Unclear stem
• Words in stem & key
• Different lengths of options
(only 1)
• Reading comprehension
Nedeau-Cayo, R., Laughlin, D., Rus, L., & Hall, J.
(2013)

26. Implausible distractors
During transport to surgery the client says to
the nurse, “I don’t know if I want the surgery
anymore.” What should the nurse do?
• Return the client to the room on the acute care unit
• Cancel the surgery and get the client discharged
• Report the incident to the hospital’s ethics committee
• Inform the surgeon about the doubts the client is
experiencing

27. Teaching in stem
A client has been administered the anti-
diarrheal agent loperamide
hydrochloride (Imodium). Which
condition is the mediation indicated for?
• Abdominal pain
• Patients with an ileostomy
• Bloody diarrhea
• Acute dysentery

28. Teaching in stem
A client has been administered the anti-diarrheal
agent loperamide hydrochloride (Imodium).
Which condition is the mediation indicated for?
• Abdominal pain
• Patients with an ileostomy
• Bloody diarrhea
• Acute dysentery

29. Unnecessary
information in stem
A client is 16-hours post cesarean section.
The client is breastfeeding her infant, and
positive bonding is noted. For which post-
operative condition should the nurse
administer this medication?
• Incisional infection
• Abdominal pain
• Atelectasis
• Emesis

30. Unnecessary information in stem
A client is 16-hours post cesarean section. The client
is breastfeeding her infant, and positive bonding is
noted. For which post-operative condition should the
nurse administer this medication?
• Incisional infection
• Abdominal pain
• Atelectasis
• Emesis

31. Unclear stem
A nurse is teaching a group of
adolescents about high-risk behaviors.
Which strategy is effective?
• Work individually with each member
• Allow the group to decide the topics
• Share personal experiences with members
• Encourage the group to ask questions

32. Unclear stem
A nurse is teaching a group of
adolescents about high-risk behaviors.
Which strategy is effective?
• Work individually with each member
• Allow the group to decide the topics
• Share personal experiences with members
• Encourage the group to ask questions

33. Different distractor lengths
(only 1)
The nursing instructor teaches a student about
inotropic medications. The nurse determines
teaching has been effective when the student makes
which statement?
• “These medications decrease afterload.”
• “These medications increases heartrate.”
• “These medications increases the force of contraction.”
• “These medications are used to treat CHF.”

34. Different distractor
lengths (only 1)
The nursing instructor teaches a student about
inotropic medications. The nurse determines
teaching has been effective when the student
makes which statement?
• “These medications decrease afterload.”
• “These medications increase heartrate.”
• “These medications increase the force of contraction.”
• “These medications are used to treat CHF.”

35. Reading comprehension
A community health nurse is visiting the home of a client
whose preschool child has a low-grade fever and an earache.
The client states transportation is a problem and there is no
one to care for the other two children if she takes this child
to the provider’s office. The client states “I will take my child
to the clinic tomorrow after the others go to a babysitter.”
Which intervention is appropriate?
• Call the clinic the next day to verify the appointment was kept
• Ask the client how she will care for the preschooler until morning
• Make an appointment for the child for the next day
• Notify the clinic the client will bring in a sick child the next day

36. Reading
comprehension
A community health nurse is visiting the home of a client
whose preschool child has a low-grade fever and an earache.
The client states transportation is a problem and there is no
one to care for the other two children if she takes this child
to the provider’s office. The client states “I will take my child
to the clinic tomorrow after the others go to a babysitter.”
Which intervention is appropriate?
• Call the clinic the next day to verify the appointment was kept
• Ask the client how she will care for the preschooler until morning
• Make an appointment for the child for the next day
• Notify the clinic the client will bring in a sick child the next day

37. Item writing
guidelines
Many rules, but not set in stone
• Planning
• Item development
• Review
Nedeau-Cayo, R., Laughlin, D., Rus, L., & Hall, J. (2013)
Kranz, C., Love, A., & Roche, C. (2019)

38. Planning
What do I want students to know
What is the educational
level/course progression
Bloom’s taxonomy

39. Fundamental course
Which actions should the nurse perform when washing
hands as part of medical asepsis before caring for a
client in an outpatient clinic? Select all that apply.
• Wash hands with the hands held higher than the elbows
• Adjust temperature of water to the hottest possible
• Scrub hands and nails with a scrub brush for 5 minutes
• Use a clean paper towel to turn off water
• Rub vigorously using firm circular motions

40. Advanced course
Which actions should the nurse take to comply with
principles of medical asepsis? Select all that apply.
• When providing tracheostomy care, touch only outer aspect of
reusable inner cannula of endotracheal tube
• Wear gloves and surgical mask when connecting peritoneal dialysis
solution to client
• Recap syringe after administering insulin
• Insert needle into rubber port of a previously used multidose vial
without swabbing the port with alcohol
• Use surgical mask while working with a client who has tuberculosis

41. Item development
• Clinical scenario & stem
• Plausible options
• Uniform options

42. Review
• Clear & complete thoughts
• No typos
• Terminology

43. Show & Tell

44. Test banks for sale
• Topic or specialty area
• Credit card, Western Union, PayPal
• $5-$25
• Instant delivery
• Many websites out there
• Standard tests available too
Madara. B., Resha, C., Krol, M., Lacey, K., Martin, E.F.,
O’Sullivan, C., et al. (2017)

45. Summary
• Item writing takes time
• Faculty need item writing training
• Faculty need to evaluate exam
items to determine need for
revision
• Faculty should then revise the
items
• Faculty need feedback from
mentors

46. Questions

47. References
• Billings, D.M. & Halstead, J.A. (2016). Teaching in nursing: A guide for faculty. St. Louis, MO: Elsevier.
• Bristol, T., Nelson, J., Sherrill, K., & Wangerin, V. (2017). Current state of test development, administration, and analysis: A study of faculty practices.
Nurse Educator, 43(2), 68-72. doi:10.1097/NNE.0000000000000425
• Bristol, T., & Brett, A. (2015). Test item writing: 3Cs for successful tests. Teaching and Learning in Nursing, 2015(10),100-103. doi:10.1016/j.teln.2015.01.004
• Glascow, M.E., Dreher, H.M., & Schreiber, J. (2019). Standardized testing in nursing education: Preparing students for NLCEX and practice. Journal of
Professional Nursing. 2019, (35), 440-446. Doi:10.1016/j.profnurs.20.19.04.012
• Kranz, C., Love, A., & Roche, C. (2019). How to write a good test question: Nine tips for novice nurse educators. Journal of Continuing Education in
Nursing, 50(1), 12-14. doi:10.3928-0022020190102-04
• Madara. B., Resha, C., Krol, M., Lacey, K., Martin, E.F., O’Sullivan, C., et al. (2017) Nursing students’ access to test banks: Are your tests secure? Journal of Nursing
Education, 56 (5), 292-294. doi:10.3928/01484834-20170421-07
• Moran, V., Wade, H., Moore, L., Israel, H., & Bultas, M. (2021). Preparedness to write items for nursing education examinations: A national survey of
nurse educators. Nurse Educator, 47(2), 63-68. doi:10.1097/NNE.0000000000001102
• Nedeau-Cayo, R., Laughlin, D., Rus, L., & Hall, J. (2013). Assessment of item-writing flaws in multiple choice questions. Journal for Nurses in Professional
Development, 29(2), 52-57. doi:10.1097/NND.0b013e318286b2f1
• Smith, S. and Geist, M. (2021). TERM model: The incorporation of mentorship as a test-item improvement strategy. Teaching and Learning in Nursing,
2021(16), 60-62. doi:10.1610/j.teln.2020.05.008
• Sutherland, K., Schwartz, J, & Dickison, P. (2012). Best practices for writing test items. Journal of Nursing Regulation, 3(2), 35-39. doi: 10.1016/S2155-
8256(15)30217-9