1. | © 2013 Regional Health Education, The Permanente Medical Group, Inc.December 30, 2016
Finding the Perfect FIT:
Designing and User Testing New FIT Kit Instructions
Jessica Alper, Graduate Student Intern
August 14, 2013

2. 2 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016
Agenda
FIT Kit User Error
Cognitive Load Theory in Health Education
User Testing
Outcomes
Conclusion

3. 2012 FIT Kit Return Rate
N=467,003
3 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016
32.1% 67.9%
Not submitted to KP
Submitted to KP
6.4%
93.6%
Unprocessed due to user error
Processed
2012 FIT Kit User Error
N=317,095

4. 4 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016
Most Common User Errors (N=20,294)
49.3%
23.5%
16.5%
7.0%
1.9%
1.1%
0.7%
No collection date
Not labeled
Too old to process
Unable to ID member
No specimen included
Specimen leaked
Expired collection tube

5. When To User Test?

6. 6 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016
Cognitive Load Theory in Health Education
 Cognitive Load Theory (Sweller, 1988)
 The Magic Number Seven, Plus or Minus Two: Some Limits
on Our Capacity for Processing Information (Miller,1956)
 “Chunking” optimizes memory’s limited capacity
Designing Communications That Support Cognitive Learning Processes

7. 7 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016

8. 8 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016

9. 9 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016

10. 10 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016

11. 11 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016

12. 12 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016

13. 13 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016

14. 14 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016
The Power of Pictures
— The Joint Commission (2010). Advancing
Effective Communication, Cultural Competence, and
Patient-Centered Care
“Using pictures, with or without written
explanations, can greatly increase
patient understanding and retention of
information, particularly among patients
with low functional health literacy
skills.”

15. 15 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016

16. Not All Pictures Are Worth a Thousand Words
Original Version Revised Version
16 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016

17. 17 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016

18. 18 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016

19. 19 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016
User Testing
 User (usability) testing—evaluates a product by testing it with
users
 “Watching users try to complete tasks with your interface is the
most effective and productive way to uncover usability
problems.”
–Jakob Nielsen, Nielsen Norman Group

20. 20 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016
Why Conduct User Testing?
— Mark Twain
“We tend to project our own rationalizations and beliefs onto the
actions and beliefs of others.”
-Don Norman, The Design of Everyday Things
“Supposing is good. Knowing is better.”
-Mark Twain

21. 21 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016
User Testing 101
 Recruit representative participants
 Conduct user testing:
– Facilitate representative tasks
 “Think out loud”
– Observe
 Rainbow spreadsheet
 Evaluate
– Standardized usability survey

22. Why You Only Need to Test With Five Users

23. 23 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016
User Testing Techniques
 Probe with open-ended questions
 User speculation unreliable: reinforce data, not opinion
– Video clip
 Think Aloud Protocol
– Concurrent
– Retrospective

24. User Test Demographics (N=7)
1
4
2
Age
42-54 yrs
55-64 yrs
65-74 yrs
2
3
1
1
Race
Hispanic/Latino
White
Asian
Black/African
American
4
3
Sex
Male
Female

25. 25 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.
Of 5 users in test one:
 5 rated instructions as “very easy to understand.”
 5 began FIT by laying out all items and comparing each with its
accompanying picture, without any prompting.
 4 hesitated or expressed confusion regarding which label they
should use.
 3 expressed discomfort or reservation about writing their name in
the return address portion of the envelope.
 2 wrote PHI directly onto bottle label instead of peel off label.
December 30, 2016
Results Summary
User Testing, Round One (N=5)

26. 26 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016

27. 27 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.December 30, 2016
Key Learnings
 Apply design principles that improve comprehension and recall
– Chunking: borders and white space
– Pictures
– Format and sequencing
 User testing is simple, robust, and effective