As plain language professionals, we think we know the rules and can recognize when a document or website is presented in plain language. But do our customers always agree?

1. Is it really plain?
A case for content testing
Annetta Cheek
Center for Plain Language
Kath Straub
Usability.org | Center for Plain Language
PLAIN
2013
|
Vancouver
CA

2. Where we are going ….
• A parable
• A process
PLAIN
2013
|
Vancouver
CA

3. The Parable
PLAIN
2013
|
Vancouver
CA

4. PLAIN
2013
|
Vancouver
CA

5. PLAIN
2013
|
Vancouver
CA

6. We identified 10 “must-know” facts.
1. Any veteran can join.
2. When you join, you give access to your
VA System electronic health record,
including
–
–
–
–
–
–
Your vital signs
Diagnoses
Doctor’s notes
Medicines
Treatment history
Test results
3. VA will use the data starting from the
time you entered the system until 10
years after you joined Donate My Data.
4. If you die during the program, the VA will
stop using your data.
5. VA will follow the laws about protecting
your data.
6. VA will use donated data to test software.
7. VA uses made-up data to test software
now.
8. Employees and contractors who build
software for the VA will have access to
your health data.
9. You can drop out at any time, by sending
an email request to a specific address.
10. If you drop out, VA will keep your
authorization and withdrawal forms, and
stop using your data.
PLAIN
2013
|
Vancouver
CA

7. The brochure didn’t test as well as we expected ….
Target
=
80%
recall
1
2
3
4
5
6
7
8
9
10
PLAIN
2013
|
Vancouver
CA

8. What we learned
• Some key facts were not clear enough
• As a result, the brochure was not as persuasive as
desired
• If we hadn’t tested,
we would never have known.
PLAIN
2013
|
Vancouver
CA

9. Why am I telling you this?
• We have entered the age of content.
PLAIN
2013
|
Vancouver
CA

10. In the Age of Content,
there are no guides.
PLAIN
2013
|
Vancouver
CA

11. In the Age of Content,
there are no guides.
PLAIN
2013
|
Vancouver
CA

12. In the age of content,
we will have to stop
blaming the victim.
*
Tes?ng
shows
that
the
F-­‐shaped
reading
behavior
is
a
symptom
of
the
relevance
of
content
not
an
determined
habit
of
humans.
PLAIN
2013
|
Vancouver
CA

13. Why am I telling you this?
• We have entered the age of content.
• Comprehension testing sounds harder than it is.
(Don’t tell anyone.)
PLAIN
2013
|
Vancouver
CA

14. Usability people
Content People
• Content is not our job,
• We are writers, not researchers
Usability people
its their job
PLAIN
2013
|
Vancouver
CA

15. Why am I telling you this?
• We have entered the age of content.
• Comprehension testing sounds harder than it is.
• We were surprised by our own results.
Don’t
tell
anyone
this
either,
ok?
PLAIN
2013
|
Vancouver
CA

16. A process
(Guerilla content testing)
PLAIN
2013
|
Vancouver
CA

17. 3 types of content testing
• “Simple” comprehension testing
• Confidence testing
• Persuasiveness testing
• Some logistics
PLAIN
2013
|
Vancouver
CA

18. “Simple” comprehension testing
1.
2.
3.
4.
PLAIN
2013
|
Vancouver
CA
Agree
on
the
facts
Decide
which
are
most
important
Create
a
ques?on
about
each
fact
Agree
on
the
answers

19. The challenge: Good questions are hard to write
PLAIN
2013
|
Vancouver
CA
Tips:
• Expect
to
revise
your
ques?ons
several
?mes
• People
remember
m/c
strategies
from
uni
(e.g.,
the
longer,
specific
answer
is
the
right
one.
• Give
par?cipants
an
alterna?ve
to
guessing
(e.g.,
“The
brochure
didn’t
say”)

20. “Simple” comprehension testing
1.
2.
3.
4.
Agree
on
the
facts
Decide
which
are
most
important
Create
a
ques?on
about
each
fact
Agree
on
the
answer
5. Pre-­‐test
your
ques?ons
PLAIN
2013
|
Vancouver
CA

21. Comprehension testing: Have a Bake-off
Tip:
No
one
version
is
going
to
be
all
beer.
By
tes?ng
mul?ple
versions
you
can
compare
and
contrast
which
works
best
(for
which
audiences.)
PLAIN
2013
|
Vancouver
CA

22. “Simple” comprehension testing
1.
2.
3.
4.
Agree
on
the
facts
Decide
which
are
most
important
Create
a
ques?on
about
each
fact
Agree
on
the
answer
5. Pre-­‐test
your
ques?ons
6. Use
the
test
to
have
a
bakeoff
PLAIN
2013
|
Vancouver
CA

23. Reporting comprehension results
PLAIN
2013
|
Vancouver
CA

24. Confidence testing
Could
you
describe
the
Donate
my
Data
program
to
a
family
member
or
friend?
a. I
can
describe
both
the
idea
and
the
details
to
someone
else,
and
get
them
all
right.
b. I
can
describe
it
mostly,
but
I
might
get
some
of
the
details
wrong.
c. I
can
describe
the
general
idea,
but
not
the
details.
d. I
wouldn’t
try
to
describe
this
to
someone
else.
PLAIN
2013
|
Vancouver
CA

25. Persuasiveness testing
Just because I can,
doesn’t mean I will.
PLAIN
2013
|
Vancouver
CA

26. Persuasiveness testing
• Motivators
• Blocks/Concerns
• Barriers
PLAIN
2013
|
Vancouver
CA

27. Mechanical Turk (Mturk).
Crowdsourcing science
• A parable
• A process
PLAIN
2013
|
Vancouver
CA

28. PLAIN
2013
|
Vancouver
CA

29. Some MTurk tips
• Start with a comprehension test
• Embed ‘catch’ questions
• Create a ‘panel’ over time
• You get what you invest and what you pay for
PLAIN
2013
|
Vancouver
CA

30. A case for testing
• Testing doesn’t have to be complicated to be robust
• Crowd-sourced testing can be quick, cheap and
effective.
• You don’t really know its plain language to your target
audience unless you’ve tested in your target audience
PLAIN
2013
|
Vancouver
CA