Is it really plain?
A case for content testing

Annetta Cheek
Center for Plain Language
Kath Straub
Usability.org | Center...
Where we are going ….
•  A parable
•  A process

PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  
The Parable

PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  
PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  
PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  
We identified 10 “must-know” facts.
1.  Any veteran can join.
2.  When you join, you give access to your
VA System electron...
The brochure didn’t test as well as we expected ….

Target	
  =	
  80%	
  
recall	
  

1	
  

2	
  

3	
  

4	
  

5	
  

...
What we learned
•  Some key facts were not clear enough
•  As a result, the brochure was not as persuasive as
desired
•  I...
Why am I telling you this?
•  We have entered the age of content.

PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  
In the Age of Content,
there are no guides.

PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  
In the Age of Content,
there are no guides.

PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  
In the age of content,
we will have to stop
blaming the victim.

*	
  Tes?ng	
  shows	
  that	
  	
  the	
  	
  

F-­‐shap...
Why am I telling you this?
•  We have entered the age of content.
•  Comprehension testing sounds harder than it is.
(Don’...
Usability people

Content People

•  Content is not our job,
•  We are writers, not researchers
Usability people
its their...
Why am I telling you this?
•  We have entered the age of content.
•  Comprehension testing sounds harder than it is.
•  We...
A process

(Guerilla content testing)

PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  
3 types of content testing
•  “Simple” comprehension testing
•  Confidence testing
•  Persuasiveness testing
•  Some logist...
“Simple” comprehension testing
1. 
2. 
3. 
4. 
	
  

PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  

Agree	
  on	
  the	
  fact...
The challenge: Good questions are hard to write

PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  

Tips:	
  
•  Expect	
  to	
  r...
“Simple” comprehension testing
1. 
2. 
3. 
4. 

Agree	
  on	
  the	
  facts	
  
Decide	
  which	
  are	
  most	
  importan...
Comprehension testing: Have a Bake-off

Tip:	
  No	
  one	
  version	
  is	
  going	
  to	
  be	
  all	
  
beer.	
  By	
  t...
“Simple” comprehension testing
1. 
2. 
3. 
4. 

Agree	
  on	
  the	
  facts	
  
Decide	
  which	
  are	
  most	
  importan...
Reporting comprehension results

PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  
Confidence testing

Could	
  you	
  describe	
  the	
  Donate	
  my	
  Data	
  program	
  to	
  a	
  family	
  member	
  or...
Persuasiveness testing

Just because I can,
doesn’t mean I will.

PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  
Persuasiveness testing
•  Motivators
•  Blocks/Concerns
•  Barriers
	
  

PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  
Mechanical Turk (Mturk).
Crowdsourcing science

•  A parable
•  A process

PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  
PLAIN	
  2013	
  |	
  Vancouver	
  CA	
  
Some MTurk tips
•  Start with a comprehension test
•  Embed ‘catch’ questions
•  Create a ‘panel’ over time
•  You get wha...
A case for testing
•  Testing doesn’t have to be complicated to be robust
•  Crowd-sourced testing can be quick, cheap and...
Upcoming SlideShare
Loading in...5
×

PLAIN 2013 - Is it really plain? A case (and process) for content testing

1,082

Published on

As plain language professionals, we think we know the rules and can recognize when a document or website is presented in plain language. But do our customers always agree?

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,082
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

PLAIN 2013 - Is it really plain? A case (and process) for content testing

  1. 1. Is it really plain? A case for content testing Annetta Cheek Center for Plain Language Kath Straub Usability.org | Center for Plain Language PLAIN  2013  |  Vancouver  CA  
  2. 2. Where we are going …. •  A parable •  A process PLAIN  2013  |  Vancouver  CA  
  3. 3. The Parable PLAIN  2013  |  Vancouver  CA  
  4. 4. PLAIN  2013  |  Vancouver  CA  
  5. 5. PLAIN  2013  |  Vancouver  CA  
  6. 6. We identified 10 “must-know” facts. 1.  Any veteran can join. 2.  When you join, you give access to your VA System electronic health record, including –  –  –  –  –  –  Your vital signs Diagnoses Doctor’s notes Medicines Treatment history Test results 3.  VA will use the data starting from the time you entered the system until 10 years after you joined Donate My Data. 4.  If you die during the program, the VA will stop using your data. 5.  VA will follow the laws about protecting your data. 6.  VA will use donated data to test software. 7.  VA uses made-up data to test software now. 8.  Employees and contractors who build software for the VA will have access to your health data. 9.  You can drop out at any time, by sending an email request to a specific address. 10.  If you drop out, VA will keep your authorization and withdrawal forms, and stop using your data. PLAIN  2013  |  Vancouver  CA  
  7. 7. The brochure didn’t test as well as we expected …. Target  =  80%   recall   1   2   3   4   5   6   7   8   9   10 PLAIN  2013  |  Vancouver  CA    
  8. 8. What we learned •  Some key facts were not clear enough •  As a result, the brochure was not as persuasive as desired •  If we hadn’t tested, we would never have known. PLAIN  2013  |  Vancouver  CA  
  9. 9. Why am I telling you this? •  We have entered the age of content. PLAIN  2013  |  Vancouver  CA  
  10. 10. In the Age of Content, there are no guides. PLAIN  2013  |  Vancouver  CA  
  11. 11. In the Age of Content, there are no guides. PLAIN  2013  |  Vancouver  CA  
  12. 12. In the age of content, we will have to stop blaming the victim. *  Tes?ng  shows  that    the     F-­‐shaped  reading  behavior  is  a   symptom  of  the  relevance  of   content  not  an  determined  habit   of  humans.   PLAIN  2013  |  Vancouver  CA  
  13. 13. Why am I telling you this? •  We have entered the age of content. •  Comprehension testing sounds harder than it is. (Don’t tell anyone.) PLAIN  2013  |  Vancouver  CA  
  14. 14. Usability people Content People •  Content is not our job, •  We are writers, not researchers Usability people its their job PLAIN  2013  |  Vancouver  CA  
  15. 15. Why am I telling you this? •  We have entered the age of content. •  Comprehension testing sounds harder than it is. •  We were surprised by our own results. Don’t  tell     anyone  this     either,  ok?   PLAIN  2013  |  Vancouver  CA  
  16. 16. A process (Guerilla content testing) PLAIN  2013  |  Vancouver  CA  
  17. 17. 3 types of content testing •  “Simple” comprehension testing •  Confidence testing •  Persuasiveness testing •  Some logistics PLAIN  2013  |  Vancouver  CA  
  18. 18. “Simple” comprehension testing 1.  2.  3.  4.    PLAIN  2013  |  Vancouver  CA   Agree  on  the  facts   Decide  which  are  most  important   Create  a  ques?on  about  each  fact   Agree  on  the  answers  
  19. 19. The challenge: Good questions are hard to write PLAIN  2013  |  Vancouver  CA   Tips:   •  Expect  to  revise  your  ques?ons  several  ?mes   •  People  remember  m/c  strategies  from  uni  (e.g.,     the  longer,  specific  answer  is  the  right  one.   •  Give  par?cipants  an  alterna?ve  to  guessing   (e.g.,  “The  brochure  didn’t  say”)  
  20. 20. “Simple” comprehension testing 1.  2.  3.  4.  Agree  on  the  facts   Decide  which  are  most  important   Create  a  ques?on  about  each  fact   Agree  on  the  answer   5.  Pre-­‐test  your  ques?ons         PLAIN  2013  |  Vancouver  CA  
  21. 21. Comprehension testing: Have a Bake-off Tip:  No  one  version  is  going  to  be  all   beer.  By  tes?ng  mul?ple  versions  you   can  compare  and  contrast  which  works   best  (for  which  audiences.)   PLAIN  2013  |  Vancouver  CA  
  22. 22. “Simple” comprehension testing 1.  2.  3.  4.  Agree  on  the  facts   Decide  which  are  most  important   Create  a  ques?on  about  each  fact   Agree  on  the  answer   5.  Pre-­‐test  your  ques?ons   6.  Use  the  test  to  have  a  bakeoff         PLAIN  2013  |  Vancouver  CA  
  23. 23. Reporting comprehension results PLAIN  2013  |  Vancouver  CA  
  24. 24. Confidence testing Could  you  describe  the  Donate  my  Data  program  to  a  family  member  or  friend?   a.  I  can  describe  both  the  idea  and  the  details  to  someone  else,  and  get  them  all   right.   b.  I  can  describe  it  mostly,  but  I  might  get  some  of  the  details  wrong.   c.  I  can  describe  the  general  idea,  but  not  the  details.   d.  I  wouldn’t  try  to  describe  this  to  someone  else.   PLAIN  2013  |  Vancouver  CA  
  25. 25. Persuasiveness testing Just because I can, doesn’t mean I will. PLAIN  2013  |  Vancouver  CA  
  26. 26. Persuasiveness testing •  Motivators •  Blocks/Concerns •  Barriers   PLAIN  2013  |  Vancouver  CA  
  27. 27. Mechanical Turk (Mturk). Crowdsourcing science •  A parable •  A process PLAIN  2013  |  Vancouver  CA  
  28. 28. PLAIN  2013  |  Vancouver  CA  
  29. 29. Some MTurk tips •  Start with a comprehension test •  Embed ‘catch’ questions •  Create a ‘panel’ over time •  You get what you invest and what you pay for PLAIN  2013  |  Vancouver  CA  
  30. 30. A case for testing •  Testing doesn’t have to be complicated to be robust •  Crowd-sourced testing can be quick, cheap and effective. •  You don’t really know its plain language to your target audience unless you’ve tested in your target audience PLAIN  2013  |  Vancouver  CA  
  1. ¿Le ha llamado la atención una diapositiva en particular?

    Recortar diapositivas es una manera útil de recopilar información importante para consultarla más tarde.

×