Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PLAIN 2013 - Is it really plain? A case (and process) for content testing

1,460 views

Published on

As plain language professionals, we think we know the rules and can recognize when a document or website is presented in plain language. But do our customers always agree?

Published in: Technology, Business
  • Be the first to comment

PLAIN 2013 - Is it really plain? A case (and process) for content testing

  1. 1. Is it really plain? A case for content testing Annetta Cheek Center for Plain Language Kath Straub Usability.org | Center for Plain Language PLAIN  2013  |  Vancouver  CA  
  2. 2. Where we are going …. •  A parable •  A process PLAIN  2013  |  Vancouver  CA  
  3. 3. The Parable PLAIN  2013  |  Vancouver  CA  
  4. 4. PLAIN  2013  |  Vancouver  CA  
  5. 5. PLAIN  2013  |  Vancouver  CA  
  6. 6. We identified 10 “must-know” facts. 1.  Any veteran can join. 2.  When you join, you give access to your VA System electronic health record, including –  –  –  –  –  –  Your vital signs Diagnoses Doctor’s notes Medicines Treatment history Test results 3.  VA will use the data starting from the time you entered the system until 10 years after you joined Donate My Data. 4.  If you die during the program, the VA will stop using your data. 5.  VA will follow the laws about protecting your data. 6.  VA will use donated data to test software. 7.  VA uses made-up data to test software now. 8.  Employees and contractors who build software for the VA will have access to your health data. 9.  You can drop out at any time, by sending an email request to a specific address. 10.  If you drop out, VA will keep your authorization and withdrawal forms, and stop using your data. PLAIN  2013  |  Vancouver  CA  
  7. 7. The brochure didn’t test as well as we expected …. Target  =  80%   recall   1   2   3   4   5   6   7   8   9   10 PLAIN  2013  |  Vancouver  CA    
  8. 8. What we learned •  Some key facts were not clear enough •  As a result, the brochure was not as persuasive as desired •  If we hadn’t tested, we would never have known. PLAIN  2013  |  Vancouver  CA  
  9. 9. Why am I telling you this? •  We have entered the age of content. PLAIN  2013  |  Vancouver  CA  
  10. 10. In the Age of Content, there are no guides. PLAIN  2013  |  Vancouver  CA  
  11. 11. In the Age of Content, there are no guides. PLAIN  2013  |  Vancouver  CA  
  12. 12. In the age of content, we will have to stop blaming the victim. *  Tes?ng  shows  that    the     F-­‐shaped  reading  behavior  is  a   symptom  of  the  relevance  of   content  not  an  determined  habit   of  humans.   PLAIN  2013  |  Vancouver  CA  
  13. 13. Why am I telling you this? •  We have entered the age of content. •  Comprehension testing sounds harder than it is. (Don’t tell anyone.) PLAIN  2013  |  Vancouver  CA  
  14. 14. Usability people Content People •  Content is not our job, •  We are writers, not researchers Usability people its their job PLAIN  2013  |  Vancouver  CA  
  15. 15. Why am I telling you this? •  We have entered the age of content. •  Comprehension testing sounds harder than it is. •  We were surprised by our own results. Don’t  tell     anyone  this     either,  ok?   PLAIN  2013  |  Vancouver  CA  
  16. 16. A process (Guerilla content testing) PLAIN  2013  |  Vancouver  CA  
  17. 17. 3 types of content testing •  “Simple” comprehension testing •  Confidence testing •  Persuasiveness testing •  Some logistics PLAIN  2013  |  Vancouver  CA  
  18. 18. “Simple” comprehension testing 1.  2.  3.  4.    PLAIN  2013  |  Vancouver  CA   Agree  on  the  facts   Decide  which  are  most  important   Create  a  ques?on  about  each  fact   Agree  on  the  answers  
  19. 19. The challenge: Good questions are hard to write PLAIN  2013  |  Vancouver  CA   Tips:   •  Expect  to  revise  your  ques?ons  several  ?mes   •  People  remember  m/c  strategies  from  uni  (e.g.,     the  longer,  specific  answer  is  the  right  one.   •  Give  par?cipants  an  alterna?ve  to  guessing   (e.g.,  “The  brochure  didn’t  say”)  
  20. 20. “Simple” comprehension testing 1.  2.  3.  4.  Agree  on  the  facts   Decide  which  are  most  important   Create  a  ques?on  about  each  fact   Agree  on  the  answer   5.  Pre-­‐test  your  ques?ons         PLAIN  2013  |  Vancouver  CA  
  21. 21. Comprehension testing: Have a Bake-off Tip:  No  one  version  is  going  to  be  all   beer.  By  tes?ng  mul?ple  versions  you   can  compare  and  contrast  which  works   best  (for  which  audiences.)   PLAIN  2013  |  Vancouver  CA  
  22. 22. “Simple” comprehension testing 1.  2.  3.  4.  Agree  on  the  facts   Decide  which  are  most  important   Create  a  ques?on  about  each  fact   Agree  on  the  answer   5.  Pre-­‐test  your  ques?ons   6.  Use  the  test  to  have  a  bakeoff         PLAIN  2013  |  Vancouver  CA  
  23. 23. Reporting comprehension results PLAIN  2013  |  Vancouver  CA  
  24. 24. Confidence testing Could  you  describe  the  Donate  my  Data  program  to  a  family  member  or  friend?   a.  I  can  describe  both  the  idea  and  the  details  to  someone  else,  and  get  them  all   right.   b.  I  can  describe  it  mostly,  but  I  might  get  some  of  the  details  wrong.   c.  I  can  describe  the  general  idea,  but  not  the  details.   d.  I  wouldn’t  try  to  describe  this  to  someone  else.   PLAIN  2013  |  Vancouver  CA  
  25. 25. Persuasiveness testing Just because I can, doesn’t mean I will. PLAIN  2013  |  Vancouver  CA  
  26. 26. Persuasiveness testing •  Motivators •  Blocks/Concerns •  Barriers   PLAIN  2013  |  Vancouver  CA  
  27. 27. Mechanical Turk (Mturk). Crowdsourcing science •  A parable •  A process PLAIN  2013  |  Vancouver  CA  
  28. 28. PLAIN  2013  |  Vancouver  CA  
  29. 29. Some MTurk tips •  Start with a comprehension test •  Embed ‘catch’ questions •  Create a ‘panel’ over time •  You get what you invest and what you pay for PLAIN  2013  |  Vancouver  CA  
  30. 30. A case for testing •  Testing doesn’t have to be complicated to be robust •  Crowd-sourced testing can be quick, cheap and effective. •  You don’t really know its plain language to your target audience unless you’ve tested in your target audience PLAIN  2013  |  Vancouver  CA  

×