Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Testing Content: Early, Often, & Well


Published on

You've heard about testing design. What about testing content? This presentation walks through a case study of testing content with people and offers key testing takeaways.

Published in: Business

Testing Content: Early, Often, & Well

  1. Testing Content Early, Often, & Well Colleen Jones Kevin O’Connor
  2. Testing design. You’ve probably heard of it. It’s good. But, it’s not enough.
  3. We’ve talked about testing content for years, but as an industry...
  4. “ Too many usability tests focus ONLY on finding information— “ not on how the information [content] itself works for people. Ginny Redish
  5. What we mean by “content.” “ Content includes the text, graphics, video, & audio “ that make up an interactive experience. Kristina Halvorson
  7. CDC Travelers’ Health wanted to assess their content.  
  8. TREND: Content is critical to the health industry. “ …providers of online health resources can increase the relevance & influence of their Web sites if they •  focus on site usability, •  bolster the credibility of their content, & “ •  bridge the gap between Web & point of care.
 Who Cares About Online Health Content? Forrester Research
  9. The site is a big opportunity to improve health decisions through content. •  Top 5 most popular CDC websites •  In 2009 •  9.5 million visits •  31 million page views
  10. “ With such rich content on our site, I was concerned about the number of people “ who were calling because they couldn’t find vaccine information. Kelly Holton Communication & Education Team Lead Travelers’ Health, CDC
  11. “ I knew it was time to do something different to help drive our content strategy. “ I felt we needed to find out what users really think about & do with our content. Kelly Holton Communication & Education Team Lead Travelers’ Health, CDC
  12. CDC had clear objectives.   •  Gather feedback about different user groups’… •  Content needs & preferences. •  Perspectives on content quality, presentation, & format. •  Create a content and usability testing model.
  14. Testing content helps you choose the right content direction.
  15. We tested content iteratively.   Baseline Test Concepts Validation Test
  17. We focused on a critical content sample: DESTINATIONS  
  18. We tested with these user groups. Travelers   Travel  Medicine   Consumers  who  do  or  do   Professionals   not  research  health-­‐related   Doctors,  clinic  admins,   informa3on  before   nurses,  &  specialists   traveling  to  foreign   provide  travel  health   countries  (N=12)   informa3on  &  care  (N=6)   Healthcare  Professionals   Travel  Professionals   Doctors,  nurses,  &   Travel  &  tourism  industry   pharmacists  who  are  in  a   representa3ves  (e.g.,  travel   posi3on  to  provide  travel   agents,  tour  organizers,   health-­‐related  informa3on   etc.)  (N=6)   &  care  (N=12)  
  19. We observed & interviewed people one-on-one.   We needed a deep understanding of WHY.
  20. Our protocol focused on key content questions…   Can users… •  Find & read the content they need? •  Understand the content? •  Act on the content?
  21. We learned a lot.   Can they find & read it? NO •  Users couldn’t find vaccinations. Do they understand it? PARTLY •  Users were not sure whether the content was specific to the destination. •  Users were distracted by & misunderstood "Travel Notices.” Can & will they act on it? NO •  Users trusted CDC but… •  Felt overwhelmed by the content. •  Were confused about which vaccinations were required, necessary, or most relevant to their level of risk. •  Were not clear on the next step they should take.
  23. Lost in Content  
  24. We recommended focusing on travelers. •  Other sections, such as Yellow Book, addressed travel Travelers   medicine professionals better. Consumers  who  do  or  do   not  research  health-­‐related   informa3on  before   •  80 / 20 rule applies. traveling  to  foreign   countries  (N=12)   •  Health professionals & travel professionals act as catalysts or referrals.
  26. Travelers need to decide…   •  Whether they are at risk. •  What shots to get. •  How to get shots. •  How to avoid common diseases. •  What health items to bring. •  More
  27. “ “ Design from the content out. Jeffrey Zeldman That’s exactly what we did.  
  28. We explored content…   •  Organization & priority •  Layering & layout •  Writing style & tone •  Calls to action
  29. 5 organization schemes Priority on vaccinations
  30. Tone & language Rhetorical / psychological framing  
  31. Layering / progressive disclosure Decision table
  32. Call to action
  33. Then, we tried variations. Concept 1 Concept 2 Layering & •  Less intro instruction •  More intro instruction layout •  Tabs •  Reveal •  More explanation in •  Less explanation in vaccination summary vaccination summary Calls to action •  Bold buttons •  Contextual text Style & tone •  More familiar •  Less familiar
  34. Does this content work better for travelers?  
  36. We tested 2 concepts + the original.   “ …by presenting users with alternative…solutions, subjective ratings are less prone to inflation & “ give rise to more & stronger criticisms. −Getting the Right Design & the Design Right CHI 2006
  37. The protocol was similar to the baseline.   •  Can they find and read it? •  Do they understand it? •  Will they act on it?
  38. We didn’t expect a “winner,” just better feedback.  
  39. We followed best testing techniques.   •  Tested on a small scale. •  Avoided order bias. •  Followed a discussion guide. •  Took time stamped notes & looked at patterns of behaviors.
  40. The concepts tested better.   Can they find it? YES Do they understand it? MOSTLY •  Users understood the vaccination content but…. •  Asked for more explanation of the ratings. •  Were still distracted by & misunderstood "Travel Notices.” Can and will they act on it? YES •  Users preferred more explanation in the decision table. •  Users knew the next step they should take. •  Despite tone & framing, users still trusted the content.
  41. Preference for Concept  
  42. What’s Next •  Testing •  Evaluation 1 •  Analysis 2 •  Concept 3 • Refinement 4 •  Quality Plan Create •  Testing Deliver •  Testing Govern •  Synthesis Control Process  Inspired  by  Content  Strategy  for  the  Web  
  43. We’re not done, but we know we’re on the right track.    
  44. Top Takeaways for Testing Content   1.  Test the content early in a project, then iteratively. 2.  Test for whether people can use, understand, and act on the content. 3.  Test using a mix of concrete tasks & exploratory questions. 4.  Test with the right people. 5.  Test more than one concept for the best feedback.
  45. Find out whether your content works for people. And learn all you can from the journey.
  46. Questions? Colleen Jones Kevin O’Connor
  47. Acknowledgments   •  Centers for Disease Control & Prevention (CDC) •  Oak Ridge Associated Universities •  Kim Ware, Content Science •  Karen Williams, User Insight