Testing Content: Early, Often, & Well

Testing Content Early, Often, & Well Colleen Jones Kevin O’Connor colleen@content-science.com koconnor@userinsight.com

Testing design. You’ve probably heard of it. It’s good. But, it’s not enough.

We’ve talked about testing content for years, but as an industry...

“ Too many usability tests focus ONLY on finding information— “ not on how the information [content] itself works for people. Ginny Redish

What we mean by “content.” “ Content includes the text, graphics, video, & audio “ that make up an interactive experience. Kristina Halvorson

THE SITUATION

CDC Travelers’ Health wanted to assess their content.

TREND: Content is critical to the health industry. “ …providers of online health resources can increase the relevance & influence of their Web sites if they •  focus on site usability, •  bolster the credibility of their content, & “ •  bridge the gap between Web & point of care.  Who Cares About Online Health Content? Forrester Research

The site is a big opportunity to improve health decisions through content. •  Top 5 most popular CDC websites •  In 2009 •  9.5 million visits •  31 million page views

“ With such rich content on our site, I was concerned about the number of people “ who were calling because they couldn’t find vaccine information. Kelly Holton Communication & Education Team Lead Travelers’ Health, CDC

“ I knew it was time to do something different to help drive our content strategy. “ I felt we needed to find out what users really think about & do with our content. Kelly Holton Communication & Education Team Lead Travelers’ Health, CDC

CDC had clear objectives. •  Gather feedback about different user groups’… •  Content needs & preferences. •  Perspectives on content quality, presentation, & format. •  Create a content and usability testing model.

OUR APPROACH

Testing content helps you choose the right content direction.

We tested content iteratively. Baseline Test Concepts Validation Test

BASELINE TEST

We focused on a critical content sample: DESTINATIONS

We tested with these user groups. Travelers Travel Medicine Consumers who do or do Professionals not research health-‐related Doctors, clinic admins, informa3on before nurses, & specialists traveling to foreign provide travel health countries (N=12) informa3on & care (N=6) Healthcare Professionals Travel Professionals Doctors, nurses, & Travel & tourism industry pharmacists who are in a representa3ves (e.g., travel posi3on to provide travel agents, tour organizers, health-‐related informa3on etc.) (N=6) & care (N=12)

We observed & interviewed people one-on-one. We needed a deep understanding of WHY.

Our protocol focused on key content questions… Can users… •  Find & read the content they need? •  Understand the content? •  Act on the content?

We learned a lot. Can they find & read it? NO •  Users couldn’t find vaccinations. Do they understand it? PARTLY •  Users were not sure whether the content was specific to the destination. •  Users were distracted by & misunderstood "Travel Notices.” Can & will they act on it? NO •  Users trusted CDC but… •  Felt overwhelmed by the content. •  Were confused about which vaccinations were required, necessary, or most relevant to their level of risk. •  Were not clear on the next step they should take.

TMI PORTABLE FORMATS? CONFUSING DATES RISK LEVELS? AMBIGUOUS LANGUAGE HARD-TO-READ FORMATS

Lost in Content

We recommended focusing on travelers. •  Other sections, such as Yellow Book, addressed travel Travelers medicine professionals better. Consumers who do or do not research health-‐related informa3on before •  80 / 20 rule applies. traveling to foreign countries (N=12) •  Health professionals & travel professionals act as catalysts or referrals.

THE CONCEPTS

Travelers need to decide… •  Whether they are at risk. •  What shots to get. •  How to get shots. •  How to avoid common diseases. •  What health items to bring. •  More

“ “ Design from the content out. Jeffrey Zeldman That’s exactly what we did.

We explored content… •  Organization & priority •  Layering & layout •  Writing style & tone •  Calls to action

5 organization schemes Priority on vaccinations

Tone & language Rhetorical / psychological framing

Layering / progressive disclosure Decision table

Call to action

Then, we tried variations. Concept 1 Concept 2 Layering & •  Less intro instruction •  More intro instruction layout •  Tabs •  Reveal •  More explanation in •  Less explanation in vaccination summary vaccination summary Calls to action •  Bold buttons •  Contextual text Style & tone •  More familiar •  Less familiar

Does this content work better for travelers?

TESTING THE CONCEPTS

We tested 2 concepts + the original. “ …by presenting users with alternative…solutions, subjective ratings are less prone to inflation & “ give rise to more & stronger criticisms. −Getting the Right Design & the Design Right CHI 2006

The protocol was similar to the baseline. •  Can they find and read it? •  Do they understand it? •  Will they act on it?

We didn’t expect a “winner,” just better feedback.

We followed best testing techniques. •  Tested on a small scale. •  Avoided order bias. •  Followed a discussion guide. •  Took time stamped notes & looked at patterns of behaviors.

The concepts tested better. Can they find it? YES Do they understand it? MOSTLY •  Users understood the vaccination content but…. •  Asked for more explanation of the ratings. •  Were still distracted by & misunderstood "Travel Notices.” Can and will they act on it? YES •  Users preferred more explanation in the decision table. •  Users knew the next step they should take. •  Despite tone & framing, users still trusted the content.

Preference for Concept

What’s Next •  Testing •  Evaluation 1 •  Analysis 2 •  Concept 3 • Refinement 4 •  Quality Plan Create •  Testing Deliver •  Testing Govern •  Synthesis Control Process Inspired by Content Strategy for the Web

We’re not done, but we know we’re on the right track.

Top Takeaways for Testing Content 1.  Test the content early in a project, then iteratively. 2.  Test for whether people can use, understand, and act on the content. 3.  Test using a mix of concrete tasks & exploratory questions. 4.  Test with the right people. 5.  Test more than one concept for the best feedback.

Find out whether your content works for people. And learn all you can from the journey.

Questions? Colleen Jones Kevin O’Connor colleen@content-science.com koconnor@userinsight.com

Acknowledgments •  Centers for Disease Control & Prevention (CDC) •  Oak Ridge Associated Universities •  Kim Ware, Content Science •  Karen Williams, User Insight

http://www.leenjones.com	77
http://www.weebly.com	44
http://lostandfound.posterous.com	14
http://www.slideshare.net	11
http://greenlifeadvocacy.weebly.com	10
http://paper.li	4

Testing Content: Early, Often, & Well

by Colleen Jones on Apr 11, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

6 Embeds 160

Statistics

Testing Content: Early, Often, & Well — Presentation Transcript