Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cognifide content usabilitytesting-csa2017-v0.1


Published on

Content usability testing - a case study in how to measure the effectiveness of your content

Published in: Business
  • Be the first to comment

Cognifide content usabilitytesting-csa2017-v0.1

  1. 1. Kate Kenyon @kate_kenyon Content Usability Testing: Getting to Good
  2. 2. Hello, I’m Kate ‘80s – bookish child of an IT dev and a PM ‘90s – linguist, magazine writer ‘00s – marketer, journalist, technologist, content strategist I design content, and I design the teams and systems that make it.
  3. 3. What we’re covering today We are here Lunch is here
  4. 4. The Goal is to deliver a better customer experience using effective content
  5. 5. Testing content shows the gap between strategy and practice
  6. 6. Testing provides data for content decision making
  7. 7. Testing content informs IA, CS, and Design Image:
  8. 8. Testing content keeps stakeholders aligned Image courtesy of Shutterstock
  9. 9. Testing content explodes expectations
  10. 10. Testing enables content to become a quantifiable, manageable asset with genuine business value
  11. 11. The Challenge is to create a content testing process that will yield regular, usable data
  12. 12. Testing is part of a cycle, not a process Plan Make TestReview Publish
  13. 13. Testing content needs to be a repeatable exercise Plan Make TestReview Publish Plan Make TestReview Publish Plan Make TestReview Publish Plan Make TestReview Publish
  14. 14. It’s not the testing, it’s what you do with the results Repeated testing builds data around how content works on your site, and what you should be making. That data needs to be part of your decision making processes Plan Make TestReview Publish
  15. 15. Old production processes + a bit of testing won’t deliver the benefits you’d like. You need a plan to change your processes, before you start testing.
  16. 16. Approach
  17. 17. How I do testing Image courtesy of Shutterstock • Content testing principles • Mapping the landscape • Testing the things that matter • Writing test cases • Running tests • Writing up findings
  18. 18. Content Testing Principles Image courtesy of Shutterstock • Content is there to do a job. Know what that job is, and test for how well the content is doing that. • Test by asking questions that are important to your key people. These could be customers or the business, or both. • Test using questions with definitive answers.
  19. 19. Mapping the landscape Image courtesy of Shutterstock
  20. 20. Test for the things that matter Image courtesy of Shutterstock • What is the significant influencing driver within the business? • How do your testing goals align with that driver? • What will you do with the results of the testing? • Who will potentially be impacted by the testing?
  21. 21. Writing test cases • Write tasks: one task per question • Questions must have ONE definitive answer • No clues in the questions • Use plain language and common speech • Be concise • Make tasks independent from each other • Make sure tasks don’t require confidential information.
  22. 22. Writing test cases: samples • How many days holidays is a new, full-time employee entitled to in UK offices? • Who should be the first person you contact if you have an accident while on site with a client? • How many bottles of hair dye did the company sell in the UK in 2015?
  23. 23. Running tests Image courtesy of Shutterstock
  24. 24. Running Tests • Choose volunteers carefully: screen for prior knowledge and bias. • Give volunteers plenty of notice, and tell them in advance what will happen. • Stress that you’re testing the site, not them. • Check your environment! Equipment, connections, facilities, room bookings • Have water, pens, paper, tissues, food available.
  25. 25. Recording results • Use VIDEO! Jing is free, easy screen recording software. There are many others, and for remote testing, GotoMeeting has worked well. • Ask participants to comment and narrate as they go • Have one test observer sit beside the participant to guide and lead the test • Have another observer sit behind them to one side, so they can look over their shoulder, and note down any behaviours or comments.
  26. 26. Writing up results Image courtesy of Shutterstock • Use a meaningful format. Powerpoint, doc, video: choose a format that will best deliver the message to the intended audience • Exec summaries are essential, and recommendations. • If quoting from participants, maintain anonymity, and quote accurately. • Provide the data in appendices
  27. 27. Some notes on things I’ve learned during testing
  28. 28. Testing: have a clear view of who is involved Image courtesy of Shutterstock
  29. 29. Be clear that this is about making hard choices Image:
  30. 30. Testing content makes you align what you’ve got with what you want
  31. 31. Case Studies
  32. 32. Mapping the landscape Image courtesy of Shutterstock • A global intranet site hosting over 100,000 pages of content • One (rather ancient) implementation of Sharepoint • Thousands of employees, across multiple countries • A head of employee comms fighting to get her message across • No way of showing the impact that the intranet was having.
  33. 33. Image courtesy of Shutterstock Image courtesy of Shutterstock
  34. 34. Mapping the landscape Image courtesy of Shutterstock How effective is the Bank’s intranet in supporting Global Standards? Does the intranet help employees implement their day-to-day activities, or actively contribute to risk within the Bank? Cognifide, a technology and consulting firm, was commissioned to conduct content usability research to look answering these questions. The research was conducted between July and August 2016, and focussed on how well Bank employees could use the intranet to answer 5 questions related to Global Standards. It found that, on average, employees were only able to find the answers to just over half (51%) of the questions. This means that for this test the failure rate was 49%, a significant exposure to risk for the Bank.
  35. 35. Writing test cases Image courtesy of Shutterstock • Asking each user to find the answers to a set of questions relating to Global Standards. They had to use the intranet only to complete the task. • Questions were worded to have an unambiguous answer. • Additional self-assessment questionnaire to provide a subjective view on how easy it was to do the tasks Outcome Meaning Metrics Success The information can be found in a reasonable amount of time. User can complete the task/give the right answer in the allotted time. Fail The information cannot be found in a reasonable amount of time. User cannot complete the task/give the right answer in the allotted time, or gives up. Disaster The information is confusing or misleading. User completes a different task/gives the wrong answer.
  36. 36. Participants & Location Image courtesy of Shutterstock • Participants were recruited among Bank new joiners who had completed their mandatory training. They took part anonymously and they are identified in this report by a progressive number (Participant 1, Participant 2, etc.). • Anonymity is a pre-requisite for this type of research, to ensure participants can freely express their opinion. All participants were already familiar with the intranet, and most of them stated that they use the intranet daily for their job. • Sessions were run at the Bank in a common meeting room, without external observers, and they were attended by the participants, one at a time. Two facilitators from Cognifide were attending as well, one running the session and another one taking notes.
  37. 37. Running tests Image courtesy of Shutterstock Task # Question Task 1 What are the goals of the Bank’s Global Standards? Task 2 If you suspect an account is being used for money laundering, can you tell me one of the ways to escalate your concern? Task 3 Can you name three unusual activities that might indicate a customer is involved in money laundering? Task 4 A customer wants to send money to their relatives in Syria: are they allowed? Task 5 Following an Unusual Activity Report, a customer asks to see their financial crime risk rating. Can you informally tell them?
  38. 38. Writing up results Image courtesy of Shutterstock • Report written, with exec summary, methodology, test cases, results per question, overall results, outcomes and recommendations • PowerPoint of recommended next actions, with timelines and rough costs
  39. 39. Results into Actions Image courtesy of Shutterstock • Data used to build business case for $10m investment in a new intranet CMS • Data used to acquire new funding for UX review, Content Strategy work and new intranet governance • Repeat testing scheduled to show changes following updates.
  40. 40. Well, that’s a wrap!
  41. 41. Plan Plan for the smallest start Design for outcomes, not problems Image courtesy of Shutterstock
  42. 42. Run Start fast Expect headwinds as results appear Image courtesy of Shutterstock
  43. 43. Repeat, often Don’t overthink it Hold your ideas lightly Expect to retest monthly, quarterly, yearly Image courtesy of Shutterstock
  44. 44. Final things! PLAN: agree on what the goal is plan how to get there know what you’ve got to work with know how you measure ‘good’ REPEAT: keep moving on, like water RUN: brief producers fully keep track of what’s live own your errors, and successes 
  45. 45. @ k a t e _ k e n y o n •