Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Collaborative usability test reviews UX Scotland 2019

243 views

Published on

How to execute a collaborative review of usability testing to facilitate design and development prioritisation consensus. Case studies of how the approach has worked at the University of Edinburgh. Presented at UX Scotland conference 2019

Published in: Design
  • Be the first to comment

  • Be the first to like this

Collaborative usability test reviews UX Scotland 2019

  1. 1. Shared research experiences RAPID REPORTING AND SHOWCASING OF USABILITY TESTING 1NEIL ALLISON @USABILITYED #UXSCOT
  2. 2. Hi I’m Neil… UX SERVICE MANAGER AT THE UNIVERSITY OF EDINBURGH EX-CONTENT MANAGEMENT SYSTEM SERVICE MANAGER SERVICE DESIGNING BEFORE I KNEW IT WAS A THING NEIL ALLISON @USABILITYED #UXSCOT 2
  3. 3. Hands up… If you’ve ever conducted a usability test If you’ve ever watched a usability test If all this is totally new to you 3NEIL ALLISON @USABILITYED #UXSCOT
  4. 4. Usability testing IT’S EASY, RIGHT? NEIL ALLISON @USABILITYED #UXSCOT 4
  5. 5. What can be challenging Getting the go ahead to use your time on usability testing Getting colleagues to take on board what you uncover Getting fixes to problems implemented NEIL ALLISON @USABILITYED #UXSCOT 5
  6. 6. These are common challenges… NEIL ALLISON @USABILITYED #UXSCOT 6
  7. 7. Standing on the shoulders of… Caroline Jarrett and Steve Krug surveyed the UX community in 2012. They blogged about why usability problems go unfixed: http://bit.ly/krug-jarrett-unfixed NEIL ALLISON @USABILITYED #UXSCOT 7
  8. 8. A story about change MY CHALLENGE IN 2013… NEIL ALLISON @USABILITYED #UXSCOT 8
  9. 9. Moving out of our comfort zone… I was the University’s CMS Service Manager, responsible for user support and training, community engagement, user research and requirements validation, co-design facilitation… We had a vendor-supplied CMS that everyone hated, with a service around it that people quite liked. Features were introduced in waterfall projects. We had a small working group delivering that was stable, working in a user-centred fashion. And we trusted each other. Then in 2012 it was announced we were switching CMS, going open source and developing our own system… NEIL ALLISON @USABILITYED #UXSCOT 9
  10. 10. Changing times, new challenges New people, new methodology, new urgency… A development team of 3 plus a business analyst, working agile to deliver a new CMS and transition the service in 2 years. We didn’t have a culture of user-centred design. The project team needed to be won over. The CMS user community wanted reassurance. Old ways of working were not going to fit this new situation… NEIL ALLISON @USABILITYED #UXSCOT 10
  11. 11. Steve Krug to the rescue Free to use observation session materials Everyone listing top 3 issues for each test participant Bringing the development team closer to the CMS user group that the service team knew so well 11NEIL ALLISON @USABILITYED #UXSCOT
  12. 12. Good, but not great (sorry Steve) Testing every 4 weeks. Live observations, the CMS user group keen to get involved. Team (broadly) happy to dedicate time. But after the tests, discussion meandered. People hung on to pet issues. Tech solution chat dominated. We would run out of time, no closure. NEIL ALLISON @USABILITYED #UXSCOT 12
  13. 13. Plus we wanted to go public We wanted to bring in the user community as contributors, not just as test participants The post-test conversations were meandering with 6-8 people. What would happen with 40? NEIL ALLISON @USABILITYED #UXSCOT 13
  14. 14. Enter David Travis “RED ROUTE” USABILITY TO THE RESCUE NEIL ALLISON @USABILITYED #UXSCOT 14
  15. 15. An objective prioritisation flowchart Three questions to grade every usability issue Post observation prioritisation complete in 30-40 minutes Scalable: works the same with 30+ people NEIL ALLISON @USABILITYED #UXSCOT 15
  16. 16. The process HOW TO DO IT YOURSELF NEIL ALLISON @USABILITYED #UXSCOT 16
  17. 17. Step-by-step 1. Get the right people in a room 2. Watch a small number of short sessions with users doing something 3. Prioritise the issues they see 4. Collaboratively consolidate their priority lists 5. Agree actions for usability issues 6. Repeat every few weeks NEIL ALLISON @USABILITYED #UXSCOT 17
  18. 18. Who are the right people? Everyone with a stake in the service If you’re making design decisions you should be there “Have you had your recommended dose of research?” http://bit.ly/gds-research-dose 18NEIL ALLISON @USABILITYED #UXSCOT
  19. 19. What do we watch? Real users doing real tasks Facilitated usability testing sessions Agree the focus of testing within the team “Research shows that teams make better services when everyone on a project team observes users first hand.” Jared Spool http://bit.ly/spool-exposure 19NEIL ALLISON @USABILITYED #UXSCOT
  20. 20. How many do we watch? As many as you can fit into the time you have Example: ◦ Intro – 10 min ◦ Test session 1 + 5 minutes ◦ Test session 2 + 5 minutes ◦ Test session 3 + 5 minutes ◦ Issue prioritisation – 40 minutes “The most striking truth of the curve is that zero users give zero insights.” Jakob Nielsen http://bit.ly/nielsen-5-users 20NEIL ALLISON @USABILITYED #UXSCOT
  21. 21. How do we prioritise? Rocket Surgery template: ◦ Individual notes while observing ◦ Distil to 3 issues after each participant “Running a usability test has been compared with taking a drink from a fire hydrant…” David Travis 21NEIL ALLISON @USABILITYED #UXSCOT http://bit.ly/usability-prioritise-travis
  22. 22. How do we consolidate? “If you prioritise usability problems using 'gut feel' or intuition, you run the risk of being exposed as a fraud…” David Travis http://bit.ly/usability-prioritise-travis 22NEIL ALLISON @USABILITYED #UXSCOT
  23. 23. NEIL ALLISON @USABILITYED #UXSCOT 23 xxx
  24. 24. Taking action on the insight This matrix is a great way to separate issues from ‘solutions’ You might put the same issue on the matrix more than once, if multiple solutions exist Agree the fastest, cheapest way to validate your solution Minor issue for users Major issue for users Easy solution available No easy solution available 24NEIL ALLISON @USABILITYED #UXSCOT
  25. 25. Top tips Testing focus ◦ Agree priority tasks together ◦ Trial test with the team ◦ No surprises! Participants ◦ Recordings are far less stressful than live review sessions ◦ Krug – “Recruit loosely, grade on a curve” Do whatever it takes to get observers in the room ◦ Start over lunch break ◦ Supply refreshments ◦ Bribery, favours, threats… Be well organised so you don’t waste the observer group’s time ◦ Test everything before hand ◦ Provide a schedule in advance Stick to the process ◦ It’s easy to digress when you’ve all seen so much Encourage collective reflection on the session ◦ Admitting usefulness is first step to getting observers to turn up next time ◦ Have the next one scheduled ASAP NEIL ALLISON @USABILITYED #UXSCOT 25
  26. 26. Everything you need Steve Krug’s Rocket Surgery resources http://bit.ly/rocket-surgery-krug David Travis’ Red Route prioritisation flowchart http://bit.ly/usability-prioritise-travis Digitise with Post-it Plus app: http://bit.ly/post-it-plus 26NEIL ALLISON @USABILITYED #UXSCOT
  27. 27. What happened next A TOOL FOR CULTURE CHANGE & ADOPTION NEIL ALLISON @USABILITYED #UXSCOT 27
  28. 28. Usability testing showcases I approached a number of business-critical services and offered usability testing for free on the condition that we reviewed as a collaborative event. This made the case for a pilot usability testing service which then grew into a pilot UX consultancy service. These ‘freebies’ have developed into longer-term funded initiatives. NEIL ALLISON @USABILITYED #UXSCOT 28
  29. 29. Learn VLE service enhancements Collaborating with schools on a rolling basis to expose teaching staff and learning technologists to student experiences. Making the case for greater consistency and developing guidelines in the open. Exposing the institution to the range of approaches in existence, and the impact on the student. http://bit.ly/VLE-UX NEIL ALLISON @USABILITYED #UXSCOT 29
  30. 30. IT self help iterative improvements Bringing frontline and second line support teams together around the highest volume support call themes. Creating a cycle of testing, analytics and content development. http://bit.ly/IT-self-service NEIL ALLISON @USABILITYED #UXSCOT 30
  31. 31. In conclusion Involve the team at every stage if you want usability testing to have impact ◦ From setting a focus through to review and prioritisation Work to establish a virtuous cycle that others can adopt ◦ Focus on inclusion and empowerment ◦ Little of this is “specialist” – it’s everyone’s responsibility ◦ This only works if you keep it going Few read your reports and fewer believe what you write ◦ Give them real exposure to users ◦ Channel the group to write their own “report” Go as public as you can get away with ◦ The greater the exposure, the bigger the organisational ripples NEIL ALLISON @USABILITYED #UXSCOT 31
  32. 32. Thank you QUESTIONS? N E I L A L L I S O N U X M A N A G E R , U N I V E R S I T Y O F E D I N B U R G H @ U S A B I L I T Y E D NEIL ALLISON @USABILITYED #UXSCOT 32

×