The Latest User Testing Techniques for All Budgets


Published on

We are often asked to outline the benefits of certain user research, engagement and testing techniques, not only from a project cost and schedule perspective but also because some people feel too much user involvement can derail plans, change specifications and lead to competing views or negative feedback.

In an update to last year's talk, Justin will look at simple, cost-effective and enjoyable techniques used to engage users. He will show examples of tools and techniques Sigma has employed for our most recent projects ranging in size, complexity and budget and present new insights particularly with regard to analysing your findings.


This presentation is from Camp Digital, a free, one-day event in Manchester exploring some of the most important and emerging themes in the digital industry. To see more presentations and videos from the day visit:

Camp Digital was brought to you by Sigma. We are a leading specialist in User Experience consulting and design, information management, and web technologies. We provide research, analysis, design, development and support services, with specific expertise in usability, accessibility, content and document management, websites, intranets and online applications.

If you think we can help you call us on 01625 427718 or email

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The Latest User Testing Techniques for All Budgets

  1. 1. ONE SIZE DOES NOT FIT ALL The latest user testing techniques for all budgetsJustin DarleyHead of Training andContent ServicesSigma UK #campdigital@just_ux
  2. 2. What is User Experience (UX)? “a person’s perceptions and responses that result fromthe use or anticipated use ofa product, system or service” — ISO definition
  3. 3. Why does UX matter? People need to see value in a site/app if you want them to use it A negative experience is lasting and hard to shake off Sites/apps that are hard to use are ignored You don’t know your users as well as you think you doYou can help to ensure a positive UX through: Research Engagement Testing
  4. 4. Why is UX often avoided? Expensive Time Complex consuming - eats into - delays - too many budget development cooksBUT there are a range of techniques you can use, not all of whichare costly or difficult and it pays off down the line…
  5. 5. Different approaches fit different projects… UX techniques – Online surveys – Card sorting – Tree testing – Rapid prototyping and Joint Application Development (JAD) – Remote, guerrilla and lab-based usability testing 3 very different customers – Citizens Advice – InterContinental Hotels – University of Brighton
  6. 6. Citizens Advice service extranet Card sorting Prototyping Guerrilla usability testing
  7. 7. AdviserNet
  8. 8. Citizens Advice AdviserNet Tree testing
  9. 9. University of Brighton Online survey Telephone interviews Card sorting Tree testing Rapid prototyping Guerrilla usability testing Remote usability testing Lab-based usability testing
  10. 10. IHG Brand Central
  11. 11. IHG Brand Central site Face-to-face workshops Telephone interviews Online survey Lab-based and remote usability testing
  12. 12. Online surveys
  13. 13. Online surveys Tools: Survey Monkey (free 10 questions, 100 responses / other plans) Good for: – Large datasets – Quantitative feedback Tips: – Clear introductory email – Maximum of 15 minutes to complete – Aim for 80-100 responses (response rates vary) – Progress bar – Multiple choice questions – Good tips on the blog ( – Use the data collected to feed into user profiles
  14. 14. Card sorting
  15. 15. Card sorting Tools: Spread sheets (free from – and XMind (free) Good for: – Insight into users’ mental categorisation of your site – Anecdotal general insight into content and user journeys Tips: – Max 4-5 people per card sort group – Fewer than 100 cards (if you can) – Give users an example first – pickled onions in a supermarket works well – Take lots of envelopes, lots of rubber bands and spare cards – Listen to what people say and ask questions
  17. 17. Tree testing
  18. 18. Tree testing Tools: Treejack ($109 per month or $990 per year) and XMind (free) Good for: – Testing navigational hierarchies Tips: – Be careful with task design – At least 30 users – Using representative, motivated users cuts down on “garbage” – Max 12 tasks – ALWAYS run a pilot – Try to arrange one or two moderated sessions – Prune “garbage” sessions – Come to a Sigma course 
  19. 19. Tree testing – results
  20. 20. Tips for analysis – the spread sheet
  21. 21. Tips for analysis “Visited first” statistics Pruning "garbage" results "Evil attractors"
  22. 22. Rapid prototyping and JAD
  23. 23. Rapid prototyping and JAD Tools: Axure (free trial / $589) Good for: – User centred User Interface (UI) design – Highly visual – Rapid feedback – Requirements gathering – Provides a prototype for usability testing Tips: – Minimum of 2 people to run the session – User, stakeholder and technical representatives – Maximum of 10 participants – Keep it as informal as possible – Break people into small groups – Make good use of post-its and whiteboards
  24. 24. Usability testing
  25. 25. Guerrilla usability testing
  26. 26. Guerrilla usability testing Tools: Silverback (free trial / $70) and Freemind (free) Good for: – Low cost usability testing – Quantitative feedback Tips: – Introductory email – Any users are better than none but aim for two rounds: 10 per round – Aim for a good cross-section – Max 30-45 minutes to complete (around 5 tasks) – Create meaningful tasks – word tasks as instructions – Draft in a scribe – Think aloud and retrospective think aloud – Discussion guide – consistency is important
  27. 27. Moderated remote usability testing Tools: Adobe Connect (free trial / $55 month) Good for: – Geographically distributed users – Qualitative feedback Tips: – As for guerrilla, plus: – Create a set up guide – Try to keep sessions a bit shorter – Have several dry runs! – Use Adobe to record sound and visual
  28. 28. Unmoderated remote usability testing Tools: Loop11 Good for: – Task based testing on existing sites – Collecting quantitative data from many users (Email invite. No plugins for users) – Identifying navigation paths across live sites with multiple routes to information – Desktop/Tablet/Mobile sites Notes: – Can work for testing prototypes, but they will need to be highly developed (Real content, only a few null links) – Canned reports are good, but often require manual analysis as well – Tasks need to be clear concise and have a defined end goal (must be achievable without guidance)
  29. 29. Lab-based usability testing Lab provisioned technology – Tech quality can vary between labs (always visit before selecting a suite) Good for: – Geographically centred users • Users tend to prefer labs, a more relaxed and friendly environment (guerrilla testing is often in a small corner office on client site) – The lab staff do the hard work, you can just observe and learn – The observation room is extremely good at engaging stakeholders and product teams. • Allows stakeholders to see how users really use their site and improves their awareness and empathy towards issues • Much more effective than watching videos
  30. 30. Text expander
  31. 31. Accessibility testing – an aside Checklists – – Listen to your sites on screen readers - (try turning your monitor off) Automated testers like Expert reviews – RNIB – See it right – Abilitynet – Independent evaluators
  32. 32. Conclusions Don’t underestimate the power of UX! There are a range of methods you can use Doesn’t have to be expensive, time consuming or complex Can be a great way to get user buy in Satisfied users means a successful project
  33. 33. ResourcesTo recap… Also of interest… SurveyMonkey (online surveys) AbilityNet (accessibility testing) Axure (rapid prototyping) EpiServer (multivariate testing) Silverback (guerrilla testing) Google Optimiser (conversion rates) Freemind (mind-mapping) Adobe Connect (remote testing) Card sorting (shaping navigation) Treejack (tree testing navigation) Phrase Express (text expander) Loop11 (unmoderated remote testing)
  34. 34. THANK YOU!ANY QUESTIONS? @WeAreSigma #campdigital
  35. 35. UX Training at SigmaSigma will be launching a range of new User Experiencetraining services over the next few months, includingcourses in: Rapid prototyping and solution design with Axure Writing for the web Intro to UX – principles, tools and techniques Tree testing with Treejack