We all know how much can be learned from a great user test. This talk will give you advice learned from years of not-so-great user tests about how to make the best out of the worst that user testing can throw at you.

1. Surviving the Train
Wreck
Advice for the worst user testing can throw at you
Andrew Hackman

4. But everything can go wrong

5. 1. Taking the Scenic Route – Incorrect knowledge or assumptions
Disasters

6. 1) Taking the
Scenic Route
/ˈtiːvoʊ/ tee-voh
/tɪˈvoʊ/ tai-voh

7. Enjoy the scenic
route
 DO NOT CORRECT YOUR USERS
 Don’t do it
 Stop it
 Unless the user being wrong is preventing you from continuing, let
it slide
 Learn from your users, don’t teach them

8. 1. Taking the Scenic Route – Incorrect knowledge or assumptions
2. Opening a cabin door at the wrong time – Users feel uncomfortable and stop
participating
Disasters

9. 2) Opening the
Cabin Door
Story Time Screenshot Goes Here

10. Knock loudly, lie
strategically
 Keep your users comfortable
 Respect the small, unexpected things that can make them
uncomfortable
 If you have to enter their personal space, knock loudly
 Be willing to bend over backwards to accommodate
 If all else fails, pretend nothing happened

11. 1. Taking the Scenic Route – Incorrect knowledge or assumptions
2. Opening a cabin door at the wrong time – Users feel uncomfortable and stop participating
3. Running out of soda – A logistical oversight or mistake
Disasters

12. Running out of
Soda

13. 3) Running Out
of Soda
 Make a checklist
 Do a dress rehearsal
 Be unnecessarily over prepared
 Be flexible and creative

15. But you can
survive it
 Taking the Scenic Route – As long as you going to the right place,
don’t tell the user they are going the wrong way
 Opening a cabin door at the wrong time – Strategically lie to avoid
letting the user’s know what happened; Knock loudly to avoid
incident in the first place
 Running out of soda – Make a checklist of what you need; run
through test at least once with an ‘outsider’
 Missed your stop - Plan the highest priority, simplest questions
ahead of time to ask if time has to be cut
 Shouting at the conductor – Give user a non-destructive way to
output opinion and feel listened to
 The train is a boat– Adapt your plan to fit what users are actually
doing with your stimuli; perform a test-test ahead of time to
double check veracity
 Train is on fire –Know when continuing is not providing any more
information and gracefully end it