Usability testing - why to do, how to do, who to invite. Examples how usability testing can improve your prodcuts. Lessons learned.

1. Have fun and improve your work
Usability Testing

2. Dario Jazbec Hrvatin
● Web Developer
● In-development Tester
● Usability Tests Organizer for Toolset
plugins by OnTheGo Systems
● author, singer, guitarist of the band
Helika (helika.org)
pictures goes here

3. Whaaaa?
What is Usability Testing?
● A way to evaluate your work by testing it
on real users.
● Even simpler: watching how people use
the things you are creating.
Guess what - it’s fun!

4. Why do it
● Nothing is made perfect.
● It will help you improve your work.
● It will highlight problems early in
the development and let you fix
them at a much lower cost.
● It is easy - if I can do it, you can do it!

5. Things you can use it for
● themes
● plugins
● websites
http://wp-types.com/home/layouts/

6. What does a typical usability test look like?
It can look
something
like this.

7. What does a typical usability test look like?
Or maybe
something
like this.

8. What does a typical usability test look like?
Or even
something
like this.
Well…
Sorta.

9. Part 1
So, what do you actually need?

10. So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

11. So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

12. ● Google hangouts
● Skype
Communication software

13. Communication breakdown
● Stable and fast-enough
internet connection for:
○ you
○ testers
○ observers
● Are your users comfortable
using the chat?

14. So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

15. Who can be your tests leader?
● Knowledge of the subject
● Interest in usability testing
● Good listener
● Patient
● Kind
© Photo by Saša Huzjak (www.sasahuzjak.com)

16. So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

17. Who can be your testers?
● Beginners versus experts
● Coworkers?
● Family members?
● Anonymous people?

18. How and where to find your testers?
Invite people
● Your website/blog
● Forums
● Email / Newsletter
● Social Networks
http://wpml.org/2013/12/invitation-join-usability-testing-new-
drag-drop-layout-tool/

19. Meet potential testers
● Organize 5-minute video chats with
potential testers.
● Explain them how it will look like.
● Explain how you will compensate them
for their time.
Make sure final participants:
● are available on the testing day,
● meet your technical requirements,
● seem like good participants for this test.

20. So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

21. Should you do it alone? The role of Observers
● Observers - people who
○ watch the test
○ take notes
○ attend the debriefing after
the tests
● They need instructions as well

22. So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

23. Why testing scenarios are so important?
● testers need
tasks to execute
● testing will be
structured and
to the point

24. Example
Scenario step
Layouts plugin

25. So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

26. Testing environment
● A ready to use testing site
● Accounts for your testers
● All required stuff working
fine and matching your
scenarios

27. So, what do you actually need?
1. Communication software
2. Tests Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

28. Follow-up doc for debriefing
Prepare a doc for
a follow-up test
review

29. Part 2
How to run usability testing?

30. How to run usability testing?
1. Get ready in advance
2. Give introduction to the user
3. Let him/her do the tasks
4. Close the meeting
5. Run debriefing session

31. How to run a usability testing?
1. Get ready in advance
2. Give introduction to the user
3. Let him/her do the tasks
4. Close the meeting
5. Run debriefing session

32. Get ready in advance
Be well prepared
● Check that the testers are ready and
coming
● Test your connection and conference
software
● Make sure that the testing site is ready
● Make sure scenarios are ready
● Turn off things which can interrupt (cell
phones etc)

33. How to run a usability testing?
1. Get ready in advance
2. Give introduction to the user
3. Let him/her do the tasks
4. Close the meeting
5. Run debriefing session

34. Usability test - introduction
● Greet the tester.
● Describe shortly what this testing
is about.
● Explain that this is the test of your
product and not of the tester.
● Ask the tester to remember to
○ verbalize his thoughts and
actions,
○ be honest and blunt.

35. Get your testers relaxed
Ask some simple questions
● Where are they from?
● What’s their occupation?
● Their favourite themes and plugins.
Testers feeling good and
comfortable is very
important for the successful
usability testing.
?

36. How to run a usability testing?
1. Get ready in advance
2. Give introduction to the user
3. Let him/her do the tasks
4. Close the meeting
5. Run debriefing session

37. Doing tasks

38. Don’t forget your role
● Your job is to guide the testers
through the usability test.
● Make sure they are talking
aloud about what they’re
thinking and doing.
● Keep a friendly tone.
● Don’t help them in finishing
their tasks.

39. Handy phrases
● What are you thinking?
● What do you think?
● What are you doing?
● Did you expect this?
● What would you normally do?
● What are you trying to do now?

40. How to run a usability testing?
1. Get ready in advance
2. Give introduction to the user
3. Let him/her do the tasks
4. Close the meeting
5. Run debriefing session

41. “Everything that has a beginning has an end.”
Use the last 5 minutes for:
● asking any additional
questions about the test
● answering any questions
they might have
● thanking them

42. How to run a usability testing?
1. Get ready in advance
2. Give introduction to the user
3. Let him/her do the tasks
4. Close the meeting
5. Run debriefing session

43. We are not done yet. Debriefing
Debriefing - a meeting with
all of the observers where
you compare notes
You will need a leader
to keep the things focused

44. Debriefing: let’s compare our notes
● Ask observers one by
one to read the issues
they noticed
● Write the notes down in
a shared document
● Shortly discuss how
problems can be solved
● Rank problems by
severity
● Create and distribute
list of fixes to
implement

45. Part 3
What we learned

46. How usability testing helped us. Examples
before after

47. How usability testing helped us. Examples
before after

48. How usability testing helped us. Examples
before after

49. How usability testing helped us. Examples

50. “Where do we go from here?”
● Usability testing is easy, cheap and effective.
● You will learn a lot about your own products.
● You can make friends and meet people.
● You will have fun.
● You will improve your work.
My advice: Go for it!

51. Further reading
● “Rocket Surgery Made Easy”, Steve Krug
● “A Practical Guide to Usability Testing”,
Joseph S. Dumas and Janice G. Redish
● The web…
● http://wp-types.com/home/layouts/ - the
Layouts plugin refined in usability testing
● This presentation will be shared and
linked in our Toolset blog at wp-types.
com

52. What we learned
“Don't cheat the process. If the
evidence shows there is a problem, fix
it. Don't make excuses.”
“Take nothing for granted. Users have
unexpected workflow. What you think
is a logical order of doing things is not
always going to be how users want to
do things.“
Amir Helzer
Project Manager
Bruce Pearson
Layouts Plugin Lead Developer
“You may find out that your perfectly
crafted interface is not so intuitive.
And that fancy border is not a drag
handle. It is just a fancy border”.
Przemek Olaf Surm
Layouts Plugin Front-end Developer

53. What we learned
“What was completely clear to us may have been
completely hidden to the user and viceversa.
It’s like when you grow up your kid and you see him / her
everyday and you’re losing the perception he’s growing,
while others that see him only every once in a while tell
him at a glance: “you grew up a lot since last time”.
Riccardo Strobbia
Layouts Plugin Developer and Javascript Ninja

54. Have a question? Fire away!
?