Have fun and improve your work
Usability Testing

Dario Jazbec Hrvatin
● Web Developer
● In-development Tester
● Usability Tests Organizer for Toolset
plugins by OnTheGo Systems
● author, singer, guitarist of the band
Helika (helika.org)
pictures goes here

Whaaaa?
What is Usability Testing?
● A way to evaluate your work by testing it
on real users.
● Even simpler: watching how people use
the things you are creating.
Guess what - it’s fun!

Why do it
● Nothing is made perfect.
● It will help you improve your work.
● It will highlight problems early in
the development and let you fix
them at a much lower cost.
● It is easy - if I can do it, you can do it!

Things you can use it for
● themes
● plugins
● websites
http://wp-types.com/home/layouts/

What does a typical usability test look like?
It can look
something
like this.

What does a typical usability test look like?
Or maybe
something
like this.

What does a typical usability test look like?
Or even
something
like this.
Well…
Sorta.

Part 1
So, what do you actually need?

So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

● Google hangouts
● Skype
Communication software

Communication breakdown
● Stable and fast-enough
internet connection for:
○ you
○ testers
○ observers
● Are your users comfortable
using the chat?

So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

Who can be your tests leader?
● Knowledge of the subject
● Interest in usability testing
● Good listener
● Patient
● Kind
© Photo by Saša Huzjak (www.sasahuzjak.com)

So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

Who can be your testers?
● Beginners versus experts
● Coworkers?
● Family members?
● Anonymous people?

How and where to find your testers?
Invite people
● Your website/blog
● Forums
● Email / Newsletter
● Social Networks
http://wpml.org/2013/12/invitation-join-usability-testing-new-
drag-drop-layout-tool/

Meet potential testers
● Organize 5-minute video chats with
potential testers.
● Explain them how it will look like.
● Explain how you will compensate them
for their time.
Make sure final participants:
● are available on the testing day,
● meet your technical requirements,
● seem like good participants for this test.

So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

Should you do it alone? The role of Observers
● Observers - people who
○ watch the test
○ take notes
○ attend the debriefing after
the tests
● They need instructions as well

So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

Why testing scenarios are so important?
● testers need
tasks to execute
● testing will be
structured and
to the point

Example
Scenario step
Layouts plugin

So, what do you actually need?
1. Communication software
2. Test Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

Testing environment
● A ready to use testing site
● Accounts for your testers
● All required stuff working
fine and matching your
scenarios

So, what do you actually need?
1. Communication software
2. Tests Leader
3. Adequate testers
4. Observers
5. Testing scenarios
6. Testing environment
7. Follow-up doc for debriefing

Follow-up doc for debriefing
Prepare a doc for
a follow-up test
review

Part 2
How to run usability testing?

How to run usability testing?
1. Get ready in advance
2. Give introduction to the user
3. Let him/her do the tasks
4. Close the meeting
5. Run debriefing session

How to run a usability testing?
1. Get ready in advance
2. Give introduction to the user
3. Let him/her do the tasks
4. Close the meeting
5. Run debriefing session

Get ready in advance
Be well prepared
● Check that the testers are ready and
coming
● Test your connection and conference
software
● Make sure that the testing site is ready
● Make sure scenarios are ready
● Turn off things which can interrupt (cell
phones etc)

How to run a usability testing?
1. Get ready in advance
2. Give introduction to the user
3. Let him/her do the tasks
4. Close the meeting
5. Run debriefing session

Usability test - introduction
● Greet the tester.
● Describe shortly what this testing
is about.
● Explain that this is the test of your
product and not of the tester.
● Ask the tester to remember to
○ verbalize his thoughts and
actions,
○ be honest and blunt.

Get your testers relaxed
Ask some simple questions
● Where are they from?
● What’s their occupation?
● Their favourite themes and plugins.
Testers feeling good and
comfortable is very
important for the successful
usability testing.
?

How to run a usability testing?
1. Get ready in advance
2. Give introduction to the user
3. Let him/her do the tasks
4. Close the meeting
5. Run debriefing session

Doing tasks

Don’t forget your role
● Your job is to guide the testers
through the usability test.
● Make sure they are talking
aloud about what they’re
thinking and doing.
● Keep a friendly tone.
● Don’t help them in finishing
their tasks.

Handy phrases
● What are you thinking?
● What do you think?
● What are you doing?
● Did you expect this?
● What would you normally do?
● What are you trying to do now?

How to run a usability testing?
1. Get ready in advance
2. Give introduction to the user
3. Let him/her do the tasks
4. Close the meeting
5. Run debriefing session

“Everything that has a beginning has an end.”
Use the last 5 minutes for:
● asking any additional
questions about the test
● answering any questions
they might have
● thanking them

How to run a usability testing?
1. Get ready in advance
2. Give introduction to the user
3. Let him/her do the tasks
4. Close the meeting
5. Run debriefing session

We are not done yet. Debriefing
Debriefing - a meeting with
all of the observers where
you compare notes
You will need a leader
to keep the things focused

Debriefing: let’s compare our notes
● Ask observers one by
one to read the issues
they noticed
● Write the notes down in
a shared document
● Shortly discuss how
problems can be solved
● Rank problems by
severity
● Create and distribute
list of fixes to
implement

Part 3
What we learned

How usability testing helped us. Examples
before after

How usability testing helped us. Examples
before after

How usability testing helped us. Examples
before after

How usability testing helped us. Examples

“Where do we go from here?”
● Usability testing is easy, cheap and effective.
● You will learn a lot about your own products.
● You can make friends and meet people.
● You will have fun.
● You will improve your work.
My advice: Go for it!

Further reading
● “Rocket Surgery Made Easy”, Steve Krug
● “A Practical Guide to Usability Testing”,
Joseph S. Dumas and Janice G. Redish
● The web…
● http://wp-types.com/home/layouts/ - the
Layouts plugin refined in usability testing
● This presentation will be shared and
linked in our Toolset blog at wp-types.
com

What we learned
“Don't cheat the process. If the
evidence shows there is a problem, fix
it. Don't make excuses.”
“Take nothing for granted. Users have
unexpected workflow. What you think
is a logical order of doing things is not
always going to be how users want to
do things.“
Amir Helzer
Project Manager
Bruce Pearson
Layouts Plugin Lead Developer
“You may find out that your perfectly
crafted interface is not so intuitive.
And that fancy border is not a drag
handle. It is just a fancy border”.
Przemek Olaf Surm
Layouts Plugin Front-end Developer

What we learned
“What was completely clear to us may have been
completely hidden to the user and viceversa.
It’s like when you grow up your kid and you see him / her
everyday and you’re losing the perception he’s growing,
while others that see him only every once in a while tell
him at a glance: “you grew up a lot since last time”.
Riccardo Strobbia
Layouts Plugin Developer and Javascript Ninja

Have a question? Fire away!
?