CREATE USABLE AND ENJOYABLE PRODUCTS - Usability testing knowledge for agile cross functional teams

1. MY PORSCHE
CREATE USABLE AND ENJOYABLE PRODUCTS
Usability testing knowledge for
agile cross functional teams

2. Insights from 9 people are
not worth anything. You need
much more people to test.
URBAN LEGENDS AND QUESTIONS
WE WANT TO CLARIFY TODAY
Your results vary from our
usage data of Google
Analytics, can they be true?
The users you have tested are
not in the target group, the
results are not appropriate.

3. USABILITY VS.
USER EXPERIENCE
Often mixed up or used as synonym
these two concepts describe two
different disciplines of human
computer interaction.
They have something in common but
must be understood to master.

4. The extent to which a product can be used
by specific users in a specific usage
context to achieve specific objectives
effectively, efficiently and
satisfactorily.
USABILITY
DIN EN ISO 9241-11. (1998). Ergonomische Anforderungen für Bürotätigkeiten mit
Bildschirmgeräten Teil 11 : Anforderungen an die Gebrauchstauglichkeit. Berlin: Beuth Verlag.

5. A person with average (or even below average) ability and
experience understands how to use the thing [so it's
learnable] to achieve something without the effort being
greater than the benefit.
— Steve Krug
Don’t Make Me Think: Revisited. Web &
Mobile Usability. Das intuitive Web.
2014

6. USER EXPERIENCE
BEFORE USE
ANTICIPATED USE
Assumptions and
ideas about
the use.
AFTER USE
PROCESSING USE
Emotional attachment or
Distance from use;
Acceptance or reactance.
Usability
PERCEIVED USE
Effective, efficient and
satisfactory use;
free of interference.
DURING USE
Sarodnick, F. & Brau H. (2011). Methoden der Usability Evaluation: Wissenschaftliche Grundlagen und praktische Anwendung. Bern: Hogrefe AG

7. WHY DO WE TEST ?

8. If you want to have a great site you have to test it. If you've
worked on a site for even just a few weeks, you can't look at
it unbiased. You know too much. The only way to find out
if it really works is to test it.
— Steve Krug
Don’t Make Me Think: Revisited. Web &
Mobile Usability. Das intuitive Web.
2014

9. COGNITIVE EFFORT
DON’T MAKE ME THINK
If it is not obvious, the user struggles
and that is exhausting. Effort is hard
work that you can't enjoy in this case.
MUDDLE THROUGH
WE DON'T MAKE GOOD CHOICES
We usually have a goal. We only look
for the chunks that match our current
interest, the rest is irrelevant.
FULL UNDERSTANDING
IF ITWORKS THE USER STAYS
… As long as there's no better offer.
Understanding creates a sense of
control which makes users like to
come back.
SOME REASONS WHY WE NEED
TO INVEST IN GOOD USABILITY

10. THE MYTH OF THE
OTTO-NORMAL-USER
Whether developer or designer, once personal
and professional beliefs collide, one usually
tries to find arguments for what most users
like or need.
The basic thesis is that most users use the
web the way I do.

11. ALL WEB-USERS ARE UNIQUE, AND THE
ENTIRE USE OF THE WEB IS BASICALLY
ARBITRARY AND INDIVIDUALISTIC*
* Despite all individuality, good design takes into account the
complexity of the arbitrariness and individuality of the web user.

12. CONCLUSIONS OR ARGUMENTS LIKE WHAT
MOST PEOPLE LIKE OR DON'T LIKE ARE
COMPLETELY USELESS

13. TEST YOUR HYPOTHESIS

14. HOW TO TEST

15. Inductive Tests
Identification of
usability problems using
a prototype
Deductive Tests
Comparison of several
prototypes to evaluate
changes made and their
contribution
PERCEIVED USE
Effective, efficient and satisfactory use, free of interference.
DURING USE
Discount Usability Tests Broad survey of use
Performance Evaluation

16. Inductive Tests
Identification of
usability problems using
a prototype
Deductive Tests
Comparison of several
prototypes to evaluate
changes made and their
contribution
PERCEIVED USE
Effective, efficient and satisfactory use, free of interference.
DURING USE
Discount Usability Tests Broad survey of use
Performance Evaluation
GOOGLE
ANALYTICS
Google Analytics can show you that
and maybe where potential usability
problems exist but not which one.
It can help you to evaluate goals
and quality but not usability
problems.

17. The ideal number of participants for each Do-It-Yourself
test round is three [...] and I will tell you why.
— Steve Krug
Don’t Make Me Think: Revisited. Web &
Mobile Usability. Das intuitive Web.
2014

18. Identified
problems you
have resources
to fix
Identified problems by
three test persons

19. NIELSEN NORMAN GROUP
CASE STUDY FROM 83 USABILITY CONSULTING PROJECTS
NumberofUsabilityFindings
Number ofTest Users in NN Group Consulting Projects
5 3015
50
200

20. IT'S NOT ABOUT PROVING ANYTHING
To prove something, you have to do a
quantitative test. With a large sample
and a strict protocol.
The tests we need are qualitative tests
whose purpose is to uncover usability
problems. The result is comprehensible
insights, not evidence.
NOT ALL ERRORS NEED TO BE FOUND
It is possible to find more errors and
problems within a session with three
test persons than can be corrected
within a month.
It is important to identify and solve
only the most serious ones, and three
subjects are very likely to encounter
them.
WHAT IS OUR NEED

21. WHO IS THE BEST
FIT AS TEST
PERSON?

22. If your website requires special expertise such as an
exchange rate website for money market professionals, you
should have at least a few people with this knowledge.
— Steve Krug
Don’t Make Me Think: Revisited. Web &
Mobile Usability. Das intuitive Web.
2014

23. EVERYONE SHOULD BE ABLE TO
USE YOUR PRODUCT
A system is never used only by
domain experts.
BASICALLY, WE ARE ALL
BEGINNERS
Even experts only muddle their way
through, but on a higher level.
EVERYONE APPRECIATES
CLARITY
If anyone can use it, then an expert
has been able to use it for a long time.
REASONS WHY EVERYONE IS A
GOOD TEST PERSON

24. !
HOW TO GENERATE INSIGHTS

25. OBSERVATION
QUESTIONING COMMENTS
FOR DISCOUNT USABILITY TESTS
TRIANGULATION IS THE WEAPON OF CHOICE

26. TO VERIFY CHANGES STATISTICAL METHODS
AND GOOGLE ANALYTICS IS BEST

27. KEY TAKE AWAYS
To identify problems in
usability it is
sufficient to observe 3-5
test persons within the
scope of Discount
Usability Tests.
1
It is not necessary to test
only with domain experts,
good design bridges the
arbitrary use of the web
and should not require any
expertise.
2
To validate improvements,
Google Analytics is
excellent because we can
compensate statistical
measurement errors, yet it
needs valid data.
3

28. COPYRIGHT AND PROOF OF OWNERSHIP
Non Porsche Background Images: pexels.com made by respective author
Porsche Related Background Images: vmmedia.porsche.de made by respective author
Icons: flaticon.com made by respective author