Workshop @UXCamp Amsterdam 2016 about User Checks, a method for agile usability testing Learning by doing a user check in 45 minutes...

1. userneeds! @anous
User Checks!
involve users always
and get to a better design fast
September 10, 2016"
Workshop UXCAMP AMSTERDAM 2016
"
Anouschka Scholten
userneeds

2. userneeds! @anous

3. userneeds! @anous
person
needs
goal
Image: source http://thenextweb.com/dd/2014/05/27/context-design-seize-magic-moment/

4. userneeds! @anous
Check with
users?!

5. userneeds! @anous

6. userneeds! @anous

7. userneeds! @anous

8. userneeds! @anous
UX insights!

9. userneeds! @anous
Quantitative methods
What happens
Where
How often
WHY
How better
?

10. userneeds! @anous

11. userneeds! @anous
Make people’s
lives better.

12. userneeds! @anous
You can’t possibly predict/find/be aware of all the issues

13. userneeds! @anous
Focus
group
Enquête
Verify
test
Remote
tracking
Usability
testing
User
driven
Marketing
driven

14. userneeds! @anous
People do not always do what !
you think !
you tell them !
they think !
they say !
they do!
!
!
Observing and asking why makes you
find out what people really !
do and need!

15. userneeds! @anous
Usability testing
is about
observing behaviour
Watch the user do what comes naturally,
while performing a task (thinking aloud).
Don't help.
Not asking about opinion.

16. userneeds! @anous
empathy!
a key function of usability testing
to really see, hear and find out
what moves your users
the ability to see an experience
through another person’s eyes
Getting to

17. userneeds! @anous
WHY??!!!
It’s not about
how often something is said or done
it’s no quantitative research, it’s not about significance
It’s about
why things happen or are said
What’s behind the behavior? Did the user expect something else, what? Is
there something in the context which is new and we have to consider? Is
there something really important, much more than we thought, an
opportunity? Or a painpoint we didn’t meet exactly?
1

18. userneeds! @anous
Usability test to!
Detect problems!
Optimize!
Innovate!
Engage!
Convice!
Focus!
!
!

19. userneeds! @anous
Usability ROI
• Increased conversion rates
• Increased productivity
• Reduced need for help desk
intervention
• Reduced fall off rate at checkout
• Reduce waste, focus on what’s needed

20. userneeds! @anous
Meeting user needs is key
UX – U = X
UX without users is a no go

21. userneeds! @anous
Picture credits: Johan Groenen http://www.jgroenen.nl/
NOT the waterfall way

22. userneeds! @anous
Picture credits: Johan Groenen http://www.jgroenen.nl/
The agile way!

23. userneeds! @anous
Improve the UX ongoing
Validated learning loop for innovation, solving problems and optimization
The Lean Startup process,
source: http://theleanstartup.com/principles

24. userneeds! @anous
user checks!
& traditional
usability testing!

25. userneeds! @anous
User Checks –
agile usability testing

26. userneeds! @anous
26

27. userneeds! @anous
Usability Test traditional
1 round of testing"
In a lab environment, eye tracking
Moderator is a UX researcher
Process based

28. userneeds! @anous
Traditional Usablity test
drawbacks
• 1 time, at end design process
• An extensive report, little or no redesign
• No retest
• Not in user context
• High costs: lab, eye tracking, special UX
researcher
• Product team indirectly involved
• Schedule well in advance (at least 1 month)

29. userneeds! @anous
http://uxpajournal.org/wp-content/uploads/pdf/JUS_Romano_Bergstrom_November_2011.pdf

30. userneeds! @anous
Test and retest within 1-4 weeks
Preferably in the users context, quick and easy set up
Team member (Designer, PO,…) conducts the test, team
directly involved
Output is a better design/ better MVP"
Value based
User Checks Agile Usability Test

31. userneeds! @anous
User Checks Agile Usability Testing
6. Evaluate
Priortize
Improvements
2. User
Checks
round 1
3. Evaluate
Priortize
Improvements
5. User
Checks
round 2
1. Design /
MVP
4. Improved
Design / MVP
7. Improved
Design /
MVP
Repeat
or develop
Based on the RITE-model: https://uxmag.com/articles/the-rite-way-to-prototype

32. userneeds! @anous
“Elaborate tests are a waste of resources”
Source: http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
75% of the problems found with 3 or 4 user tests

33. userneeds! @anous
Outsourcing your user research work
is like outsourcing your vacation.
It gets the job done, but probably
won’t have the effects you were
seeking.
Jared Spool
http://www.uie.com/brainsparks/2011/08/02/outsourcing-your-user-research-is-like-outsourcing-your-vacation/
“

34. userneeds! @anous
A better design
fast!

35. userneeds! @anous
With User Checks
• Checking with users on a regular bases
Effective problem detection
• Ensures that users can successfully reach their goals and shows where
their satisfaction can be improved
Boosts optimalization
• Offers insights about the proposition or concept weaknesses and
unknown opportunities
Enables innovation
• Ensures the whole team understands how users use their product
Empathy and commitment
• Emphasizes where a design change is needed, reduces waste
Brings focus, avoids feature creep

36. userneeds! @anous
Let’s just do it!

37. userneeds! @anous
Live user check test session !
additional participants in role ‘observer’!
!
!

38. userneeds! @anous
Collect en prioritize findings!
learing by doing!

39. userneeds! @anous
Come up with solutions and
prioritize!
learing by doing!
!

40. userneeds! @anous

41. userneeds! @anous

42. userneeds! @anous
The 4 basics of a User Check
1. Do a short interview upfront
Ask for experiences; how did they… the last time. Let the
user formulate the goal they have, the situation, the task.
Use that for the user check
2. Let your user experience the prototype/
MVP/site
Observe don’t tell. Put your prototype/site in the user’s
hands (or your user in the prototype) and give just the
minimum context so they understand what to do.
Give them the situation and task they formulated in the
interview

43. userneeds! @anous
3. Have them talk through their experience
(talk aloud) & Observe actively
Watch how they use (and misuse) what you have given them.
Never immediately “correct” what your user is doing.
Only stimulate to talk aloud: “what are you thinking”
4. Evaluate afterwords: follow up with
questions
This is important; often this is the most valuable part of testing.
You mentioned… what did you mean with that? I saw you
doing… why? What did you expect? How would you call it?
Answer questions with questions (i.e “well, what do you think
that button does”)
The 4 basics of a User Check

44. userneeds! @anous
I need 1 volunteer:
the user

45. userneeds! @anous
Case uxcampams.com
• The main goal(s) of the site
Promote the concept, get people interested to apply and be
part of the event
• Target users
UX experts and related expertises to join as attendees,
speakers or volunteers
• Main tasks, main questions of the users
For Speakers: how does it work, what level of experience is
expected, who attends/speaks, what can I expect, do I fit in?
• Context of use
Re-visiting the site after buying a ticket, considering to speak
and find out if this is doable

46. userneeds! @anous
Observe the user’s
behaviour
1 sticky per issue/observation
Note every occasion the user:
• hesitates, worries, or is forced to think
• misunderstands something
• gets frustrated or annoyed
• gives up
• is surprised

47. userneeds! @anous

48. userneeds! @anous
The findings, issues
• Collect Findings
• Prioritize issues to be
solved: prioritize from the
business/user goals
perspective, rate them
Do this with thewhole team

49. userneeds! @anous
• Collect all findings
• Re-arrange, categorize
them, label them
• Dot-vote (3 per person):
what is most important,
what should we tackle
The problem dot-voted
most: concentrate on that
one for the solutions…

50. userneeds! @anous
Improvements
• Improvements on every level, note down
Labeling, content, categorization, visual assets,
interaction, concept, flow, persuasion > but keep the
business/user goals perspective in mind (focus)!
• Prioritize by Impact & Effort (matrix) or dot-
vote
Choose improvement(s) to be re-tested
Do this with thewhole team

51. userneeds! @anous
Impact & Effort matrix
Impact: creates high/low value for users
effort: high/low effort in terms of time, knowledge, tools, people…
Source: http://www.innovationgames.com/impact-effort-matrix/
Do this with the
whole team
incl. decision
makers!

52. userneeds! @anous
• Implement the solutions
that have the most positive
impact/value for the user
and take the least effort to
realize
And re-test!

53. userneeds! @anous
A better design
fast!!

54. userneeds! @anous
Lets make people’s
lives better!
!
Just do it!!

55. userneeds! @anous
Thank you!
anouschka.scholten@userneeds.nl
www.userneeds.nl
Twitter: @anous
LinkedIn: https://nl.linkedin.com/in/anouschkascholten

56. userneeds! @anous
Extra’s

57. userneeds! @anous
User Checks set up
• Test script with real tasks
• Real target users as participants
• Ideally conducted in context user on their own
device
• Record audio and screen capture
• 1 test moderator, 1 observer
(more observers are possible, but not too much
and really on the background)

58. userneeds! @anous
Key: moderating the test
If they…
Say…
Are not talking
“What are you thinking?”
Ask you a question (e.g. “Is that
what I should do here?”)
Rephrase the question (e.g “ What do
you think you should do?”)
Get a task right or wrong
“Thank you , that is very helpful”
“Thanks for the feedback”
Mess up
“Remember, you can’t make any
mistakes. You’re doing a great job.”
Are unsure if they have
completed a task and ask you
“Is this what you would do if you were
doing X at home?”
Criticize the design
“Thanks for the feedback”

59. userneeds! @anous
Test script-agenda
1. Introduction and questions up front
2. Situation sketch + think aloud instruction
3. Think aloud session
4. Evaluate session, open questions
5. End session - thank participant

60. userneeds! @anous
1. Example Introduction
Thank you for participating, we really appreciate it!
I’m <name> and there is one person besides me <name> is also here.
We’re helping a company to develop a new service / to optimize a website
This company cares about their customers and thinks it’s very important
to involve potential customers when developing new services. They want
the new service to be valuable for you.
That’s why we want to ask you some questions and along the way we’d
like show you the service/site. You can walk through it.
The interview takes about a 20 minutes. If you do not understand
something, or you want to add something, do not hesitate to interrupt me!
Do you mind if we record the session for future reference within our
organisation? Everything we discuss and record is confidential; we will not
share things without your permission!
Any questions?

61. userneeds! @anous
Task-think aloud instruction
You are going to see: a draft proposal of the service, which has
not yet been developed/ the site now.
<if a proptotype> The proposal is only images. It looks finished
and you can click through it, but most of it is not working yet.
It’s also not as fast as a normal app and you can’t do anything
wrong.
<Situation sketch and task>
Please open the link on your mobile phone and walk through
the app.
Can you please say things aloud? Not to me, but for
yourself? Everything you do or say is fine, eg. Also "what the
hell is this ?! “.
You can start now by clicking the link

62. userneeds! @anous
Evaluation questions
After the think aloud session, zoom in to what you observed, like:
• You mentioned… what did you mean with that?
• I saw you doing… why? What did you expect? How would you call
it?
• You didn’t go to … why not? What do you expect to find?
Also ask service, product or content specific questions, e.g.:
• You read this text thoroughly, why?
• You were going back and fourth between product X and Y, why?
• You choose X and said something about .., what did you mean
with that ?