1
USER TESTING
Diana Adorno & Valeria Spirovski
December 3, 2014
#bamperth
@dianaadorno @charmedquark
Business Analyst Meetup
Photo credit: Diana Adorno
FORMAT
•  Questions before we start
•  Intro & frame
•  Usability test demo
•  Practice facilitating a test
•  Questions & Discussion
2
HOUSEKEEPING
•  Emergency procedure
•  Bathrooms
•  Considerations
•  Questions
3
WHAT IS USER
TESTING? 4
WHY DON’T WE JUST ASK PEOPLE WHAT THEY THINK?
5
Usability testing
is the killing field of
cherished notions.
DAVID ORR
43
If it costs $1 to make a
change on paper, then it
costs $10 in code and
$100 when the site is up.
THERESA CUNNINGTON
COMPUTERWORLD 2006
37
DEMO: USER TEST (OPEN)
8
Photo credit: Diana Adorno
DEMO: USER TEST (CLOSED)
9
www.margaretriver.com
BASIC FORMAT FOR A USER TEST SESSION
welcome
a few background questions
think aloud
the test
the gift
10
TRY THIS
11
TRY THIS…
WORK IN PAIRS
One user, one test facilitator
■  1 scenario * 2 tasks * Speaking aloud * Take a few notes
SCENARIO
Planning a holidays to Broome with a few
friends
12
THE TASKS
!  Task 1: Find 4 places to visit around
Broome
!  Task 2. Find 4 places to visit around
Broome using the WA Tourism site
(http://www.westernaustralia.com/au/Pages/
Welcome_to_Western_Australia.aspx)
13
SHAREBACK
Success or failure
14
KEY QUESTIONS
ABOUT USER
TESTING
15
APPEARANCE
Visual and UI Design
INTERACTION
Navigation and interactive elements
INFORMATION
Content & Information Architecture
STRUCTURE
The relationship between the product
components and the organisation
PROPOSITION
How product provides value to meet the
users needs, while being financially viable
& technically feasible
CONCEPT
The idea of how the value is delivered
to the user/customer
WHAT ARE YOU TESTING FOR
TYPES OF TESTING
Exploratory (formative)
!  Use competitor sites
!  Paper ideas
!  Your existing site
Validation (summative)
!  Test the actual system
!  Test as you go
17
What are you looking for…
Effectiveness
•  Do they reach their goal?
Efficiency
•  How easy was it to reach the goal?
•  time to complete, mistakes
•  What made it difficult, what was confusing, backwards
and forwards-ing, needing help to complete
Satisfaction
•  Emotional response
18
3-clicks rule is
dead
19
WHO TO TEST WITH
20
HOW MANY PEOPLE TO TEST WITH?
Benefit
Cost
5
http://www.nngroup.com/articles/how-many-test-users/
BUDGET FOR USABILITY STUDIES
20 users
5 5 5 5
WHAT SYSTEM TO
TEST WITH
23
PAPER
PROTOTYPES 24
ASK PARTICIPANTS TO INTERACT
http://www.nngroup.com/reports/prototyping/video_stills.html
25
26
MOCKUPS
INTERACTIVE PROTOTYPES
27
28
29
IMAGES WITH
LINKS
30
EXISTING
SYSTEM OR
COMPETITOR
31
INVISIBLE
SYSTEM
WHAT SHOULD YOU
DO?
32
33
WHERE TO TEST
In the project room
34
IN DESIGN WORKSHOPS
35
IN THE FIELD
GUERRILLA USER TESTING
37
RECORD
WHAT YOU
SEE
o credit: Diana Adorno
RECORD WITH A VIDEO
CAMERA
38
Photo credit: Diana Adorno
OR JUST USE THE CAMERA IN YOUR
LAPTOP
39
SILVERBACK & OTHER RECORDING
TOOLS
40
41
Take notes on
a laptop
OR POST-IT NOTES
42
43
On sticky notes
(observer out
of shot)
WHAT TO CAPTURE IN NOTES
•  Only what you see (don’t summarise)
•  Direct quotes
•  Directly observed actions or problems
44
NOTE: Summarise later
when you distill with
your team
DISTILLING RESULTS
45
POST-IT NOTES GROUPED INTO THEMES/INSIGHTS
46
STRAIGHT ONTO THE SCREENS
47
Share back with your team as
soon as possible
48
SHARING THE
FINDINGS
49
HOW SHARE WHAT YOU DISCOVER
•  Take video of the session
•  Photos
•  Share back the relevant parts
•  Include different members of your team
or stakeholders as observer
•  Share direct quotes
“It is not about the time it takes, it is about annoyances. If it is
annoying, I will spend a minute being annoyed”
50
WHAT CAN GO
WRONG
51
AGGRESSIVE PARTICIPANT
Thank them for their input
Give them their gift
Stand up
Move to the door
Open the door
52
NO TASKS
Ask a few questions so that they describe
actual tasks
Ask them about the last time, they actually
did x and use that
Work with the business stakeholders to
come up with realistic scenarios and likely
tasks
53
FLYING SOLO
Record – audio or video
Run through the test and then do a
walkthrough with the participant and mark
up the designs together
54
FORGET WHAT TO DO
Write a script
Helps if there are several things to test
55
EXAMPLES &
READING & TOOLS
56
READ & WATCH
Rocket Surgery Made Easy, Steve Krug
http://www.sensible.com/
Video of a User Test (Steve Krug)
www.peachpit.com/promotions/promotion.aspx?promo=137602
57
TOOLS
RECORDING
Silverback http://silverbackapp.com/ (Mac)
REMOTE TESTING
www.usertesting.com
http://www.loop11.com/usability-testing/
58
THANK YOU
diana.adorno@thoughtworks.com
valeria.spirovski@woodside.com.au

User testing