DESIGN FOR USABILITY
TESTING
by
Mumin Yu, User Experience Consultant, USER XING
Jen Wang, User Experience Specialist, MiTAC

What Are We Gonna Do Today

Don’t worry.
It is not that bad.

A TYPICAL USABILITY TEST

http://reportwritingskills.com/images/report_writing_skills11.jpg

http://www.pod.usace.army.mil/images/I-Want-You-Flat.jpg

http://theotherthomasotter.files.wordpress.com/2007/06/asug07-moraescreengrabs01.png

http://reportwritingskills.com/images/report_writing_skills11.jpg

PLANNING RECRUITING TESTING ANALYZING REPORTING

PLANNING RECRUITING TESTING ANALYZING REPORTING

GET OTHERS INVOLVED

http://www.cyber-cinema.com/gallery/others.jpg

Designers
Top Managers Product Managers
Sales
Marketings
Engineers
Customer Services

The more/earlier you communicate your plan,
the better the result will be.

GIVE ME A BREAK

If you are feeling tired, so is the participant!

Make no session longer than 2 hours.

Do no more than 4 sessions a day.

Allow at least 60 minutes break between sessions.

Projects Sun Aug 30 – Sat Sep 5, 2009 (Taipei)
Sunday 8/30 Monday 8/31 Tuesday 9/1 Wednesday 9/2 Thursday 9/3 Friday 9/4 Saturday 9/5
9am
10am
Test Session Test Session Test Session
10am - 12pm 10am - 11:30am 10am - 11am
11am
12pm
Test Session
12pm - 1pm
1pm
Test Session
1pm - 2:30pm
Test Session
2pm 1:30pm - 3:30pm
Test Session
3pm 2:30pm - 3:30pm
Test Session
4pm 3:30pm - 5pm
Test Session Test Session
5pm 4:30pm - 6:30pm 4:30pm - 5:30pm
6pm
7pm
8pm

Projects Sun Aug 30 – Sat Sep 5, 2009 (Taipei)
Sunday 8/30 Monday 8/31 Tuesday 9/1 Wednesday 9/2 Thursday 9/3 Friday 9/4 Saturday 9/5
9am
10am
Test Session Test Session Test Session
10am - 12pm 10am - 11:30am 10am - 11am
11am
12pm
Test Session
12pm - 1pm
1pm
Test Session
1pm - 2:30pm
Test Session
2pm 1:30pm - 3:30pm
Test Session
3pm 2:30pm - 3:30pm
Test Session
4pm 3:30pm - 5pm
Test Session Test Session
5pm 4:30pm - 6:30pm 4:30pm - 5:30pm
6pm
7pm
8pm

Projects Sun Aug 30 – Sat Sep 5, 2009 (Taipei)
Sunday 8/30 Monday 8/31 Tuesday 9/1 Wednesday 9/2 Thursday 9/3 Friday 9/4 Saturday 9/5
9am
10am
Test Session Test Session Test Session
10am - 12pm 10am - 11:30am 10am - 11am
11am
12pm
Test Session
12pm - 1pm
1pm
Test Session
1pm - 2:30pm
Test Session
2pm 1:30pm - 3:30pm
Test Session
3pm 2:30pm - 3:30pm
Test Session
4pm 3:30pm - 5pm
Test Session Test Session
5pm 4:30pm - 6:30pm 4:30pm - 5:30pm
6pm
7pm
8pm

Projects Sun Aug 30 – Sat Sep 5, 2009 (Taipei)
Sunday 8/30 Monday 8/31 Tuesday 9/1 Wednesday 9/2 Thursday 9/3 Friday 9/4 Saturday 9/5
9am
10am
Test Session Test Session Test Session
10am - 12pm 10am - 11:30am 10am - 11am
11am
12pm
Test Session
12pm - 1pm
1pm
Test Session
1pm - 2:30pm
Test Session
2pm 1:30pm - 3:30pm
Test Session
3pm 2:30pm - 3:30pm
Test Session
4pm 3:30pm - 5pm
Test Session Test Session
5pm 4:30pm - 6:30pm 4:30pm - 5:30pm
6pm
7pm
8pm

THINK OUT OF THE LAB

On Treadmill
http://www.bodyqueststore.com/Commercial/Treadmills/Nautilus%20718%20Treadmill.jpg

On Foot

In Car

PLANNING RECRUITING TESTING ANALYZING REPORTING

PLANNING RECRUITING TESTING ANALYZING REPORTING

SHOW THEM THE MONEY

http://blogs.edmunds.com/greencaradvisor/money_stack.jpg

Enough to motivate them,
but not enough for them to pretend

NT$1000 per hour seems to work just fine for
now.

What if you don’t have the money?

GIFT CERTIFICATE

FOOD
http://www.singleandhappy.net/.a/6a00e554e9a52b88330112797c5b5328a4-320wi

SCHOOL CREDIT

THANK YOU
http://upload.wikimedia.org/wikipedia/commons/thumb/6/6a/Heart_left-highlight_jon_01.svg/491px-Heart_left-highlight_jon_01.svg.png

SNOWBALL EFFECT

http://wpcontent.answers.com/wikipedia/commons/thumb/2/23/Giant_snowball_Oxford.jpg/180px-Giant_snowball_Oxford.jpg

You still need to do the screening yourself

PLANNING RECRUITING TESTING ANALYZING REPORTING

PLANNING RECRUITING TESTING ANALYZING REPORTING

A GOOD EXAMPLE OF A BAD EXAMPLE
http://www.breadwig.com/uploads/sketches/bunnies/000554.jpg

Shut Up and Listen/Watch!

RULES OF ENGAGEMENT

You need participants to move on to the next task
because time is limited.

You need to bring participants back on track
because they have gone too far down the wrong
path.

Participants are stuck. You think you can learn
more if they continue.

Moderator, please give them more time.

WATCH YOURSELF

WATCH OUT FOR
Less-than-neutral questions
Unfocused sessions
Annoying gestures
Repeating words
Facial expressions
more...

PLANNING RECRUITING TESTING ANALYZING REPORTING

PLANNING RECRUITING TESTING ANALYZING REPORTING

METRICS

(!"
'!"
!"#$%&'%!()*+,)$-.+
&!"
%!"
$!"
#!"
!"
)*+,-"#" )*+,-"$" )*+,-"%" )*+,-"&" )*+,-"'" )*+,-"(" )*+,-"."
PERFORMANCE METRICS

PERFORMANCE
Task success
Time-on-task
Errors
Efficiency
Learnability

)"
("
!"#$%&'()'*+&,-.*+/01'
'"
&"
%"
$"
#"
!"
*+,"-./"011" 7.-8501,"9:" *+,"-./"4-.?"3.?" 7.-8501,";9.5/"
23124.5/"6+-4" -;<+=;>.-"23.+210" /."@1/5@-"/." 03+AA+-=".A>.-0"
03.AA+-="
ISSUED-BASED METRICS

ISSUES
Producing an error
Assuming a task is complete when it is not
Preventing task completion
Creating some level of confusion
Not seeing something that should be noticed
Assuming something is correct when it is not

()*+,-.*))"
'!!"
&!"
%!"
$!"
#!"
607)+0871." !" /0)*"1+"()*"
/0)*"1+"2*03.4.5"
SELF-REPORTED METRICS

SELF REPORTS
After-Scenario Questionnaire (ASQ)
Post-Study System Usability Questionnaire (PSSUQ)
Computer System Usability Questionnaire (CSUQ)
System Usability Scale (SUS)
Questionnaire for User Interface Satisfaction (QUIS)
Usefulness, Satisfaction, and Ease of Use Questionnaire (USE)

BEHAVIORAL & PHYSIOLOGICAL METRICS
http://www.simpleusability.com/images/heuga-home-eye-plot.jpg

BEHAVIOR & PHYSIOLOGY
Verbal behaviors
Facial expression
Eye-tracking
Pupillary
Skin conductance
Heart rate

PLANNING RECRUITING TESTING ANALYZING REPORTING

PLANNING RECRUITING TESTING ANALYZING REPORTING

Number of usability reports to read 5
4
3
2
1
1 2 3 4 5
Hours slept during office hours
USABILITY REPORTS VS OFFICE HOURS SLEPT
http://www.slideshare.net/gregbell/usability-20

Not everyone needs to read a long report.

So you need to have a more effective way to
communicate your findings

A VIDEO IS WORTH A THOUSAND WORDS

CASE STUDY