Remote User Testing

Evaluation for 2008
Remote User Testing
Michael Gaigg
March, 18th 2009

Agenda
Show how
User-centered Design helps teams to save:
Time
Money
Aggravation from clients
Usability testing early in the design process identifies:
Areas of confusion and frustration
Where the user can’t complete a task
While it is still cheap and easy to mitigate these problems
03/18/2009
Remote User Testing
2

03/18/2009
Remote User Testing
3
Agenda
User-centered Design
User Testing
Output (Qualitative & Quantitative)
Software & Usage
Time Estimates & Cost
Advantages & Disadvantages

Introduction to User-centered Design
UCD is a design philosophy and a process in which the needs, wants, and limitations of the end user of an interface or document are given extensive attention at each stage of the design process.
Its measures are:
Effectiveness (accuracy and completeness to achieve goals)
Efficiency (resources expended in relation to the accuracy and completeness)
Satisfaction (comfort and acceptability of the work system to its users)
03/18/2009
Remote User Testing
4

User Testing is a UCD Method
03/18/2009
Remote User Testing
5

Overview
User testing involves:
A design
A user (participant)
An observer (facilitator).
Observe user behavior & receive user feedback.
Test tasks (not functions).
03/18/2009
Remote User Testing
6
User
Design
Observer

Qualitative Output
Understand mental model of the user (navigation).
Validate requirements and design assumptions (focus on important functionality early).
Discover non-functional design mistakes and areas of improvement (unforeseen user requirements).
Evaluate effectiveness of design elements.
03/18/2009
Remote User Testing
7

Quantitative Output
Success rates / Error rates
Task time
Satisfaction ratings
03/18/2009
Remote User Testing
8
Task Success
Task Time

User Testing Software
TechSmithMorae
Manager: $ 1,295.-
Bundle: $ 1,495.-
Morae Manager uses the collected data (observation markers and notes, video, keyboard and mouse inputs) to analyze and calculate task times, error quotes and other common measurements.
03/18/2009
Remote User Testing
9

Remote User Testing Software
TechSmithUserVue
$ 149.- / Month or
$ 1,495. / Year
Web-based app enabling a Facilitator to remotely observe (and record) a Participant using a phone line for communication.
Multiple Observers can passively join the Session and share their observations with the Facilitator.
03/18/2009
Remote User Testing
10

The Session
Setting up a session.
Connecting (download of
software bundle required).
Calling participant.
Starting the session.
Viewing (and recording) the participant.
Removing software bundle from computers of involved parties.
03/18/2009
Remote User Testing
11

Time Estimates & Cost
Preparation (18 hours)
User screening: 8 hours
Task creation: 8 hours
Environment: 2 hours
User testing (10 hours)
2 hours per user
5 users per round
Post-test (32 hours)
Test report: 16 hours
Implementation: 12 hours
Communication: 4 hours
03/18/2009
Remote User Testing
12
Our time:
1 Round = 60 hours
2 Rounds = 110 hrs.
3 Rounds = 160 hrs.
+ User comps
+ Time additional
observers

Advantages of remote UT
Reduced (or no) travel time and expenses.
Higher exposure through easy screen sharing (managers can sneak in easily).
Actual user environment, familiar and comfortable.
Possibly fewer drop-outs.
03/18/2009
Remote User Testing
13

Disadvantages of remote UT
Facilitator not physically present (degree of separation can be challenging).
Can't see facial expressions or non-verbal cues.
Difficult to build rapport and trust.
Difficult to control environment.
Possibly technical difficulties (firewall, etc.).
Setup and use of software or usability lab might be challenging and requires a liaison.
03/18/2009
Remote User Testing
14

Summary
Increased usability
Higher degree of client satisfaction
Continued business = Higher revenues
Lesser client aggravation (“This is completely unusable”)
Assist PM during requirements and design
Focus and prioritize on important functionality
Identify unforeseen user requirements early!
Reduced costs
Training and service
Viewer Help-Desk calls
03/18/2009
Remote User Testing
15

http://www.michaelgaigg.com	71
http://www.mgitsolutions.com	32
http://www.slideshare.net	6
http://webcache.googleusercontent.com	1

Remote User Testing

by Michael Gaigg on Oct 12, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

4 Embeds 110

Statistics

Remote User Testing — Presentation Transcript