Barcamp Bangalore 2 - On User Experience and Usability Testing

Amit Pande Manager, Oracle User Experience ( http:// www.flickr.com/photos/amitpande ) Dec 02, 2006 Barcamp Bangalore 2 On User Experience & Usability testing

User expectations

Users have natural expectations about how things work

User Interfaces and experiences need to match this expectation

What happens when products don’t match users’ expectations?

Lets compare two products/experiences and see how they match up to users’ expectations

http://destraynor.com/serendipity/index.php?/categories/6-Usability-Reviews

Excellent role play and commentary by Des Traynor highlighting extreme usability differnces between Yahoo Calendar and 30boxes

What this presentation is about

Overview of User experience and usability testing

What is User Experience, where does it fit in?

What is Usability testing

What gets measured in usability testing

Formal and informal lab setups

Links to interesting stuff

What is User Experience?

Slide courtesy: http://www.idemployee.id.tue.nl/g.w.m.rauterberg/

Integrating User Experience into SDLC

4 main stages: Requirements, Design, Construct, Testing

User Experience activities occur through the development lifecycle

What is Usability Testing?

… a method by which users are asked to perform certain tasks in an effort to measure the product's ease-of-use, task time, and the user's perception of the experience

… is done throughout the development lifecycle and can be used to measure all aspects of the interface from 'jargon comprehension' to 'task flow' for critical tasks

… 4 Usability Test Methods

Exploratory

Comparison

Assessment

Validatory

… requires formal training, great sensitivity to user psychology, and adherence to protocol

Video: The Good, Bad, Ugly with end users

What’s happening in this video?

Users performing online tasks

Buying flowers

Buying car accessories

Evaluating a company’s site

Users following ‘Think Aloud’ protocol

Users walking through and expressing different usability problems

http://www.boltpeters.com/videos/bp_large.wmv

Courtesy: http:// www.boltpeters.com

Data gathered through usability testing

On Visual Design

On Layout

On Terminology

On Messaging

On Navigation

On Task Flow

On Task Support

Informal usability testing

Informal Lab setups: Participant and host sit together, and the host logs the participant's activities

Remote testing

Remote Lab setups:

Participants and host communicate over web collaboration tool

VOIP/phones

Screen sharing (WebEx or others)

Formal lab testing

Walkthrough of a sample usability test

Task name: Turn Off Login Music ( www.betterdesktop.org )

Task was performed using Suse 9.3 in a Portable Lab on the GNOME desktop. The test was administered in English. The following is a description of the task:

'Please pretend that you do not like the login music which played when you logged in. Please fix the computer so that you don't have to hear the login music again. '

Subjects are asked to rate their experience (Exp Level) with computers on a scale from 1 to 10, with 10 being very experienced and 1 being no experience. The breakdown of the subjects responses according experience is below:

5 3 6 6 9 3 10 1 # of Subjects Exp Level

Test users Task: Turn off login music - Summer 2006 Mpeg (22 Mb) Theora (9 Mb) Female 6/10 18-30 Subject84 Mpeg (23 Mb) Theora (9 Mb) Female 9/10 18-30 Subject82 Mpeg (16 Mb) Theora (6 Mb) Female 8/10 18-30 Subject80 Mpeg (209 Mb) Theora (89 Mb) Male 9/10 18-30 Subject79 Mpeg (81 Mb) Theora (34 Mb) Female 9/10 18-30 Subject76 Mpeg (31 Mb) Theora (13 Mb) Male 9/10 30-40 Subject72 Mpeg (32 Mb) Theora (13 Mb) Female 8/10 30-40 Subject70 Mpeg (25 Mb) Theora (10 Mb) Female 8/10 18-30 Subject48 Mpeg (19 Mb) Theora (7 Mb) Male 8/10 30-40 Subject29 Video Downloads Gender Experience Age Subject

Test videos

http://www.betterdesktop.org/wiki/index.php?title=Data

Sample tasks:

Task: Find the email from chelsea@chillonia.org

Task: See if you are free at the time of the party

Task: Enter the party information into your calendar

Task: Send Chelsea your RSVP

Task: Change the time of the party in your calendar

Task: Change the system language to Spanish

Task: Turn off the firewall

Task: Turn off login music - Summer 2006

Task: Move a window to a different desktop

… .and others

Test results

5 subjects passed this task successfully. 6 subjects failed to complete the task.

The max time for this task was 14 minutes 11 seconds. The minimum time for this task was 1 minutes 28 seconds. The average time for this task was 5 minutes 0 seconds.

Other usability testing methods