Usability Testing- An Introduction & Tips for Effective Usability Testing in India By Abhay Rautela http://www.conetrees.com , http://www.theuxbookmark.com , @conetrees

Agenda Introduction Usability? What is Usability Testing? Why conduct Usability Testing? When and What to perform Usability Testing on? How many users to test? The Usability Testing Process Applicability Tips for Effective Usability Testing in India Questions & Feedback

Introduction

Usability?

What is Usability Testing?

Why conduct Usability Testing?

When and What to perform Usability Testing on?

How many users to test?

The Usability Testing Process

Applicability

Tips for Effective Usability Testing in India

Questions & Feedback

What is Usability? Usability is what you may know as user friendliness- ease of use Usability is used in context of any object or a product a human might interact with A product with good usability is one that is: Effective- in helping the user accomplishing their goal for which they are using the product Efficient (can help the user accomplish their goals goal quickly) Satisfying to use Easy to approach and low error rate and criticality

Usability is what you may know as user friendliness- ease of use

Usability is used in context of any object or a product a human might interact with

A product with good usability is one that is:

Effective- in helping the user accomplishing their goal for which they are using the product

Efficient (can help the user accomplish their goals goal quickly)

Satisfying to use

Easy to approach and low error rate and criticality

What is Usability Testing Usability Testing is a methodology To evaluate how easy a product is to use By giving tasks that are commonly performed or intended to be performed on the product To people who are representative of the product’s target audience Types Formative/ Exploratory Summative/ Assessment/ Verification or Validation Towards the beginning of the PDLC, it’s good if you can conduct usability tests on your competitions’ products

Usability Testing is a methodology

To evaluate how easy a product is to use

By giving tasks that are commonly performed or intended to be performed on the product

To people who are representative of the product’s target audience

Types

Formative/ Exploratory

Summative/ Assessment/ Verification or Validation

Towards the beginning of the PDLC, it’s good if you can conduct usability tests on your competitions’ products

When and What to perform Usability Testing on You can and should test at most phases of the PDLC/ SDLC: Requirements Gathering Design Development Testing Release

You can and should test at most phases of the PDLC/ SDLC:

Requirements Gathering

Design

Development

Testing

Release

When and What to perform Usability Testing on 2. You can test from concept to ready product Prototype Level of detail Lo-fi to hi-fi (paper to interactive) Length and breadth of functionality Horizontal- all features, limited functionality Vertical- limited feature(s), full functionality within those feature(s) Product Section Complete

2. You can test from concept to ready product

Prototype

Level of detail

Lo-fi to hi-fi (paper to interactive)

Length and breadth of functionality

Horizontal- all features, limited functionality

Vertical- limited feature(s), full functionality within those feature(s)

Product

Section

Complete

How many users? Research (Virzi, 1992, Nielsen & Landauer,1993) indicates that 5 users uncover approximately 80% of a product’s usability problems Other research suggest different numbers The Truth: It depends A simple answer: For those interested in DIY usability testing: 5 is good to uncover an appreciable number of usability issues in Increasing number of participants means significant increase in time and cost due to increased complexity of planning, participant management and data analysis amongst other factors Else consult with experienced professionals

Research (Virzi, 1992, Nielsen & Landauer,1993) indicates that 5 users uncover approximately 80% of a product’s usability problems

Other research suggest different numbers

The Truth: It depends

A simple answer:

For those interested in DIY usability testing: 5 is good to uncover an appreciable number of usability issues in

Increasing number of participants means significant increase in time and cost due to increased complexity of planning, participant management and data analysis amongst other factors

Else consult with experienced professionals

Why conduct Usability Testing In order to make a product user friendly you need to uncover usability issues in it You can’t do that on you own since: You might not be the end user Even if you belong to the target audience, you are probably not of the same user expertise level as most of or a section of the users are. You need to validate you assumptions Make informed decisions. Have data by your side and make decisions on facts, not hunches or opinions

In order to make a product user friendly you need to uncover usability issues in it

You can’t do that on you own since:

You might not be the end user

Even if you belong to the target audience, you are probably not of the same user expertise level as most of or a section of the users are.

You need to validate you assumptions

Make informed decisions. Have data by your side and make decisions on facts, not hunches or opinions

The benefits of Usability Testing Being penny wise, pound foolish does not pay. Don’t look at saving a few days because of which you end up overshooting your plan by months Speed up development and cut costs by getting it right in the least time and money possible by making it easy to use for the user and designed in a way that best helps them accomplish their goals PDLC integration: To continually identify and act upon usability issues throughout the process at a low cost rather than trying to at the end when it is more or less impossible

Being penny wise, pound foolish does not pay. Don’t look at saving a few days because of which you end up overshooting your plan by months

Speed up development and cut costs by getting it right in the least time and money possible by making it easy to use for the user and designed in a way that best helps them accomplish their goals

PDLC integration: To continually identify and act upon usability issues throughout the process at a low cost rather than trying to at the end when it is more or less impossible

The Usability Testing Process Plan Participant Recruitment Scenario/ Task creation Pilot Test Execution/ Test sessions Data Analysis Report Checkpoint

Plan

Participant Recruitment

Scenario/ Task creation

Pilot Test

Execution/ Test sessions

Data Analysis

Report

Checkpoint

Applicability Specifically applies to: Usability testing in India in organizations following an Agile SDLC More towards participants belonging to or residing in metros You should be interested if you are in India and: Are interested in learning about usability testing or are a usability engineer A product manager interested in learning about usability testing A start up interested in how usability testing can benefit your product Even if you are outside India: You should still be able to derive value from the tips if you are conducting usability tests in countries outside India

Specifically applies to:

Usability testing in India in organizations following an Agile SDLC

More towards participants belonging to or residing in metros

You should be interested if you are in India and:

Are interested in learning about usability testing or are a usability engineer

A product manager interested in learning about usability testing

A start up interested in how usability testing can benefit your product

Even if you are outside India:

You should still be able to derive value from the tips if you are conducting usability tests in countries outside India

Applicability of Usability Testing Tips Why tips on India are different from the books? India has a different cultural system as compared to the west Culture Value system Language Ways of working and interfacing with people Hofstede’s cultural dimensions There is a distinction between the working of Indian internet companies and multinational corporations or foreign firms

Why tips on India are different from the books?

India has a different cultural system as compared to the west

Culture

Value system

Language

Ways of working and interfacing with people

Hofstede’s cultural dimensions

There is a distinction between the working of Indian internet companies and multinational corporations or foreign firms

Tip 1 (Planning) Create a Plan that Will be Read Suggest plan formats to the stakeholders Finally, create a plan according to their preference

Suggest plan formats to the stakeholders

Finally, create a plan according to their preference

Tip 2 (Planning) Participant Compensation that Works Kids- Toys, Chocolates, Confectionary School- Movie tickets, Cash College- Cash, Movie tickets, Shoppers Stop/ Life Style Coupons, Big Bazaar coupons Working Professionals- Lunch/ Dinner coupons, Shoppers Stop/ Life Style Coupons, Big Bazaar coupons Elders- Big Bazaar coupons Recruitment during festival season is tough. Reverse the flow by offering festival related gifts as compensation instead of the usual

Kids- Toys, Chocolates, Confectionary

School- Movie tickets, Cash

College- Cash, Movie tickets, Shoppers Stop/ Life Style Coupons, Big Bazaar coupons

Working Professionals- Lunch/ Dinner coupons, Shoppers Stop/ Life Style Coupons, Big Bazaar coupons

Elders- Big Bazaar coupons

Recruitment during festival season is tough. Reverse the flow by offering festival related gifts as compensation instead of the usual

Tip 3 (Recruitment) Talk like the User When calling prospective participants over the phone, speak in the language you feel the user is comfortable in, and can communicate and understand better to increase your chances of recruitment. When the person at the other end is comfortable with you, and more importantly gets some feeling of trust, the better your chances or successful recruitment Mix language and use slang when appropriate Empathize with all users Affection with children and respect to the elderly is required and just as beneficial

When calling prospective participants over the phone, speak in the language you feel the user is comfortable in, and can communicate and understand better to increase your chances of recruitment.

When the person at the other end is comfortable with you, and more importantly gets some feeling of trust, the better your chances or successful recruitment

Mix language and use slang when appropriate

Empathize with all users

Affection with children and respect to the elderly is required and just as beneficial

Tip 4 (Execution) Record using Inbuilt Laptop Camera Inbuilt laptop cameras are hardly noticeable* Reduce awkwardness and intimidation for the camera conscious, especially effective with participants from non-metros Decrease introduction of bias, increase accuracy of test results * The user must be briefed about recording them. Their permission is required before proceeding. Get a CDA/ NDA signed.

Inbuilt laptop cameras are hardly noticeable*

Reduce awkwardness and intimidation for the camera conscious, especially effective with participants from non-metros

Decrease introduction of bias, increase accuracy of test results

* The user must be briefed about recording them. Their permission is required before proceeding. Get a CDA/ NDA signed.

Tip 5 (Execution) Simulate as much as possible Use a bandwidth shaper E.g.- You might want to limit speed to 56 kbps if this is the internet speed available to most of your users rather than using high speed internet connectivity available at your office Same goes for OS, browsers and resolution

Use a bandwidth shaper

E.g.- You might want to limit speed to 56 kbps if this is the internet speed available to most of your users rather than using high speed internet connectivity available at your office

Same goes for OS, browsers and resolution

Tip 6 (Execution) Use ‘Think Aloud’ only if Appropriate Use ‘Think Aloud’ protocol only if participants are able to adapt to it easily, Feels unnatural for many thus increasing bias Adds to the pressure and increases bias, may affect performance Retrospective evaluation is an alternative but is time consuming

Use ‘Think Aloud’ protocol only if participants are able to adapt to it easily,

Feels unnatural for many thus increasing bias

Adds to the pressure and increases bias, may affect performance

Retrospective evaluation is an alternative but is time consuming

Tip 7 (Execution) Speak & Dress like the participant Try and dress like your participants, don’t be intimidating Speak in the language you feel the participant is comfortable in, and can communicate and understand better to increase your chances of recruitment This will help decrease bias introduced due to intimidation or awkwardness The more comfortable the participant, the more your chances of accurate results Caution! Don’t hand the reins over to the user

Try and dress like your participants, don’t be intimidating

Speak in the language you feel the participant is comfortable in, and can communicate and understand better to increase your chances of recruitment

This will help decrease bias introduced due to intimidation or awkwardness

The more comfortable the participant, the more your chances of accurate results

Caution! Don’t hand the reins over to the user

Tip 8 (Execution) Stakeholders as observers Let the stakeholders get a first hand experience of how the user actually uses the product Observation by stakeholders will help increases transparency and their belief validity of a methodology they are new to If they can not be observers, provide access to all the video recordings of the sessions or even better, a highlights video Stakeholders observing participants is more influential in them understanding usability issues rather than handing out a report

Let the stakeholders get a first hand experience of how the user actually uses the product

Observation by stakeholders will help increases transparency and their belief validity of a methodology they are new to

If they can not be observers, provide access to all the video recordings of the sessions or even better, a highlights video

Stakeholders observing participants is more influential in them understanding usability issues rather than handing out a report

Tip 9 (Execution) Usability testing at Elders homes In the field, try to restrict team to 2 members One facilitator and one observer is good enough Let them know how many will be coming in advance Show your ID/ business card- Most elders are not comfortable with the idea of strangers entering their homes

In the field, try to restrict team to 2 members

One facilitator and one observer is good enough

Let them know how many will be coming in advance

Show your ID/ business card- Most elders are not comfortable with the idea of strangers entering their homes

Tip 10 (Reporting) Prioritization also by LOE Prioritize by criticality and frequency of issues so stakeholders can decide on incorporation as required When time and resources are scarce, stakeholders will pick recommendations requiring least effort to implement

Prioritize by criticality and frequency of issues so stakeholders can decide on incorporation as required

When time and resources are scarce, stakeholders will pick recommendations requiring least effort to implement

Tip 11 (Reporting) Provide quotes in original language Don’t translate quotes in English. When you translate a quote, you loose quality of the insight gained through it. (Think JPG- lossy) You can provide a translation (in the rarer case) that you have stakeholders who don’t understand the language the quote is in

Don’t translate quotes in English. When you translate a quote, you loose quality of the insight gained through it. (Think JPG- lossy)

You can provide a translation (in the rarer case) that you have stakeholders who don’t understand the language the quote is in

Tip 12 (Reporting) Create a Report that will be read Discuss in advance the format of the report with stakeholders offering suggestions to them on it Don’t deliver the report in a format that the stakeholder won’t read Avoid an unpleasant surprise with a format that contains or misses out on information (and sorting) that the stakeholders wanted

Discuss in advance the format of the report with stakeholders offering suggestions to them on it

Don’t deliver the report in a format that the stakeholder won’t read

Avoid an unpleasant surprise with a format that contains or misses out on information (and sorting) that the stakeholders wanted

Tip 13 (Reporting) Present Report Submitting it does not guarantee it being read or stakeholders acting upon it Always try and present the report To ensure comprehension of the entire report to stakeholders To understand why certain recommendations can not be incorporated To prevent information distortion

Submitting it does not guarantee it being read or stakeholders acting upon it

Always try and present the report

To ensure comprehension of the entire report to stakeholders

To understand why certain recommendations can not be incorporated

To prevent information distortion

Tip 14- Checkpoint It is not uncommon in an Agile SDLC to see reports disappear over time without any incorporation of agreed recommendations Checkpoint at appropriate intervals of time to keep track of and follow through on recommendations agreed to be incorporated

It is not uncommon in an Agile SDLC to see reports disappear over time without any incorporation of agreed recommendations

Checkpoint at appropriate intervals of time to keep track of and follow through on recommendations agreed to be incorporated

Tip 15: Overall Process Involve stakeholders/ Product Department Absolutely important if usability is just beginning to get recognized in your organization or you are part of the few who are part of the group busy evangelizing A must if you are at Stage 2 to 5 of Nielsen’s Corporate Usability Maturity Stages 1 to 3 of Trump’s Usability Maturity Model Why? They make or break usability within the organization Ultimately, they give the nod, allocate resources and budget for usability If you shut doors on them, they can shut you down since you are probably conducting usability tests because they are allowing you to

Involve stakeholders/ Product Department

Absolutely important if usability is just beginning to get recognized in your organization or you are part of the few who are part of the group busy evangelizing

A must if you are at

Stage 2 to 5 of Nielsen’s Corporate Usability Maturity

Stages 1 to 3 of Trump’s Usability Maturity Model

Why?

They make or break usability within the organization

Ultimately, they give the nod, allocate resources and budget for usability

If you shut doors on them, they can shut you down since you are probably conducting usability tests because they are allowing you to

Questions?

Feedback

Thank you Website: http://www.conetrees.com Email: hello at conetrees dot com Twitter: http://www.twitter.com/conetrees The UX Practitioners’ resource: http://www.theuxbookmark.com

Join the UX Book Club, New Delhi http://uxbookclub.org/doku.php?id=new_delhi As creator and lead coordinator of the UX Book Club, New Delhi, I’d like you to consider joining us if you are work in user experience in and around New Delhi Join the SlideShare Accessibility Group slideshare.com /group/web-accessibility View and share useful presentations on web accessibility- Join the group