Practical Usability Testing

Practical Usability Testing
Session in Association with
techgig.com / Nimbuzz
Ravindran Antonysamy
QA Head, Nimbuzz

http://www.techgig.com/expertspeak/Practical-Usability-Testing-432
Before we fall in love with our own designs, lets find out what ‘really’ our users are thinking about it !!

1

Who Am I
Target Audience
What is Usability testing ?
PET – Persuasion / Emotion / Trust
ISO 9241
How to hire usability testers ?
Screener / Consent form / recording
Test protocol
Execution / Prototypes
Presenting Reports
MobileQAZone.com
Links (tools / videos)
Nimbuzz !!

10/30/2013

Go get Nimbuzz from
http://get.nimbuzz.com

2

Usability testing largely misunderstood with User acceptance testing which we do
with customers (UAT). Usability testing is NOT UAT. This webinar will give you the
basics of Usability testing. This webinar will give you the types of usability testing
(Hallway testing, Remote usability testing, Expert review, Automated Expert review
and A/B testing).
Usability testing measures the usability, or ease of use, of a specific object or set of
objects, whereas general human-computer interaction studies attempt to
formulate universal principles.
Why Usability test your product? How to conduct usability testing? What is a
Screener? How to recruit users? What is a test protocol and why you need to make
it, who should make it?

10/30/2013

Go get Nimbuzz from

3

How to conduct the usability testing with appropriate users? What to focus
on, when recruiting users? How to reward them or motivate them?
What are all the challenges when conducting usability testing with real
users? How to tackle them?
What is the hardware and software requirements when conducting
usability testing?
This webinar addresses all, to help you kick-start your usability testing
initiative!

10/30/2013

Go get Nimbuzz from

4

Key Discussion Points:
Types of Usability testing
How to perform usability testing in practical?
How to present the data to the Product Management ?
How to follow up the usability test results helps or transforms into
business?

Who Should Attend?
Mobile Application Testers, Mobile Handheld Testers, Mobile Usability
testers, Product Managers, Test and Tech Leads

10/30/2013

Go get Nimbuzz from

5

Who Am I & What I love

10/30/2013

Go get Nimbuzz from

6

Who Am I & What I love

10/30/2013

Go get Nimbuzz from

7

All testing types

Hallway testing
Remote Usability Testing
Expert review
Automated expert review
A/B Testing

8

Best testing types
Few Famous methodologies, and why we should use or
why we should not use them:
Hallway testing: set of people invited to test the product. Engaged in the
early stages of design / product. We shall invite new nimbuzz employees to
perform this test. This is a pilot test before performing the actual usability
testing (this will evaluate our test protocol and test methodology)
Other formats such as Expert review, automated expert review, A/B testing
shall not be engaged at this point (considering the cost involved for tools
and resources)
Moderated Usability testing : We shall hire usability testers / real users via
social media channels / friends circles. Evaluate them. Invite them to
Nimbuzz. Follow the test protocol reviewed by Kapil, Harsh and approved
Jammy + Respective PM. Conduct the test and circulate the results.

9

Buzz about Sample Size

More users,
Same
problem

According to the industry, the more number of users we use, great possibility is that
they will come across the same kind of usability problems. So we shall use 5 test
samples in the beginning. At the max, we shall use 7 or 10 people for this.
10

Test Execution
Method
-

-

-

Participant evaluation sheet shared with the PM and
he/she provide inputs for selecting the relevant participants.
On Scheduled test day, participant arrives to the test lab,
briefed about the product he/she will be using
Usability test session conducted by Test moderator and an Observer is present
to observe the events as It happens.
Moderator follows the test protocol (Reviewed by Functional team and
approved by Product manager) and request the tester to perform those actions.
User / usability tester “think aloud “ and speaks as he/she performs actions on
the specific task.
User / usability testers face & tapping action & think aloud voice recorded as it
goes (prior consent had been obtained in written form from the tester)
The Data then presented to the Product management with a easy to use –
decision making format (PPT presentation + consolidated videos of facial and
finger tapping along with voice)

11

Features for
Usability
Testing
-

Go with Core Features
Go with mostly visible to the user
Go with the feature that you want the user to focus on when entering /
navigating in your product
Go with features that fetches money

12

Test Protocol
Task / Question

Expected Reply /
User Action (User
think aloud when
performing this)

You downloaded
Nimbuzz client and
you do not have an
username. Please
create a Nimbuzz
account in this
phone.

-

-

-

Problems
Comments by
faced by the the Moderator
User

User opens
Application and
touches I am a new
user
Feeding the number is
mandatory ?
Can I skip this ?
How much time I have
to wait for an SMS ?
Can I use call me
Option now ?
Do I have to Use
numbers in the
password ?
Allowing Phone book
upload is mandatory ?

* Sample test protocol. Usability test moderator will carry a print of this to the usability test lab (or
wherever the usability test been executed) and use this as a key to request the user to perform various
test actions. We shall have 10 – 15 tests during one usability session with a user – maximum duration 1
hour to 1.5 hours.

13

Resource / Tools and Costs
For Hiring Resources, in Indian context, we shall pay 1K per user.
Considering we hire 5 people for Phase 1, we spend 5K.
In International context, upto 100 USD / 100 EURO’s paid. Some times companies
issue gift cards / gift vouchers / calling cards
Phase 1 Usability testing – 5 users : 5000 Rs (90 USD)
One time purchase of tools (http://www.techsmith.com/morae.html) – Ex for
Windows : 15,000 Rs (or 299 USD)
Heatmap tools : http://www.testing-web-sites.co.uk/2013/01/23/20-heatmaptools-for-testing-your-websites-usability/

14

Screener

To Inform the consultancy company
or Human resources team to get
the right people
15

Persona
Personas
The purpose of personas is to create reliable and realistic
representations of your key audience segments for
reference. These representations should be based on
qualitative and some quantitative user research and web
analytics. Remember, your personas are only as good as
the research behind them.
http://www.usability.gov/how-to-and-tools/methods/personas.html

16

Participants
1.
2.
3.
4.
5.
6.
7.
8.

Engineers
Non Engineers
Government Employees
Kids
Students (College / School)
Travelers
Low / High economical backgrounds
Plumbers, Doctors, Labors

17

http://www.youtube.com/watch?v=9wQkLthhHKA

18

Should user Cam to record user expressions and voice, separate cam for recording user actions
(such as tapping on the product). If required we will procure some simple stands to keep the
mobile / cam.
19

Presentation / Report Format
Humm…Now I am
trying to create an
account

-Sample-

Details: User is really
comfortable in
downloading the
app. But she faced
trouble in receiving
the pin and feeding
in. Please watch 2:35
min in the video.

User (1 to n) Data will be presented with a video (of the user when performing action, video of
the Device under test during the test, audio recoding when they “think aloud”.
27

Overall Execution Path
Initial test plan
created &
circulated to
Product Leadership
and management

Execution Plan
and Test Protocol
Approved by
Product
Leadership and
Management

Hiring resources,
Scheduling & Lab
setup completed

Test Execution &
video editing /
analysis report
created

Resource cost +
Tools cost
approved by
management

Final
Presentation to
Product
Leadership* and
Management
28

Links:
Documents / Info : http://www.usability.gov
Eye Tracking : http://www.tobii.com/en/eye-trackingresearch/global/research/
Usability testing of a Fruit : http://www.youtube.com/watch?v=3Qg80qTfzgU
7 Principles : http://www.youtube.com/watch?v=0oS-tvCUbBs
10 AB testing tools : http://www.practicalecommerce.com/articles/4133-10A-B-Testing-Tools-for-Small-Businesses
Remote Usability testing and UX Research Tools :
http://remoteresear.ch/tools/
PET : http://www.slideshare.net/QuiffBoy/pet-designing-for-persuasionemotion-and-trust
http://www.nngroup.com/ and http://www.nngroup.com/books/designingweb-usability/
http://www.amazon.in/dp/0321344758/?tag=googinhydr2506-21 (DON’T
make me think – steve krug)
29

Thanks, time for QA
Connect with me ra2012vi@gmail.com if required..
Youtube Link of this Session : http://www.youtube.com/watch?v=XXNXNSrlZcs
Nimbuzz Chat ID : ravindran2454
Get Nimbuzz Mobile Application for Android, iPhone, Windows Phone, PC,
Symbian, Java Phones from http://get.nimbuzz.com or
www.nimbuzz.com

30

Practical Usability Testing

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Practical Usability Testing