A deck detailing a case study on how UX and usability testing was used to evaluate candidate features for a mobile banking application roadmap

1. Case Study: Evaluating candidate
features through usability testing
and the KANO model
Joseph Dickerson

2. Goals of the test
• Objective was to identify what (if any)
proposed mobile banking features should be
prioritized for the next release of the
product
• This was done by testing the features with
participants and using the KANO Model to
the identify what features are positively (or
negatively) received by participants

3. The KANO Model
• The KANO Model is used in product development and
looks at customer preferences in five key categories
– Delighters (or “Attractive”)
– Satisfiers (or “Value-Adds”)
– Expected (or “Must-Have”)
– Unimportant (or “Indifferent”)
– Detractors

4. Delighters
Baseline
Satisfiers
Detractors tend to occur when
a basic need is implemented
badly or a unneeded feature
“overwhelms” an experience

5. Process
• 12 Participants were recruited based on legacy
personas
– All had to be current mobile banking users
• The participants were stepped through a series of
screens, one set for each of the six proposed features
– All features were described by the facilitator using
the same objective verbiage
– The order in which features were presented were
different for each participant

6. Proposed features evaluated
• Click to Dispute (a transaction)
• Scan a bill (to pay it)
• Open a new account (through app)
• Setup a budget
• Search Transactions
• Deposit a Check (aka Remote Deposit Capture)
– This was a “control” – we already know this feature
scored highly with customers

7. Process
• After being walked through each feature the
participant was asked to rate the feature and discuss
their reactions if it was offered by their bank
– I like it
– I expect it
– I’m neutral
– I can tolerate it
– I dislike it
• Then ask their reaction if it was NOT offered by their
bank
• The responses were then analyzed and classified
using the KANO Model

8. Analysis Method
Negative Question
Like Expect Neutral Tolerate Dislike
PositiveQuestion
Like - Attractive Attractive Attractive
One-
Dimensional
Expect Undesired Unimportant Unimportant Unimportant Must-Have
Neutral Undesired Unimportant Unimportant Unimportant Must-Have
Tolerate Undesired Unimportant Unimportant Unimportant Must-Have
Dislike Undesired Undesired Undesired Undesired -
8
Responses were averaged and weighed for both
questions and also analyzed based on persona mapping

9. Results
• Had one “No-show”, so only 11 participant’s
feedback was captured
• Of the six features reviewed, only two features were
indicated as “Must Haves”
– Search Transactions
– Deposit a Check
• Two features were indicated as “Attractive”
– Scan a bill (to pay it)
– Setup a budget
• Two features were indicated as “Unimportant”
– Open a new account
– Click to dispute (a transaction)

10. Observations
• Search Transactions was a potential “Delighter”
– Many positive comments about the perceived usefulness
of this feature
• Participants liked the idea of setting up a new account
on the phone, but doubted they would use it
• The majority of participants said they would never
use the Click to Dispute (a Transaction) feature – they
would call the bank’s support number to do this task
• Scan a Bill (to pay it) was well received but many
participants doubted they would use it
• None of the features were perceived to be potential
Detractors
10

11. Recommended features and focus
• Based on these results, we recommended a
focus on adding Search Transactions for the
next release of the mobile banking product
• Additional evaluation of Scan a Bill (to pay it)
and Setup a Budget was needed
• Open a New Account should be considered as
a feature to be added to the tablet banking
application

12. Participant Responses
How would you feel if this feature existing in
your mobile banking app?
Participant Reaction
Feature Like Expect Neutral Tolerate Dislike
Click to Dispute (a transaction) 2 2 6 1 0
Scan a bill 6 1 3 0 1
Open a new account (through
app) 2 0 6 0 3
Setup a budget 4 1 5 1 0
Search Transactions 8 2 1 0 0
Deposit a Check 4 6 1 0 0
12

13. Participant Responses
13
How would you feel if this feature didn't exist
in your mobile banking app?
Participant Reaction
Feature Like Expect Neutral Tolerate Dislike
Click to Dispute (a transaction) 0 1 8 0 2
Scan a bill 1 4 3 2 1
Open a new account (through
app) 1 3 7 0 0
Setup a budget 2 4 3 1 1
Search Transactions 0 2 3 2 4
Deposit a Check 0 1 2 2 6