Qualitative Measuring of UX- presentation at Maria 01 UX Design Meetup April 2017

Maria 01 UX Design
Meetup April 2017
–
Qualitative
Measuring of UX
Outi Kotala

http://fundersandfounders.com/how-airbnb-started/

@okotala @GoforeOy www.gofore.com
Outi Kotala
Lean Startup philosophy and measuring UX
• ”Be first” and ”fail fast”
• No time to do extensive market research or usability studies
• But you can measure and validate your User Experience whatever the project
phase, thru out the entire product or service life span!

Outi Kotala
Quantitative vs Qualitative UX measuring
Quantitative
• For example data analysis and online surveys, feedback forms
• Pro’s:
• No time-consuming test sessions or interviews
• A lot of data about the users and their behaviour (semi)automatically collected
• Statistically accurate
• Con’s
• Cannot give deep insight to users’ attitudes, needs, expectations, prejudices etc.
• Finding usability issues difficult

Outi Kotala
Problem?
“They are not using my app or service!”
“They’re using just parts of the app?”
“We have this idea of a new feature, but we are not sure if it works for
them”
They = the users

Outi Kotala
Problem?
“They are not using my app or service!”
• Are you targeting the wrong target user groups?
• Do your users find you service?
• Does your app or service answer to the users needs and
expectations?

Outi Kotala
Problem?
“They’re using just parts of the app?”
• Does your service or app have a designed customer journey with clear
service touchpoints, that corresponds to users goals and the context of
use?
• Are there any gaps or usability issues that affect your UX?
• Are there factors outside your user interface that affect the service UX?

Outi Kotala
Problem?
“We have this idea of a new feature, but we are not sure if it works for
them”
• Should we just design and implement the new feature and get user
feedback on-the-go?
• Are there any fast and light ways to design and validate a concept
before implementation work starts?

Customer journey map examples
Kuudes.fi service design
concept for Helsinki City
Library

Customer Journey Map through Red & White grocery store via kdsketchbook.tumblr.com

• Introduction to user journey and experience
maps by UX Lady:
www.ux-lady.com/experience-maps-user-journey-and-more-
exp-map-layout/

UX Best Practices
Lean UX
Measuring & Validating
Product-Market Fit
Are we making the
right thing?

Lean UX
www.smashingmagazine.com/2011/03/lean-ux-getting-out-of-the-deliverables-business/

Qualitative methods
for measuring UX

Outi Kotala
Heuristic analysis
• Means a walk-through of your service / user interface using a heuristic usability checklist
• Pros:
• Quick (takes approx. one day to do)
• No real users needed
• Can find most of the navigational issues and problems related to the UI / component level
usability
• Cons:
• Cannot give deep insight to users’ attitudes, needs, expectations, prejudices etc.
• Cannot validate whether your assumed customer journey or service concept refer to actual
user needs: heuristic walk-through can only analyse the in UI usability, not the concept
itself!

Outi Kotala
Interviews
• Semi-structured interviews
• Good to find out user’s attitudes, prejudices, expectations, what they are
interested in…
• Usually 5-6 interviewees is enough
• The most critical issues and themes are found with 5 test persons
• If you have several target groups that differ from each other a lot
demographically, recruit 2-3 per each group

Outi Kotala
Interviews
• You can use interviewing in all of the phase of you project
• Market research: get information on users needs and context
• Concept or ideation phase: validate your idea with users
• Implementation: check on some features or possible usability issues, if
there are wider factors involved in the problem
• Launch / MVP: enrich your feedback and survey results with more insightful
knowledge about the use of your service

Outi Kotala
Questionnaires, surveys + qualitative interviews
• Sometimes you can start with a questionnaire / survey study and deepen your
understanding about the users based on the survey results
• Sometimes you first need to interview people to find out what to ask in a
survey!
• Wrong or too vague questions in a survey provide no usable results!

Outi Kotala
Focus groups
• Group interviews or facilitated workshops are a good way to get even deeper
level insight to users and their attitudes
• Group discussion usually generates more ideation and richer feedback from
the participants than single-person interviews
• You can have group size from 4-5 up to a dozen
• Larger group may prevent fluent discussion, so break it to smaller groups (for
example using the World Café workshop method)

Outi Kotala
Focus groups
• Facilitation is very important:
• Test persons should represent all your services target groups
• The groups should be designed so that their dynamics support fluent discussion
and themes indented for the workshop / group interview
• In some cases you need to have several focus groups depending on the
interest areas and target user profiles
• Make sure that all the participants get their say on the conversation!
• You can also consider co-design sessions with the users to come up with needs
and new ideas towards you service

Lean Usability testing
ux.stewdean.com/ux-and-the-engineering-mindset-evidence-based-design

Outi Kotala
Lean UX measurement and testing
1. Assume: make hypotheses
• Ditch “requirements” and use a “problem statement”
2. Test your assumptions
• Research users
• Test the usability!
• Make it quick
• Validate often and in smaller chunks
• Keep with the agile development process, validate UX in each sprint!
• Iterate design according to results as soon as possible
• -> Test fast – “succeed fast”

Outi Kotala
Lean usability testing
• 4-6 users is enough!
• You can also recruit “from the streets” (see later section User recruitment)
• Online testing is an option, too
• Online testing services or teleconference with screen capture
• Tools:
• Lookback
• Silverback
• Userbrain
• UXPin
• More tools: http://www.creativebloq.com/ux/best-user-testing-software-61515337

Outi Kotala
Combine methods!
• Usability testing + focus group day
• Interview + usability testing

Avoid these
• “Corridor testing”
• Avoid the “tech” bubble / [insert your own demographic group here] bubble!
• Testing only your own assumptions or just single features
• Constricting UX measurements to testing a single feature might miss some important user
information. Test your whole service too!
• Let your users surprise you: they may see some functionality more important than the one
you thought was the “key thing”
• Doing it too heavy
• Recruit tens of users (4-6 is enough!)
• Wait too long (test fast = fail/succeed fast)
• Skipping UX measuring altogether
• Cost-saving in UX measurement can cost you much more later
• You can prevent major hiccups with relatively small usability validation

Outi Kotala
Recruiting users

Outi Kotala
Recruit volunteers
1. Decide (or research!) what you target groups are
2. Define the user profiles of these groups, for example using “personas” tool
3. Recruit volunteer test persons so that you have representation from each
personas /user profiles
• Recruit approx. 20% more people than you need. For example 7 people if
you need 6. There are always some who doesn’t show up or cancels.
However, be prepared for 100% show-up.
• Don’t forget to include a gratuity or reward for participating
• A gift card or a discount / free sample of your service should do nicely

Outi Kotala
Ways to recruit
• Recruit yourself
• In many cases you have the best channels to your users. Make the best of
them
• Make sure you have a good representation from all the different user
profiles / target groups
• ”Personas” or user profile documentation help in this
• Include also people who have never heard about your service or product, or
at least have not used it before

Outi Kotala
Ways to recruit
• Snowball sampling
• Existing test persons recruit future volunteers from among their
acquaintances
• Get a larger sample of volunteers than you need, and hand-pick most
suitable test persons to represent all of your major target groups

Outi Kotala
Ways to recruit
• Let the “pro” do it
• If you don’t have ready channels for recruiting test persons with the right target
group profile, it’s really time consuming to recruit volunteers for interviews or
usability testings
• Someone who has done this before can do it a lot quicker
• You can also consider using professional user test recruitment companies

Outi Kotala
What to google? Concepts and methods for
further reading
• Lean UX
• Agile UX
• Guerilla usability testing
• Focus group
• Co-creation / Co-design / Participatory design
• Heuristic analysis
• Personas
• Customer / user journey

And remember…
It’s never too late to measure your UX!

THANK YOU!
outi.kotala@gofore.com
@okotala
www.gofore.com
@GoforeOy

Qualitative Measuring of UX- presentation at Maria 01 UX Design Meetup April 2017

Recommended

Recommended

More Related Content

Similar to Qualitative Measuring of UX- presentation at Maria 01 UX Design Meetup April 2017

Similar to Qualitative Measuring of UX- presentation at Maria 01 UX Design Meetup April 2017 (20)

Recently uploaded

Recently uploaded (20)

Qualitative Measuring of UX- presentation at Maria 01 UX Design Meetup April 2017