design research for
‘everyday’ projects
UX London 09
leisa reichelt
disambiguity.com
@leisa

this is not really about user centred design

to solve a problem you must
first understand it
- kim goodwin

it’s about good design

who is this
workshop for?

‘everyday’ projects?
the design research projects you hear about most often are
enormous, time consuming and expensive.
most projects we work on can’t bear that.
research can/should be customised to
suit the project requirements

what we’ll cover
PART ONE: Designing Design Research
PART TWO: Conducting Design Research
PART THREE: Analysing Design Research
LOTS of hands on exercises.
Lots of your questions & shared experiences
Focus on QUALITATIVE Research
(esp. interview format)
Focus more on practical ‘what you can actually
do’ than ‘best practice’

part 1: designing
design research

what is
design research?
activities that seek insight into
user behaviour, goals and needs
that might be supported by the
design of products/services

why do
design research?
‘we tend to project our own
rationalisations and beliefs onto
the actions and beliefs of others’
- don norman, the design of everyday things

why do
design research?
design research helps you uncover,
understand and design for
real user needs

real user needs
= design for good usability
but also
= insight for inspiration/validation

when to do
design research?

generative research
prototype
evaluative research

design research toolkit
Qualitative
Stakeholder Interviews Co-design/Participatory Design
Subject Matter Expert (SME) Interviews Group/Social Research
Competitive Reviews Longitudinal Research
- diary study
Literature Reviews -twitter
- flickr
Depth Interviews (Customer&User)
Focus Groups
Direct Observation (Ethnography) Quantitative
Contextual Inquiry Surveys
Usability Testing Remote ‘Testing’ Tools
‘Snap’ Interviews Stats/Analytics

quantitative Vs
qualitative

mathematics and natural sciences
Quantitative
statistically sound, scientific
large random samples
questionnaires, surveys, tests
Quantitative research is the systematic scientific
investigation of quantitative properties and
phenomena and their relationships. The objective
of quantitative research is to develop and employ
mathematical models, theories and/or hypotheses
pertaining to natural phenomena. The process of
measurement is central to quantitative research
because it provides the fundamental connection
between empirical observation and mathematical
expression of quantitative relationships.
http://en.wikipedia.org/wiki/Quantitative_research

social sciences
insight gathering, exploratory
small sample sizes
Qualitative participatory, observational,
interviews, analysis of
documents & materials.
Qualitative research ...aim(s) to gather an in-depth
understanding of human behavior and the reasons that
govern human behavior. Qualitative research relies on
reasons behind various aspects of behavior.
Simply put, it investigates the why and how of decision
making, not just what, where, and when.
http://en.wikipedia.org/wiki/Qualitative_research

‘To design an easy-to-use
interface, pay attention to what
users do, not what they say.
Self-reported claims are
unreliable, as are user
speculations about future
behaviour.’
- jakob nielson

choose qualitative for insight
but ideally, combine quantitative
and qualitative inputs
quantitative can be great for problem
identification & solution validation

how to choose?

‘stock standard’ / garden variety plan
45-60 minute interviews
6-8 participants.
but...

understanding the problem:
what are you trying to learn?
- the problem is not always
what it first appears to be
- define your research questions
- what might be the best ways
to answer those questions?

understanding the problem:
what is the context for the
problem?
- care not to narrow the context early
- are their multiple relevant contexts?
- WHERE can we learn about the people’s behaviour
in relation to this problem?
- research as close to context as possible.

remember:
the power of ‘artifacts’
- memory aid / detail
- cross check for accuracy

...don’t be afraid of
getting a bit creative

...ethical research

EXERCISE! Part 1
Your client is a grocery store.
They want you to do some research and make
recommendations about how they can provide a
better online grocery shopping experience for
their customers.
You have unlimited time & budget
for your research.
What research activities would you suggest?

design research toolkit
Qualitative
Stakeholder Interviews Co-design/Participatory Design
Subject Matter Expert (SME) Interviews Group/Social Research
Competitive Reviews Longitudinal Research
- diary study
Literature Reviews -twitter
- flickr
Depth Interviews (Customer&User)
Focus Groups
Direct Observation (Ethnography) Quantitative
Contextual Inquiry Surveys
Usability Testing Remote ‘Testing’ Tools
‘Snap’ Interviews Stats/Analytics

squeeze to fit
START:
design the research project you’d love to do if
time/money/resources were no barrier
THEN:
work out what is achievable. Be creative!

ways to squeeze
- fewer participants
- fewer activities
- shorter sessions
- less proximate to context (use artifacts)
- use technology (phone, web video, twitter)
- less complex research activities
- more readily accessible participants
Even the smallest amount of
data beats none.
(yes, I’m quoting Jakob again - Guesses vs Data as the basis for
Design Recommendations)

let’s take a break!

part 2: conducting
design research

recruitment

who to recruit?
‘persona hypotheses’
Think about:
- the different user roles
- factors most likely to affect behaviour
Don’t over complicate it!

how many?
as few as possible.
if given the choice to more research
studies with few participants.
why?
- diminishing returns
- speed to action (design)

(yes, I’m quoting Jakob again - Why you only need to test with 5 users)
In earlier research, Tom Landauer and I [Jakob Nielsen]
showed that the number of usability problems found in a
usability test with n users is:
N(1-(1-L)n)
where N is the total number of usability problems in the
design and L is the proportion of usability problems
discovered while testing a single user. The typical value of L
is 31%, averaged across a large number of projects we
studied. Plotting the curve for L=31% gives the following
result:

The most striking truth of the curve
is that zero users give zero insights.
http://www.useit.com/alertbox/20000319.html

how to find them?
DIY vs Using Professionals
- how much time/money do you have?
- how easily accessible are participants?
is using friends/family network really bad?

Incentives. Pay them.

CAUTION:
researching young
people & kiddies

logistics
- where to research
- timing your research
- factoring in ‘no shows’ (& ‘floaters’)
- consent forms (privacy, their rights
& incentive acknowledgement)

discussion guides
-writing a discussion guide

discussion guide
> define your research questions
> start as wide as possible, narrow slowly
> keep it contextual, not speculative
> show, don’t tell (observation = good)
(show me how you do that?)
> uncover mental models
(if you clicked there, what do you think would happen?)
> ask OPEN questions
(ask questions to get them talking NOT yes/no answers)
> take care not to lead
(you can have leading questions AND leading structures)
> don’t outsource design to your participant!
(remember, you’re the designer!)

EXERCISE! Part 2a
write a discussion guide for a depth
interview for your grocery store client
(who wants to design a better online
grocery shopping experience)

interview technique
getting the most from your
research participants
(quite possibly the most important part!)

capturing data

what to capture on
sticky notes
> *anything* interesting/relevant said in the
course of your interview, in as close to direct
quotes as possible.
> your design ideas
> questions for the future
> capturing on the fly is a *real* skill. I still live
transcribe to a text file then extract affinity
notes from the transcription.

interview technique
introductions & getting started
- introduce yourself (and any colleagues)
- what are we doing here?
- what time will we finish
- forms & incentives
- ask permission to record

interview technique
it’s not a test!
- allay their nerves
- distance yourself from the design
- we only care about *their* opinion, not people
they know or ‘most people’
- there’s no right answer or smart answer
- their opinion counts - will really help shape the
design of the product/service

interview technique
building rapport
*everyone* has something that makes them
either very interesting or passionate.
find it. talk about it.
the investment in building rapport is repaid by
the quality/quantity of insight given.

interview technique
- Active Listening - nodding, smiling & paraphrasing
- Focus - don’t let yourself get distracted
- Look for physical clues - additional information,
and do they match the words?
- Don’t Rush - take time to get your thoughts
together & prepare your next question
- Keep it open - Who, What Where, When, Why, How
and my favourite ‘Tell me about...’
- Follow the flow - don’t stick to your script, mix up
the order if it flows better for that participant.

interview technique
Have a great closing question.
Some of my favourites are:
- Do you know someone you might
recommend this to? Who/Why
- How would you rate it out of 10
- What would you tell the designers
Try to summarise the parting sentiment
(notes that it is not more or less important than the initial sentiment)

interview technique
say thank you.
always remember the participant
is HELPING YOU.
be appreciative.

if you’re feeling nervous, remember...
research participants
are like dangerous
animals...
they’re usually just as scared of you as you are of them.

EXERCISE! Part 2b
An interview!
in pairs, take turns to conduct a 10 minute
interview using your discussion guide from
the previous exercise

part 3: analysing
design research

rule of thumb:
allow at least a day of analysis
for each day of research

start with
single case analysis
(the story of each participant)

then move to
multiple case analysis
(trends in the data)
eg. affinity sorting

a mind map can be a
digital affinity sort

collaborative
affinity sorting?
I’ve had great results with this but
only if participants have observed the research

using research data in design:
audience modelling

using research data in design:
audience modelling

using research data in design:
experience strategy & design
principles

using research data in design:
audience modelling
Today I’m pissed
HI at British Gas
I’m passionate
about Education
Proximity
am I experiencing once my billing the more I learn
‘it’ right now? problem is fixed, the more I care
I’m fine.
LOW HI
Commitment
how much will I care next month?

Keith ‘I just want to know enough to buy well’
Keith is planning to pop the question soon - in about a weeks time
The engagement ring will be his first big jewellery purchase.
He knows virtually nothing about diamonds or jewellery. He doesn’t want to become
and expert, he just wants to know enough to buy well and wants to feel reassured
using research data in that he’s getting what he’s paying for.
He has a general idea of his girlfriend’s preferred style but is not really confident
about choosing the right design. Some of his mates have been engaged recently and
models: personas
he’s asked them a few questions about the process.
He popped into Goldsmiths last week to tentatively start investigating his purchase
32yrs and spent most of the time telling assistants he was ‘just looking’ - he left quite
Jnr Mgr, Lloyds TSB quickly, not liking the ‘pressure’ of the store experience. He doesn’t know (but wants
Income approx £30K p.. to) what makes one ring so much more expensive than another.
investment emotion Purchase Lifecycle
STAGE ONE - ‘RADAR’ STAGE TWO - ‘INTENSIVE RESEARCH’ STAGE THREE - MAKE PURCHASE
novice expert knows that a potential actively seeking information to find and purchase the ring
purchase is on the cards, inform purchase (qualities of
has heightened awareness diamond and metal, price etc.) the right ring at the right
for self for other price from a company he can
of information that crosses Gaining enough knowledge to buy
his path but not actively well. trust.
need it quickly willing to wait seeking information.
10 days to weeks in advance approx. 1 week in advance of
weeks/months in advance. in store, google for information proposal
£100 £2000 possible sources: social
networks, media/content

Recommended Reading:
Rapid About Face 3 Designing for
Contextual Cooper, the Digital Age
Design Reinman Kim Goodwin
Holzblatt, & Cronin
Wendell & Wood

thank you :)
Leisa Reichelt
disambiguity.com
leisa@disambiguity.com
@leisa