Almost anything can be measured. Your website’s usability, the success of an app, the value of your business card, the engagement of your customers. Everything can be researched. Studies suggest that for every one dollar that is invested in user experience, it can bring up to one hundred dollars in return. But there are hundreds of research methods! How do you know which to use? How do you quantify “fuzzy things” like feelings? How do you choose what to measure, when to measure it, and how to analyze the results? Lots of questions to explore during this overview! Join us to learn more about Qual vs. Quant with Think Company's Keren Toledano! Keren is a User Experience Design Lead at Think Company. She has worked across applications and websites for clients including Netflix, The Moth, Comcast, Baume et Mercier, and The Olympic Games. She believes very strongly in liberal arts majors working in tech—that algorithms, interfaces and most digital advances present social challenges that need critical thinking about their human context. A lover of dystopian fiction, museums, video games (RPGs), movie soundtracks, strolling aimlessly, and reuniting our Left Brains and Right Brains, Keren is always working on her next side project.

1. Qualitative v. Quantitative
November 6, 2018
Research Rewind

2. Keren Toledano
● Academic background in American
Studies, Museums, and Nonprofit
Management
● Didn’t know User Experience,
User-Centered Design or Human
Factors were fields of study
● Built websites as a kid, but didn’t
think it was something “girls did”

3. Root Meaning
Words matter! Why do these two words even exist? How do they differ?
Qualitative
Qualitative v. Quantitative
Quantitative
Concerning qualities not quantities.
Based on comparisons.
Expressible as a quantity or relating to
measurement. Numeric.
Related terms: subjective, conditional,
dependent, partial, emotions, behaviors,
social, human, empirica, firsthand
Related terms: objective, calculable,
surveyable, gaugeable, statistical,
mathematic, monitor, observable
3

4. Quantitative is the ‘what’, and qualitative is the ‘why’.
Both are equally important for the success of User Experience design.
Qualitative v. Quantitative

5. Qualitative research typically answers broad questions, such as “what
features should our app have?” or “why aren’t our users buying widgets?”
Quantitative research typically answers explicit, and specific questions,
such as “how many users registered for our website in February 2017?”

6. Quantitative
Born: 1581 CE / 437 years old
Other words from that year:
● Analysis 📊
● Academic 📚
● Countable 🔢
● Deficient 📉
● ad infinitum ∞
● Caliper
● Diary 📔
6

7. Qualitative
Born: 1607 CE / 411 years old
Other words from that year:
● Antelope
● Slobber
● Frightful 😱
● Earshot 👂
● Unbiased ⚖
● Emasculate ♂
● Misapplication ❌
7

8. The Beatles
Qualitative v. Quantitative

9. Qualitative v. Quantitative
Meet The Beatles
A mixed-approach summary:
The Beatles were an English rock band, started in
Liverpool, England in 1960 until they broke up in 1970.
They are the most successful and influential band in the
history of music. The Beatles drew influences from many
music genres through their career, including 1950s rock
and roll, rhythm and blues, classical, psychedelia, and
Indian music, and their songs ranged from pop ballads,
psychedelic rock to hard rock.
There were four members of the band: John Lennon
(rhythm guitar), Paul McCartney (bass guitar), George
Harrison (lead guitar), Ringo Starr (drums)
Key: Qual / Quant
9

10. Number of Albums: 13Longest Song: 8 minutes and 22 seconds
Note that in 1964 and 1965, The Beatles wrote,
recorded and released two studio albums and two
singles, made a film, and toured extensively.
Years Active: ~8 years
The Beatles released their catalog for streaming in
2015. Although Spotify was founded in 2006, and The
Beatles were created in 1960, The Beatles were the
sixth most streamed band on Spotify in 2017. They
continue to have high active listener counts in 2018.
Qualitative v. Quantitative
Quantitative: The Beatles
At 23 seconds long, "Her Majesty" is the shortest
song in the Beatles' repertoire, their longest song is
the avant-garde "Revolution 9", clocking in at 8:22.
If counting years from first studio contract to
dissolution, The Beatles existed from 1962 until
1970. Although they performed previous to those
dates with different names and lineups.
Spotify Monthly Listeners: 14,323,587
10

11. What did that information tell you? What can you infer?
What else would you like to know?
Qualitative v. Quantitative

12. Results:
Ab e R d - 10
The es - 3
Hel ! - 5
Sg . Pep ’s Lo y H a s C u n - 4
A Har ’s Ni t - 7
Rev r - 2
Rub S ul - 2
Mag My er T - 6
Yel S b a n - 1
Qualitative v. Quantitative
Quantitative: The Beatles
Let’s say you’ve gathered 40 people together in a
room and you’ve asked them all, “What is your
favorite Beatles album?”
You tally the results and they look like this….
12

13. Qualitative v. Quantitative
Quantitative: The Beatles
● Please open your phone and navigate to https://pollev.com/ktoledano352 or
text KTOLEDANO352 to 22333
● Please follow the prompts and select which option closest matches your opinion
● Wait for my instructions
Strongly
Disagree
Disagree Slightly
Disagree
Slightly
Agree
Agree Strongly
Agree
1 2 3 4 5 6
50% Negative 50% Positive
13

14. How did specificity of the question change the results? If you changed your
answer between the first and the second question, why?
Qualitative v. Quantitative

15. Why do you like the band? ….
Qualitative v. Quantitative
Qualitative: The Beatles
“I s e t n Sa r ni t m e n a s, no d e ,
bu s in B at n an t ad . W A wo h e
ca n ve f h o t la n f e w , an r
w i Th e s a 9 so s h o 15! Eac c e
as at t a r ul mo f o t y i .”
“The r o t e r b s o t w in h or f
t e s o t re d , s o n co c ni h an
me r d 't u t to r y a s, wi h n ri s
be d s e s i g r a r .”
“The r a t i n pa d. The h u t oc s
un p ed. The r o r u ad yo se t
wa c ib . One r e b h i w ev h e s
ne b om ut — us y u t me Ch t a . Whe
on n of m t e d t o l l a w fo
hi-te d re y er c lo .”
“Joh , Pa l, Ge r , an R g a h h i m . The r e
re al p e t er s e m s o h . Eve
ot c ef he B es p on r a f he y
ma me n h a l , bu t Be t .”
15

16. The key to making a good
forecast is not in limiting
yourself to quantitative
information.
“
-Nate Silver

17. Quantitative data and qualitative data are inherently related.
They work together to tell a story.
The Beatles sold 1.6 billion albums in the USA! But does that capture
the hysteria of the fans? The influence of their music? The place the
band holds in American culture and our communal psyche?

18. OK, so what are the many methods exactly?

19. Qualitative v. Quantitative
Landscape of Research Methods
What People Say
What People Do
Why?
How to?
How Many?
How Much?
BEHAVIORAL
ATTITUDINAL
QUALITATIVE
(DIRECT)
QUANTITATIVE
(INDIRECT)
DataSource
Approach
Adapted from Christian Rohrer

20. Types of Quantitative Methods
A / B Tests*
Analytics Surveys &
Questionnaires*
Benchmarking
Eye Tracking
Studies
Desirability
Studies
Clustering or
Affinity Mapping*
Usability Tests*
Qualitative v. Quantitative
*Can be both quant/qual
20

21. Types of Qualitative Methods
A / B Tests*
Interviews Surveys &
Questionnaires*
Tree Tests*
Card Sorts*
Usability Tests*
Contextual
Inquiry
Heuristic Analysis
Qualitative v. Quantitative
*Can be both quant/qual
21

22. Qualitative v. Quantitative
Landscape of Research Methods
What People Say
What People Do
Why?
How to?
How Many?
How Much?
BEHAVIORAL
ATTITUDINAL
QUALITATIVE
(DIRECT)
QUANTITATIVE
(INDIRECT)
DataSource
Approach
Adapted from Christian Rohrer
Eye-Tracking
Usability Lab
Studies
A/B Analysis
Ethnographic
Field Studies
Focus Groups
Interviews
Diary Studies
Usability
Benchmarking
Card Sorting
Customer Feedback
Unmoderated UX Studies
Intercept Surveys
Email Surveys
Concept Testing Desirability Studies
True Intent Analysis
User Field Surveys
Business Intelligence /
Data Mining

23. We are not ethnographers. We are not anthropologists.
We are not sociologists. We are not scientists.
We are not engineers.
Qualitative v. Quantitative
23

24. ...And That’s OK
We get to borrow from (and temporarily visit) fields
● Human Factors: Employs psychological and
physiological principles
● Design-Thinking: Based in C-K theory, lateral
thinking, method engineerings and the social
psychology idea of “sense-making.”
● User Research: Borrows heavily from
anthropological methods like fieldwork,
participant-observation, field notes, and coding
● Consumer Research: Grounded in direct
observation, surveys, and interviews. 24

25. Qualitative v. Quantitative
Batman's Utility Belt
At last count, Batman’s most recent belt
held 28 elements. Using all tools would
make for a long comic book; and a very
technical movie. He uses his tools
carefully.
Don’t always use freeze grenades.

26. Qualitative v. Quantitative
Batman is not “nothing without
his utility belt”

27. When should we use each type of research?
Qualitative v. Quantitative
27

28. Design Cycle or Product Development Phase
Strategy Execution Assessment
Goal: Inspire, explore
and brainstorm
opportunities
Inform feature and function
decisions, optimize designs
in order to reduce risk,
improve usability
Measure product
performance against
itself or its
competition
Approach: Qualitative and
Quantitative
Mostly Qualitative Mostly Quantitative
(summative)
Typical
methods:
Stakeholder
interviews, field
studies, diary
studies,
open-ended
surveys, data
mining, heuristic
analysis, analytics
Card sorting, participatory
design, paper prototype,
and usability studies,
desirability studies,
Usability
benchmarking, task
assessments,
analytics, intercepts
surveys, A/B testing
Qualitative v. Quantitative
28

29. What did we learn?
● Qualitative and quantitative research are
complementary, answering different goals
● Real behavior is informed by, and made sense
of, using a combination of both methods
● It is just as important to understand the why
and how, as it is to understand the how many
and how much
● It’s ok to use only a few types of research
TL;DR - Try to always use a mix of both
29

30. Thank You