Researchers generally see quantitative methods as very different to qualitative methods. However with modern analytics and the advent of ‘big data’ our insight approaches are undergoing a shift. The numbers of data points and activity measured in a typical digital product is growing, providing increasingly rich and complete sets of insightful data. This gives us the information we need to understand what people are doing and more importantly, why they are doing it. However researchers are not typically asked to get involved with the design of analytics measures. Digital teams are lacking skills on how to extract qualitative meaning from analytics, opting for traditional qualitative research approaches that are not fully integrated with their existing user-base. We took the challenge on and built our first Big Data System bringing Quant and Qual together.

1. August 2015
UCD CONFERENCE - HUMANITY IN DIGITAL LANDSCAPES
@LolaOye
Quant = Qual
Why we should all love data

2. ‣ My background is primarily Qual.
‣ Depending on your leaning (Qual or Quant), you may feel
my points are unfair.
‣ I’m more interested in what’s next, than what was.
As you listen, bear in mind:
2

3. 3
“I assumed that the time would come when there would be a science in which things could be
predicted on a probabilistic or statistical basis.
Isaac Asimov, Author of the Foundation Series, inventor of Psychohistory

4. Quantitative
4
/ˈkwɒntɪˌtətɪv,-ˌteɪtɪv/
Relating to, measuring, or measured by the quantity
of something rather than its quality.
In UX Research measured by:
• Numbers
• Amounts
• Trends
• Increments
• Statistics
We treat quantitative data as inherently summative.
It lack’s the nuance that experiences are built on.
Qualitative
/ˈkwɒntɪˌtətɪv,-ˌteɪtɪv/
Relating to, measuring, or measured by the quality
of something rather than its quantity.
In UX Research measured by:
• Narratives
• Stories
• Superlatives
• Inferences
We treat qualitative data as both formative and
summative. But it is open to bias and slow to analyse.

5. 5
UX research is losing
relevance.
It takes too long and is too expensive.
Too many “UX people” don’t have research skills.
We no longer have the right skills for the
emergent business & technology context

6. 6
“Given two or three data points, our minds can construct an
alternate reality in which all of those data points make
flawless sense. Five UX Research Pitfalls, UXMagazine. Elaine Wherry, 2010

7. Indication not Inference
Causation not Correlation

8. “Gary King, Harvard University, Director of
Institute for Quantitative Social Science, 2013
Big Data is not about
the data.
8

9. 9
We have a lot of quantitative data
that we need to analyse better.
We have a lot of qualitative data
and research but we can’t harness
these insights in real time.
Our quantitative data and
qualitative data are not joined up.
We can’t keep waiting 3-6 weeks
for data analysis.
UX & Product people need to
stop burning budget on research!
We need big data. We got a
Hadoop, please bring instructions.

10. HEY, WHAT CAN YOU TELL ME
ABOUT OUR CUSTOMERS’
BEHAVIOUR? WELL WHAT DO YOU WANT TO
KNOW?

11. I WANT TO BE OPEN, WHAT
CAN YOU TELL ME?
LOTS. YOU NEED TO BE MORE
SPECIFIC.

12. OK, WHAT IS OUR CUSTOMERS
MOBILE BEHAVIOUR?
DO YOU WANT EVERYTHING?

13. ERM, YEAH…?
OK, THAT’S GOING TO TAKE 6
WEEKS. EXCEL FILE OK?

14. ARE YOU SERIOUS OR JOKING?
I CAN’T TELL. I’M NOT JOKING.

15. ‣ They didn’t know how to get the data they wanted, how it
was stored or how we would be able to use it
‣ The ‘user’ was us…UX & product folks who need to make
informed decisions to prioritise services and features
‣ We’d never built a big data system before, so we had to
learn quickly!
This brief had spiky bits:
15

16. 16
User Experience Architect
Statistician/Data Scientist
Back-End Developer
Front-End Developer
UI Designer
“Unicorn” Developer
Client
Bringer of whisky.

17. Nobody really knew what they
could get. So they didn’t know
what they could ask for.

18. You have to start with
qualitative questions you’ve
always wanted to answer.
There will be huge gaps in
insight because some
databases don’t play nice.
The Data Protection Act.
It sucks.
1. 2. 3.

19. What the business has
always wanted to know, but
never knew it could get:
19
PERFORMANCE
• App downloads / app usage / app feedback (1view)
• Activity in digital products over time (spot trends, dips and peaks)
• Customer activity (frequency) across channels
• Cross product offering and cross channel purchase behaviour (360
view)
• Touchpoint usage: feature use, feedback on features, drop off
points and completed journeys
• Segments x value earned by channel
EXPERIENCE
• Customer’s comments over a determined period of time
• Behaviour across channels over a period of time
Use data to segment users by behaviour, not spend
• Impact of launches or push notifications on behaviour over time
• Movement between transactional segments: crossing demographics
with purchase value/frequency
…..and lots more.

20. POS
Website Clickstream
On-Site Customer Reviews Loyalty Card
External Product
Reviews
Daily Sentiment
Analysis
Social Media
Search Data
App Analytics
Postcode Lookup
Dozens of UX Insights
DATA SOURCES

21. 21
Red: Data sources we don’t have access to or don’t
know how to access, or for which the source itself is
unintelligible to us or totally unknown.
Amber: Data sources we have access to but which
have problems that severely limit their use.
Green: We have access to these sources today and
the data is clean.
53%
10%
37%
30 Data Sources | RealTime | Periodic | Historical

22. 22
Their quant stuff
we could plug in
Qualitative (Unstructured) Data
The bits we made…all re-usable
and open!
ACIXOM

23. 23
REAL TIME SALES
SITE CATALYST
HISTORICAL SALES
AXIOM
DEMOGRAPHICS
TWITTER
APP FIGURES
OPINION LABS

24. 24
Can I see how many vouchers have been redeemed
this week and how that compared to last week?
REAL TIME SALES
SITE CATALYST
HISTORICAL SALES
AXIOM
DEMOGRAPHICS
TWITTER
APP FIGURES
OPINION LABS

25. 25
Can I monitor the impact of the app launch on
customer’s voucher redemption behaviour?
REAL TIME SALES
SITE CATALYST
HISTORICAL SALES
AXIOM
DEMOGRAPHICS
TWITTER
APP FIGURES
OPINION LABS
Can I monitor the impact of the app launch on
customer’s voucher redemption behaviour?

26. Can I monitor the impact of the app launch on
customer’s voucher redemption behaviour?
26
REAL TIME SALES
SITE CATALYST
HISTORICAL SALES
AXIOM
DEMOGRAPHICS
TWITTER
APP FIGURES
OPINION LABS
Can I monitor engagement across different
channels in relation to feature releases and I can I
overlay that with channel specific sentiment?
Can I monitor the impact of the app launch on
customer’s voucher redemption behaviour?

27. 27
Can I monitor app downloads over time, find out
who is using them and what the overall sentiment
is about the app?
REAL TIME SALES
SITE CATALYST
HISTORICAL SALES
AXIOM
DEMOGRAPHICS
TWITTER
APP FIGURES
OPINION LABS
Can I monitor the impact of the app launch on
customer’s voucher redemption behaviour?
Can I monitor engagement across different channels
in relation to feature releases and I can I overlay that
with channel specific sentiment?
Can I monitor the impact of the app launch on
customer’s voucher redemption behaviour?

28. 28

29. We’re not allowed to know everything
we would like to know. Get over it.

30. It’s really difficult to separate
interesting behavioural data
from “Personally Identifiable
Information”.
30
THE DATA ITSELF
• Geography vs location
• Individual & household vs personalisation
• MAC addresses
THE ANALYSIS
• Sentiment + Sales by geography
• Usage + Ratings + Sales by product
….the more you cross and the more ‘accurate’ it is, the closer you get
to effectively breaking the Data Protection Act.

31. QUANT
QUAL

32. 32
“The secret of genius is to carry the spirit of the child into old age, which means never losing
your enthusiasm.
Aldous Huxley, Author & Philosopher

33. @LolaOye
lola.oyelayo@headlondon.com
THANK YOU