Finding the Narrative in Numbers: Making the Most of Metrics [UX Immersion 2017]

tweet-friendly
Hello!
@katerutter

*
*
**
*
*
*
*
*
*
*
*
*
*
*

how do we know 
our work is…
WORKING?
*

3 questions
1.What are we trying to 
accomplish?
2.What does that look like?
3.How will we measure the work  
& track progress?
Words
Pictures
Numbers

the languages
Words
Pictures
3
Numbers

UX expertise
huh? literacy ﬂuency mastery
Words
Pictures
Numbers

Numbers
1. What are we trying to accomplish?
2. What does that look like?
3. How will we measure the work 
& track progress?
* * * *

Numbers
“ ~ Joshua Porter
Your metrics will be as
unique as your business.

Numbers
“Over 10,000 downloads!”
“Avg time on site is 22 minutes!”
“+450 new sign-ins this week!”

Numbers
None of these numbers help us know 
our product is working for our users.
“Over 10,000 downloads!”
“Avg time on site is 22 minutes!”
“+450 new sign-ins this week!”
Nope.

Numbers
User Research
Surveys
Usability
Testing
Interviews
Observation
Field
research
Metrics &
Analytics
Quantitative
tells you what is
happening
Qualitative
tells you why
things happen
Generative
helps you come up
with new ideas
Evaluative
helps you assess or
validate something
Big data?
W
ords
Pictures
Numbers
WHITEBOARDABLE
best source
of possible
metrics

Numbers

Numbers
Metrics &
Analytics
• Click tracking
• Analytics packages
• A/B testing
• KPIs / OKRs
• Usage stats
• Conversion

Numbers Common metrics
DAU Daily Active Users
MAU Monthly Active Users
CPA Cost per Acquisition
ARPDAU Average Revenue per Daily Active User
Retention How long they stay & how  
frequently they interact
Churn How many users you lose vs. keep
LTV LifeTime Value
NPS Net Promoter Score

Numbers Frameworks
The PULSE of a Pirate’s HEART

Numbers
PULSE Metrics
Page views
Uptime
Latency
Seven-day  
active users
Earnings
Frameworks

Numbers
Acquisition
Activation
Retention
Referral
Revenue
Frameworks
Happiness
Engagement
Adoption
Retention
Task success
Pirate Metrics HEART Metrics

Numbers
Product
Trafﬁc
Retention
Frameworks
WHITEBOARDABLE

Numbers
Product
Trafﬁc
Retention
use
Frameworks
WHITEBOARDABLE

Numbers
…measures the
usage of your 
product by a
person.
a good metric…
Clear &
speciﬁc
Comparative
Normalized
Actionable
Changes 
your behavior

Numbers
• Know what interactions represent
the desired user behaviors.
• Identify the speciﬁc interactions
that support the action or behavior.
• Find the numbers to track  
these interactions.
Clear & speciﬁc
a good metric…
share a task
*

Numbers
Normalized 
(rate or ratio)
• Percentages are
inherently comparative
• They balance out big
changes in total
quantities.
a good metric…
782 total users
420 share a task
1,275 share a task
6,835 total users
18.6%
53.7%

Numbers
Comparable 
(time-stamped)
• Captured at regular
intervals
• Directly comparable
over a span of time
a good metric…
WTFJHT?
6

Numbers
Actionable, not vanity
• Total number of users
• Downloads
• Page views
• Likes
• Funding press releases
• Articles in media
• etc…
a good metric…

Numbers
“ ~ Alistair Croll 
with Ben Yoskovitz
Don’t just ask questions. Know how
the answers to the questions will 
change your behavior.
a good metric…

Numbers
vanity
total number
of registered
users
unhelpful
sign-ups
% of users
who share a
task 3+  
times a day, 
per week
awesome
better
% of users
who sign in 3+
times a day, 
per week
good
% of new
users per
week
a good metric…

Numbers
• Identify and design for key interactions  
that deliver measurable value to your users.
• Have a clear set of numbers 
that indicate progress.
• Know what interactions you need to track 
to measure user behaviors.

So…
How do we get
our awesome
metric?

What does our user
need to do? Where
in the interface can
they do that?
Ask:

Features
Users
Uses
Product
Needs
Company
Purpose
sketches,
prototypes,
pixels
UI
via @luxrco
UX Stack WHITEBOARDABLE

Words
“
Your product must be a bridge
between your customers 
and their dreams.
~ Christina Wodke

Words
Meet Erika, a socially engaged college student.
Erika needs to feel closely connected with friends
near and far, every day.
using status updates on her “wall”, messages,
comments and “likes”
in a way that’s universal, clean, consistent, and fast.
With Facebook, Erika can share her latest
thoughts and see what her many friends are up to,
Facebook

Words
Meet Erika, a socially engaged college student.
Erika needs to feel closely connected with friends
near and far, every day.
using status updates on her “wall”, messages,
comments and “likes”
in a way that’s universal, clean, consistent, and fast.
With Facebook, Erika can share her latest
thoughts and see what her many friends are up to,
Key Use
Facebook

Words
Meet Mary, a working professional with two kids.
Mary needs to share the load of household tasks
with her family, and get help completing tasks.
using the task scheduler, task sharer and done
notiﬁcations.
in a way that’s friendly, clear, visual, and fast.
With Task-a-doodle, Mary can create tasks, share
them with her spouse and kids, and know when
something gets done
Task-a-doodle

Words
Meet Mary, a working professional with two kids.
Mary needs to share the load of household tasks
with her family, and get help completing tasks.
using the task scheduler, task sharer and done
notiﬁcations.
in a way that’s friendly, clear, visual, and fast.
With Task-a-doodle, Mary can create tasks, share
them with her spouse and kids, and know when
something gets done
Key Use
Task-a-doodle

What can our
customer do with our
product that they
can’t do without it?
Ask:

Words Key Use
Lead with the punch line
With _________, ________ can 
________________________.
persona
name
goal statement
product
feature
interaction

Who is our user?
Where do they go &
what do they do? What
do they think/feel/say?
Ask:

Pictures
Key Use

Pictures Key Use 6-up
Via Alexa Andrzejewski

Pictures Key Use 6-up
(Be a SCAM artist)
Speciﬁc
Clear
Accessible
Memorable

Pictures Key Use
WHITEBOARDABLE

Pictures
“
We humans often need to see something
to clearly understand it, and so those of
us with the ability to effectively convey an
experience in a visual way
possess a special and
important skill indeed.
~ Julie Zhuo

Numbers
Key
Metric

Numbers
One can be enough.
OMTM
The One Metric That Matters.
a good metric…

Numbers
= 7+ friends within 10 days of  
account creation
= 1 ﬁle in one folder
= user comes back the day after  
signing up for a game
= The third run
a good metric…

Numbers
Leading,
not lagging
= 7+ friends within 10 days of  
account creation
= 1 ﬁle in one folder
= user comes back the day after  
signing up for a game
= The third run
a good metric…

Numbers fun with funnels

Numbers
A series of steps in
a process.
via Laura Klein
fun with funnels
You can count progress
at each step and ﬁnd
friction points.

Pictures Storyboards
Via Brandon Schauer Via Christina Wodke Via https://wordsofasuperheroine.wordpress.com/tag/airbnb/

Pictures
Via Adaptive Path
Journey Maps

Numbers
Sign-ins
Create a task
Share a task
Have it completed
fun with funnels
WHITEBOARDABLE

Numbers
“ ~ Laura Klein
Once you know what needs to happen
at each step of the funnel, you can
gather real user metrics to 
ﬁnd the friction points.
fun with funnels

Numbers
Three ways to gather data: 
• Non-technical 
• Technical “by hand” 
• Instrumented with software
The process of setting up the technology to capture
the numbers that matter to you. This often means
using an analytics package or writing custom code.
Track over time
Instrumentation

Numbers Track over time
Instrumented in software and packages
http://www.businessofapps.com/big-list-app-analytics-tools/
https://blog.bufferapp.com/social-media-analytics-tools

Non-technical & technical “by hand”

Nope.

3. How will we measure it 
& track progress?
* * * *
you need a team

3. How will we measure it 
& track progress?
* * * *
you need a team
this ain’t easy

Engage the team
UX doesn’t
“own” the
numbers.
Why it’s hard
It’s challenging to
collaborate across
different business units.
We are already so
busy so so many
other things…

Engage the team Why it’s hard
“ ~ Cynnedd Bowles
Senior designers and leaders can’t withdraw
from the data discourse, but they are well
placed to question its ideological power. 
Data is a valuable adviser 
but a tyrannical master […]
Dataﬁcation and ideological blindness
https://medium.com/@cennydd/dataﬁcation-and-ideological-blindness-ca15f9e41494

Do it for yourself.
Do it with your team.
Be the change you
want to see in your
company.

1 thing you can do tomorrow
Why wait?

http://www.dear-data.com
Boondoggle

Erin Malone
Boondoggle

Let’s do something now
What’s your goal?
What does
it look
like?
How will you
measure it?
How will you
track it?
What’s your goal?
What does it 
look like?
How will you measure
and track it?
5 min

how do we know
our work is…
WORKING?
*

3 questions
1.What are we trying to accomplish?
2.What does that look like?
3.How will we measure the work  
& track progress?

Words
Pictures
Numbers

Geek out & learn more

Products that
make our
users’ dreams
become real

Questions
Rants
Discussion
Debate

Kate Rutter
@katerutter
http://intelleto.com
Thanks!
Swearing, drinking & UX…give it a listen!
http://bit.ly/whatiswrongwithux

Attributions
Intro:
• Fitbit screenshot, via @feralbeagle
• UX Stack: Janice Fraser, Luxr
Engaging the Team:
• Cennydd Bowles: https://www.cennydd.com/writing/datafication-and-ideological-
blindness
• Book cover images via Amazon: https://www.amazon.com/
Nice reads:
• Great metrics article: https://medium.com/the-year-of-the-looking-glass/metrics-
versus-experience-a9347d6b80b
• KISSmetrics blog: https://blog.kissmetrics.com/
• Joshua Porter, Bokardo slideshare: https://www.slideshare.net/andrew_null/metrics-
driven-design-by-joshua-porter/28-Saturday_March_12_2011
Words:
• Christina Wodke: http://eleganthack.com/
• Robert McKee: http://www.bang2write.com/wp-content/uploads/2014/07/mckee_worldbuilding.jpg
• Purpose statements: Ikea via http://www.ikea.com/ms/en_SG/about_ikea/our_business_idea/index.html;
Facebook via https://investor.fb.com/resources/default.aspx; Snapfood via Alexa Adnrzejweski; Luxr via
Kate Rutter
Pictures:
• Dan Roam: http://www.danroam.com/
• Sketches via : Alexa Andrzejewski, foodspotting (now OpenTable);  
Capra Glaves, Brandon Shauer, Christina Wodke, Kate Rutter
• Sketching photos via Tradecraft: http://tradecrafted.com/
• Product as Hero Storyboard: http://eleganthack.com/product-as-hero-storyboard/
• Airbnb Storyboards: https://twitter.com/aclaytonbupa; Vhttps://wordsofasuperheroine.wordpress.com/tag/
airbnb/
• Julie Zhuo: https://medium.com/the-year-of-the-looking-glass
Numbers:
• I <3 Metrics pin: https://www.zazzle.com/i+heart+metrics+gifts
• Joshua Porter: http://bokardo.com/ Quote via https://www.slideshare.net/andrew_null/metrics-driven-design-
by-joshua-porter/28-Saturday_March_12_2011
• PULSE metrics:
• Pirate metrics: Dave McClure via http://500hats.typepad.com/500blogs/2007/09/startup-metrics.html
• HEART metrics: Kerry Rodden via http://www.rodden.org/kerry/heart/
• Ancient screenshots: http://www.telegraph.co.uk/technology/6125914/How-20-popular-websites-looked-
when-they-launched.html
• OMTM citations: Lean Analytics via http://leananalyticsbook.com/
• Minard's Visualization Of Napoleon's 1812 March via https://www.edwardtufte.com/tufte/minard
• Laura Klein: https://www.usersknow.com/
• Funnel images: http://rosenfeldmedia.com/books/build-better-products/
• OpenWeb Analytics: http://www.openwebanalytics.com/?attachment_id=43
• MeasureMap: https://www.flickr.com/photos/veen/51884282/
• Packages & platforms: http://www.businessofapps.com/big-list-app-analytics-tools/
• https://blog.bufferapp.com/social-media-analytics-tools
• Feltron report: http://feltron.com/FAR14.html
• Dear Data: http://www.dear-data.com/, cover image via Amazon
• Erin Malone: A year of reading: https://www.flickr.com/photos/erinmalone/32128324745/sizes/c/

Finding the Narrative in Numbers: Making the Most of Metrics [UX Immersion 2017]

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Finding the Narrative in Numbers: Making the Most of Metrics [UX Immersion 2017]

Similar to Finding the Narrative in Numbers: Making the Most of Metrics [UX Immersion 2017] (20)

More from Kate Rutter

More from Kate Rutter (20)

Recently uploaded

Recently uploaded (20)

Finding the Narrative in Numbers: Making the Most of Metrics [UX Immersion 2017]