SlideShare a Scribd company logo
1 of 77
Download to read offline
1
Josh Aberant
CMO
t.co/Josh
@jaberant
Being Data Driven
@jaberant 2
Bio - Josh Aberant
SparkPost CMO & Growth
Twitter Postmaster
RestEngine CoFounder
Marketo Director of Privacy
@jaberant 3
Postmaster Role @Twitter
• Growth team
Early member of formalized Growth team built around acquihires
• Mixer Labs, Summify, Fluther, RestEngine & more
• Build & optimize growth surfaces
• Work on Growth surfaces
• NUX, RUX, messaging, logged out, distribution/viral
• Provide growth services across enterprise
Data as a service
Experiments as a service
• Lots of teams needed/consumed growth services
Led to a lot of security work
Security discussion is for another day. We are hear to talk #Growth
4
#GrowthHacking
Data in Action
Being Data Driven
@jaberant 5
@jaberant 6
#GrowthHacking - Data in Action
Growth
User State
Models
Data Experiments
7
@jaberant 8
#GrowthHacking for CX
#GrowthHacking is a capable CX framework (Customer Experience)
• CX is marketing teams embracing both data & empathy for the user
• The whole breadth of user interactions
• Acquisition, Engagement, Loyalty & Advocacy
• Listening to and co-designing with the users
• #GrowthHacking is methodology for scaling empathy
• #GrowthHacking measurable & repeatable way to get at the
essence of what the user wants, enjoys & needs
9
You are all #GrowthHackers
here, even if you don’t
know it yet
@jaberant 10
The Growth Role
Blend data & deep understanding of user
• Needs, habits, perception, tastes, preferences
Answer questions on user experience. e.g.:
• Why are users dropping out of the sign up
experience?
• Why don’t users come back to the application after
the initial download?
• Why aren’t users responding to special offers?
Experiment to collect real meaningful feedback
• Innovate through experimentation
@jaberant 11
The Growth Role
Data & Experiments Infrastructure Services
• Analytics & experiments
Roll your own (work with the Data scientist & devs) or
Commercially available solutions - Google Analytics to
Optimizely
Dashboards
• State Models
Growth Plan
• Define the growth objectives
• Decide where in the tornado (funnel) to focus
e.g. User acquisition or fighting churn
@jaberant 12
Funnels Have Become Tornados
• Funnels have been blown to bits
• User’s don’t take linear paths to and
through apps or products
• They get to your app or product how they
want to, when they want to and do
whatever they want to do
• Users own the relationship
@jaberant 13
Managing Tornadoes
• Breakdown all user mechanics
• Measure & instrument
• Experiment to improve
The classic funnel stages still matter - users just don’t go
through them linearly
• Acquisition, activation, engagement, retention,
resurrection
@jaberant 14
Experimental Results = User Insights
Through experiments we use real world data that to
generate user insights that are fed into the enterprise
• Empathy
• Thinking like the users think
• Understanding essence of users environment, tasks &
goals
• Living in the world of users & enhancing their experience
This isn’t just polling & surveys - experiments enable to see
what users truly do when changes are made.
15
There’s no magic code
“There is no code. Ultimately growth is a
function of the right mix of a product that
meet important user needs and tenacious
creative data driven execution that
leverage principles of growth and often
exploits psychological triggers.”
-Sean Ellis
#GrowthHacking is
getting users hooked on
your apps & products
More
scientifically...
18
#GrowthHacking = Science
Growth Hacking is a framework to study how users
engage so as to give them a more meaningful app or
product that they'll want to use & share.
If you want to build apps or products that people are
hooked to then make growth hacking core to your
development.
@jaberant 19
Systematic & Repeatable
There’s really 3 main things #GrowthHacking does
• See how users activate & engage
• Get them to activate & engage more
• Get them to share
@jaberant 20
To Grow
Instrument & measure
• Observe & understand
Innovate through experimentation
• Try things
• Optimize things
• Throw out what doesn’t work, do what does
Be a champion of the user
21
The1%experiments
@jaberant 22
#BeingDataDriven
Life inside a data driven organization
• Anyone can do a 1% experiment
• Indeed, don’t show up to present to execs without
experimental data
• Innovate through experimentation!
23
Experiments
24
Experiments are what
make #GrowthHacking
scientific
@jaberant 25
Experiments Made Easy
@jaberant 26
Experiments Made Easy
Test Group
@jaberant 27
Experiments Made Easy
Test Group
Run the change on the test group
Leave the rest alone (they are the control)
Measure the change in the metrics you care about
@jaberant 28
Working With Experiments
#BeingDataDriven means a regular cadence of test &
reviews
• Daily standups, weekly team review, set schedule of
reviews around initiatives, backlog mgmt
Learn to have instincts point to experimental success
• Often an experiment need more time to show results
• Or it needs to be expanded beyond 1%
Make decisions on how to proceed
• Expand, reformulate, place on hold, kill
@jaberant 29
Focus on what works
Double down on what already works
• With limited resources you don’t have time to work on
everything
• It’s easier to build on a strength than to improve a
weakness
• It’s often easier to improve a conversion rate that’s
already doing well
• Easier to increase engagement of active users compared
to people who are inactive
Instrumentation &
User State Models
31
Instrument the metrics
that tell you if your users
are healthy
“Based on experience of which companies stuck with us and
which didn't, we decided that any team that has exchanged
2,000 messages in its history has tried Slack — really tried
it. For a team around 50 people that means about 10 hours’
worth of messages. For a typical team of 10 people, that’s
maybe a week’s worth of messages. But it hit us that,
regardless of any other factor, after 2,000 messages, 93% of
those customers are still using Slack today.”
- Stewart Butterfield, Slack Founder
32
Healthy User Example
@jaberant 33
Example Healthy User Metrics
Facebook
• Getting a user to 7 friends in 10 days
Dropbox
• Getting a new user to link a computer and add a file
@jaberant 34
Some Common Engagement Metrics
• Logins past 30 days
• Days since last login
• Session length
• Session interval frequency (how often user launches app)
• App screens per session
• Interactions
• Post
• Shares
• Views
@jaberant 35
Metrics That Matter
Ask
• Do metrics indicate that a user is healthy?
• Which phase of the tornado (funnel)?
Activation, activation, engagement, vitality,
resurrection
Build
• User state models
Specific to your user paths (tornado)
@jaberant 36
User State Models
User state models simplify working with a lot of metrics by
aggregating metrics them into meaningful models that can
be used to quickly make decisions on the health of an
experimental result
• Examples
Healthy vs Unhealthy
Casual or Core
37
User State Models
Growth is all about the quality
and quantity of the user base.
User state models provide a
way of measuring the quality.
@jaberant 38
New
Casual Core
At
Risk
User State Models Examples
@jaberant 39
Measuring User State Transitions Example
Example from Pinterest: Measuring flows through the Tornado
• New signup: User joins
• New -> Dormant: New users doesn’t engage for 28 days
• MAU -> Dormant: User was an MAU, but doesn’t engage for 28
days
• Dormant -> MAU: Users engages after not using app for 28 days
@jaberant 40
Days since last login
Logins
in past
30 days
Healthy Users
At risk:
focus
engagement
experiments
here
Transitioning out:
focus Resurrected User
Experience (RUX)
experiments here
User State Models Examples
@jaberant 41
RUX while you can
• User transitioning out are about to be churned users. Apply
resurrection messaging & experiments to them before they
leave while it is still much easier to reach them.
• RUX is generally a subset - the best parts of - the NUX
(New User Experience)
@jaberant 42
User State Models Example
Pyramid example (for Twitter):
• Heavy tweeter
• Heavy non-tweeter
• Medium tweeters
• Medium non-tweeters
• Light users
• Very light
• Nearly light (new users)
@jaberant 43
User State Models Example
Xd28s example (from Pinterest):
# of users Pinterest in past 28 days
• 4d28s+ = # of users that used Pinterest 4 or more days in
last 28
Major categories
• Core = 14d28s+
• Casual = 4d28s+
• Marginal = <4d28
Example provided by Pinterest Eng Blog
@jaberant 44
User State Models Consideration
There’s a lot of debate about whether “middle” user states
are meaningful
• Medium, casual
Or wether is best to always think of these states as
transitioning out
• “I’m about out of here” group
@jaberant 45
User State Models Consideration
There’s no one model that meets all experimental cases
• All models have limits
• Different models work better in different cases
Always consider whether a given model is right for the
experiments
@jaberant 46
Counter Metrics
• Don’t lose sight of the forest in the trees. There’s a risk
your user state model doesn’t apply well to the
experiment
• Best practice: Pair every metric with appropriate counter
metric
Signups with activations
Activations with churn
New paid customers with total revenue
This encourages a holistic approach to growth & will help
keep you on the right track on when your user state models
or serving you well or when you should be using different
user state models.
@jaberant 47
More Meaningful Cohorts w/ User States
Often cohorts are looked at by individual metrics
• In any given experiment the metrics might move different
metrics in different directions
• Difficult to evaluate an experiment’s results
Use user state models in your cohort analysis
• Scale decision making
• Incorporate multiple metrics
• Tells you what to do next
If the experiment makes more users healthy its a good
experiment
@jaberant 48
Standard Cohort Example
A cohort is a group of people that share something in common
Default cohort analysis often means clustering users by when they
started & seeing how behavior changes over time
• E.g. 1 week old cohorts in Feb vs March when a big app change was
made at beginning of March
Lots of great commercial solutions
• Google Analytics, Kissmetrics, Keen.io, Segment, Mixpanel
@jaberant 49
Cohorts with Users States
If the experiment makes more users healthy it’s a good
experiment
Cohort
Health
Session
length
Session
interval
App
Screes
per
session
Interactions Views Post
+ + + + + - +
- - - + + + +
- + - - - - -
+ + + - - - -
@jaberant 50
Interactive Dashboard Best Practices
Use dashboards!
• Relevant metrics
• Visually engaging
You’ve to to get everyone on the team looking at & using
these dashboards
• Intuitive default selections
• Super easy basics analytical tasks
Filtering view, adjusting parameters, drill to examine
underlying data
Allow team members to understand data and drill
down in 1-2 clicks
@jaberant 51
Interactive Dashboard Best Practices
Keep it current!! - up-to-date data
• Min, hour, week, month, quarter - whatever the right
timeline is for best managing this growth function
Label metrics meaningfully
• Meaningful to others
• Make sure metrics support the objective & the audience)
Stay away from chart junk
• Low value graphics or unintelligible widgets
@jaberant 52
Interactive Dashboard Best Practices
Alerts
• Altering on growth systems
• E.g. email volume & push volume (on per type basis.)
• In addition to core product alerting
• Need good dashboards & alerting so when alerts go off
you can figure out what went wrong
@jaberant 53
Keep Dashboards Relevant
Key metrics can shift as growth happens and challenges
change
• Maintain your dashboards
• Are they meeting the current needs of the team?
@jaberant 54
Growth Surfaces
NUX
RUX
• The best parts of NUX
Messaging
• Email, Push, app updates
Logged out/ unauthed user state
• SEO
• Signups
Deeplinking (distribution)
• App indexing
Viral mechanism
@jaberant 55
New User Experience (NUX)
Often the Growth team owns the NUX and it is a focus of
intense experimentation
• Activate Users
Quickly orient & teach users to use your app our
service
• Move users to a healthy state
E.g. Follow people, add friends, link a file
@jaberant 56
Activation in the NUX
What is the “aha” moment or “must have” experience?
• How do you get users to this point as fast as possible (i.e.
in seconds)?
• What actions should users take to put them in a healthy
state?
57
Growth comes from
delivering the core product
value to users as early and
often as possible.
@jaberant 58
Resurrected User Experience (RUX)
Often the Growth team own the RUX and it is a focus of
another focus intense experimentation
• How do you bring users back and reactivate them?
• What’s changed since the users was last there that they’ll
want to stick around longer?
• Do you have a way of reaching churned users?
Email
Notifications
App store updates
Custom channels
59
Resurrection is hard
It’s very hard to resurrect people without being
pushy. But if done right emails, notifications and app
store updates can do much for bringing users back
#KeepItClassy
@jaberant 60
Messaging
The Growth team often provides messaging capabilities
across the enterprise
• Infrastructure, Services & best practices
• Email, push, sms, in-app, app store updates, custom
channels
• Policy & controls (based on fatigue models) to prevent
different teams from over-mailing or over-pushing
Prevent user fatigue
61
App Indexing
via
Deeplinking
@jaberant 62
SEO is back
With App Indexing Google will deeplink from search results
into content in your app.
• 30+ billion deeplinks & counting
• Powerful distribution
• Think about the landing experience
• The years sent on developing best practices for landing
pages on the web...
App Indexing + Adwords = Search download ads
@jaberant 63
• Why would users want to share your app or product’s
atomic unit?
• How can users share the atomic unit?
• What are the mechanics of sharing?
• How do you reduce sharing friction?
• How do you get users to recommend your app?
Virality
@jaberant 64
Vitality - Your app’s atomic unit
What is the fundamental object that people are consuming
and/or creating?
Some examples:
• Twitter – tweet
• Soundcloud - audio track
• Instagram – photo
• Tumblr – post
• Dropbox – file
• Codecademy – lesson
65
@jaberant 66
Social Proof - Virality Best Practice
Example from Nuzzel App - nuzzel.com
@jaberant 67
Social Proof Best Practices
Pictures
People
• Connections
• Experts
• Celebrities
• Similar people
Don’t include if poor quality
68
“We bet heavily on Twitter. Even if
someone is incredibly enthusiastic about a
product, literal word of mouth will only
get to a handful of people — but if
someone tweets about us, it can be seen
by hundreds, even thousands.”
- Stewart Butterfield, Slack Founder
Share via Twitter
69
Listen
70
@jaberant 71
Listen To Your Users
App & product users will communicate
• They want the stuff they are using to get better
Give them ways to talk back to you
• Button to talk to you, in-app prompt, surveys
Find guidance on what they want
• Then experiment to confirm
Tag, ticket & study messages from users
• Be fastidious
72
“If you put that all together, we probably
get 8,000 Zendesk help tickets and 10,000
tweets per month, and we respond to all
of them.”
” - Stewart Butterfield, Slack Founder
Listen with Twitter
73
Finally a word of warning
74
Don’t Spam
@jaberant 75
Don’t Spam
Spam ruins the internet
Users don’t like it
Can easily waste so much good work
Virality doesn’t matter until you can deliver a compelling
benefit to users
• Nothing worth sharing means just that
Avoid gimmicks that drive sign-ups in the short-term but
alienate users in the long-run
76
Now go forth & #GrowthHack
77
Josh Aberant
CMO
t.co/Josh
@jaberant
Being Data Driven

More Related Content

What's hot

Bluebird University Core Competencies
Bluebird University Core CompetenciesBluebird University Core Competencies
Bluebird University Core Competencies
slamplat
 
Set up, grow and successfully engage members of your panel final
Set up, grow and successfully engage members of your panel   finalSet up, grow and successfully engage members of your panel   final
Set up, grow and successfully engage members of your panel final
QuestionPro
 
Pardot Elevate 2011 (slides): Developing Customer-centric Content (by Brainri...
Pardot Elevate 2011 (slides): Developing Customer-centric Content (by Brainri...Pardot Elevate 2011 (slides): Developing Customer-centric Content (by Brainri...
Pardot Elevate 2011 (slides): Developing Customer-centric Content (by Brainri...
Pardot
 

What's hot (20)

Case Study: The Human Factor of UNICEF'S Digital Transformation - Carolina Ra...
Case Study: The Human Factor of UNICEF'S Digital Transformation - Carolina Ra...Case Study: The Human Factor of UNICEF'S Digital Transformation - Carolina Ra...
Case Study: The Human Factor of UNICEF'S Digital Transformation - Carolina Ra...
 
Meredith MXM
Meredith MXMMeredith MXM
Meredith MXM
 
Building and Leveraging Content Frameworks and Collateral Maps
Building and Leveraging Content Frameworks and Collateral MapsBuilding and Leveraging Content Frameworks and Collateral Maps
Building and Leveraging Content Frameworks and Collateral Maps
 
LavaCon 2017 - Future-proof Your Content: Beyond Traditional Publishing for S...
LavaCon 2017 - Future-proof Your Content: Beyond Traditional Publishing for S...LavaCon 2017 - Future-proof Your Content: Beyond Traditional Publishing for S...
LavaCon 2017 - Future-proof Your Content: Beyond Traditional Publishing for S...
 
Bluebird University Core Competencies
Bluebird University Core CompetenciesBluebird University Core Competencies
Bluebird University Core Competencies
 
Planning For The Personalization Journey: From Empathy to Engagement
 Planning For The Personalization Journey: From Empathy to Engagement Planning For The Personalization Journey: From Empathy to Engagement
Planning For The Personalization Journey: From Empathy to Engagement
 
Set up, grow and successfully engage members of your panel final
Set up, grow and successfully engage members of your panel   finalSet up, grow and successfully engage members of your panel   final
Set up, grow and successfully engage members of your panel final
 
Burrows orofino unified_cx_04042016
Burrows orofino unified_cx_04042016Burrows orofino unified_cx_04042016
Burrows orofino unified_cx_04042016
 
Driving Brand Loyalty Through Superior Customer Experience
Driving Brand Loyalty Through Superior Customer Experience Driving Brand Loyalty Through Superior Customer Experience
Driving Brand Loyalty Through Superior Customer Experience
 
Realizing the Digital Technology Promise: How to operationalize for success a...
Realizing the Digital Technology Promise: How to operationalize for success a...Realizing the Digital Technology Promise: How to operationalize for success a...
Realizing the Digital Technology Promise: How to operationalize for success a...
 
Presentation for Exl Pharma Conference about Future of Search
Presentation for Exl Pharma Conference about Future of SearchPresentation for Exl Pharma Conference about Future of Search
Presentation for Exl Pharma Conference about Future of Search
 
Create a Collaborative External Community with Jive Software
Create a Collaborative External Community with Jive SoftwareCreate a Collaborative External Community with Jive Software
Create a Collaborative External Community with Jive Software
 
Website Redesign and CMS Migrations - 2013 AMP Annual Meeting
Website Redesign and CMS Migrations - 2013 AMP Annual MeetingWebsite Redesign and CMS Migrations - 2013 AMP Annual Meeting
Website Redesign and CMS Migrations - 2013 AMP Annual Meeting
 
Google Optimize: How Mass.gov Builds Great Government UX
Google Optimize: How Mass.gov Builds Great Government UXGoogle Optimize: How Mass.gov Builds Great Government UX
Google Optimize: How Mass.gov Builds Great Government UX
 
Pardot Elevate 2011 (slides): Developing Customer-centric Content (by Brainri...
Pardot Elevate 2011 (slides): Developing Customer-centric Content (by Brainri...Pardot Elevate 2011 (slides): Developing Customer-centric Content (by Brainri...
Pardot Elevate 2011 (slides): Developing Customer-centric Content (by Brainri...
 
How to Build a Social Online Research Panel
How to Build a Social Online Research PanelHow to Build a Social Online Research Panel
How to Build a Social Online Research Panel
 
Driving purchase decisions in the social media era
Driving purchase decisions in the social media eraDriving purchase decisions in the social media era
Driving purchase decisions in the social media era
 
Contently & Airportels
Contently & AirportelsContently & Airportels
Contently & Airportels
 
Toward a Unified Customer Experience
Toward a Unified Customer ExperienceToward a Unified Customer Experience
Toward a Unified Customer Experience
 
Search engine optimization
Search engine optimizationSearch engine optimization
Search engine optimization
 

Similar to Josh Aberant - Data-Driven Digital Growth

Similar to Josh Aberant - Data-Driven Digital Growth (20)

Validating hypotheses with user research
Validating hypotheses with user researchValidating hypotheses with user research
Validating hypotheses with user research
 
How To Drive Data Driven Change In A Legacy Organization
How To Drive Data Driven Change In A Legacy OrganizationHow To Drive Data Driven Change In A Legacy Organization
How To Drive Data Driven Change In A Legacy Organization
 
10 Ways to Prove the Business Value of Social
10 Ways to Prove the Business Value of Social10 Ways to Prove the Business Value of Social
10 Ways to Prove the Business Value of Social
 
CLA Outcomes 2015 Conference - Digital Marketing Master Class
CLA Outcomes 2015 Conference - Digital Marketing Master ClassCLA Outcomes 2015 Conference - Digital Marketing Master Class
CLA Outcomes 2015 Conference - Digital Marketing Master Class
 
Intro to Product Management
Intro to Product Management Intro to Product Management
Intro to Product Management
 
How to Build Winning Products by Microsoft Sr. Product Manager
How to Build Winning Products by Microsoft Sr. Product ManagerHow to Build Winning Products by Microsoft Sr. Product Manager
How to Build Winning Products by Microsoft Sr. Product Manager
 
UX Field Research Toolkit - Updated for Big Design 2018
UX Field Research Toolkit - Updated for Big Design 2018UX Field Research Toolkit - Updated for Big Design 2018
UX Field Research Toolkit - Updated for Big Design 2018
 
6 Pillars of Proving the Business Value of Social - CollabCon15.pptx
6 Pillars of Proving the Business Value of Social - CollabCon15.pptx6 Pillars of Proving the Business Value of Social - CollabCon15.pptx
6 Pillars of Proving the Business Value of Social - CollabCon15.pptx
 
Optimising Your Content for findability
Optimising Your Content for findabilityOptimising Your Content for findability
Optimising Your Content for findability
 
Simple Ways of Planning, Designing and Testing Usability of a Software Produc...
Simple Ways of Planning, Designing and Testing Usability of a Software Produc...Simple Ways of Planning, Designing and Testing Usability of a Software Produc...
Simple Ways of Planning, Designing and Testing Usability of a Software Produc...
 
How to use your data science team: Becoming a data-driven organization
How to use your data science team: Becoming a data-driven organizationHow to use your data science team: Becoming a data-driven organization
How to use your data science team: Becoming a data-driven organization
 
Building innovative products
Building innovative productsBuilding innovative products
Building innovative products
 
Building innovative products
Building innovative productsBuilding innovative products
Building innovative products
 
Optimizing Dev Portals with Analytics and Feedback
Optimizing Dev Portals with Analytics and FeedbackOptimizing Dev Portals with Analytics and Feedback
Optimizing Dev Portals with Analytics and Feedback
 
Agile metrics at-pmi bangalore
Agile metrics at-pmi bangaloreAgile metrics at-pmi bangalore
Agile metrics at-pmi bangalore
 
BuckeyeSPUG Triggering Cultural Metamorphisis wide_final.pptx
BuckeyeSPUG Triggering Cultural Metamorphisis wide_final.pptxBuckeyeSPUG Triggering Cultural Metamorphisis wide_final.pptx
BuckeyeSPUG Triggering Cultural Metamorphisis wide_final.pptx
 
Designing Change for Sustained User Adoption
Designing Change for Sustained User AdoptionDesigning Change for Sustained User Adoption
Designing Change for Sustained User Adoption
 
Enterprise digital transformation
Enterprise digital transformationEnterprise digital transformation
Enterprise digital transformation
 
Lean analytics
Lean analyticsLean analytics
Lean analytics
 
Digital Learning Sample Metrics
Digital Learning Sample MetricsDigital Learning Sample Metrics
Digital Learning Sample Metrics
 

More from Digital Experience (DX) Summit 2016

More from Digital Experience (DX) Summit 2016 (20)

Tony Byrne - Digital: Keeping It "Real"
Tony Byrne - Digital: Keeping It "Real"Tony Byrne - Digital: Keeping It "Real"
Tony Byrne - Digital: Keeping It "Real"
 
Alan Porter - Redefine the Customer Journey
Alan Porter - Redefine the Customer JourneyAlan Porter - Redefine the Customer Journey
Alan Porter - Redefine the Customer Journey
 
Laurel Fitts - The Power of Mixed Reality
Laurel Fitts - The Power of Mixed RealityLaurel Fitts - The Power of Mixed Reality
Laurel Fitts - The Power of Mixed Reality
 
Jorge Trevino - Marketing Intelligence
Jorge Trevino - Marketing IntelligenceJorge Trevino - Marketing Intelligence
Jorge Trevino - Marketing Intelligence
 
Jen Adamski-Torres - Empowering Retailers: Eliminating the Gap Between Physic...
Jen Adamski-Torres - Empowering Retailers: Eliminating the Gap Between Physic...Jen Adamski-Torres - Empowering Retailers: Eliminating the Gap Between Physic...
Jen Adamski-Torres - Empowering Retailers: Eliminating the Gap Between Physic...
 
Cleve Gibbon - Design Principles to Deliver Your Own Customer Experience Plat...
Cleve Gibbon - Design Principles to Deliver Your Own Customer Experience Plat...Cleve Gibbon - Design Principles to Deliver Your Own Customer Experience Plat...
Cleve Gibbon - Design Principles to Deliver Your Own Customer Experience Plat...
 
Pierre Debois - Using Analytics for Sustainable Customer Experiences
Pierre Debois - Using Analytics for Sustainable Customer ExperiencesPierre Debois - Using Analytics for Sustainable Customer Experiences
Pierre Debois - Using Analytics for Sustainable Customer Experiences
 
Deb Lavoy - How Powerful Narratives Drive Great DX
Deb Lavoy - How Powerful Narratives Drive Great DXDeb Lavoy - How Powerful Narratives Drive Great DX
Deb Lavoy - How Powerful Narratives Drive Great DX
 
Steve Walker & Seth Earley - Understanding the DX Ecosystem & Developing a Ma...
Steve Walker & Seth Earley - Understanding the DX Ecosystem & Developing a Ma...Steve Walker & Seth Earley - Understanding the DX Ecosystem & Developing a Ma...
Steve Walker & Seth Earley - Understanding the DX Ecosystem & Developing a Ma...
 
Fabio Matsui - Virtual Reality and Augmented Reality Applications for Immersi...
Fabio Matsui - Virtual Reality and Augmented Reality Applications for Immersi...Fabio Matsui - Virtual Reality and Augmented Reality Applications for Immersi...
Fabio Matsui - Virtual Reality and Augmented Reality Applications for Immersi...
 
Brice Dunwoodie - DX Summit Welcome
Brice Dunwoodie - DX Summit WelcomeBrice Dunwoodie - DX Summit Welcome
Brice Dunwoodie - DX Summit Welcome
 
Gerry McGovern - Measuring the Customer Experience with Top Tasks
Gerry McGovern - Measuring the Customer Experience with Top TasksGerry McGovern - Measuring the Customer Experience with Top Tasks
Gerry McGovern - Measuring the Customer Experience with Top Tasks
 
Jeff Litvak - The Microsoft.com Global Operating Model: A Home Page Case Study
Jeff Litvak - The Microsoft.com Global Operating Model: A Home Page Case StudyJeff Litvak - The Microsoft.com Global Operating Model: A Home Page Case Study
Jeff Litvak - The Microsoft.com Global Operating Model: A Home Page Case Study
 
Meghan Walsh - From the Trenches: The Value of Transforming Roles and Technology
Meghan Walsh - From the Trenches: The Value of Transforming Roles and TechnologyMeghan Walsh - From the Trenches: The Value of Transforming Roles and Technology
Meghan Walsh - From the Trenches: The Value of Transforming Roles and Technology
 
Arjan van Rooijen & Arjen van den Akker - The Headless CMS and Website Deliv...
Arjan van Rooijen &  Arjen van den Akker - The Headless CMS and Website Deliv...Arjan van Rooijen &  Arjen van den Akker - The Headless CMS and Website Deliv...
Arjan van Rooijen & Arjen van den Akker - The Headless CMS and Website Deliv...
 
Sheryl Kingstone - How to Create Insight-Driven Customer Experiences
Sheryl Kingstone - How to Create Insight-Driven Customer ExperiencesSheryl Kingstone - How to Create Insight-Driven Customer Experiences
Sheryl Kingstone - How to Create Insight-Driven Customer Experiences
 
Scott Brinker - Hacking Marketing: The Amazing Convergence of Marketing and S...
Scott Brinker - Hacking Marketing: The Amazing Convergence of Marketing and S...Scott Brinker - Hacking Marketing: The Amazing Convergence of Marketing and S...
Scott Brinker - Hacking Marketing: The Amazing Convergence of Marketing and S...
 
Gerry McGovern - Measuring Customer Experience
Gerry McGovern - Measuring Customer ExperienceGerry McGovern - Measuring Customer Experience
Gerry McGovern - Measuring Customer Experience
 
Kristina Podnar - Delivering Digital Excellence with Global Compliance and In...
Kristina Podnar - Delivering Digital Excellence with Global Compliance and In...Kristina Podnar - Delivering Digital Excellence with Global Compliance and In...
Kristina Podnar - Delivering Digital Excellence with Global Compliance and In...
 
Vab Dwivedi - Real World E-Commerce and Customer Experience Optimization Prac...
Vab Dwivedi - Real World E-Commerce and Customer Experience Optimization Prac...Vab Dwivedi - Real World E-Commerce and Customer Experience Optimization Prac...
Vab Dwivedi - Real World E-Commerce and Customer Experience Optimization Prac...
 

Recently uploaded

Microsoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdfMicrosoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdf
Overkill Security
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
FIDO Alliance
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
Wonjun Hwang
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
FIDO Alliance
 

Recently uploaded (20)

Microsoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdfMicrosoft BitLocker Bypass Attack Method.pdf
Microsoft BitLocker Bypass Attack Method.pdf
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptx
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch Tuesday
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 
JohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptx
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 

Josh Aberant - Data-Driven Digital Growth

  • 2. @jaberant 2 Bio - Josh Aberant SparkPost CMO & Growth Twitter Postmaster RestEngine CoFounder Marketo Director of Privacy
  • 3. @jaberant 3 Postmaster Role @Twitter • Growth team Early member of formalized Growth team built around acquihires • Mixer Labs, Summify, Fluther, RestEngine & more • Build & optimize growth surfaces • Work on Growth surfaces • NUX, RUX, messaging, logged out, distribution/viral • Provide growth services across enterprise Data as a service Experiments as a service • Lots of teams needed/consumed growth services Led to a lot of security work Security discussion is for another day. We are hear to talk #Growth
  • 6. @jaberant 6 #GrowthHacking - Data in Action Growth User State Models Data Experiments
  • 7. 7
  • 8. @jaberant 8 #GrowthHacking for CX #GrowthHacking is a capable CX framework (Customer Experience) • CX is marketing teams embracing both data & empathy for the user • The whole breadth of user interactions • Acquisition, Engagement, Loyalty & Advocacy • Listening to and co-designing with the users • #GrowthHacking is methodology for scaling empathy • #GrowthHacking measurable & repeatable way to get at the essence of what the user wants, enjoys & needs
  • 9. 9 You are all #GrowthHackers here, even if you don’t know it yet
  • 10. @jaberant 10 The Growth Role Blend data & deep understanding of user • Needs, habits, perception, tastes, preferences Answer questions on user experience. e.g.: • Why are users dropping out of the sign up experience? • Why don’t users come back to the application after the initial download? • Why aren’t users responding to special offers? Experiment to collect real meaningful feedback • Innovate through experimentation
  • 11. @jaberant 11 The Growth Role Data & Experiments Infrastructure Services • Analytics & experiments Roll your own (work with the Data scientist & devs) or Commercially available solutions - Google Analytics to Optimizely Dashboards • State Models Growth Plan • Define the growth objectives • Decide where in the tornado (funnel) to focus e.g. User acquisition or fighting churn
  • 12. @jaberant 12 Funnels Have Become Tornados • Funnels have been blown to bits • User’s don’t take linear paths to and through apps or products • They get to your app or product how they want to, when they want to and do whatever they want to do • Users own the relationship
  • 13. @jaberant 13 Managing Tornadoes • Breakdown all user mechanics • Measure & instrument • Experiment to improve The classic funnel stages still matter - users just don’t go through them linearly • Acquisition, activation, engagement, retention, resurrection
  • 14. @jaberant 14 Experimental Results = User Insights Through experiments we use real world data that to generate user insights that are fed into the enterprise • Empathy • Thinking like the users think • Understanding essence of users environment, tasks & goals • Living in the world of users & enhancing their experience This isn’t just polling & surveys - experiments enable to see what users truly do when changes are made.
  • 15. 15 There’s no magic code “There is no code. Ultimately growth is a function of the right mix of a product that meet important user needs and tenacious creative data driven execution that leverage principles of growth and often exploits psychological triggers.” -Sean Ellis
  • 16. #GrowthHacking is getting users hooked on your apps & products
  • 18. 18 #GrowthHacking = Science Growth Hacking is a framework to study how users engage so as to give them a more meaningful app or product that they'll want to use & share. If you want to build apps or products that people are hooked to then make growth hacking core to your development.
  • 19. @jaberant 19 Systematic & Repeatable There’s really 3 main things #GrowthHacking does • See how users activate & engage • Get them to activate & engage more • Get them to share
  • 20. @jaberant 20 To Grow Instrument & measure • Observe & understand Innovate through experimentation • Try things • Optimize things • Throw out what doesn’t work, do what does Be a champion of the user
  • 22. @jaberant 22 #BeingDataDriven Life inside a data driven organization • Anyone can do a 1% experiment • Indeed, don’t show up to present to execs without experimental data • Innovate through experimentation!
  • 24. 24 Experiments are what make #GrowthHacking scientific
  • 26. @jaberant 26 Experiments Made Easy Test Group
  • 27. @jaberant 27 Experiments Made Easy Test Group Run the change on the test group Leave the rest alone (they are the control) Measure the change in the metrics you care about
  • 28. @jaberant 28 Working With Experiments #BeingDataDriven means a regular cadence of test & reviews • Daily standups, weekly team review, set schedule of reviews around initiatives, backlog mgmt Learn to have instincts point to experimental success • Often an experiment need more time to show results • Or it needs to be expanded beyond 1% Make decisions on how to proceed • Expand, reformulate, place on hold, kill
  • 29. @jaberant 29 Focus on what works Double down on what already works • With limited resources you don’t have time to work on everything • It’s easier to build on a strength than to improve a weakness • It’s often easier to improve a conversion rate that’s already doing well • Easier to increase engagement of active users compared to people who are inactive
  • 31. 31 Instrument the metrics that tell you if your users are healthy
  • 32. “Based on experience of which companies stuck with us and which didn't, we decided that any team that has exchanged 2,000 messages in its history has tried Slack — really tried it. For a team around 50 people that means about 10 hours’ worth of messages. For a typical team of 10 people, that’s maybe a week’s worth of messages. But it hit us that, regardless of any other factor, after 2,000 messages, 93% of those customers are still using Slack today.” - Stewart Butterfield, Slack Founder 32 Healthy User Example
  • 33. @jaberant 33 Example Healthy User Metrics Facebook • Getting a user to 7 friends in 10 days Dropbox • Getting a new user to link a computer and add a file
  • 34. @jaberant 34 Some Common Engagement Metrics • Logins past 30 days • Days since last login • Session length • Session interval frequency (how often user launches app) • App screens per session • Interactions • Post • Shares • Views
  • 35. @jaberant 35 Metrics That Matter Ask • Do metrics indicate that a user is healthy? • Which phase of the tornado (funnel)? Activation, activation, engagement, vitality, resurrection Build • User state models Specific to your user paths (tornado)
  • 36. @jaberant 36 User State Models User state models simplify working with a lot of metrics by aggregating metrics them into meaningful models that can be used to quickly make decisions on the health of an experimental result • Examples Healthy vs Unhealthy Casual or Core
  • 37. 37 User State Models Growth is all about the quality and quantity of the user base. User state models provide a way of measuring the quality.
  • 39. @jaberant 39 Measuring User State Transitions Example Example from Pinterest: Measuring flows through the Tornado • New signup: User joins • New -> Dormant: New users doesn’t engage for 28 days • MAU -> Dormant: User was an MAU, but doesn’t engage for 28 days • Dormant -> MAU: Users engages after not using app for 28 days
  • 40. @jaberant 40 Days since last login Logins in past 30 days Healthy Users At risk: focus engagement experiments here Transitioning out: focus Resurrected User Experience (RUX) experiments here User State Models Examples
  • 41. @jaberant 41 RUX while you can • User transitioning out are about to be churned users. Apply resurrection messaging & experiments to them before they leave while it is still much easier to reach them. • RUX is generally a subset - the best parts of - the NUX (New User Experience)
  • 42. @jaberant 42 User State Models Example Pyramid example (for Twitter): • Heavy tweeter • Heavy non-tweeter • Medium tweeters • Medium non-tweeters • Light users • Very light • Nearly light (new users)
  • 43. @jaberant 43 User State Models Example Xd28s example (from Pinterest): # of users Pinterest in past 28 days • 4d28s+ = # of users that used Pinterest 4 or more days in last 28 Major categories • Core = 14d28s+ • Casual = 4d28s+ • Marginal = <4d28 Example provided by Pinterest Eng Blog
  • 44. @jaberant 44 User State Models Consideration There’s a lot of debate about whether “middle” user states are meaningful • Medium, casual Or wether is best to always think of these states as transitioning out • “I’m about out of here” group
  • 45. @jaberant 45 User State Models Consideration There’s no one model that meets all experimental cases • All models have limits • Different models work better in different cases Always consider whether a given model is right for the experiments
  • 46. @jaberant 46 Counter Metrics • Don’t lose sight of the forest in the trees. There’s a risk your user state model doesn’t apply well to the experiment • Best practice: Pair every metric with appropriate counter metric Signups with activations Activations with churn New paid customers with total revenue This encourages a holistic approach to growth & will help keep you on the right track on when your user state models or serving you well or when you should be using different user state models.
  • 47. @jaberant 47 More Meaningful Cohorts w/ User States Often cohorts are looked at by individual metrics • In any given experiment the metrics might move different metrics in different directions • Difficult to evaluate an experiment’s results Use user state models in your cohort analysis • Scale decision making • Incorporate multiple metrics • Tells you what to do next If the experiment makes more users healthy its a good experiment
  • 48. @jaberant 48 Standard Cohort Example A cohort is a group of people that share something in common Default cohort analysis often means clustering users by when they started & seeing how behavior changes over time • E.g. 1 week old cohorts in Feb vs March when a big app change was made at beginning of March Lots of great commercial solutions • Google Analytics, Kissmetrics, Keen.io, Segment, Mixpanel
  • 49. @jaberant 49 Cohorts with Users States If the experiment makes more users healthy it’s a good experiment Cohort Health Session length Session interval App Screes per session Interactions Views Post + + + + + - + - - - + + + + - + - - - - - + + + - - - -
  • 50. @jaberant 50 Interactive Dashboard Best Practices Use dashboards! • Relevant metrics • Visually engaging You’ve to to get everyone on the team looking at & using these dashboards • Intuitive default selections • Super easy basics analytical tasks Filtering view, adjusting parameters, drill to examine underlying data Allow team members to understand data and drill down in 1-2 clicks
  • 51. @jaberant 51 Interactive Dashboard Best Practices Keep it current!! - up-to-date data • Min, hour, week, month, quarter - whatever the right timeline is for best managing this growth function Label metrics meaningfully • Meaningful to others • Make sure metrics support the objective & the audience) Stay away from chart junk • Low value graphics or unintelligible widgets
  • 52. @jaberant 52 Interactive Dashboard Best Practices Alerts • Altering on growth systems • E.g. email volume & push volume (on per type basis.) • In addition to core product alerting • Need good dashboards & alerting so when alerts go off you can figure out what went wrong
  • 53. @jaberant 53 Keep Dashboards Relevant Key metrics can shift as growth happens and challenges change • Maintain your dashboards • Are they meeting the current needs of the team?
  • 54. @jaberant 54 Growth Surfaces NUX RUX • The best parts of NUX Messaging • Email, Push, app updates Logged out/ unauthed user state • SEO • Signups Deeplinking (distribution) • App indexing Viral mechanism
  • 55. @jaberant 55 New User Experience (NUX) Often the Growth team owns the NUX and it is a focus of intense experimentation • Activate Users Quickly orient & teach users to use your app our service • Move users to a healthy state E.g. Follow people, add friends, link a file
  • 56. @jaberant 56 Activation in the NUX What is the “aha” moment or “must have” experience? • How do you get users to this point as fast as possible (i.e. in seconds)? • What actions should users take to put them in a healthy state?
  • 57. 57 Growth comes from delivering the core product value to users as early and often as possible.
  • 58. @jaberant 58 Resurrected User Experience (RUX) Often the Growth team own the RUX and it is a focus of another focus intense experimentation • How do you bring users back and reactivate them? • What’s changed since the users was last there that they’ll want to stick around longer? • Do you have a way of reaching churned users? Email Notifications App store updates Custom channels
  • 59. 59 Resurrection is hard It’s very hard to resurrect people without being pushy. But if done right emails, notifications and app store updates can do much for bringing users back #KeepItClassy
  • 60. @jaberant 60 Messaging The Growth team often provides messaging capabilities across the enterprise • Infrastructure, Services & best practices • Email, push, sms, in-app, app store updates, custom channels • Policy & controls (based on fatigue models) to prevent different teams from over-mailing or over-pushing Prevent user fatigue
  • 62. @jaberant 62 SEO is back With App Indexing Google will deeplink from search results into content in your app. • 30+ billion deeplinks & counting • Powerful distribution • Think about the landing experience • The years sent on developing best practices for landing pages on the web... App Indexing + Adwords = Search download ads
  • 63. @jaberant 63 • Why would users want to share your app or product’s atomic unit? • How can users share the atomic unit? • What are the mechanics of sharing? • How do you reduce sharing friction? • How do you get users to recommend your app? Virality
  • 64. @jaberant 64 Vitality - Your app’s atomic unit What is the fundamental object that people are consuming and/or creating? Some examples: • Twitter – tweet • Soundcloud - audio track • Instagram – photo • Tumblr – post • Dropbox – file • Codecademy – lesson
  • 65. 65
  • 66. @jaberant 66 Social Proof - Virality Best Practice Example from Nuzzel App - nuzzel.com
  • 67. @jaberant 67 Social Proof Best Practices Pictures People • Connections • Experts • Celebrities • Similar people Don’t include if poor quality
  • 68. 68 “We bet heavily on Twitter. Even if someone is incredibly enthusiastic about a product, literal word of mouth will only get to a handful of people — but if someone tweets about us, it can be seen by hundreds, even thousands.” - Stewart Butterfield, Slack Founder Share via Twitter
  • 70. 70
  • 71. @jaberant 71 Listen To Your Users App & product users will communicate • They want the stuff they are using to get better Give them ways to talk back to you • Button to talk to you, in-app prompt, surveys Find guidance on what they want • Then experiment to confirm Tag, ticket & study messages from users • Be fastidious
  • 72. 72 “If you put that all together, we probably get 8,000 Zendesk help tickets and 10,000 tweets per month, and we respond to all of them.” ” - Stewart Butterfield, Slack Founder Listen with Twitter
  • 73. 73 Finally a word of warning
  • 75. @jaberant 75 Don’t Spam Spam ruins the internet Users don’t like it Can easily waste so much good work Virality doesn’t matter until you can deliver a compelling benefit to users • Nothing worth sharing means just that Avoid gimmicks that drive sign-ups in the short-term but alienate users in the long-run
  • 76. 76 Now go forth & #GrowthHack