This presentation was given as part of an invitation to lecture at Wharton, University of Pennsylvania.

1. Growth Marketing
Science
@datasartoriasf

2. Introduction
Behavior Science +
Statistics
Statistical
Programming
Ecomm
Marketplace
Rideshare Hyper-Local
Social
Platform
B2B SaaS
Volunteer

3. Setting the Table
● Marketing Science
○ The Scientist’s Toolkit
○ The Importance of Measurement
○ Growth Metrics
● Storytelling with Experiment Design
○ A/B Testing
○ Market Matching
○ Causal Inference
● Attribution and Incrementality
● Career Development
● Resource Repo

4. “Truth emerges more readily from error than from confusion.” –Francis Bacon
● Reproducibility can reduce uncertainty and help us discover solutions to
problems.
● EDA uses pictures to listen to the data so that a likely story can be told.
The Scientist’s Toolkit

5. “More than anything, EDA is a state of mind. During the initial phases of EDA
you should feel free to investigate every idea that occurs to you. Some of these
ideas will pan out, and some will be dead ends… ” “As your exploration
continues, you will hone in on a few particularly productive areas that you’ll
eventually write up and communicate to others.”
- Hadley Wickham, R for Data Science
EDA is a state of mind

6. ggplot2: a language for building graphs
for exploration and
communication.
Tools for EDA

7. Pictures force us to notice what we did
not expect to see.
● Geometric objects to represent
measurements.
● Mapping to segment our
populations
● Statistical modeling to estimate
relationships.
Tools for EDA

8. Nextdoor Case
● EDA is a rapid,
reproducible and iterable
tool for surfacing
targeting and
optimization
opportunities.
Tools for EDA

9. Analytical frameworks define objective
and success numerically
● How will we analyze the data?
● What is our outcome of interest to
define success?
● What sample sizes are needed for
various levels of confidence in our
learnings?
The Importance of Measurement

10. Balancing growth and revenue
● Are your customers successful
in the product?
● Customer retention is a
leading indicator of success.
B2B Growth Metrics

11. Measuring Performance
● Acquisition:
○ CAC, LTV, LTV/CAC ratio.
● Retention:
○ 30 / 60 / 90 day
retention.
○ Engagement:
■ % active
■ Sessions Per Month
B2B Growth Metrics

12. Know your business model
● What is the average lifecycle
of a user?
● How do user’s generate
revenue?
● What is the distribution of
user generated revenue?
B2C Growth Metrics

13. Measuring Performance
● Acquisition:
○ CPA
○ LTV/CPA ratio
○ Payback period
○ ARPU
● Retention:
○ W4W, MAU, Impressions Per
User.
B2C Growth Metrics

14. Storytelling with Experiments
A/B Testing
● Benefits
○ Direct incrementality
■ CA: Incremental Acq
■ CPIA: Cost Per
Incremental Acq
○ High reproducibility

15. Causal Inference
● Benefits:
○ Explainable, Indirect incrementality, Moderate reproducibility
Storytelling with Experiments

16. Storytelling with Experiments
Bayesian Regression: Market
Matching
● Benefits
○ Direct incrementality
■ CA: Incremental Acq
■ CPIA: Cost Per
Incremental Acq
○ Moderate reproducibility

17. Attribution
Attribution
● Map your stakeholders:
Marketing, Product, Growth,
Engineering and Finance.
● Define needs, motivations and
objectives.
● Methodologies:
○ First touch
○ Last touch
○ MMM

18. Incrementality
Incrementality
● Questions to answer:
○ What would have happened
without your marketing?
○ What is the absolute growth that
was driven by marketing?
○ What is the relative percentage of
growth was driven by marketing?

19. Career Development
Career Development
● Identify the right organization for you.
● Make the interview your discovery phase.
● Right fitting your role.
● Gaining visibility and socializing your work.

20. Resources
Download the data science resourcing library here
Contact: Peter Spangler