- Why customer analytics is complex now?
- One metric answers all the question
- Predictive customer lifetime value prediction
- Campaign analytics and DiD methods
1. 1 David Huang, Senior Data Scientist at Migo
Customer Analytics
Using Data Science to Acquire Valuable Users and Hook
Them for The Long Haul
Data Science Meetup Taipei
CONFIDENTIAL AND PROPRIETARY
Any use of this material without specific permission of
David’s Perspective is strictly prohibited.
2. 2
David Huang
FB: ⼤大⿐鼻觀點 - 統計與資料科學
Hello! I’m David
• MSc. In Statistics, NTU
• BSc. In Quantitative Finance, NTHU
• Summer Student, Academia Sinica
• Summer Researcher, Peking University
Academics
• Senior Data Scientist, Migo
• Consultant, Mastercard Data & Services
• Business Consultant, APT
• Data Scientist, InrayTek Corporation
Experience
Activity
• R and Text Mining Course
https://hahow.in/cr/rtextmining
• David’s Perspective
https://taweihuang.hpd.io
3. 3
Multiple Touchpoints Make Customer Journey Complex
Example: Cross-border Credit Card Holder (1)
Frame the Dream
Holiday Trip in
United Kingdom
Search on Internet
Browse Facebook,
Blogs, Ptt, etc.
Select Tickets
Skyscanner, Travel
Agency, Airline
Website, etc.
Select Hotels Go Abroad
Restaurant
Book living places
on Agoda or Airbnb
Customer Acquisition
• Is this customer valuable to acquire?
• Which hook is most effective to acquire
this customer?
Customer Usage & Habit Formation
• Which merchant should we partner with to activate the first
credit card transaction? Which offer should we send?
• How do we incentivize customers to put their card on files?
Uber /
Grab
Shopping
4. 4
Advocacy
Share reviews with
friends / online
Continuous Usage Go Abroad II
Visit United States
after 4 months
Online Shopping
Domestic
Purchase
Customer Churn Resurrection
Win back the
customer
Choose another
credit card
Customer Loyalty
• How to make customers sticky to our products & services?
• How to push customers to advocate our products?
• How to leverage referral programs to acquire customers?
Customer Retention
• How can we prevent customer churn
for risky customers?
• Is our resurrection program effective?
Multiple Touchpoints Make Customer Journey Complex
Example: Cross-border Credit Card Holder (2)
5. 5
CustomerAnalytics Framework
One Metric GivesAllYou Need to Know
Lifetime Value (LTV)
Acquisition Cost (CAC)
Predictive
=
Prescriptive
• (Acquisition) Which
customers should we prioritize
to acquire? How much should
we invest in them?
• (Revenue) Which types of campaigns can increase the spend? Which
can maximize the ROI?
• (Retention) How do we activate dormant customers and constantly
drive the retention probability?
6. 6
Predictive Customer Lifetime Value
3 Target Metrics for Lifetime Value Prediction
Typical Questions:
• Which channels are easier for us to acquire high value customers?
• When will large sign-up or referral bonuses pay back?
DormantAcquire
DormantAcquire
DormantAcquire
Acquire
Lifetime
Prediction
Spend
Prediction
Frequency
Prediction
7. 7
Predictive Customer Lifetime Value
Why Predictive CLV Is Important?
40%
55%
70%
85%
100%
M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12
Organic
Referral
Digital Ad
Below-the-line
Survival
Probability
$0.0
$2.3
$4.5
$6.8
$9.0
M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12
Organic
Referral
Digital Ad
Below-the-line
Spend per
Active Month
Active Share
0%
25%
50%
75%
100%
M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12
Organic
Referral
Digital Ad
Below-the-line
Referral program seems to attract customers with
long lifetime potential!
(How about the drop in Month 3?)
Referral programs seems to attract many cherry
pickers for the bonus. It’s bad!
(Really? We only have 3-month data!)
Referral programs seems to have higher active
share! Sounds promising!
(But they spend so less in Month 2 & 3!)
8. 8
Predictive Customer Lifetime Value
Decompose CLTV into Metrics withAction Steps
DormantAcquire
t = 1 t = 2 t = 15 t = T
Spend2Spend1 Spend15
Modeling Strategy
• Assume customer lifetime (T ) is a stopping time (i.e. determined by past and current information)
• By Wald’s equation, we have the following formula:
• Hence, we transform the global prediction problem into a single period prediction problem:
Survival Model
Classification Regression
9. 9
Predictive Customer Lifetime Value
Model Training Strategy for Predictive CLTV
Demographic
Feature
Product
Preference
Brand Loyalty
Root Cause
Acquisition
Touchpoint
Source of
Difference
Observable Customer Attributes
Early Purchase Behavior
Customer Adaptation
Customer Month Acquisition M1 Spend …
1001 2018-09 Digital Ad $3.50
1002 2018-09 Below-the-line $2.70
1003 2018-09 Organic $5.90
1004 2018-10 Referral $12.60
1005 2018-10 Referral $9.80
1006 2018-10 Below-the-line $5.30
1007 2018-11 Organic $2.10
Cohort as
Training Set
Validation
Strategy
0%
25%
50%
75%
100%
M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12
Predict Real
10. 10
CampaignAnalytics
Challenge for CampaignAnalytics
Cause-Effect Analysis
Targeted customers for a campaign are
different than average customers
• How to select similar test & control
customers who are representative of the
target roll-out pool?
Target on Right Customers
Multiple versions of campaigns were run
simultaneously on multiple segments
Cashback
Double Points
Coupons
• How do we target on customers who will
respond? How do we make sure that they
respond profitably?
11. 11
CampaignAnalytics
Select Similar Test & Control Customers
Define Population
• Clearly define the
target roll-out pops
Identify Drivers
• Identify customer
attributes that may
cause bias
Test & Control
• Test & control shave
the same distribution
on key drivers
Campaign Target
People who has at least
one transaction on our
website in last 30 days
Define Metrics
• Define objectives and
key performance
metrics for the test
Objective Metric
• Monthly spend
• Transaction count
• Average ticket size
Driver for Success
• Transaction frequency
• Favorite categories
• Gender / Occupation
Liner / Logistic
Regression
Assess Distribution
Avoid the selection bias
by assessing distribution
of key drivers
Kolmogorov–
Smirnov Test
13. 13
CampaignAnalytics
Targeted Roll-out Decision by Prediction Models
Response
Rate
Incremental
Spend
Campaign
Cost
Natural
Active Rate
Target
Propensity Model
Heterogenous Treatment Effect Model
14. 14
CampaignAnalytics
Incremental Spend - Heterogeneous Treatment Effect Model
Homogeneous relationship
between test entity and
similar control entities
Spend Model
Incremental spend
(heterogeneous treatment
effect) for test entity i
Average
treatment effect
Incremental spend depends
on customer attributes
Effect Model
15. 15
Closing Remark
One Metric GivesAllYou Need to Know
Lifetime Value (LTV)
Acquisition Cost (CAC)
=
Predictive Prescriptive
• (Acquisition) Which
customers should we prioritize
to acquire? How much should
we invest in them?
• (Revenue) Which types of campaigns can increase the spend? Which
can maximize the ROI?
• (Retention) How do we activate dormant customers and constantly
drive the retention probability?
16. 16
Migo’s Vision
Bridge the Digital Divide for Next Billion Users
Our product is an internet-independent, node network of neighborhood download stations that cache
thousands of hours of content and provide content-to-go to customers via fast and easy WiFi downloads
17. 17
Join Migo asAn Imagineer
Our Data Team is Hiring!
Senior Data Scientist Senior Data Engineer
• 3+ years related work experience
and Master’s degree
• SQL/NoSQL, Python / R, etc.
• Experience in customer analytics
or recommendation system
• 5+ years related work experience
• Python / SQL / Distributed System
• Strong data architecture, data
modeling, schema design, and
project management skills
