Customer Intelligence:
A Machine Learning
Approach
Ilya Katsov
Head of Practice, Industrial AI
Grid Dynamics
SEATTLE
MAY 21 2019
ML-based Decision Automation in Marketing Operations
● Billions of micro-decisions in real-time: who, when, how, what, ...
● Complex environment: human behavior, complex business models, hidden factors
● Many building blocks: propensity scoring, recommendation algorithms, multi-armed bandits, etc.
● How to design a system that can make micro-decisions based on business objectives?
Case Study: Environment
Retailer 1 Manufacturer 1
Manufacturer NRetailer M
purchases, clicks, loyalty IDs
...
...
Promotion targeting
system
● Drive traffic
● Improve loyalty
● Increase market share
● Acquire/grow/retain clients
● Improve loyalty
Case Study: Decisions to be Automated
● Targeting – who
○ Exploits variability in tastes, price sensitivity, propensity to buy
○ Optimize short-term or long-term outcomes
● Timing – when
○ Exploits variability in price sensitivity
○ Exploits individual purchasing cycles
● Outreach/budgeting – how many
○ Exploits variability in propensity
● Promotion properties – what
○ Aggregated view on a promotion calendar
Approach
Retailers
Brands
Product
• Willingness to pay
• Stages of journey
• Affinities to brands
• Affinities to channels
Predictive Models
(Digital Twins)
• Propensity
• Life-time
value
• Demand
Economic Models
• What-if analysis
• Optimization
• Opportunity
finding
• Business
objectives
• Constraints
Controls
• Offers
• Channels
• Messages
• Prices
Signals Decisions
Targeting and Timing
Models
7
Incremental revenue
Acquisition Maximization Retention
time
New Cardholder
$/brand
current non-buyers
+
high propensity to buy new product
current buyers
+
high propensity to buy more
current buyers
+
high propensity to buy less
Product Trial
Replenishment
Category Stretch
Retention Alarm
Com
petitive Defence
Look Alike Modeling and Survival Analysis
8
Look Alike Modeling and Survival Analysis
time
no purchase
Model training
Model scoring
purchase
no purchase
behavioral history outcome
Customer
profiles for
training
Customer
profile to be
scored
score
9
Look Alike Modeling and Survival Analysis: Target Metric Design
behavioral history outcome
Unconditional propensity:
Expected LTV:
click/purchase/CTR
3-month spend
Response/value uplift:
Challenges with Basic Propensity Scoring
10
Checking
Account
Credit
Card
Brokerage
Account
Banking /
Telecom
Customer maturity
Product maturity level
Retail
● Does not take into account
product sequences
● Does not optimize offer
sequences (i.e. not strategic)
● Requires separate models
for different
products/offers/objectives
time
profile
profile
Next Best Action Model - Naive Approach
11
profile value (LTV / ROI)
value (LTV / ROI)
value (LTV / ROI)
M
M
M
Time
Offer 1 Offer 2 Offer 3
Offer 3
Offer 2
Offer 1
Offer 3
Offer 2
Offer 3
Offer 2
Offer 1
Offer 1
Next Best Action with Reinforcement Learning
12
Customer state, t
action1
action2
action3
reward32
reward33
reward34
Customer state, t+1 Customer state, t+2 Customer state, t+3
Expected LTV / ROI
Q(s, a)
One
timer
Churner
Repeater
Loyal
customer
Multi
product
● Need to estimate an action-value
function given a certain offer policy:
State
(customer feature vector up to moment t)
Action
(offer feature vector)
● Use Q-function to optimize the offer
policy
s1
s2
s3
s4
s5
Next Best Action with Fitted Q Iteration (FQI)
13
Purchase
Visit
No action
Offer 1 Offer 2 Offer 3
1. Generate a batch of transitions
(each trajectory corresponds to 4 transitions):
{ (state, action, reward, new state) }
2. Initialize approximate
3. Initialize training set
4. For each
5. Learn new from training data
repeat
A simplified test dataset is shown for illustration
Next Best Action with FQI
14
Offer 3
Offer 2
Offer 1 (default)
Low state V
High state V
Customers who got
Offer 3 in early
Customers who did
not get offers or got
Offer 1
Customers who got
Offer 2 early
Customers who got
Offer 2 -> Offer 3
● Max value for each state:
● Next best action for each state (policy):
A simplified test dataset is shown for illustration
Next Best Action with FQI
15
● A generalization of the look alike modeling for multi-step and/or multi-choice strategies
● More control over LTV/ROI metrics
● Can evaluate performance of a new policy based on historical trajectories
● Batch-online learning trade-off: multi armed bandits
Budgeting Models and
Decision Automation
Privileged and Confidential 17
Targeting Thresholds: Static Optimization
High
propensity
Low
propensity
Privileged and Confidential 18
Targeting Thresholds: Dynamic Optimization
time
$$
campaign
duration
target budget
Decrease
propensity
threshold
Increase
propensity
threshold
19
Campaign Parameters Optimization
Purchase
trigger
buy <X buy X+
buy 0 buy 1+
Announcement
Buy X or more units
and save on your
next shopping trip!
Promotion
Y% off
1. Estimate demand elasticity
2. Estimate how many
consumers will buy more,
how many will redeem offers
3. Do break-even analysis for
costs and benefits
20
Solution Design: Technical Perspective
Marketing
Manager
Campaign Template
● Steps
● Offer types
● Forecasting logic
Targeting Score
(Look Alike or Next Best
Action)
Timing Score
(Replenishment)
LTV Score
(Monetary)
Offer Database
Profile Database
Campaign
Planner
Targeting Server
Forecasting
Optimization
Targeting decisions
Budgeting decisions
request response
Marketing
Manager
(merchant)
Decision
automation
Customer
models
Objective Selection
Plan and Forecast
Review
User Experience
Execution and
Measurement
Privileged and Confidential 21
Solution Design: Marketer’s Perspective
Thank you!

Customer intelligence: a machine learning approach 5/21/2019

  • 1.
    Customer Intelligence: A MachineLearning Approach Ilya Katsov Head of Practice, Industrial AI Grid Dynamics SEATTLE MAY 21 2019
  • 2.
    ML-based Decision Automationin Marketing Operations ● Billions of micro-decisions in real-time: who, when, how, what, ... ● Complex environment: human behavior, complex business models, hidden factors ● Many building blocks: propensity scoring, recommendation algorithms, multi-armed bandits, etc. ● How to design a system that can make micro-decisions based on business objectives?
  • 3.
    Case Study: Environment Retailer1 Manufacturer 1 Manufacturer NRetailer M purchases, clicks, loyalty IDs ... ... Promotion targeting system ● Drive traffic ● Improve loyalty ● Increase market share ● Acquire/grow/retain clients ● Improve loyalty
  • 4.
    Case Study: Decisionsto be Automated ● Targeting – who ○ Exploits variability in tastes, price sensitivity, propensity to buy ○ Optimize short-term or long-term outcomes ● Timing – when ○ Exploits variability in price sensitivity ○ Exploits individual purchasing cycles ● Outreach/budgeting – how many ○ Exploits variability in propensity ● Promotion properties – what ○ Aggregated view on a promotion calendar
  • 5.
    Approach Retailers Brands Product • Willingness topay • Stages of journey • Affinities to brands • Affinities to channels Predictive Models (Digital Twins) • Propensity • Life-time value • Demand Economic Models • What-if analysis • Optimization • Opportunity finding • Business objectives • Constraints Controls • Offers • Channels • Messages • Prices Signals Decisions
  • 6.
  • 7.
    7 Incremental revenue Acquisition MaximizationRetention time New Cardholder $/brand current non-buyers + high propensity to buy new product current buyers + high propensity to buy more current buyers + high propensity to buy less Product Trial Replenishment Category Stretch Retention Alarm Com petitive Defence Look Alike Modeling and Survival Analysis
  • 8.
    8 Look Alike Modelingand Survival Analysis time no purchase Model training Model scoring purchase no purchase behavioral history outcome Customer profiles for training Customer profile to be scored score
  • 9.
    9 Look Alike Modelingand Survival Analysis: Target Metric Design behavioral history outcome Unconditional propensity: Expected LTV: click/purchase/CTR 3-month spend Response/value uplift:
  • 10.
    Challenges with BasicPropensity Scoring 10 Checking Account Credit Card Brokerage Account Banking / Telecom Customer maturity Product maturity level Retail ● Does not take into account product sequences ● Does not optimize offer sequences (i.e. not strategic) ● Requires separate models for different products/offers/objectives time
  • 11.
    profile profile Next Best ActionModel - Naive Approach 11 profile value (LTV / ROI) value (LTV / ROI) value (LTV / ROI) M M M Time Offer 1 Offer 2 Offer 3 Offer 3 Offer 2 Offer 1 Offer 3 Offer 2 Offer 3 Offer 2 Offer 1 Offer 1
  • 12.
    Next Best Actionwith Reinforcement Learning 12 Customer state, t action1 action2 action3 reward32 reward33 reward34 Customer state, t+1 Customer state, t+2 Customer state, t+3 Expected LTV / ROI Q(s, a) One timer Churner Repeater Loyal customer Multi product ● Need to estimate an action-value function given a certain offer policy: State (customer feature vector up to moment t) Action (offer feature vector) ● Use Q-function to optimize the offer policy s1 s2 s3 s4 s5
  • 13.
    Next Best Actionwith Fitted Q Iteration (FQI) 13 Purchase Visit No action Offer 1 Offer 2 Offer 3 1. Generate a batch of transitions (each trajectory corresponds to 4 transitions): { (state, action, reward, new state) } 2. Initialize approximate 3. Initialize training set 4. For each 5. Learn new from training data repeat A simplified test dataset is shown for illustration
  • 14.
    Next Best Actionwith FQI 14 Offer 3 Offer 2 Offer 1 (default) Low state V High state V Customers who got Offer 3 in early Customers who did not get offers or got Offer 1 Customers who got Offer 2 early Customers who got Offer 2 -> Offer 3 ● Max value for each state: ● Next best action for each state (policy): A simplified test dataset is shown for illustration
  • 15.
    Next Best Actionwith FQI 15 ● A generalization of the look alike modeling for multi-step and/or multi-choice strategies ● More control over LTV/ROI metrics ● Can evaluate performance of a new policy based on historical trajectories ● Batch-online learning trade-off: multi armed bandits
  • 16.
  • 17.
    Privileged and Confidential17 Targeting Thresholds: Static Optimization High propensity Low propensity
  • 18.
    Privileged and Confidential18 Targeting Thresholds: Dynamic Optimization time $$ campaign duration target budget Decrease propensity threshold Increase propensity threshold
  • 19.
    19 Campaign Parameters Optimization Purchase trigger buy<X buy X+ buy 0 buy 1+ Announcement Buy X or more units and save on your next shopping trip! Promotion Y% off 1. Estimate demand elasticity 2. Estimate how many consumers will buy more, how many will redeem offers 3. Do break-even analysis for costs and benefits
  • 20.
    20 Solution Design: TechnicalPerspective Marketing Manager Campaign Template ● Steps ● Offer types ● Forecasting logic Targeting Score (Look Alike or Next Best Action) Timing Score (Replenishment) LTV Score (Monetary) Offer Database Profile Database Campaign Planner Targeting Server Forecasting Optimization Targeting decisions Budgeting decisions request response Marketing Manager (merchant) Decision automation Customer models
  • 21.
    Objective Selection Plan andForecast Review User Experience Execution and Measurement Privileged and Confidential 21 Solution Design: Marketer’s Perspective
  • 22.