This is a brief review of net lift models based on the presentation I did at Truecar in June, 2013 after I attended the training provided by SAS Institute. Recently, I added a few things to the slides after I reviewed several online examples and papers.

1. A REVIEW OF
NET LIFT MODELS
J U N E , 2 0 1 3 Z I X I A W A N G
S U M M E R Z W A N G @ G M A I L . C O M

2. BACKGROUND
• The true effectiveness of a market Champaign should be measured by the incremental
impact, which are the purchases that would not have taken place in absence of the
campaign rather than the gross number of purchase.
• Traditional propensity models (response model)
maximize the gross purchase rate
• Net lift models (incremental model, uplift model or true lift model)
Maximize the incremental impact/lift
Incremental Impact=Test group purchase rate (Gross purchase rate) - Control Group
Purchase Rate (self-selection purchase rate)

3. BACKGROUND
• There are four types of customers in the test and
control sets based on their response to the
marketing campaign.
• We can see that the traditional propensity models
are focus on all people that make purchases which
includes both self-selectors and swing clients but
net lift models will be able to identify swing clients
who are make the most incremental sale from you
marketing campaigns.
• Net models can significantly increase the net
impact of marketing campaigns when you have a
large number of self-selectors.

4. BASIC CONCEPT
• There are a few statistical concepts need to be known in order to understand the
fundamental of net lift model.
• Weight of evidence (WOE):
Describes the relationship (pattern) between a binary variable and a predictor
WOE>0 : positive impact
WOE=0: no impact
WOE<0: negative impact
Calculation methods:
Kernel density estimators: http://en.wikipedia.org/wiki/Kernel_density_estimation
Histogram estimator

5. BASIC CONCEPT
• Information value (IV):
Measure the strength of relationship
• WOE=ln(0.05/0.06)=-0.182 IV=-0.182*(0.05-0.06)=0.002
• Usually 0.02 or 0.05 are used as the cut point for the IV value to determine if the variable has significant
impact.

6. BASIC CONCEPT
• Penalized IV (PIV):
Measure the robustness of WOE and IV
For each bin, penalty is calculated as the difference of WOE between training and
validation sample.
Total Penalty= Sum (Penalty in each bin * (% responders-% non-responders))
Penalized IV= IV - Total Penalty. The suggested cutoff for PIV is 0.1 for variable
selection.
If the Total Penalty is relative small to IV then we can consider the variable is robust.
Only include variables that with relative large penalized IV in the final model.
• Net Weight of Evidence (NWOE):
NWOE=WOE(test)-WOE(control)
• Net Information Value (NIV):
Net IV describes the net strength.
• Penalized NIV:
measured the robustness of a variable.

7. MODELING METHOD
• Regression-based methods:
1) DSM (Different score models)
Method 1: Build two separate logistic regression models
Incremental lift score= P(purchase | treatment)-P(purchase | control)
Method 2: A single logistic regression model (the bifurcated logistic model)
Logit(P(reponse|X) = a + b*X + g*treatment + l* treatment *X
score = P(response|X,treatment =1) - P(response|X,treatment =0)
1) PDM (Probability decomposition models)
When the test and control group are equally sized:
P(purchase due to treatment)=P(purchase | treatment)*(2-1/P(treatment | purchase))
Otherwise :
P(purchase due to treatment)=P(purchase | treatment)*(1+Nt/Nc*1/P(treatment |
purchase))

8. MODELING METHOD
• Non-regression methods:
1) uplift Radom forest:
This method estimate personalized treatment effects by binary recursive partitioning. The
estimated personalized treatment effect is obtained by averaging the predictions of the
individual trees in the ensemble.
2) KNN( K-nearest-neighbors) classifiers
This method use the net purchase rate calculated from a nearest neighborhood of customers
form the training set to estimate the net score for observations in the validation dataset.
3) Net Naive ( and Semi-Naïve) Bayes classifier
Naive bayes classifier assumes that all predictors are conditionally independent given the
target variable Y. The net score using net naive Bayes method would just be the net weight
of evidence (NWOE). The generalized version Net naïve Bayes method rotated the WOE table
and make them more orthogonal to each other.

9. METHOD COMPARISON
• Regression-based Methods:
1) No attempt to maximize the incremental purchase rate directly
2) Subtracting two independent models can present a black box
3) Little control over the smoothness of the final prediction functions
• Non-regression methods:
1) Fitting the incremental purchase rate more directly
2) If NBC or SNBC methods are used, the prediction functions can be interpreted directly
and we can control the smoothness of these functions. For KNN, it's still a black box.
3) It is fitting an inherently unstable target( double variable) and can be over-fitting.
Therefore, full validation or forward validation are needed.

10. EVALUATING THE EFFECTIVENESS OF NET MODEL
• It’s still an area of on-going research.
• A commonly used way is to use the top two deciles or top 10% as a measure of
success.
• Based on the case example provided by Kim Larsen in 12th Annual data mining
conference, net difference score with bifurcated adaptive logistic regression works the
best followed by generalized net naive bayes, net naive bayes, KNN classifier and net
difference score with two linear logistic regressions.
• For your result, the clients have the high net score are swing clients describe in slices 3
and clients with 0 or even negative score are self-selector, no purchase or do not
disturb( sleeping dog).

11. IMPLEMENTATIONS
• A lot of examples that available online are build using the a series of macros coded in
SAS.
• There is uplift package for R using causal conditional inference trees to estimate
personalized treatment effects
• There are also R Package smbinning and R package WOE allow you to quickly
calculated the WOE and IV values.

12. APPLICATION
• Various of Marketing analysis:
Direct Mail
Email
Sweepstake
A/B testing
• There are different ways of to define the profit per purchase depends on your goal and
time windows. Measurements can be the net profit margin per sale, Net present value
(NPV), life time value (LTV).
• Net modeling has also been applied to personalized medicine.

13. REFERENCE
• Net Lift Models: Optimizing the Impact of Your Marketing Efforts by SAS institute
• Net Models presentation in 12th Annual data mining conference by Kim Larsen
https://www.youtube.com/watch?v=JN3WE8IZNVY
• Analyzing Collection effectiveness using Incremental Response Modeling by Ryan
Burton etc.
http://www.mwsug.org/proceedings/2014/BI/MWSUG-2014-BI06.pdf
• What are uplift models by Jeffrey Strickland
http://www.analyticbridge.com/profiles/blogs/what-are-uplift-models