Using Customer Satisfaction toModel LoyaltyBruce IngrahamIngraham ConsultingSF Data MiningPredicting Consumer BehaviorJune...
Overview•   Background•   Theoretical framework•   Method•   Results•   Discussion•   Questions2
Background• Financial services company “Golden Investments”• Over 30,000,000 customers• Client had a very good model to pr...
Theoretical Framework• Mittal, V., & Kamakura, W. A. (2001).  Satisfaction, repurchase intent, and  repurchase behavior: I...
Theoretical Framework• Differential satisfaction thresholds    – Customers have different “pain thresholds” or      tolera...
ILLUSTRATION: Same Satisfaction ThresholdsIf two groups of customers which differ in one characteristic, e.g. tenure, have...
ILLUSTRATION: Differential Satisfaction ThresholdsIf the groups have different thresholds, their response graphs have para...
Theoretical Framework• Response bias    – Likert-scale satisfaction ratings are error-prone      measures of unobservable ...
ILLUSTRATION: Response BiasIf two groups of customers which differ in one characteristic have differential thresholds andt...
Theoretical Framework• Non-linear functional form     – Likert-scale data is ordinal categorical at best     – The relatio...
ILLUSTRATION: NonlinearityWhen using a Likert scale to measure satisfaction, the assumption is often made that the differe...
Theoretical Framework• Binary probit model• Main effects parameters γi capture the  different thresholds for each customer...
BACKGROUND: Probit ModelsProbit models are used to model a binary response variable, e.g. retained/defected, when it isass...
BACKGROUND: Probit ModelsInthis model, the input variable is the satisfaction rating. Individual variation is modeled bych...
Method• Data     – Responses from 12 monthly surveys were       combined to reach a sample of N=9,105     – The survey que...
Method• Data     – Using the predictive defection model, each       customer was assigned to a defection segment     – Ris...
Method• The dependent variable was if the customer  had defected within six months of the survey  response• Model estimati...
Method• Test for addition information about defection     – χ2 goodness-of-fit test to compare actual and       predicted ...
Results• Main effects parameters γi     – Intercept 2.9 (>.001)     – Rating      γp       •   1       -1.3    >.001      ...
Results• Main effects     – Risk          γp        •   VH       -1.5 >.001        •   H        -1.1 >.001        •   M   ...
EXAMPLE: Find the retention rate forrisk=High, rating=2.We start with the intercept, 2.9, which tells us the position of t...
Next, the estimate forrisk= High is -1.11. Add this to 2.97 to get the newposition, 1.86. This reduces the retention rate ...
Finally, the estimate for rating = 2 is -0.89. Add this to 1.86 to get the newposition, 0.97. This reduces the retention r...
Start at 99.8                                     risk moves to 96.8     rating moves to 83.124
Results• Interactions     – None of the interaction estimates were       statistically significant     – Suspect Type II e...
Results• χ2goodness-of-fit tests performed for the defection model  and for the probit model. The expected values for the ...
Discussion• Differential satisfaction thresholds     – Statistically significant main effects are evidence for       this ...
Discussion• Additional information about defection     – χ2 tests are evidence that the survey data provides       additio...
Questions?29
Thanks for coming!30
Upcoming SlideShare
Loading in …5
×

Satisfaction and loyalty

476 views

Published on

Bruce Ingraham (Ingraham Consulting) gave a talk on Satisfaction and Loyalty at the SF Data Mining event: http://www.meetup.com/Data-Mining/events/68283282/

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
476
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Satisfaction and loyalty

  1. 1. Using Customer Satisfaction toModel LoyaltyBruce IngrahamIngraham ConsultingSF Data MiningPredicting Consumer BehaviorJune 19, 2012San Francisco, CA1
  2. 2. Overview• Background• Theoretical framework• Method• Results• Discussion• Questions2
  3. 3. Background• Financial services company “Golden Investments”• Over 30,000,000 customers• Client had a very good model to predict risk of defection based on customer attributes and transaction data• Client also had monthly customer satisfaction surveys• Does satisfaction data provide any additional information about defections?3
  4. 4. Theoretical Framework• Mittal, V., & Kamakura, W. A. (2001). Satisfaction, repurchase intent, and repurchase behavior: Investigating the moderating effect of customer characteristics. Journal of Marketing Research, 38, 131-142.• Three components in model – Differential satisfaction thresholds – Response bias – Non-linear functional form4
  5. 5. Theoretical Framework• Differential satisfaction thresholds – Customers have different “pain thresholds” or tolerance levels with respect to the decision to defect – For example, new customers may be sensitive to customer service issues – Satisfaction thresholds vary systematically with customer characteristics5
  6. 6. ILLUSTRATION: Same Satisfaction ThresholdsIf two groups of customers which differ in one characteristic, e.g. tenure, have the samethresholds, their response graphs have nearly identical lines, reflecting similar retention behaviorfor the same satisfaction level.6
  7. 7. ILLUSTRATION: Differential Satisfaction ThresholdsIf the groups have different thresholds, their response graphs have parallel lines, since given thesame rating, customers with lower thresholds are more likely to remain customers.7
  8. 8. Theoretical Framework• Response bias – Likert-scale satisfaction ratings are error-prone measures of unobservable (latent) true satisfaction – Harsh-raters and easy-raters – Response bias varies systematically with customer characteristics8
  9. 9. ILLUSTRATION: Response BiasIf two groups of customers which differ in one characteristic have differential thresholds andthe same response bias, then their response graphs look like the previous chart—parallel lines.However, if the response bias differs systematically between groups, the additional variation inretention appears as unequal slopes. 9
  10. 10. Theoretical Framework• Non-linear functional form – Likert-scale data is ordinal categorical at best – The relationship between satisfaction and percent defecting is not interval data – For example, the difference in percent defecting between a satisfaction rating of 1 and a rating of 2 may be different from that between a 4 and a 510
  11. 11. ILLUSTRATION: NonlinearityWhen using a Likert scale to measure satisfaction, the assumption is often made that the differencein response between rating levels is constant, i.e. the functional form is linear. When thisassumption is true, the response graph is a straight line, and the ratings can be treated as intervaldata in modeling. When the assumption is false, the ratings need to be modeled as ordinalcategorical data. 11
  12. 12. Theoretical Framework• Binary probit model• Main effects parameters γi capture the different thresholds for each customer group• Interaction parameters δj capture the different response biases for each customer group by rating12
  13. 13. BACKGROUND: Probit ModelsProbit models are used to model a binary response variable, e.g. retained/defected, when it isassumed that the response depends an individual’s threshold being surpassed by an inputvariable, such as satisfaction. The threshold is assumed to vary among individuals, and to benormally distributed within the population.13
  14. 14. BACKGROUND: Probit ModelsInthis model, the input variable is the satisfaction rating. Individual variation is modeled bycharacteristics, e.g.loyaltyscore, and interactions with the rating. The resulting threshold valuegives the customer’s position in the normal distribution, e.g. z-score, and the CDF is used to findthe corresponding probability of retention.Probability ofretention = .84 Threshold = 114
  15. 15. Method• Data – Responses from 12 monthly surveys were combined to reach a sample of N=9,105 – The survey question of interest was worded “How likely are you to continue doing business with Golden Investments?” – Ratings were on a 5-point Likert scale, where 1 indicated highly unlikely, and 5 indicated highly likely15
  16. 16. Method• Data – Using the predictive defection model, each customer was assigned to a defection segment – Risk % of population • Very high 10% • High 10% • Moderate 15% • Low 15% • Very low 50%16
  17. 17. Method• The dependent variable was if the customer had defected within six months of the survey response• Model estimation – Parameter estimation and hypothesis testing for the probit model were carried out using SAS PROC PROBIT – The reference levels • rating: 5 • risk: very low17
  18. 18. Method• Test for addition information about defection – χ2 goodness-of-fit test to compare actual and predicted frequencies for the defection model and the probit model18
  19. 19. Results• Main effects parameters γi – Intercept 2.9 (>.001) – Rating γp • 1 -1.3 >.001 • 2 -0.9 >.001 • 3 -0.05 >.001 • 4 -0.2 .066 • 5 reference19
  20. 20. Results• Main effects – Risk γp • VH -1.5 >.001 • H -1.1 >.001 • M -0.8 >.001 • L -0.4 .01 • VL reference20
  21. 21. EXAMPLE: Find the retention rate forrisk=High, rating=2.We start with the intercept, 2.9, which tells us the position of the referencegroup in the normal distribution. The reference group,risk= Very Low, rating =5, has a retention rate of 99.8%. 99.8% 2.921
  22. 22. Next, the estimate forrisk= High is -1.11. Add this to 2.97 to get the newposition, 1.86. This reduces the retention rate to 96.8%. 96.8% 1.8622
  23. 23. Finally, the estimate for rating = 2 is -0.89. Add this to 1.86 to get the newposition, 0.97. This reduces the retention rate to 83.1%, which is the answer!If significant interactions between risk segment &rating had been found, theywould be added in also. 83.1% 0.9723
  24. 24. Start at 99.8 risk moves to 96.8 rating moves to 83.124
  25. 25. Results• Interactions – None of the interaction estimates were statistically significant – Suspect Type II error: n too small in some cells relative to the variance, resulting in large s.e.25
  26. 26. Results• χ2goodness-of-fit tests performed for the defection model and for the probit model. The expected values for the risk model were obtained by averaging the risk scores for all of the survey customers in the segment, and then multiplying by the segment frequency. The expected values for the probit models were calculated similarly, using probabilities estimated by the model.• The null hypotheses was rejected for the risk model (p=0.035). The large residuals which indicate lack of fit belonged to the Very High and Low segments.• The null hypotheses was not rejected for the probit model (p=.99). The residuals were very small for all segments, and the fit was nearly perfect.26
  27. 27. Discussion• Differential satisfaction thresholds – Statistically significant main effects are evidence for this claim• Response bias – Statistically non-significant interaction terms do not support this claim – May be Type II error due to small n• Non-linear functional form – Statistically non-significant estimate for rating of 4 is evidence for this claim27
  28. 28. Discussion• Additional information about defection – χ2 tests are evidence that the survey data provides additional information about defection• Issues – Non-response bias and propensity to respond. Are unhappy customers less likely to respond? – Overfitting – Predicted risk as independent variable• Business application is to target retention programs to highest-risk segments28
  29. 29. Questions?29
  30. 30. Thanks for coming!30

×