When Predictive Models Join Forces in Customer Intelligence (Sas Forum Belux 2011 Presentation Koen W. De Bock)

6,758 views
6,565 views

Published on

When Predictive Models Join Forces in Customer Intelligence - On the How and Why of Ensemble Learning For Customer Intelligence. Presented during the Spicing Up Analytics track at the SAS Forum Belux 2011 (6 October 2011, Louvain-La-Neuve, Belgium) by Koen W. De Bock, Ph.D., assistant professor at the IESEG School of Management (Lille & Paris, France).

In customer intelligence, predictive modeling is a key instrument. Applications such as customer churn prediction, response modeling, cross-sell (Next-Product-To-Buy) analysis or customer lifetime value analysis all depend upon inferences about expected future customer behavior or characteristics in order to make marketing campaigns more targeted and effective. While the success of these activities depends on decisions made in several phases of the modeling process, it is widely acknowledged that the choice of the modeling technique is a prominent one. In this presentation, light is shed upon the advantages of letting predictive models in CI join forces, whereby several models are combined into new and more powerful models. These so-called ensembles have consistently emerged as winning entries in data mining contests, such as the Teradata/Duke CRM competition, KDD Cup or the Netflix Prize since many years. However, despite their strength and intuitive nature, their applications in real-life business are still scarce. This talk will untangle the topic of ensemble learning, include an overview of the most important techniques, how they can be tailored to get the most out of your CI applications. The advantages of the techniques are demonstrated throughout several academic experiments on real-life datasets.

Published in: Business, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,758
On SlideShare
0
From Embeds
0
Number of Embeds
4,117
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

When Predictive Models Join Forces in Customer Intelligence (Sas Forum Belux 2011 Presentation Koen W. De Bock)

  1. 1. When Predictive Models Join ForcesOn the How and Why of Ensemble Learning for Customer Intelligence<br />Koen W. De Bock<br />IéSEG School of Management<br /> Lille & Paris, France<br />
  2. 2. Private business school / grande école de commerce<br />Ranking in France<br />Overall: 8th<br />Post-Bac: 1st<br />AboutIéSEG School of Management<br />2400 students (20% int’l)<br />2 Campuses: Lille & Paris<br />75% internationalstaff<br />
  3. 3. Who Am I?<br />
  4. 4. Objective: researchcolloborationswith businesses in the field of database marketing / customer intelligence<br />Can takeany of the followingforms<br />Student graduation projects <br />Joint research projects around a specific objective <br />Executive education with specialist courses related to database marketing <br />Single- or multi year company sponsorship of a Ph.D. researcher <br />Current project partners<br />More info: www.ieseg-databasemarketing.com<br />Introducing the IESEG Expertise Center for Database Marketing<br />
  5. 5. KDD Cup 2009<br />Three marketing tasks: predict the propensity of customers<br />To switch provider: Churn<br />To buy new products or services: Cross-selling<br />To buy upgrades or new options proposed to them: Up-selling<br />Two tracks<br />Fast track (5 days)<br />Slow track (several weeks)<br />Introduction<br />
  6. 6. KDD Cup 2009<br />And the winners are… (EL = Ensemble Learning, Yes/No)<br />Introduction<br />
  7. 7. Othercontests<br />KDD Cup ’05, ’06, '07, ’08, '10<br />NetflixPrize2007 and 2008<br />NetflixPrize $1 MillionGrand Prize 2009<br />Duke University Center forCRM Competition 2002 (churnprediction)<br />2010 UC San Diego Data Mining Contest (Customer acquisition)<br />...<br />TropheeCupboard of Ensemble Learning<br />Introduction<br />
  8. 8. Agenda (i)<br />PredictiveAnalytics in Customer Intelligence<br />WhatIs Ensemble Classification?<br />WhyDoesIt Work?<br />Classical Ensemble Classifiers<br />Suitabilityfor Customer Intelligence<br />Superior Model Performance<br />Model Interpretability<br />Remedying Class Imbalance<br />Maximising Profits<br />Ensemble Learning in SAS<br />Wrap-Up<br />
  9. 9. PredictiveAnalytics in Customer Intelligence<br />Key disciplines<br /> Customer Relationship Management<br />Increase of firm value throughimprovingrelationshipswithcustomers<br />3 pillars: acquisition, development and retention<br />Increaseeffectiveness of marketing actions using the companydatabase<br />Customer Acquisition<br />Customer Development<br />Customer Retention<br />Database Marketing / Customer Intelligence<br />
  10. 10. PredictiveAnalytics in Customer Intelligence<br />Key disciplines<br /> Customer Relationship Management<br />Increase of firm value throughimprovingrelationshipswithcustomers<br />3 pillars: acquisition, development and retention<br />Increaseeffectiveness of marketing actions using the companydatabase<br />Createmodels to predict future customerbehaviour<br />Customer Acquisition<br />Customer Development<br />Customer Retention<br />Database Marketing<br />Predictivemodeling<br />
  11. 11. PredictiveAnalytics in Customer Intelligence<br />Date<br />The Predictive Classification Process <br />Project definition<br />Setting objectives<br />Data preprocessing<br />Project definition<br />Defining outcome to predict<br />Feature creation<br />Model building<br />Outlier detection<br />Missing value treatment<br />Classification technique choice<br />Variable transformations<br />Model deployment<br />Model training<br />Model validation<br />Feature creation automatisation<br />Technique comparison<br />Performance criteria<br />Scoring automatisation<br />Improving model quality<br />
  12. 12. PredictiveAnalytics in Customer Intelligence<br />The Predictive Classification Process <br />Project definition<br />Setting objectives<br />Data preprocessing<br />Defining outcome to predict<br />Feature creation<br />Model building<br />Outlier detection<br />Missing value treatment<br />Classification technique choice<br />Variable transformations<br />Model deployment<br />Model training<br />Model validation<br />Feature creation automatisation<br />Technique comparison<br />Performance critera<br />Scoring automatisation<br />Improving model quality<br />
  13. 13. Data set<br />PredictiveAnalytics in Customer Intelligence<br />The Predictive Classification Process <br />features (variables)<br />E.g.<br />Customer churn prediction<br />Response modeling<br />NPTB<br />X (Independent information)<br />Y (dependnet outcome)<br />Instances (observations)<br />E.g.<br />Transactional data (purchase history)<br />Customer information (demographics, psychographics)<br />Textual information<br />Network information<br />
  14. 14. PredictiveAnalytics in Customer Intelligence<br />The Predictive Classification Process <br />features (variables)<br />X (Independent information)<br />Y (dependnet outcome)<br />Instances (observations)<br />F(X) = Y<br />
  15. 15. PredictiveAnalytics in Customer Intelligence<br />Modeling technique<br />Statistical<br />Logistic regression (e.g.,Buckinx & Van den Poel, 2005)<br />Generalized additive models (GAMs) (Coussement et al., 2010)<br />Survival analysis (Van den Poel & Larivière, 2004)<br />Data mining / machine learning<br />Decision trees (CART, C4.5) (e.g., Smith et al., 2000)<br />Support Vector Machines (SVMs) (Coussement & Van den Poel, 2008)<br />Neural nets (ANNs) (e.g., Pendharkar, 2009)<br />Ensemble classifiers (Random Forest, AdaBoost, Rotation Forest) (e.g., Lemmens & Croux, 2006, Burez & Van den Poel, 2007)<br />“Differences are managerial meaninigful” (Neslinet al., 2006, Journal of Marketing Research)<br />The Predictive Classification Process<br />F(X) = Y<br />
  16. 16. What Is Ensemble Classification?<br />Philosophy<br />
  17. 17. What Is Ensemble Classification?<br />Single expert versus team of experts<br />Philosophy<br />VS<br />
  18. 18. What Is Ensemble Classification?<br />Hybrid ensemble<br />An ensemble consisting of multiple algorithms<br />E.g. an ensemble of a logistic regression, a decision tree and a neural network<br />Philosophy<br />VS<br />
  19. 19. What Is Ensemble Classification?<br />Non-hybrid ensemble<br />An ensemble consisting of members thatbelongto the samealgorithm class<br />E.g. Ensemble of 100 decision trees<br />Philosophy<br />VS<br />
  20. 20. What Is Ensemble Classification?<br />Typology<br />Data set<br />DIVERSITY<br />The Data Level<br />Training set<br />member 1<br />Training set<br />member 2<br />Training set<br />member m<br />...<br />Memberclassifier 1<br />Memberclassifier2<br />Memberclassifierm<br />The Member Classifier Level<br />Combiner<br />The CombinationLevel<br />Ensemble predictions<br />
  21. 21. Key concept in ensemble learning<br />Refers to disagreementamong ensemble members<br />What Is Ensemble Classification?<br />Diversity<br />
  22. 22. Why does it work?<br />Dietterich's 3 arguments of ensemble classifier success<br />1. Statistical 2. Computational 3. Representational<br />D4<br />°<br />D4<br />°<br />D4<br />°<br />D3<br />°<br />D3<br />°<br />D3<br />°<br />D2<br />°<br />D2<br />°<br />D*<br />°<br />D*<br />°<br />D*<br />°<br />D2<br />°<br />D1<br />°<br />D1<br />°<br />D1<br />°<br />
  23. 23. Well-KnownMethods<br />Bagging(Breiman, 1996)<br />Bootstrap samples <br />Decision Trees<br />Majorityvoting<br />Data set<br />Data level<br />Classifier level<br />Combination level<br />Training set<br />member 1<br />Training set<br />member 2<br />Training set<br />member m<br />...<br />Memberclassifier 1<br />Memberclassifier2<br />Memberclassifierm<br />Combiner<br />Ensemble predictions<br />
  24. 24. Well-KnownMethods<br />Nickname: Attribute Bagging<br />RandomSubspaceMethod(Ho, 1998)<br />Random Feature Subsets<br />Decision Trees<br />Majorityvoting<br />Data set<br />Data level<br />Classifier level<br />Combination level<br />Training set<br />member 1<br />Training set<br />member 2<br />Training set<br />member m<br />...<br />Memberclassifier 1<br />Memberclassifier2<br />Memberclassifierm<br />Combiner<br />Ensemble predictions<br />
  25. 25. Classical Ensemble Classifiers<br />(Real) AdaBoost(Freund & Schapire, 1996)<br />Iterativelyreweighted data sets:<br />Mistakesincreaseweights<br />Decision trees / stumps<br />Weightedmajorityvote<br />Data set<br />Data level<br />Classifier level<br />Combination level<br />.<br />Training set<br />ensemble member 1<br />Training set<br />ensemble member 2<br />Training set<br />ensemble memberm<br />...<br />Memberclassifier 1<br />Memberclassifier2<br />Memberclassifierm<br />Combiner<br />Ensemble predictions<br />
  26. 26. Suitability for Customer Intelligence<br />
  27. 27. 1. Strong Model Performance<br />Evidence exhibit # 2: Customer ChurnPrediction data of a EuropeanSupermarket<br />Suitability for Customer Intelligence<br />
  28. 28. 1. Strong Model Performance<br />Evidence exhibit # 2: Customer ChurnPrediction in Telecom ( Lemmens & Croux, 2006)<br />Gini Coefficient and Top Decile Lift <br />Baggingand Stochastic Gradient Boosting versus Logisticregression<br />Suitability for Customer Intelligence<br />
  29. 29. 1. Strong Model Performance<br />Evidence exhibit # 3: Hybrid ensembles for customerchurnprediction<br />Top-decile lift<br />Suitability for Customer Intelligence<br />
  30. 30. 1. Strong Model Performance<br />Evidence exhibit # 3: Hybrid ensembles for customerchurnprediction<br />Top-decile lift<br />Suitability for Customer Intelligence<br />
  31. 31. 2. Model Interpretability<br />Suitability for Customer Intelligence<br />+ + = ?<br />
  32. 32. 2. Model Interpretability<br />Strategy # 1: VariableImportanceMeasures<br />The Black Box Revelation<br />
  33. 33. 2. Model Interpretability<br />Strategy # 2: Bootstrap Confidence Bands forSmoothingSplines(De Bock et al., 2010)<br />The Black Box Revelation<br />
  34. 34. 3. Facilitating Data Preprocessing<br />Diversity is important<br />Keep as much variables as possible<br /> No feature selection!<br />Keep your members as complex as possible<br /> No tree pruning!<br />Treatment for class imbalance<br />Usually: weighting or sampling (over/under)<br />Several ensemble algorithms deal withimbalance in a natural way, byusing the full data set<br />E.g. EasyEnsemble, BalanceCascade<br />Suitability for Customer Intelligence<br />
  35. 35. 4. Optimizing for Profit<br />Optimizing your model fortotalprofitabilityinstead of classification performance<br />E.g. Weighted RF, AdaCost<br />Suitability for Customer Intelligence<br />features (variables)<br />X (Independent information)<br />Y (dependnet outcome)<br />Customer Profitability (CLV)<br />Instances (observations)<br />F(X) = Y<br />
  36. 36. Software implementations<br />SAS Enterprise Miner<br />Bagging and Boosting: model accumulation<br />Hybrid ensembles: model combination<br />Ensemble learners in SAS<br />
  37. 37. Software implementations<br />SAS PROC IML & R (SAS/IML 9.22 and up)<br />Easy access toall ensemble algorithms in R<br />E.g. GAMens, randomForest, adabagpackages<br />Access toall WEKA algorithmsthroughRweka package in R<br />Ensemble learners in SAS<br />
  38. 38. Wrap-up<br />Ensemble learners combine multiple modelsintoaggregateones<br />Great results in data miningcontestandacademicliterature<br />Advantagesfor CI<br />Strong model performance<br />Without loss of model interpretability<br />Facilitation of data preprocessing<br />Optimization of profit<br />Which ensemble algorithmshouldyou use?<br />Good algorithms to startwith<br />Ensemble options available in SAS EM<br />More advancedalgorithms via PROC IML + R (e.g. RF, GAMens)<br />Unfortunately, no algorithmis best for all problems need to experiment!<br />
  39. 39. Contact<br />Interested in a collaboration?<br />Web: www.ieseg-databasemarketing.com(from 10/10/2011)<br />E-mail: k.debock@ieseg.fr <br />The Database Marketing Blog: www.dbmarketingblog.com<br />Thankyou for your attention!<br />

×