L’Union Fait La Force!Recent Developments and Applications of EnsembleModels in Customer IntelligenceProf. Dr. Koen W. De ...
About IÉSEG School of Management                                                     Private business school / grande éco...
Introducing the IÉSEG ExpertiseCenter for Database Marketing Objective: research colloborations with businesses in the fi...
KDD Cup 2009Introduction Three marketing tasks: predict the propensity of customers     To switch provider: Churn     T...
KDD Cup 2009Introduction And the winners are…                    Fast Track                          EL? Slow Track      ...
Trophee Cupboard of Ensemble LearningIntroduction                               Other contests                         KD...
Agenda1.   Predictive Analytics in Customer Intelligence2.   What Is Ensemble Classification?3.   Why Does It Work?4.   Cl...
Predictive Analytics in Customer IntelligenceKey disciplinesCUSTOMER RELATIONSHIP MANAGEMENT                              ...
Predictive Analytics in Customer IntelligenceKey disciplinesCUSTOMER RELATIONSHIP MANAGEMENT                              ...
Predictive Analytics in Customer IntelligenceThe Predictive Classification Process DateProjectdefinition  Setting objecti...
Predictive Analytics in Customer IntelligenceThe Predictive Classification ProcessProjectdefinition   Setting objectives  ...
Predictive Analytics in Customer IntelligenceThe Predictive Classification ProcessData set                                ...
Predictive Analytics in Customer IntelligenceThe Predictive Classification Process                                     fea...
Predictive Analytics in Customer IntelligenceThe Predictive Classification Process                                        ...
What Is Ensemble Classification?Philosophy             Koen W. De Bock             L’Union Fait La Force! - Méthodes d’Ens...
What Is Ensemble Classification?Philosophy Single expert versus team of experts                      VS             Koen ...
What Is Ensemble Classification?Philosophy                     VS Hybrid ensemble    An ensemble consisting of multiple ...
What Is Ensemble Classification?Philosophy                      VS Non-hybrid ensemble    An ensemble consisting of memb...
What Is Ensemble Classification?Typology                                Data setDIVERSITY                                 ...
What Is Ensemble Classification?Diversity Key concept in ensemble learning Refers to disagreement among ensemble members...
Why does it work? Dietterichs 3 arguments of ensemble classifier success    1. Statistical                 2. Computatio...
Well-Known MethodsBagging (Breiman, 1996)                Data set                                                        D...
Well-Known MethodsRandom Subspace Method (Ho, 1998)Nickname: Attribute Bagging                Data set                    ...
Well-Known MethodsRandom Forests (Breiman, 2001)                Data set                                                  ...
Well-Known Methods(Real) AdaBoost (Freund & Schapire, 1996)                  Data set                                     ...
Suitability for Customer Intelligence          1. Strong Model Performance          2. Model Interpretability          3. ...
1. Strong Model Performance1. Strong Model Performance                                                    2. Model Interpr...
1. Strong Model Performance1. Strong Model Performance                                                    2. Model Interpr...
1. Strong Model Performance1. Strong Model Performance                                                     2. Model Interp...
1. Strong Model Performance    1. Strong Model Performance                                                         2. Mode...
1. Strong Model Performance1. Strong Model Performance                                                     2. Model Interp...
1. Strong Model Performance2. Model Interpretability                                                    2. Model Interpret...
2. Model InterpretabilityGAMens (De Bock et al., 2010)                Data set                                            ...
1. Strong Model Performance2. Model Interpretability                                                      2. Model Interpr...
1. Strong Model Performance2. Model Interpretability                                                      2. Model Interpr...
1. Strong Model Performance3. Facilitating Data Preprocessing                                           2. Model Interpret...
1. Strong Model Performance4. Optimizing for Profit                                                                       ...
Software implementationsEnsemble learners in SAS SAS Enterprise Miner    Bagging and Boosting: model accumulation    Hy...
Software implementationsEnsemble learners in SAS SAS PROC IML & R (SAS/IML 9.22 and up)    Easy access to all ensemble a...
Wrap-up Ensemble learners combine multiple models into aggregate ones Great results in data mining contest and academic ...
Book release: Advanced Database MarketingInnovative Methodologies & Applications For Managing Customer Relationships Comi...
Contact Interested in a collaboration?     Web: www.ieseg-databasemarketing.com     E-mail: k.debock@ieseg.fr     Pers...
Upcoming SlideShare
Loading in …5
×

L'Union Fait La Force (Sas Forum France 2012 Presentation by Koen W. De Bock

986 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
986
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

L'Union Fait La Force (Sas Forum France 2012 Presentation by Koen W. De Bock

  1. 1. L’Union Fait La Force!Recent Developments and Applications of EnsembleModels in Customer IntelligenceProf. Dr. Koen W. De Bock IÉSEG School of Management Lille & Paris, France
  2. 2. About IÉSEG School of Management  Private business school / grande école de commerce  EQUIS-accredited  Academic ranking in France (L’étudiant)  Overall: 8th  Post-Bac: 1st 2700 students (20% int’l) 2 Campuses: Lille & Paris 75% international staff Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  3. 3. Introducing the IÉSEG ExpertiseCenter for Database Marketing Objective: research colloborations with businesses in the field of database marketing / customer intelligence Can take any of the following forms  Single- or multi year company sponsorship of a Ph.D. researcher  Joint research projects around a specific objective Current project partners Open job position for a Ph.D. candidate More info: www.ieseg-databasemarketing.com Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  4. 4. KDD Cup 2009Introduction Three marketing tasks: predict the propensity of customers  To switch provider: Churn  To buy new products or services: Cross-selling  To buy upgrades or new options proposed to them: Up-selling Two tracks  Fast track (5 days)  Slow track (several weeks) Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  5. 5. KDD Cup 2009Introduction And the winners are… Fast Track EL? Slow Track EL? 1st Prize Hybrid Ensemble Selection Boosting with classification trees and shrinkage, using Bernoulli loss 2nd Prize Filter & Wrapper FS Stochastic Gradient Boosting Stochastic Gradient Boosting & Bagging 3rd Prize Grouping of Hybrid ensemble: regularized modalities/discretization, filter maximum entropy model with FS, ensemble of decision balanced AdaBoost and selective trees. Naive Bayes (EL = Ensemble Learning, Yes/No) Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  6. 6. Trophee Cupboard of Ensemble LearningIntroduction Other contests  KDD Cup ’05, ’06, 07, ’08, 10  Netflix Prize 2007 and 2008  Netflix $1 Million Grand Prize 2009  Duke University Center for CRM Competition 2002 (churn prediction)  2010 UC San Diego Data Mining Contest (Customer acquisition) ... Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  7. 7. Agenda1. Predictive Analytics in Customer Intelligence2. What Is Ensemble Classification?3. Why Does It Work?4. Classical Ensemble Classifiers5. Suitability for Customer Intelligence 1. Superior Model Performance 2. Model Interpretability 3. Facilitating Model Preprocessing 4. Maximizing Profits6. Ensemble Learning in SAS7. Wrap-Up Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  8. 8. Predictive Analytics in Customer IntelligenceKey disciplinesCUSTOMER RELATIONSHIP MANAGEMENT Increase of firm value through improving relationships with customers Customer Customer Customer Acquisition Development Retention 3 pillars: acquisition, development and retention DATABASE MARKETING / CUSTOMER Increase effectiveness of INTELLIGENCE marketing actions using the company database Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  9. 9. Predictive Analytics in Customer IntelligenceKey disciplinesCUSTOMER RELATIONSHIP MANAGEMENT Increase of firm value through improving relationships with customers Customer Customer Customer Acquisition Development Retention 3 pillars: acquisition, development and retention DATABASE MARKETING / CUSTOMER Increase effectiveness of INTELLIGENCE marketing actions using the company database PREDICTIVE MODELING Create models to predict future customer behaviour Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  10. 10. Predictive Analytics in Customer IntelligenceThe Predictive Classification Process DateProjectdefinition Setting objectives Data Defining outcome to predict preprocessing Feature creation Outlier detection Model building Missing value treatment Classification technique choice Variable transformations Model training Model Model validation deployment Technique Feature creation comparison automatisation Performance criteria ScoringImproving model quality automatisation Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  11. 11. Predictive Analytics in Customer IntelligenceThe Predictive Classification ProcessProjectdefinition Setting objectives Data Defining outcome to predict preprocessing Feature creation Outlier detection Model building Missing value treatment Classification technique choice Variable transformations Model training Model deployment Model validation Technique comparison Feature creation automatisation Performance criteria Scoring automatisationImproving model quality Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  12. 12. Predictive Analytics in Customer IntelligenceThe Predictive Classification ProcessData set E.g. features (variables) Customer churn prediction X (Independent information) Y churn, no churn Instances (observations) Y (dependnet outcome) Response modeling Y response, no response NPTB Y prod1, prod2, none E.g. • Transactional data (purchase history) • Profile information (demographics, psychographics) • Textual information • Network information Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  13. 13. Predictive Analytics in Customer IntelligenceThe Predictive Classification Process features (variables) X (Independent information) Instances (observations) Y (dependnet outcome) F(X) = Y Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  14. 14. Predictive Analytics in Customer IntelligenceThe Predictive Classification Process F(X) = Y Modeling technique Statistical  Logistic regression (e.g.,Buckinx & Van den Poel, 2005)  Generalized additive models (GAMs) (Coussement et al., 2010)  Survival analysis (Van den Poel & Larivière, 2004) Data mining / machine learning  Decision trees (CART, C4.5) (e.g., Smith et al., 2000)  Support Vector Machines (SVMs) (Coussement & Van den Poel, 2008)  Neural nets (ANNs) (e.g., Pendharkar, 2009)  Ensemble classifiers (Random Forest, AdaBoost, Rotation Forest) (e.g., Lemmens & Croux, 2006, Burez & Van den Poel, 2007) “Differences are managerial meaninigful” (Neslin et al., 2006, Journal of Marketing Research) Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  15. 15. What Is Ensemble Classification?Philosophy Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  16. 16. What Is Ensemble Classification?Philosophy Single expert versus team of experts VS Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  17. 17. What Is Ensemble Classification?Philosophy VS Hybrid ensemble  An ensemble consisting of multiple algorithms  E.g. an ensemble of a logistic regression, a decision tree and a neural network Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  18. 18. What Is Ensemble Classification?Philosophy VS Non-hybrid ensemble  An ensemble consisting of members that belong to the same algorithm class  E.g. Ensemble of 100 decision trees Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  19. 19. What Is Ensemble Classification?Typology Data setDIVERSITY The Data Level Training set Training set Training set member 1 member 2 member m ... Member Member Member The Member Classifier Level classifier 1 classifier 2 classifier m Combiner The Combination Level Ensemble predictions Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  20. 20. What Is Ensemble Classification?Diversity Key concept in ensemble learning Refers to disagreement among ensemble members Not enough  ensemble members are close to identical  no synergy  no improvement of performance Too much  ensemble members contradict each other too much  average level of expertise decreases  low ensemble performance Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  21. 21. Why does it work? Dietterichs 3 arguments of ensemble classifier success  1. Statistical 2. Computational 3. Representational D4 D4 D4 D3 ° D3 ° D3 ° °D °D ° 2 2 ° D* ° D* D* D2 ° ° ° ° D1 D1 D1 ° ° ° Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  22. 22. Well-Known MethodsBagging (Breiman, 1996) Data set Data level Bootstrap samples Training set Training set Training set member 1 member 2 member m Member Member ... Member Classifier level classifier classifier classifier Decision Trees 1 2 m Combiner Combination level Majority voting Ensemble predictions Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  23. 23. Well-Known MethodsRandom Subspace Method (Ho, 1998)Nickname: Attribute Bagging Data set Data level Random Feature Training set Training set Training set Subsets member 1 member 2 member m Member Member ... Member classifier classifier classifier Classifier level Decision Trees 1 2 m Combiner Combination level Majority voting Ensemble predictions Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  24. 24. Well-Known MethodsRandom Forests (Breiman, 2001) Data set Data level Bootstrap samples Training set Training set Training set member 1 member 2 member m Member Member ... Member classifier classifier classifier Classifier level Randomized Decision 1 2 m Trees Combiner Combination level Majority voting Ensemble predictions Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  25. 25. Well-Known Methods(Real) AdaBoost (Freund & Schapire, 1996) Data set Data level Iteratively reweighted Training set ensemble Training set ensemble . Training set ensemble data sets: member 1 member 2 member m Mistakes increase weights Member Member ... Member classifier 1 classifier 2 classifier m Classifier level Decision trees / stumps Combiner Combination level Weighted majority vote Ensemble predictions Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  26. 26. Suitability for Customer Intelligence 1. Strong Model Performance 2. Model Interpretability 3. Facilitating Data Preprocessing 4. Optimizing for Profit Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  27. 27. 1. Strong Model Performance1. Strong Model Performance 2. Model InterpretabilitySuitability for Customer Intelligence 3. Facilitating Data Evidence exhibit # 1: Customer churn prediction data of Preprocessing a European supermarket 4. Optimizing for Profit Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  28. 28. 1. Strong Model Performance1. Strong Model Performance 2. Model InterpretabilitySuitability for Customer Intelligence 3. Facilitating Data Evidence exhibit # 2: Customer churn prediction in Preprocessing telecom (Lemmens & Croux, 2006) 4. Optimizing for Profit  Gini Coefficient and Top Decile Lift  Bagging and Stochastic Gradient Boosting versus Logistic regression Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  29. 29. 1. Strong Model Performance1. Strong Model Performance 2. Model InterpretabilitySuitability for Customer Intelligence 3. Facilitating Data Evidence exhibit # 3: Hybrid ensembles for Preprocessing customer churn prediction (Lessmann, Coussement & De Bock, 2012) 4. Optimizing for Profit  Top-decile lift Data Set ES LogR Duke 1 1.471 1.330 Duke 2 1.612 1.419 Duke 3 2.444 2.159 Duke 4 1.838 1.500 EuroOp 2.622 2.446 KDD09 1.885 1.837 Operator 3.770 3.673 UCI 6.821 3.500 Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  30. 30. 1. Strong Model Performance 1. Strong Model Performance 2. Model Interpretability Suitability for Customer Intelligence 3. Facilitating Data  Evidence exhibit # 3: Hybrid ensembles for Preprocessing customer churn prediction (Lessmann, Coussement & De Bock, 2012) 4. Optimizing for Profit  Top-decile liftData Set ES RLR ANN SVM-Lin SVM-Rbf NB kNN QDA LDA CARTDuke 1 1.471 1.325 1.248 1.317 1.337 1.219 1.276 1.294 1.331 1.120Duke 2 1.612 1.425 1.505 1.422 1.477 1.042 1.371 1.332 1.424 1.116Duke 3 2.444 2.221 2.402 2.107 2.345 1.388 2.138 1.905 2.133 1.942Duke 4 1.838 1.500 1.576 1.523 1.452 1.294 1.446 1.394 1.493 1.513EuroOp 2.622 2.289 2.133 2.456 2.055 1.624 1.908 2.201 2.387 1.272KDD09 1.885 1.823 1.748 1.851 1.213 0.932 1.542 1.707 1.775 1.200Operator 3.770 1.363 3.520 1.628 3.088 1.085 3.450 3.269 3.673 2.379UCI 6.821 3.143 5.893 2.786 5.857 1.000 4.321 3.643 3.179 4.429Avg. rank 1.000 5.125 3.750 5.250 4.875 9.750 6.375 6.750 4.500 7.625 Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  31. 31. 1. Strong Model Performance1. Strong Model Performance 2. Model InterpretabilitySuitability for Customer Intelligence 3. Facilitating Data Evidence exhibit # 3: Hybrid ensembles for Preprocessing customer churn prediction (Lessmann, Coussement & De Bock, 2012) 4. Optimizing for Profit  Top-decile lift Data Set ES BagDT BagNN RF AdaB SGB LoB Duke 1 1.471 1.457 1.382 1.466 1.406 1.435 1.415 Duke 2 1.612 1.590 1.495 1.601 1.565 1.554 1.560 Duke 3 2.444 2.392 2.423 2.387 2.330 2.247 2.278 Duke 4 1.838 1.811 1.651 1.800 1.671 1.760 1.728 EuroOp 2.622 2.407 2.368 2.358 2.417 2.642 2.661 KDD09 1.885 1.542 1.775 1.707 1.864 1.878 1.899 Operator 3.770 3.172 3.812 3.575 3.895 3.631 3.700 UCI 6.821 6.750 5.964 6.786 4.214 4.214 4.571 Avg. rank 1.625 4.125 5.000 4.000 4.563 4.688 4.000 Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  32. 32. 1. Strong Model Performance2. Model Interpretability 2. Model InterpretabilitySuitability for Customer Intelligence 3. Facilitating Data Preprocessing 4. Optimizing for Profit + + = ? Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  33. 33. 2. Model InterpretabilityGAMens (De Bock et al., 2010) Data set Data level Bootstrap samples with Training set Training set Training set random feature subsets member 1 member 2 member m Member Member ... Member classifier classifier classifier Classifier level GAMs (semi-parametric 1 2 m additive logistic regressions + smoothing splines) Combiner Combination level Average aggregation Ensemble predictions Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  34. 34. 1. Strong Model Performance2. Model Interpretability 2. Model InterpretabilityThe Black Box Revelation 3. Facilitating Data Strategy # 1: Variable Importance Measures Preprocessing (Breiman, 20001; De Bock & Van den Poel, 2012) 4. Optimizing for Profit Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  35. 35. 1. Strong Model Performance2. Model Interpretability 2. Model InterpretabilityThe Black Box Revelation 3. Facilitating Data Strategy # 2: Bootstrap Confidence Bands for Smoothing Preprocessing Splines (De Bock & Van den Poel, 2012) 4. Optimizing for Profit Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  36. 36. 1. Strong Model Performance3. Facilitating Data Preprocessing 2. Model InterpretabilitySuitability for Customer Intelligence 3. Facilitating Data Diversity is important Preprocessing  Keep as much variables as possible 4. Optimizing for Profit  No feature selection!  Keep your members as complex as possible  No tree pruning! Treatment for class imbalance  Usually: weighting or sampling (over/under)  Several ensemble algorithms deal with imbalance in a natural way, by using the full data set  E.g. EasyEnsemble, BalanceCascade, IRUS Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  37. 37. 1. Strong Model Performance4. Optimizing for Profit 2. Model InterpretabilitySuitability for Customer Intelligence 3. Facilitating Data Optimizing your model for total profitability Preprocessing instead of classification performance 4. Optimizing for Profit E.g. Weighted RF, AdaCost (Glady et al., 2009) E.g. Ensemble selection (Lessmann, Coussement & De Bock, 2012) features (variables) X (Independent information) Instances (observations) Y (dependnet outcome) Customer Profitability (CLV) F(X) = Y Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  38. 38. Software implementationsEnsemble learners in SAS SAS Enterprise Miner  Bagging and Boosting: model accumulation  Hybrid ensembles: model combination Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  39. 39. Software implementationsEnsemble learners in SAS SAS PROC IML & R (SAS/IML 9.22 and up)  Easy access to all ensemble algorithms in R  E.g. GAMens, randomForest, adabag packages  Access to all WEKA algorithms through Rweka package in R Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  40. 40. Wrap-up Ensemble learners combine multiple models into aggregate ones Great results in data mining contest and academic literature Advantages for CI 1. Strong model performance 2. Without loss of model interpretability 3. Facilitation of data preprocessing 4. Optimization of profit Which ensemble algorithm should you use?  Good algorithms to start with  Ensemble options available in SAS EM  More advanced algorithms via PROC IML + R (e.g. RF, GAMens)  Experiment! Big data  Big models Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  41. 41. Book release: Advanced Database MarketingInnovative Methodologies & Applications For Managing Customer Relationships Coming March 2013 Editors: Kristof Coussement (IESEG), Koen W. De Bock (IESEG), Scott A. Neslin (Tuck/Dartmouth, USA) Contributors A. Ansari, B. Baesens, R.E. Bucklin, A. Ghose, S. Lessmann, D. Martens, D. Mayzlin, W.W. Moe, S.A. Neslin, O.J. Rutz a.o. Topics Data preprocessing Recommendation engines Text mining Mobile marketing Bayesian networks Banner advertising targeting Quantile regression Paid search advertising Ensemble learning Social media management Rule-based learning Dynamic customer optimization Direct marketing for the non-profit sector Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs
  42. 42. Contact Interested in a collaboration?  Web: www.ieseg-databasemarketing.com  E-mail: k.debock@ieseg.fr  Personal web site: www.koendebock.be Job opening: Ph.D. in database marketing / customer intelligence Thank you for your attention! Koen W. De Bock L’Union Fait La Force! - Méthodes d’Ensemble en Intelligence des Consommateurs

×