Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)

In this session, we take a specific business problem—predicting Telco customer churn—and explore the practical aspects of building and evaluating an Amazon Machine Learning model. We explore considerations ranging from assigning a dollar value to applying the model using the relative cost of false positive and false negative errors. We discuss all aspects of putting Amazon ML to practical use, including how to build multiple models to choose from, put models into production, and update them. We also discuss using Amazon Redshift and Amazon S3 with Amazon ML.

  • Be the first to comment

AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)

  1. 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Denis V. Batalov, PhD AWS Solutions Architect, EMEA November 30, 2016 Predicting Customer Churn with Amazon Machine Learning @dbatalov MAC307
  2. 2. Customer churn
  3. 3. Machine learning Science • Computer Science • Statistics • Neuroscience • Operations Research Artificial Intelligence • Rule extraction from data • Inspired by human learning • Adaptive algorithms Engineering • Training: Data  Models • Prediction: Models  Forecast • Decision: Forecast  Actions
  4. 4. ML: Robotics
  5. 5. ML: Robotics
  6. 6. ML: Image recognition
  7. 7. Supervised learning
  8. 8. Supervised learning Input Outcome
  9. 9. Supervised learning Input Outcome Input Input Input Outcome Outcome Outcome
  10. 10. Supervised Learning Input Outcome Input Input Input Outcome Outcome Outcome Supervised Learning known historical data Amazon ML
  11. 11. Supervised learning Input Outcome Input Input Input Outcome Outcome Outcome Supervised Learning Unseen Input Same Outcome known historical data Amazon ML
  12. 12. Amazon Machine Learning service
  13. 13. Amazon Machine Learning service
  14. 14. Amazon Machine Learning service
  15. 15. Amazon Machine Learning service
  16. 16. Telco churn dataset • US telco customers, their cell phone plans and usage • 21 attributes, 3333 rows: • Customer: State, Area_Code, Phone • Plan: Intl_Plan, VMail_Plan • Behavior: VMail_Messages, Day_Mins, Day_Calls, Day_Charge, Eve_Mins, Eve_Calls, Eve_Charge, Night_Mins, Night_Calls, Night_Charge, Intl_Mins, Intl_Calls, Intl_Charge • Other: Account_Length, CustServ_Calls, Churn
  17. 17. Telco churn dataset • US telco customers, their cell phone plans and usage • 21 attributes, 3333 rows: • Customer: State, Area_Code, Phone • Plan: Intl_Plan, VMail_Plan • Behavior: VMail_Messages, Day_Mins, Day_Calls, Day_Charge, Eve_Mins, Eve_Calls, Eve_Charge, Night_Mins, Night_Calls, Night_Charge, Intl_Mins, Intl_Calls, Intl_Charge • Other: Account_Length, CustServ_Calls, Churn
  18. 18. Telco churn dataset KS, 128, 415, 382-4657, 0, 1, 25, 265.100000, 110, 45.070000, 197.400000, 99, 16.780000, 244.700000, 91, 11.010000, 10.000000, 3, 2.700000, 1, 0 OH, 107, 415, 371-7191, 0, 1, 26, 161.600000, 123, 27.470000, 195.500000, 103, 16.620000, 254.400000, 103, 11.450000, 13.700000, 3, 3.700000, 1, 0 NJ, 137, 415, 358-1921, 0, 0, 0, 243.400000, 114, 41.380000, 121.200000, 110, 10.300000, 162.600000, 104, 7.320000, 12.200000, 5, 3.290000, 0, 0 OH, 84, 408, 375-9999, 1, 0, 0, 299.400000, 71, 50.900000, 61.900000, 88, 5.260000, 196.900000, 89, 8.860000, 6.600000, 7, 1.780000, 2, 0 OK, 75, 415, 330-6626, 1, 0, 0, 166.700000, 113, 28.340000, 148.300000, 122, 12.610000, 186.900000, 121, 8.410000, 10.100000, 3, 2.730000, 3, 0 AL, 118, 510, 391-8027, 1, 0, 0, 223.400000, 98, 37.980000, 220.600000, 101, 18.750000, 203.900000, 118, 9.180000, 6.300000, 6, 1.700000, 0, 0
  19. 19. Console: Creating datasource for Amazon ML
  20. 20. Console: Creating datasource for Amazon ML
  21. 21. Console: Building the Amazon ML model
  22. 22. Recipe { "groups": { "NUMERIC_VARS_NORM": "group('Intl_Charge','Night_Calls','Day_Calls','Eve_Calls','Eve_Mins','Int l_Mins','VMail_Message','Intl_Calls','Day_Mins','Night_Mins','Day_Charge', 'Night_Charge','Eve_Charge','Account_Length')” }, "assignments": {}, "outputs": [ "ALL_BINARY", "State", "Area_Code", "normalize(NUMERIC_VARS_NORM)", "CustServ_Calls" ] }
  23. 23. Recipe: normalize() function Account_Length Normalized Value 128 0.808771865 107 -0.047574816 137 1.175777586 84 -0.985478323 75 -1.352484044 118 0.400987732
  24. 24. Building the Amazon ML model
  25. 25. Cost of errors • Cost of customer churn and acquisition (false negative): • Foregone cash flow • Advertising costs • POS and sign-up admin costs • Customer retention cost (false + true positive) • Discounts • Phone upgrades • Etc.
  26. 26. Financial outcome of applying a model Prior Churn Churn Cost Cost without ML 14.49% $500.00 $72.46
  27. 27. Financial outcome of applying a model Prior Churn Churn Cost Cost without ML 14.49% $500.00 $72.46 False Negative True + False Pos Retention Cost Cost with ML 4.80% 12.10% + 14.30% $100.00 $50.40
  28. 28. Financial outcome of applying a model Prior Churn Churn Cost Cost without ML 14.49% $500.00 $72.46 False Negative True + False Pos Retention Cost Cost with ML 4.80% 12.10% + 14.30% $100.00 $50.40 • Threshold 0.3  0.17 • $22.06 of savings per customer • With 100,000 customers over $2MM in savings with ML
  29. 29. What’s next? • https://aws.amazon.com/getting-started/projects/build- machine-learning-model/ • https://aws.amazon.com/machine-learning/developer- resources/ • https://github.com/dbatalov/cost_based_ml
  30. 30. Thank you! Denis V. Batalov, PhD AWS Solutions Architect, EMEA @dbatalov
  31. 31. Remember to complete your evaluations!

×