DAs: - Vizualisation - Preprocessing DE: sys admin DS: - Baseline - Feature engineering - Model development Captain - Code review - Team Data Science Process

1. ©2022 Databricks Inc. — All rights reserved
Datathon
Retail – Churn Prediction
7th - 9th June 2022
1

2. ©2022 Databricks Inc. — All rights reserved
The team
• DAs:
• Vizualisation
• Preprocessing
• DE: sys admin
• DS:
• Baseline
• Feature engineering
• Model development
• Captain
• Code review
• Team Data Science Process
2

3. ©2022 Databricks Inc. — All rights reserved
The architecture
3
Preprocessing
members_table
Preprocessing
account_transacti
ons
Preprocessing
user_activity
Learning and
scoring
Using the model
on test set
Inference on the
predict set
Machine Learning
Feature
engineering
Split into
train/val/test
Data Engineering & Data Sciences

4. ©2022 Databricks Inc. — All rights reserved
The results
4
Model Score on train set Score on test set Score on test set provided
XGBoost 0.96 0.91 0.82
Decision Tree 0.9 0.9 0.75
Logistic Regression 0.83 0.82 0.49
• Our goal was to score on the True Positive (ie, a churn predicted as a churn).
Benchmark of different algorithms used, with the same features and same splits.

5. ©2022 Databricks Inc. — All rights reserved
5
Challenge Ideas to answer the issue
Features are the hardest component to get right Feature store provides support for feature storage, serving
and management for ML scalability
DS models can be slow and choosing the right architecture
is primordial
GPUs and even TPUs are better choices (but higher cost)
for parallel processing and complex matrix manipulation over
CPUs
Framework versions and dependency problems with ML
deployment
Model registry keeps ML models encapsulated and easy to
deploy across all environments for easy scalability
Large ML application is never a one-time deal, it requires
efficient optimization methodologies to fine-tune or retrain
through iterative processes
Bayesian optimization offers a computational-efficient
benefit to scale up ML hyper-parameter tuning
Friction between ML and business experts CI/CD platform to get direct feedback from business
stakeholders
Vertical scaling remains the most pragmatic solution (not always the best one), especially for applications that have been
in production for many years. However, in a distributed environment, horizontal scaling by adding more machines to your
pool of resources can be a better idea.
How to scale

6. ©2022 Databricks Inc. — All rights reserved
Next steps you recommend
• Capitalize within our company
• AGILE Framework
• Data analysis Methodology
• Statistics based Exploratory Data Analysis
• Business stakeholders feedbacks
• Data drift
• Engineering
• Improve features creations especially for user and activity table
• Optimize model’s hyperparameters
• Benchmark deep learning
6

7. ©2022 Databricks Inc. — All rights reserved
What is for you the value of Azure
Databricks?
• Triple level Parallelization
• Models benchmark
• Hyperparameters search within algorithms
• Map-reduceable algorithms
• Team Data Science Process Ready (Collaboration)
• One-stop-shop (Easy hands-on complex tasks)
• Parallel by design (Cutting-edge technology)
• Pyspark optimized (Done for and by ML practitioners)
7