1.
A practical approach to continuous deployment of Machine Learning pipelines
Machine Learning in Production
Christos Dimitroulas
Fullstack developer & DevOps
Luca Palmieri
Data Scientist & Machine Learning
#rapids2018

2.
Prototyping Production
#rapids2018

3.
ProductionOperational
requirements
=> Reproducibility!
● Robustness
● Availability
● Monitoring
● Debugging & auditability

4.
ReproducibilityIngredients
● Code
● Environment
● Data
Software

5.
ReproducibilityData
● Training
● Input
● Serialized dataset
● Database queries
● Output
● Serialized models
● Serialized transformers
● Inference...

6.
Use caseSkill recommendation
● Training
● Input
● Users’ skills (MongoDB)
● Output
● Serialized SVD model
● Serialized transformers
● Automated retraining
● Automated deployment on changes

7.
AutomationData Science Culture
● Code quality delivers value
● DataOps?
● Data scientist ownership needs to
get closer to production

8.
#rapids2018

9.
#rapids2018

10.
Requirements
● Strict dependency management
● Model performance analysis
● Model versioning
● Infrastructure (as code)
● Data versioning
● Continuous integration/delivery
#rapids2018

11.
Dependency Management
Lockfile! Managed virtualenv! Reproducible environments!
#rapids2018

12.
Model Performance Analysis
1. Does the model meet the basic
performance requirements?
2. How does the performance
compare to the latest model in
production?
#rapids2018

13.
Model Versioning
Two levels of versioning:
1. Which version of the code
produced this model?
2. Which data produced this model?
#rapids2018

14.
Infrastructure
● Needs to scale (ideally
automatically)
● Should be easy to maintain,
change or upgrade
● Must be easily reproducible (for
other environments e.g. staging)
● Make it easier for other team
members to develop infrastructure
#rapids2018

15.
Data Versioning
Depends largely on how the data input for the model is created.
In our use case, input comes from a database query.
● MongoDB automated backups (snapshots)
● Save relevant snapshots alongside models
#rapids2018

16.
Continuous Integration tool
● Trigger a pipeline from code
changes
● Schedule jobs
● Run tests & scripts
#rapids2018

17.
Continuous Delivery
serve-skill-recommender-system
Build training container
skill-recommender-system
Private PyPi server
Train new models
Assess models
Build serving API container
Upload models
#rapids2018

18.
● S3 is “primitive” - where is git diff for data?
● Painful transition: prototyping is chaos, production is order sancta sanctorum
● Generalize our process to a proper internal tool
#rapids2018