Creating ML models is just the starting of a long journey. In this presentation which was given as a talk on e2e AI talks, I talk about the various challenges in the machine learning life cycle

1. Ramjee Ganti, dblue Inc.
Machine Learning Lifecycle

2. Ramjee Ganti
● Founder dblue.ai
● Engineer
● Scaled Tech at Startups
● No Sugar Advocate
● @gantir

3. Machine Learning Lifecycle
“Machine learning lifecycle refers to the iterative
process that spans cross functional teams which
define, build, deploy, monitor, operate and improve
ML systems”.

4. Perception
Machine Learning Algorithms

5. Reality
ML
Algorithms
Data Collection
Configuration
Serving
Infrastructure Feature Extraction
Data
Validation
Monitoring
Process
Management Tools
Resource
Management
Analysis Tools
https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf
15%-20%

7. People

8. Why is it hard?

9. ● Make it Discoverable and Accessible
● Version your data
● Feature Repository
Data Processing

10. Model Exploration
● Training Multiple Models
● Dependent Models
● Serving the Model
○ Embedded
○ Service
○ Data

11. Testing and Validation
● Validating Data
● Validating Component Integration
● Validating model quality
● Validating model bias and fairness

12. Deployment
● Multiple models
● Shadowing
● Split serving
● Online learning models

13. Model Monitoring & Observability
● Model inputs
● Model outputs and decisions
● User actions
● Model fairness
● Model interpretability outputs

14. Take Away
1. It’s hard, accept it.
2. Take software engineering approach
3. Implement reproducible and auditable process
4. Version Data, Models, Code
5. Think Platforms

15. References/Credits
● Continuous Delivery for Machine Learning
● Hidden Technical Debt in Machine Learning Systems
● Doing Machine Learning at Uber

16. Thank You