The document outlines the machine learning lifecycle, highlighting its iterative nature involving cross-functional teams to define, build, deploy, monitor, and improve ML systems. It discusses various challenges such as data discoverability, model validation, and the importance of adopting software engineering practices. Key takeaways emphasize the need for reproducibility, versioning of data and models, and understanding the complexities of deploying ML models.

1. Ramjee Ganti, dblue Inc.
Machine Learning Lifecycle

2. Ramjee Ganti
● Founder dblue.ai
● Engineer
● Scaled Tech at Startups
● No Sugar Advocate
● @gantir

3. Machine Learning Lifecycle
“Machine learning lifecycle refers to the iterative
process that spans cross functional teams which
define, build, deploy, monitor, operate and improve
ML systems”.

4. Perception
Machine Learning Algorithms

5. Reality
ML
Algorithms
Data Collection
Configuration
Serving
Infrastructure Feature Extraction
Data
Validation
Monitoring
Process
Management Tools
Resource
Management
Analysis Tools
https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf
15%-20%

7. People

8. Why is it hard?

9. ● Make it Discoverable and Accessible
● Version your data
● Feature Repository
Data Processing

10. Model Exploration
● Training Multiple Models
● Dependent Models
● Serving the Model
○ Embedded
○ Service
○ Data

11. Testing and Validation
● Validating Data
● Validating Component Integration
● Validating model quality
● Validating model bias and fairness

12. Deployment
● Multiple models
● Shadowing
● Split serving
● Online learning models

13. Model Monitoring & Observability
● Model inputs
● Model outputs and decisions
● User actions
● Model fairness
● Model interpretability outputs

14. Take Away
1. It’s hard, accept it.
2. Take software engineering approach
3. Implement reproducible and auditable process
4. Version Data, Models, Code
5. Think Platforms

15. References/Credits
● Continuous Delivery for Machine Learning
● Hidden Technical Debt in Machine Learning Systems
● Doing Machine Learning at Uber

16. Thank You