For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/11/selecting-tools-for-developing-monitoring-and-maintaining-ml-models-a-presentation-from-yummly/ Parshad Patel, Data Scientist at Yummly, presents the “Selecting Tools for Developing, Monitoring and Maintaining ML Models” tutorial at the May 2023 Embedded Vision Summit. With the boom in tools for developing, monitoring and maintaining ML models, data science teams have many options to choose from. Proprietary tools provided by cloud service providers are enticing, but teams may fear being locked in—and may worry that these tools are too costly or missing important features when compared with alternatives from specialized providers. Fortunately, most proprietary, fee-based tools have an open-source component that can be integrated into a home-grown solution at low cost. This can be a good starting point, enabling teams to get started with modern tools without making big investments and leaving the door open to evolve tool selection over time. In this talk, Patel presents a step-by-step process for creating an MLOps tool set that enables you to deliver maximum value as a data scientist. He shares how Yummly built pipelines for model development and put them into production using open-source projects.

1. © Yummly, Inc (a Whirlpool subsidiary)
Selecting Tools for
Developing, Monitoring
and Maintaining ML
Models
Parshad Patel
Data Scientist
Yummly, Inc (part of Whirlpool)

2. © Yummly, Inc (a Whirlpool subsidiary) 2
● Navigating the landscape
● Building minimum viable pipelines
● Use-case at Yummly
● Takeaway
Agenda

3. © Yummly, Inc (a Whirlpool subsidiary)
A data scientist’s workflow
3

4. © Yummly, Inc (a Whirlpool subsidiary)
Navigating landscape - The paradigms
4
CI/CD/CT
Versioning
Stream Processing
Tracking Model Registry
Hyperparameter Optimization
Inference
Feature Store
Labeling
Metadata Store
Visualization
Observability Batch Prediction
AutoML

5. © Yummly, Inc (a Whirlpool subsidiary)
Navigating landscape - Tools and platforms
5

6. © Yummly, Inc (a Whirlpool subsidiary)
Start with first principles
6

7. © Yummly, Inc (a Whirlpool subsidiary)
© Yummly, Inc (a Whirlpool subsidiary)
Enter: Minimum viable pipelines
7

8. © Yummly, Inc (a Whirlpool subsidiary)
Minimum viable pipelines - Phase 1
8
Business impact
Experiment tracking Model registry

9. © Yummly, Inc (a Whirlpool subsidiary)
Minimum viable pipelines - Implementation
9
Experiment tracking and model registry

10. © Yummly, Inc (a Whirlpool subsidiary)
Minimum viable pipelines - Phase 2
10
Business impact
Hyperparameter
tuning
Data ingestion

11. © Yummly, Inc (a Whirlpool subsidiary)
Minimum viable pipelines - Implementation
11
Hyperparameter tuning and data ingestion

12. © Yummly, Inc (a Whirlpool subsidiary)
Minimum viable pipelines - Phase 3
12
Business impact
Dataset and artifact
tracking

13. © Yummly, Inc (a Whirlpool subsidiary)
Minimum viable pipelines - Implementation
13
Dataset and artifact tracking

14. © Yummly, Inc (a Whirlpool subsidiary)
Minimum viable pipelines - Together
14

15. © Yummly, Inc (a Whirlpool subsidiary)
Minimum viable pipelines - Together
15

16. © Yummly, Inc (a Whirlpool subsidiary)
Minimum viable pipelines - Towards scaling
16

17. © Yummly, Inc (a Whirlpool subsidiary)
© Yummly, Inc (a Whirlpool subsidiary)
Use-case: Yummly
17

18. © Yummly, Inc (a Whirlpool subsidiary)
Use case - Yummly
18

19. © Yummly, Inc (a Whirlpool subsidiary)
Use case - Journey
19

20. © Yummly, Inc (a Whirlpool subsidiary)
Minimum viable decision - Pause
20

21. © Yummly, Inc (a Whirlpool subsidiary)
Use case - Journey
21
Infrastructure
Aware
Agnostic

22. © Yummly, Inc (a Whirlpool subsidiary)
Migration from build to buy
22
Pipelines
Prediction
Metadata
Workbench
Feature Store
Matching Engine
Deep Learning Containers

23. © Yummly, Inc (a Whirlpool subsidiary)
© Yummly, Inc (a Whirlpool subsidiary)
Takeaway
23

24. © Yummly, Inc (a Whirlpool subsidiary)
In gist
24

25. © Yummly, Inc (a Whirlpool subsidiary)
In gist
25

26. © Yummly, Inc (a Whirlpool subsidiary)
© Yummly, Inc (a Whirlpool subsidiary)
Thank you!
26

27. © Yummly, Inc (a Whirlpool subsidiary)
Q/A
27

28. © Yummly, Inc (a Whirlpool subsidiary)
Resources
28
Ray Datasets
Ray Tune
Ray Train
MLFlow Tracking
MLFlow Registry
DVC Docs
Vertex AI Platform