Tech talk: https://www.aicamp.ai/event/eventdetails/W2022052410 What happens after your machine learning models are deployed in production? How do you make sure that your model performance does not degrade as data and the world change? The constantly changing data creates challenges for data scientists and engineering teams on how to detect which models have been affected and how to get their ML applications up and running seamlessly. In this session we will take a deep dive into the new ML model monitoring and drift detection technology. We will discuss: - How to track the ongoing accuracy of their models in production - How to immediately detect drift before it causes significant damage to the business - How to locate the cause of model drifting in live environments. We will also discuss how data scientists and ML engineers can collaborate effectively using their respective tools to identify issues and take the necessary actions with a live demo and a real world use case. Speaker: Younes Amar, Head of Product Wallaroo AI. Resources: https://docs.wallaroo.ai/

1. Redefining MLOps
May24Tue
AICamp

2. Younes Amar
Younes is the head of product at Wallaroo. Prior to
joining Wallaroo, Younes was the data science and AI
product lead at Tempus Labs, working with a large
team of data scientists, on the mission of leveraging
large scale multimodal clinical and genomic data to
develop AI-enabled products that help accelerate
scientific discovery and improve patients’ outcomes.
Younes has extensive experience in software
engineering and product development with focus on
delivering high impact analytics and ML platforms in
various industries such as ESG, Government,
logistics, healthcare and insurance.
Head of Product, Wallaroo

3. Proprietary and Confidential, Wallaroo Labs, Inc. (2022)
What is MLOps?
A cross-functional collaboration with one mission: Operationalize Machine Learning
Data Scientist
Data Engineer ML Engineer
ML Infrastructure
Data Infrastructure
ML tooling
ML/AI research labs

4. Proprietary and Confidential, Wallaroo Labs, Inc. (2022)
How is MLOps applied today?
Data load & prep Model Training Model deployment
-
Model Monitoring
Model Development Model Productionization

5. Proprietary and Confidential, Wallaroo Labs, Inc. (2022)
Last mile of ML
Objectives
• Close the gap between insight creation and
value realization
• Transition from Experimental AI to
Industrialized AI
Challenges
• Scale: Deploying and Serving thousands of
models
• Operational efficiency: Managing and
Measuring models at scale
• Repeatability: Consistent process and
technology
• Actionability: Timely preventative and
corrective actions

6. How does Wallaroo think about the last Mile of ML?

7. Proprietary and Confidential, Wallaroo Labs, Inc. (2022)
Wallaroo is breakthrough performance
for the last mile of ML
Seamless integration into any data and ML ecosystem

8. Proprietary and Confidential, Wallaroo Labs, Inc. (2022)
1 purpose-built platform.
3 integrated components.
Self-Service Toolkit Blazingly Fast Compute Engine
Stream of full audit logs + advanced
model monitoring and explainability
insights to help drive results
Written in Rust-Lang for breakthrough
scalability and performance
Easy-to-use SDK, UI, and API
designed for expert data scientists and
ML Engineers
Advanced Observability

9. Proprietary and Confidential, Wallaroo Labs, Inc. (2022)
The last mile of ML in Wallaroo
Model Deployment Model Management Model Observability Model Optimization
Goals
Empower data scientists and Machine Learning engineers to:
✓ Collaborate effectively to launch faster
✓ Scale deployment, management and observability of ML in production
✓ Deliver Actionable ML

10. Proprietary and Confidential, Wallaroo Labs, Inc. (2022)
Model Deployment
Key capabilities
• Model Upload with open format
conversion across all ML frameworks
• Self-service ML deployment
Pipelines Configuration
• Single and batch inferencing
• Compute utilization auto-scaling for
optimal performance and user
experience

11. Proprietary and Confidential, Wallaroo Labs, Inc. (2022)
Model Management
Key capabilities
• Workspace management & Collaboration
• Integrated Jupyter environments for
experimentation and validation
• Configurable model rollout strategies
(Canary deploys, Shadow deploys, A/B
testing, blue/green deployment)
• Model (Hot) Swapping
• Model Versioning

12. Proprietary and Confidential, Wallaroo Labs, Inc. (2022)
Model Observability
Key capabilities
• Performance and health metrics: Latency
and throughput
• Model + Data Anomaly and Drift detection
with configurable validation checks and
alerts (Wallaroo assays ™)
• Model explainability and troubleshooting
reports
• Native and integrated 3rd party reporting
tools

13. Proprietary and Confidential, Wallaroo Labs, Inc. (2022)
Model Optimization
Key capabilities
• Actionable insights
• Proactive model tuning
• Automated retraining and redeployment

14. Demo

15. Q&A

16. Proprietary and Confidential, Wallaroo Labs, Inc. (2022)
Need more information?
https:/
/docs.wallaroo.ai
www.wallaroo.ai/blog
community@wallaroo.ai
www.wallaroo.ai

17. portal.wallaroo.community
Sign Up Here!