This presentation was made on June 30th, 2020. Recording of the presentation is available here: https://youtu.be/9LajqAL_CU8 As enterprises “make their own AI”, a new set of challenges emerge. Maintaining reproducibility, traceability, and verifiability of machine learning models, as well as recording experiments, tracking insights, and reproducing results, are key. Collaboration between teams is also necessary as “model factories” are created for enterprise-wide model data science efforts. Additionally, monitoring of models ensures that drift or performance degradation is addressed with either retraining or model updates. Finally, data and model lineage in case of rollbacks or addressing regulatory compliance is necessary. H2O ModelOps delivers centralized catalog and management, deployment, monitoring, collaboration, and administration of machine learning models. In this webinar, we learn how H2O can assist with operationalizing, scaling and managing production deployments. Speaker's Bio: Felix is a part of the Customer Success team in Asia Pacific at H2O.ai. An engineer and an IIM alumni, Felix has held prominent positions in the data science industry.

1. 1 Confidential1
H2O Model Ops
Scaling and Managing
Deployments
H 2 O . a i G u i d a n c e
Felix A
Customer Success, H2O.ai

2. 4 Confidential4 Confidential
Success of AI
Business Impact
Aid Decision Making
Operationalize and consume ML models

3. 3
Agenda Overview
Challenges around
machine learning
operations
02
01
03
04
Introduction to
Model Ops
How H2O.ai can help?
Your turn – questions
and answers

4. 3
ML Lifecycle
Define
Data
BuildFinalize
Execute
Problem statement
Solution Definition
Acquisition
Preparation
Feature Engineering
Algorithm selection
Hyper parameter optimization
Validation
Documentation
Unit Testing
Deployment
In production
Monitoring performance
Replacement
Retirement

5. 3
ML Lifecycle - Bottleneck
Define Data Build Execute
Define Data Build Execute
Define Data Build Execute
Define Data Build Execute
Pre Big Data
Era
Pre Auto ML
Era
Current
State

6. 66 Confidential
Current State
Data Scientists and IT not living in harmony
“Perhaps not surprisingly, only 15% have deployed AI broadly into
production—because that is where people and process issues come into
play.” - NVP Survey of 70 industry leading firms you would recognize
http://newvantage.com/wp-content/uploads/2020/01/NewVantage-Partners-Big-Data-and-AI-Executive-Survey-2020-1.pdf
Different
Mindsets
Different
competencies
Limited
Resources
Lack of
Ownership

7. 66 Confidential
Deployments today
Data Scientists productivity gets hit
Lots of custom codes and brittle structure
IT Ops is uncomfortable in scaling such a process

8. 10
The Solution Model Ops
AKA MLOps or ML Ops
New set of technology and practices
Actors doing the right role
Collaboration between actors
Allows organizations to scale AI efforts

9. 11
ModelOps
Production Model
Deployment
Putting models into production
Model Lifecycle
Management
Keeping models healthy and
running in production
Production Model
Governance
Maintaining control of
production models and
environments
© H2O.ai 2020

10. 12
3 rules of production model deployment
1. Use the technology makes it easy to
scale
2. Make machine learning immediately
visible to deployment teams
3. Allow for a Dev – Test – Prod
approach

11. 11
ModelOps
Production Model
Deployment
Putting models into production
Model Lifecycle
Management
Keeping models healthy and
running in production
Production Model
Governance
Maintaining control of
production models and
environments
© H2O.ai 2020

12. 13
Model Lifecycle Management
Monitor Alert Failover and
Fallback
Update Testing
© H2O.ai 2020

13. 11
ModelOps
Production Model
Deployment
Putting models into production
Model Lifecycle
Management
Keeping models healthy and
running in production
Production Model
Governance
Maintaining control of
production models and
environments
© H2O.ai 2020

14. 14
Production Model Governance
Production Model Governance
Role Based Access
Control
Version control (registry)
and rollback
Audit trail for access and
changes
Controls in place to
manage risk and comply
with regulations
© H2O.ai 2020

15. 3
Demo

16. 3
ML Ops – So what
Easy
Collaboration
Reduced
resources
Cost Savings
Control
measures
Governance
Minimize
risk

17. 3
ML Lifecycle
Define
Data
BuildFinalize
Execute
Problem statement
Solution Definition
Acquisition
Preparation
Feature Engineering
Algorithm selection
Hyper parameter optimization
Validation
Documentation
Unit Testing
Deployment
In production
Monitoring performance
Replacement
Retirement

18. Confidential18 Confidential18
• Automatic feature engineering,
machine learning and interpretability
• Fully automated machine learning
from ingest to deployment
• User licenses on a per seat basis
annually
• GUI-based interface for end-to-end
data science
• A new and innovated
platform to make your own
AI apps
• Enterprise commercial
software
• Easy and intuitive platform to
have AI answer your
question
H2O.ai: AI Platforms
In-memory, distributed
machine learning algorithms with
H2O Flow GUI
Open Source H2O Driverless AI H2O Q
• 100% open source – Apache
V2 Licensed
• Integration with Apache Spark
• Enterprise support subscriptions
• Interface using R, Python on
H2O Flow
H2O Model Ops
• AI deployment platform built
for DevOps and MLOps
• Scalable to support high
throughput and low latency
model scoring environments
• Comprehensive model
monitoring
• Drift Detection and retrain

19. 3
ML Lifecycle
Define
Data
BuildFinalize
Execute

20. Confidential17
Over to you – Questions and Answers

21. Thank You