It's slide talking about how to build scalable MLOps system on AWS, made by Yunrui LI, Ray

1. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
TAIWAN | AUGUST 02-03, 2023

2. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How to build MLOps System on
AWS
Yunrui Li, Ray
Former Data Scientist, Shopee Singapore
Former Machine Learning Engineer, DAZN
Netherlands

3. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Outline
• Why do we use MLOps?
• Breaking Down MLOps System
• Implement MLOps Architecture

4. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
● Machine Learning is an iterative process.
○ How can we enable this process to be more efficient to
shorten ML Model's time-to-market?
● Data → Concept Drift
○ How can we effectively ensure the production model
considers the latest data pattern?
● MLOps-streamline the end-to-end machine learning process
○ Model Performance
○ Infrastructure
■ Scalable
■ Reliable
■ Maintainable
Why do we use MLOps?

5. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning is an iterative process
Data
Processing
Feature
Engineering
Model
Training
Model
Evaluation
Model
Inference
Model
Monitoring
Deployment Phase
Model Building Phase
Post-Deployment Phase
Predictions
Re-trigger Training
Model and Data
Quality Baseline

6. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Breaking Down MLOps system
Stream
Computing
Batch
Computing
Feature Store
Model Deployment
Computing Layer
Model
Training
Model
Inference
Model
Monitoring

7. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
MLOps Architecture-Data Sources
Stream
Computing
Batch
Computing
Computing Layer
Event-Based Database(Kafka Cluster)
OLAP Database(Data Warehouse)
OLTP Database
Kafka Connector

8. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
● Hive on EMR
○ EMR → It's Hadoop Cluster.
○ Cloud-based distributed big data processing engine(Spark,
Hive, Flink, etc)
○ Hive sits on top of data processing engine(Spark) → SQL
interface to query underlying data
● Redshift
○ AWS solution to data warehouse
● Athena
○ ServerLess to analyze underlying data on S3
MLOps Architecture-Data Warehouse

9. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
● Batch Pipeline
○ Large volume of data
○ Regular scheduling
● Streaming Pipeline
○ Real-time processing for time-sensitive features
○ One minute/second interval
Automated feature engineering pipeline

10. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
MLOps Architecture-Stream Computing
Stream
Computing
Automated Streaming Data Processing Pipeline

11. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
MLOps Architecture-Batch Computing
Batch
Computing
Automated Batch Data Processing Pipeline
Option1
Option 2/3/4
Option 2
Option 3
Option 4

12. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
● Reuse and share features across different models and teams
● Shorten the time required for ML experiments
● Minimize inaccurate predictions due to severe training-serving
skew
Why do we adopt feature store

13. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
MLOps Architecture-Feature Store
Stream
Computing
Batch
Computing
real-time ingestion
batch ingestion
Real-time
Inference
Batch
Prediction
Model
Training
Feast
sync
Option 1
Option 2
Fetch Features
Extract Training Data
Feature
Store
Get Scoring Dataset

14. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
MLOps Architecture-Model Training
Offline Feature Store
Model Training
Model Metadata Artifact
Extract Training Data
Using Point-in-time
query
Env
Code Artifact
Model Artifact
1.Difficult to version and automate code
2.Cost issue and lack of reproducibility

15. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
MLOps Architecture-Model Deployment
Online Feature Store
Fetch Features
Model Deployment
Real-Time
Batch Prediction
Online Feature Store
Scoring dataset
Deployment Orchestration

16. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
MLOps Architecture-Model Monitoring
data and model drift
baseline
Event-based alerting to trigger re-training
Model Re-training
Model
Training
automated or manual approval needed
feedback loop
Real-time
Inference
Model Monitoring
Feature Monitoring
Monitoring

17. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Summary
Stream
Computing
Batch
Computing
Computing Layer
Event-Based Database(Kafka Cluster)
OLAP Database
OLTP Database
Kafka
Connector
Automated Streaming Data
Processing Pipeline
Automated Batch Data Processing Pipeline
Option1
Option 2/3/4
Option 2
Option 3
Option 4
Feast
sync
Option 1
Option 2
Feature Store
Fetch Features
Real-Time
Batch Prediction
Scoring dataset
Deployment Orchestration
Model Deployment
Model Training
Model Metadata Artifact
Env
Code Artifact
Model Artifact
data and model drift
baseline
Event-based alerting to trigger re-training
Model Re-training
automated or manual approval needed
feedback loop
Model
Monitoring
Feature
Monitoring
Monitoring
Extract Training Data
Using Point-in-time query

18. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Summary
● More robust pipelines in case of failures
● Increased flexibility in choosing which tools to implement
● Independently-scalable components

19. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Yunrui Li, Ray
1.ray811030@gmail.com
2.https://www.threads.net/@
yunrayli
ˇ