AI Infra Day Oct. 25, 2023 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Lu Qiu (Machine Learning Engineer, @Alluxio) - Shawn Sun (Software Engineer, @Alluxio) This hands-on session will discuss best practices for using PyTorch and Alluxio during model training on AWS. Chunxu and Lu will provide a step-by-step demonstration of how to use Alluxio on EKS as a distributed cache to accelerate computer vision model training jobs that read datasets from S3. This architecture significantly improves the utilization of GPUs from 30% to 90%+, archives ~5x faster training, and lower cloud storage costs.

1. Get Started with CV Model Training
using PyTorch & Alluxio on Kubernetes
Lu Qiu, Shawn Sun
1

2. Machine Learning Engineer
Alluxio PMC
lu@alluxio.com
Software Engineer
shawn.sun@alluxio.com
Lu Qiu Shawn Sun

3. Agenda
● Introduction of Alluxio in AI/ML stack
● CV model training demo
● CV data loading demo
● Alluxio on Kubernetes demo
3

4. Alluxio in the AI Generation
4
Storage - Data/Model storage
Alluxio - High performance data access layer
ML Framework - Model training/inference
Unified Compute - ML pipeline orchestration

5. Architecture Overview
Online ML platform
Alluxio
Inference cluster
Models
Models
Training Data
Models
1
2
3
4
5
Offline training platform
Alluxio
Training cluster
Training Data
2
5

6. AI Training with Alluxio
6
POSIX API (FUSE)
RESTful API
S3 API
Alluxio
GPU (CV, NLP)
Dataset Storage
Kubernetes
Interactive
Notebook
Alluxio
Operator
Visualization
Dashboard
Alluxio
Dashboard

7. AI Training Demo (CV)
● Algorithm - ResNet
● Dataset - ImageNet (subset)
● Dataset loading
○ POSIX API (FUSE)
7

10. 10
After using Alluxio
- Reduce Data Loader Rate from 82% to 1% (82X)
- Increase GPU Utilization Rate from 17% to 93% (5X)
10

11. CV Data Loading Demo (CV)
● CV data loading performance between
○ Alluxio POSIX API (FUSE)
○ S3FS-FUSE
○ Boto3
11

12. 12

14. Alluxio is 5 times faster than S3FS-FUSE
And > 10 times faster than Boto3 S3 API

15. Alluxio on Kubernetes Live Demo
● Alluxio Kubernetes deployment
● Alluxio POSIX API utilization
● Alluxio Dashboard
15

16. 16
Q & A