Building an Efficient AI Training Platform at bilibili with Alluxio

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

bilibili Coeus ML
Platform with Alluxio
Lei Li, AI Platform Lead, bilibili
Zifan Ni, Sr. SWE, bilibili

$whoami
Lei Li
• Worked in
• Working in
• Wide interest in cloud-native,
kubenetes, MLOps, serverless, etc.
• ✈ ♠ 🏃

$whoami
Zifan Ni
• Worked in
• Working in
• Lovely Puppy, Da Vinci.
• 👨🍳 🍹 🎮

2 3 4
1CONTENTS
Best
Practice
Alluxio Meets
Coeus
bilibili Coeus Alluxio Perf

bilibili
• Leading video community
• MAU reached 230 million in last
financial report
• Aim to enrich everyday life of you...

bilibili Coeus
• cloud-native AI platform
• supporting
ü ADs
ü CV
ü NLP
ü Speech
ü e-commerce
ü etc.

model dev model training model storage model serving
VPA Hawkeye
Cloud-Native Observability System of bilibili
Alluxio + F...

2
3 4
1
CONTENTS
Best
Practice
Alluxio Meets
Coeus
bilibili Coeus Alluxio Perf
2
1

Coeus without Alluxio
OSS HDFS
Scenario 1 Container Crash
Training Container

Coeus without Alluxio
Scenario 1 Container Crash
Training Container
OSS HDFS
Download Data

Coeus without Alluxio
Scenario 1 Container Crash
Training Container
OSS HDFS
Download Data
Has to be Downloaded Again…

Coeus without Alluxio
Data is too huge to fit into one single machine, Users have to
• (Re)write difficult APIs for differ...

Alluxio meets Coeus
Cache Wanted!
• Distributed
• Hides difficult details of reading data!

• Holds huge data in distributed workers
• Fuse simplifies data reading (os.open())
Alluxio meets Coeus
Cache Wanted!
• Di...

2
2 4
CONTENTS
Best
Practice
Alluxio Meets
Coeus
bilibili Coeus Alluxio Perf
1 3

Serverless Deployment
Pod0
1. Why Serverless
GPU0 GPU1

Serverless Fuse
Pod0 Alluxio Master0
GPU1
Alluxio Worker0
Alluxio Fuse
Pod1
1. Why Serverless
GPU0

Serverless Fuse
Pod0 Alluxio Master0
Pending…
GPU1
Alluxio Worker0
Alluxio Fuse
Pod1
1. Why Serverless
GPU0

Serverless Fuse
2. Serverless Solution
Pod0 Alluxio Master0
Alluxio Worker0
Pod1
Alluxio Fuse
GPU0 GPU1

Serverless Fuse
Worker Worker
Data Panel
Control Panel
Sidecar Control Panel
Runtime Controller
Pod
Application
Master
Run...

Alluxio Tuning
Problem Solution
1.Master stop world gc 30s / 30m
• Java: MaxGCPauseMillis 👇 , ParallelGCThreads👆
• Alluxio...

2 4
CONTENTS
Best
Practice
Alluxio Meets
Coeus
bilibili Coeus Alluxio Perf
1 3

Framework Pytorch
Network TDNN neural network model
Total Epoches 20
Number of Files ~2.55Millions
File Size ～300Ki/file; ...

Speed Comparison
OSS S3Fuse Local SSD OSS Alluxio Cache
Completion Time 242.56h 63.48h 64.17h
Speed-up Ratio 1 3.82 3.78
A...

Case 2 : Video Portrait Matting
Framework Pytorch
Network TDNN neural network model
Total Epoches 50
Number of Files ~20Mi...

• not enough disks to support large-
scale training in local storage
• S3Fuse: bad performance and
availability with large...

Q&A
Lei Li, lilei06@bilibili.com
Zifan Ni, nizifan@bilibili.com
Thanksto
LuQiu@alluxio
BaoyouChen,XinguiZeng@bilibili
Yang...

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

2024 Trend Updates: What Really Wor... by Search Engine Jou... 1059379 views
Storytelling For The Web: Integrate... by Chiara Aliotta 976667 views
Artificial Intelligence, Data and C... by OECD Directorate ... 914599 views
How to Leverage AI to Boost Employe... by SocialHRCamp 387831 views
2024 State of Marketing Report – by... by Marius Sescu 219508 views
Everything You Need To Know About C... by Expeed Software 236350 views

View on Slideshare

1 of 27 Ad

View on Slideshare

1 of 27 Ad