During this talk, we will discuss the Kubernetes stack's drawbacks in the context of AI and show how dstack addresses them in the training and deployment of GenAI models. Recording of the webinar: https://dataphoenix.info/exploring-infrastructure-management-for-genai-beyond-kubernetes/

1. Exploring AI
infrastructure
management beyond
Kubernetes
Andrey Cheptsov
Founder at
Data Phoenix, March, 2024

2. Why manage AI
infrastructure?
➔ Control
➔ Customization
➔ Privacy
➔ Costs
➔ Dependencies

3. ➔ Reliability
➔ Portability
Challenges of managing
AI infrastructure

4. ➔ Reliability
➔ Portability
➔ Open-source
Why use Kubernetes for
managing
infrastructure?

5. ➔ Tailored for deployment
➔ Poor support for GPU
clouds (e.g. Lambda,
RunPod, TensorDock,
Vast.ai, etc)
Challenges of using
Kubernetes for AI
infrastructure

6. What is dstack?
➔ Container orchestration
engine
➔ Tailored for AI
development and
deployment
➔ Native support for GPU
clouds
➔ Open-source

7. Concepts of dstack
➔ Dev environments
➔ Tasks
➔ Services
➔ Pools

8. Dev environments
Configure
➔ Environment
➔ Resources
➔ IDE

9. Tasks
Configure
➔ Environment
➔ Commands
➔ Ports
➔ Resources
➔ Nodes*

10. Services
Configure
➔ Environment
➔ Resources
➔ Commands
➔ Port
➔ Model mapping
➔ Replicas*

11. Pools
Configure
➔ Resources
➔ Idle duration

12. How does it work?

13. Advantages of dstack
over Kubernetes
➔ Tailored for AI
development and
deployment
➔ Native support for GPU
clouds

14. Q&A
Learn more at
https://dstack.ai/docs