http://envd.tensorchord.ai/

1. 󰞵-Friendly Data Science Stack
With 🏕 envd
2022-08-24
@gaocegege

2. Ce Gao @TensorChord
- kubeflow co-chair, mainly focued on
training-operator and katib
- tensorchord/envd maintainer
@gaocegege
gaocegege.com

3. Developer Experience
󰞵
@gaocegege

4. Developer Experience 󰞵

5. Developer Experience 󰞵

6. Developer Experience
for DSML Engineers
󰳘
@gaocegege

7. Developer Experience 󰳘

8. Developer Experience 󰳘
https://livebook.manning.com/book/effective-data-science-infrastructure/chapter-1/v-1/20

9. Developer Experience 󰳘
https://livebook.manning.com/book/effective-data-science-infrastructure/chapter-1/v-1/20

10. Developer Experience 󰳘

11. Developer Experience 󰳘

12. Goals 🎯
- Rapid feedback loop ⏩
- Low cognitive load 🤯
- Clear path to production 🌟

13. envd
🏕
@gaocegege

14. envd 🏕

15. envd 🏕

16. envd 🏕

17. envd 🏕 - Advanced

18. envd 🏕 - Cache

19. envd 🏕
- Develop in build()
- Deploy in serve()

20. envd 🏕

21. How to use envd 🏕 internally
Integration

22. envd 🏕 - Remote build

23. Under the Hood
🚢
@gaocegege

24. Under the Hood 🚢

25. Buildkit
https://earthly.dev/blog/compiling-containers-dockerfiles-llvm-and-buildkit/
BuildKit is a toolkit for converting source code to build artifacts in an efficient,
expressive and repeatable manner.

26. What is Buildkit LLB
BuildKit builds are based on a binary intermediate format called LLB that is used
for defining the dependency graph for processes running part of your build. tl;dr:
LLB is to Dockerfile what LLVM IR is to C.
DAG

27. Buildkit Internals - solver (really complicated!)

28. Future
🔮
@gaocegege

29. Future 🔮

30. Future 🔮
- Clear boundaries and expectations for AI infra

31. Future 🔮
- Clear boundaries and expectations for AI infra

32. Future 🔮
- Clear boundaries and expectations for AI infra
- Production, production, and production.

33. Thanks 🎉
Q&A ❔
github.com/tensorchord/envd
@gaocegege