Download to read offline
Uploaded byAlluxio, Inc.
AI/ML Infra Meetup | Perspective on Deep Learning Framework
The document outlines a presentation by Triston Cao at the Alluxio meetup, focusing on deep learning frameworks, their evolution, and key technologies like TensorRT and CUDA. It highlights the distinctions between training and inference, the use of GPU acceleration, and the integration of compiler technologies into frameworks. Overall, it emphasizes the complexity and rapid evolution of deep learning software projects.
More Related Content
Similar to AI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning Framework
- 1. Triston Cao, forAlluxio Meetup on May 23, 2024 PERSPECTIVE ON DEEP LEARNING FRAMEWORK
- 2.
- 3. 3 COMPUTATION GRAPH ANDGRADIENT DECENT Image credit to Deniz Yuret's Homepage: Alec Radford's animations for optimization algorithms
- 4. 4 OPEN-SOURCE FRAMEWORKS 2014 20172020 2016 2019 2015 2018 2024 ChatGPT AlexNet ResNet Transformer
- 5. 5 WHAT DOES AFRAMEWORK LOOK LIKE A Hybrid Programming Language Environment
- 6.
- 7. 7 OPS, TENSORS, ANDPARALLEL EXECUTION System Level Optimization https://mxnet.apache.org/versions/1.9.1/api/architecture/note_engine https://www.oreilly.com/library/view/elegant-scipy/9781491922927/ch01.html
- 8.
- 9.
- 10. 10 CUDNN 10TH ANNIVERSARY April 2014– April 2024
- 11. 11 CUDA TensorRT NCCL DALI cuDNN cuBLAS … DeepLearning Frameworks CPU Libraries
- 12. 12 SYMBOLIC VS EAGER(IMPERATIVE) Performance vs Easy of use Imperative Static graph Eager mode JIT Hybrid
- 13. 13 TENSOR CORES ANDMIXED PRECISION
- 14.
- 15. 15 DATA LAYOUT MATTERSTOO Reference: Convolutional Layers User's Guide - NVIDIA Docs
- 16.
- 17. 17 NCCL FOR MULTI-NODETRAINING https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/usage/operations.html
- 18.
- 19.
- 20. 20 FRAMEWORK + TENSORRTFOR INFERENCE
- 21. 21 INFERENCE WITH INT8 Ref:Achieving FP32 Accuracy for INT8 Inference Using Quantization Aware Training with NVIDIA TensorRT | NVIDIA Technical Blog
- 22. 22 COMPILER BASED FRAMEWORK https://tvm.apache.org/docs/tutorial/relay_quick_start.html https://www.linkedin.com/pulse/exploring-jax-googles-high-performance-py thon-library-nagilla-hwauc/ Thundercan optimize Pytorch module with • torch.compile • nvFuser • cuDNN • Apex • TransformerEngine • PyTorch eager • Custom CUDA kernels through PyCUDA, Numba, CuPy • Custom kernels written in OpenAI Triton https://github.com/Lightning-AI/lightning-thun der
- 23. 23 TAKE AWAYS • Deeplearning frameworks are large software projects • NVIDIA keeps making libraries to server deep learning frameworks for GPU acceleration • Training and inference have different challenges • More stabilized by still fast evolving • Compiler technology getting more integrated into the framework