The document discusses various design techniques for deep learning accelerators (DLA). It covers topics such as convolution layers, fully-connected layers, CNN accelerators, filter decomposition, model compression through pruning and retraining, tensor cores, systolic arrays, burst fetching, analog computing, thermal management, memory bandwidth optimization, and zero-copy techniques.

1. Design Techniques
for DLA
* DLA stands for Deep Learning Accelerator
(draft)

2. https://en.wikipedia.org/wiki/The_Boss_Baby

3. Dark Silicon
https://www.publicdomainpictures.net/en/view-image.php?image=44607&picture=portrait-of-the-dark-sides-man

4. Roofline Model

5. Layer Behaviors

6. Convolution
• computation intensive
• Around 1x1xC ~ 11x11xC
• Variant
• depthwise Separable Convolution
• sparse
•

7. Fully-Connected
• Most weights

8. CNN Accelerator

9. Hardware Accelerator Design for Machine Learning, 2016

10. Hardware Accelerator Design for Machine Learning, 2016

11. Filter Decomposition

12. * A Reconfigurable Streaming Deep Convolutional Neural Network Accelerator for Internet of Things
For Larger Convolution Kernels

13. Hardware Accelerator Design for Machine Learning, 2016

14. Model Compression

15. pruned and retrained
Hardware Accelerator Design for Machine Learning, 2016

16. pruned and retrained
Deep compression: Compressing DNNs with pruning, trained quantization and huffman coding, 2015

17. Tensor Core
"SIMD" for GPU
https://devblogs.nvidia.com/programming-tensor-cores-cuda-9/

18. Systolic Array

19. GotoBLAS library

20. General Tricks
• Burst/fetch continues blocks
•

21. Analog computing
Hardware Accelerator Design for Machine Learning, 2016

22. Hardware Accelerator Design for Machine Learning, 2016

24. Thermal
Dark Memory and Accelerator-Rich System Optimization in the Dark Silicon Era, 2016

25. Memory Bandwidth
Dark Memory and Accelerator-Rich System Optimization in the Dark Silicon Era, 2016

26. • most of the energy is consumed not in computation but in
moving data to and from memory
• moving from 16- to 64-bit fetches only changes the
energy by 1.5x
•

27. • ZERO copy

28. • Conv memory remapping

29. • Embedded Binarized Neural Networks