HiPEAC2022-DL4IoT workshop_ Muhammad Waqar Azhar
•Download as PPTX, PDF•
0 likes•15 views
Co-design of DL Accelerators in VEDLIoT. Muhammad Waqar Azhar. Workshop on Deep Learning for IoT (DL4IoT), co-located with HiPEAC 2022, Budapest, Hungary, June 2022.
Recommended
Recommended
More Related Content
Similar to HiPEAC2022-DL4IoT workshop_ Muhammad Waqar Azhar
Similar to HiPEAC2022-DL4IoT workshop_ Muhammad Waqar Azhar (20)
More from VEDLIoT Project
More from VEDLIoT Project (20)
Recently uploaded
Recently uploaded (20)
HiPEAC2022-DL4IoT workshop_ Muhammad Waqar Azhar
- 1. Muhammad Waqar Azhar, Pedro Trancoso Chalmers University of Technology 20. June 2022 Co-design of DL Accelerators in VEDLIoT
- 2. 2 Which one is the best? Which one is the best for me? DL Accelerators
- 3. 3 DL Acceleration Hardware Software CPU GPU TPU NPU FPGA Models Quantization Pruning
- 4. 4 DL Accelerator Co-Design Design for… Map to… Software Hardware
- 5. 5 DL Accelerator Co-Design Design for… Map to… Software Hardware
- 6. 6 DL Accelerator Co-Design Design for… Map to… Software Hardware
- 7. 7 DL Accelerator Co-Design Design for… Map to… Software Hardware
- 8. 8 ▪ Model case-study: MobileNet ▪ Observation: generic HW not efficient ▪ Challenge: Depthwise convolution Co-Design Example - Motivation ● Heterogeneity at different levels: ○ Model layers of different type (e.g. depthwise and pointwise convolution) ○ Within same layer type (e.g. activation and filter sizes and shapes) ○ Determines: buffer sizes, reuse, parallelism Layer-specific hardware to capture heterogeneity!
- 9. 9 Co-Design Example: Open Questions & Approaches Approach A: one-HW-for-all DOG Approach B: one-HW-per-layer-type Approach C: one-HW-per-layer + Runs any model - Suboptimal efficiency + Matches layer types - Suboptimal utilization + Best efficiency - Resource-hungry
- 10. 10 Co-Design Example: Open Questions & Approaches Approach A: one-HW-for-all DOG Approach B: one-HW-per-layer-type Approach C: one-HW-per-layer TVM-VTA PYNQ-Z2 Unique Kernels ZCU102 Xilinx FINN ZCU102 • MobileNet requires aggressive quantization (4b) • Performance: 35 GOPS and 68 GOPS using MobileNetsV1 1x and 0.5x • Su, Jiang, et al. "Redundancy- reduced mobilenet acceleration on reconfigurable logic for imagenet classification." • Performance: approx 90 GOPS • Resnet34 • Performance: approx 8 GOPS • (DPU performance > 20 GOPS)
- 11. 11 Proposed Solution Co-design: ▪ Approach B: ▪ Mapping is good but throughput is below threshold… ▪ Approach C: ▪ FINN requires large HW to support original model -> more aggressive quantization ▪ Quantized model fits in HW but accuracy is below threshold… ? B + C
- 12. 12 ▪ Current situation: ▪ Zoo of DNN models ▪ Zoo of HW accelerators ▪ Heterogeneity in the model -> Heterogeneity in the hardware The need for Co-Design! Co-Design with both generic and layer-specific HW modules Conclusions
- 13. 13 Thank you for your attention.