For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/arm/embedded-vision-training/videos/pages/may-2018-embedded-vision-summit-iodice For more information about embedded vision, please visit: http://www.embedded-vision.com Gian Marco Iodice, Senior Software Engineer in the Machine Learning Group at Arm, presents the "Even Faster CNNs: Exploring the New Class of Winograd Algorithms" tutorial at the May 2018 Embedded Vision Summit. Over the past decade, deep learning networks have revolutionized the task of classification and recognition in a broad area of applications. Deeper and more accurate networks have been proposed every year and more recent developments have shown how these workloads can be implemented on modern low-power embedded platforms. This presentation discusses a recently introduced class of algorithms to reduce the arithmetic complexity of convolution layers with small filter sizes. After an introduction to the latest optimizations techniques for the most common solutions such as GEMM, the talk dives deeply into the design of Winograd algorithms, analyzing the complexity and the performance achieved for convolutional neural networks.

2. ▪
▪
▪
▪
▪
▪
▪
Agenda

3. Arm: Extraordinary Growth From Sensors to Server
50 billion
chips shipped
50 billion
chips shipped

5. conv1
17%
conv2
22%
conv3
18%
conv4
18%
conv5
18%
Layer breakdown for AlexNet
Embedded Vision Summit 2016

6. Even Smaller Convolution Kernels…

7. CNN Layer Breakdown

8. Fully Connected Layer Issue (1)

9. •
–
•
•
Fully Connected Layer Issue (2)

11. GEMM-based Convolution

12. •
•
•
•
•
•
•
•
What Did We Do to Improve the Performance?

13. device
OpenCL Concepts: Platform Model

14. •
•
•
•
OpenCL Concepts: Compute Unit

15. OpenCL concepts: work-items/work-groups
•
•
•
•
OpenCL Concepts: Work-items/work-group

16. Improving L1 Cache Utilization: Memory Coalescing

17. Improving L2 Cache Utilization Tuning LWS (1)

18. •
•
•
•
•
•
•
•
Improving L2 Cache Utilization Tuning LWS (2)

19. Improving L2 Cache Utilization Tuning LWS (3)

21. Goal

22. 𝑟0 = 𝑑00 ∙ 𝑤0 + 𝑑01 ∙ 𝑤1 + 𝑑02 ∙ 𝑤2
𝑟1 = 𝑑10 ∙ 𝑤0 + 𝑑11 ∙ 𝑤1 + 𝑑12 ∙ 𝑤2
Introduction

23. Winograd’s Minimal Filtering Algorithm (1)

24. 𝑚1 = 𝑘1 + 𝑘2 ∙
𝑤0 + 𝑤1 + 𝑤2
2
𝑚2 = 𝑘2 − 𝑘1 ∙
𝑤0 − 𝑤1 + 𝑤2
2
𝑚3 = 𝑘1 − 𝑘3 ∙ w2
𝑚0 = 𝑘0 − 𝑘2 ∙ w0
Winograd’s Minimal Filtering Algorithm (2)

25. 2D Case: Nest Minimal 1D Algorithms (1)

26. 2D Case: Nest Minimal 1D Algorithms (2)

27. 2D Case: Nest Minimal 1D Algorithms (3)

28. 𝑴1 = 𝑲1 + 𝑲2 ∙
𝑾0 + 𝑾1 + 𝑾2
2
𝑴2 = 𝑲2 − 𝑲1 ∙
𝑾0 − 𝑾1 + 𝑾2
2
𝑴3 = 𝑲1 − 𝑲3 ∙ 𝐖2
𝑴0 = 𝑲0 − 𝑲2 ∙ 𝐖0
Complexity Reduction

29. 𝑌 = 𝐴𝑇 𝐺𝑤𝐺 𝑇 ⊙ 𝐵 𝑇 𝑘𝐵 𝐴
•
•
•
•
Algorithm Design (1)

30. Algorithm Design (2)

31. Input Transform

32. Filter Transform

33. Element-wise Multiplication as Batched GEMM

34. Output Transform

35. •
•
Memory Footprint: GEMM-based vs Winograd-based

36. •
•
•
•
Optimizing Input/Output Transform

37. •
•
•
•
Optimizing batched-GEMM

38. VGG16 Convolution Layers Breakdown (CPU)

39. VGG16 convolution layer breakdown (GPU)
VGG16 Convolution Layers Breakdown (GPU)

40. GEMM-based vs Winograd-based Convolution (1)

41. GEMM-based vs Winograd-based Convolution (2)

42. Extending Winograd-based Convolution: F(4x4,3x3)

43. VGG16 Convolution Layers Breakdown (GPU)

44. •
•
GEMM-based vs Winograd-based Convolution (1)

45. •
•
GEMM-based vs Winograd-based Convolution (2)

46. •
•
Accuracy: Absolute Error

47. •
•
Accuracy: ILSVRC2012

48. •
•
•
•
•
Current Investigations

49. •
•
•
•
•
Conclusion

50. •
•
•
•
•
•
References

52. 5656 © 2018 Arm Limited
The Arm trademarks featured in this presentation are registered trademarks or
trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere. All
rights reserved. All other marks featured may be trademarks of their respective
owners.
www.arm.com/company/policies/trademarks