NGC17 Talk @ Samsung Research America - June 9, 2017

1. Politecnico di Milano
Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB)
marco.bacis@mail.polimi.it
Marco Bacis, Giuseppe Natale, Emanuele Del Sozzo,
Marco Domenico Santambrogio
CNN Dataflow implementation on
FPGAs
Samsung Research
Friday, 9th June 2017

2. Introduction 2

3. Issues
3
Challenges
Huge set of weights and data
Memory bounded computation
Need to have a scalable design in terms of
memory and resources
without losing in performance
+
=

4. ● Exploitation of the dataflow pattern of CNN operations
● Independent modules with parametric level of parallelism
● Streaming + Dataflow computational paradigm with
efficient memory access
Our Solution 4
Methodology for CNN acceleration on FPGA with

5. 5
Iterative Stencil Loops
Spatial dependencies
Memory bound
Enable efficient solutions in term of
performance and power

6. 6
● Independent modules communicating over FIFOs
● Concurrent memory access and optimal full buffering
● Scalable without increasing external memory use
Streaming StencilTimestep

7. 7
Proposed Approach

8. Implementation 8
1. Convolution Module Structure
2. Fully Connected Module Structure
3. Network Design

9. Convolution Module Structure 9
SST

10. Convolution Module - Parameters 10
• Input/Output Height
• Input/OutputWidth
• Number of Input Feature Maps
• Number of Output Feature Maps

11. Convolution Module - Parameters 11
• Kernel Height
• KernelWidth
• Number of Input Ports
• Number of Output Ports
# Input FMs received per cycle
# Output FMs sent per cycle

12. Implementation 12
1. Convolution Module Structure
2. Fully Connected Module Structure
3. Network Design

13. Fully Connected Module Structure 13
● Treated as a 1x1 convolution
● “Compressed” streaming approach
● 1 input port, 1 output port
● Low latency Floating point accumulation
● Issue for pipelining
● Multiple accumulators + Loop Unrolling

14. Implementation 14
1. Convolution Module Structure
2. Fully Connected Module Structure
3. Network Design

15. Network Design 15
● Convolutional Module
● Memory structure based on I/O ports
● Single vs Multi channel memory cores
● Pooling Module
● Independent from channel
● One module for each previous output port
● Fully-Connected Module -> single pipelined core

16. Experimental Evaluation 16
● Two evaluation designs
● CIFAR-10 network
Conv -> Pool -> Conv -> Pool -> Lin ->Lin
● USPS network
Conv -> Pool -> Conv -> Lin
● Different design choices as a proof-of-concept of the
methodology
● Tested on a XilinxVC707 board

17. CIFAR-10 Network 17
5 x 5
3 in FMs
12 out FMs
32 x 32
Conv 1
2 x 2
12 in FMs
12 out FMs
28 x 28
Pool 1
5 x 5
12 in FMs
36 out FMs
14 x 14
Conv 2
2 x 2
36 in FMs
36 out FMs
10 x 10
Pool 2
900 in
36 out
Lin 1
36 in
10 out
Lin 2

18. USPS Network 18
5 x 5
1 in FMs
6 out FMs
16 x 16
Conv 1
2 x 2
6 in FMs
6 out FMs
12 x 12
Pool 1
5 x 5
6 in FMs
16 out FMs
6 x 6
Conv 2
64 in
10 out
Lin 1

19. Experimental Results 19
Performance improvements with increased batch size

20. Experimental Results 20
Dataset GFLOPS GFLOPS/W Images/s
Test Case 1 USPS 5.2 0.25 172414
Test Case 2 CIFAR-10 28.4 1.19 7809
MSR Work [1] CIFAR-10 - - 2318
Flips Flops LUTs BRAM DSP Slices
Test Case 1 41.10% 50.86% 3.50% 55.04%
Test Case 2 61.77% 71.24% 22.82% 74.32%
Performances and Power Efficiency Results
FPGA Resources Usage
[1] K. Ovtcharov et al., “Accelerating deep convolutional neural network using specialized hardware”, Microsoft Research
Whitepaper, 2015

21. Conclusions 21
● Modular and scalar methodology to accelerate CNNs on
FPGAs using a dataflow approach
● Performance improvement over large batches
● High level pipeline between layers
● Improved memory bandwidth utilization
● High scalability given limited resources

22. FutureWorks 22
Multi-FPGA / Split layers approach
Automatic DSE / CADTool
Different precision / data type

23. 23
Questions?
Marco Bacis
M. Bacis, G. Natale, E. Del Sozzo, and M. D. Santambrogio
“A Pipelined and Scalable Dataflow Implementation of Convolutional Neural Networks on FPGA”
IPDPS Workshops (RAW), May 2017
M. Bacis, G. Natale, and M. D. Santambrogio
“On how to design dataflow FPGA-based accelerators for Convolutional Neural Networks”
ISVLSI Conference, July 2017 – To Appear
References
marco.bacis@mail.polimi.it

24. RelatedWorks● Previously based on
○ Matrix of Processing Elements [1]
○ LoopTiling + Roofline Model [1,2]
○ Parallel fixed convolvers [3]
[1] C.Zhang et al., “Optimizing fpga-based accelerator design for deep convolutional neural networks”, ISFPGA 2015
[2] M.Peemen et al., “Memory-centric accelerator design for convolutional neural networks”, ICCD 2013
[3] M.Sankaradas et al., “A massively parallel coprocessor for convolutional neural networks”, ASAP 2009
24
RelatedWorks
● Issues
○ Communication overhead
○ Suboptimal exploitation of on-chip memory
○ Execution in time (control flow) vs space (dataflow)

25. Convolutional Neural Networks 25
● State-of-art image recognition/classification component
● Chain of multiple layers to extract, transform and merge
features from the data
● Unfortunately, they are compute intensive and require
specific optimizations