Download as PDF, PPTX
Uploaded byMindos Cheng
Why Systolic Architectures
The document discusses systolic arrays and their use in hardware acceleration. It describes how systolic arrays can be used to efficiently implement algorithms like convolution, sorting, filtering, and priority queues in a massively parallel manner. Systolic arrays arrange processing elements in a grid and propagate data through the grid to perform computations. They are well-suited for hardware due to their regular structure and local data movement. The document also discusses Google's Tensor Processing Unit (TPU) which uses a systolic array design to accelerate deep learning workloads.
More Related Content
![Data Structures - Lecture 1 [introduction]](https://cdn.slidesharecdn.com/ss_thumbnails/lecture-1introduction-141217054305-conversion-gate02-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Why Systolic Architectures
![[Harvard CS264] 03 - Introduction to GPU Computing, CUDA Basics](https://cdn.slidesharecdn.com/ss_thumbnails/cs264201103-cudabasicsshare-110209024624-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
![[Harvard CS264] 02 - Parallel Thinking, Architecture, Theory & Patterns](https://cdn.slidesharecdn.com/ss_thumbnails/cs264201102-archtheorypatternshare-110206154047-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
Why Systolic Architectures
- 1.
- 2.
- 3.
- 4.
- 6. Systolic Array • Easyto describe in software language. • easy to program w some kind of domain specific language. • Elegant • Layout friendly
- 7. Memory Bandwidth grows slowly SpecYear MB/s DDR 2000 2667 DDR2 2003 5333 DDR3 2007 12800 DDR4 2014 19200
- 8. Increasing Operations /IO H. T. Kung 1982
- 9.
- 10. y1 = y1+w1x1 ϵ= ϵ+w2x1 ϵ = ϵ+w3x1 ϵ = ϵ+w4x1 y2 = y2+w1x2 y1 = y1+w2x2 ϵ = ϵ+w3x2 ϵ = ϵ+w4x2 y3 = y3+w1x3 y2 = y2+w2x3 y1 = y1+w3x3 ϵ = ϵ+w4x3 y4 = y4+w1x4 y3 = y3+w2x4 y2 = y2+w3x4 y1 = y1+w4x4 y5 = y5+w1x5 y4 = y4+w2x5 y3 = y3+w3x5 y2 = y2+w4x5 output : y1 time(broadcast) space(move) H. T. Kung 1982
- 11. H. T. Kung1982 Better precision of summation, if MAC has more digit than bus Require separate bus for collecting output from individual cells
- 12.
- 13. Without global datacommunication Better precision of summation (same as B2) Systolic output path (or use next row in 2D) Nodes are activated half of the time.
- 14. x1 x1 x1 x2 x2 x2 x2 x2x3 x3 x3 x3 x3 x4 x4 x4 w1 w1 w1 w2 w2 w2 w3 circle0 circle 1 circle 2 circle 3 circle 4 circle 5 circle 6 Half nodes are activated at any given time.
- 15. Without global datacommunication 1 node / cycle 1 node / 2 cycles Register to keep w Better precision of summation
- 16.
- 20.
- 21. Odd-Even Transposition Sort Active Comp& Swap O((n/k)log(n/k)) + O(k(n/k)) H. T. Kung 1979
- 22.
- 23. In Matrix Form H.T. Kung 1979
- 24. H. T. Kung1979
- 25. H. T. Kung1979 0 0 0 0
- 26.
- 27. • insert() • delete() •extract_min() Priority Queue Operations For n operations: O(n log n) O(n) Key : One operation can be issued after another in time.O(1)
- 28. Priority Queue Operations insert(k)delete(k) extract_min() Sink down the element with key k A) Sink down a fake element with key k to find target. B) Remove the target. C) Bubble up the below ones. A) Take first element B) Bubble up the below ones.
- 29.
- 30. xi = R(xi−1,. . . , xi−k) xi = axi−1 + bxi−2 + cxi−k + d
- 31.
- 32.
- 35. Cloud TPU Google CloudPlatform Blog https://cloud.google.com/tpu/ TPU V3TPU V2 TPU V2 Pod
- 36. TPU Programming • Acloud TPU has 4 chips x 2 cores x 1 or 2 MXU • MXU • 128x128 systolic array • 16K MAC / cycle • bfloat16 • TPU memory prefer 8 bytes alignment. • 8 or 16GB HBM2 / core https://cloud.google.com/tpu/docs/tpus https://www.nvidia.com/en-us/geforce/products/10series/titan-x-pascal/ Titan X has 3.5K cuda cores So, each TPU V3 card has 4 chips x 2 cores x 2 MXU x 16K MAC / cycle = 256K MAC / cycle at most.
- 37.
- 38. TPU Programming • XLAcompiler for TensorFlow programs. • Tiling => Need reshape • Shape => No dynamic batch • Padding => under utilize TPU, more memory usage • op_profile tool
- 39. TPU Programming • Densevector and matrix computations are fast • M x M, M x v, Convolution • Data movement on PCIe is slow. • Only dense parts of the model, loss and gradient subgraphs are on TPU. • I/O, reading data, writing checkpoint, preprocessing data is on CPU. • decoding compressed images, randomly sampling/cropping, assembling training minibatches • Non-matrix operations will likely not achieve high MXU utilization. • add, reshape, or concatenate • feature dimension => 128 x • Batch dimension => 8 x
- 40. TPUEstimator • TPUEstimator provides agraph operator to build and run a replicated computation https://www.tensorflow.org/api_docs/python/tf/contrib/tpu/TPUEstimator
- 41.
- 42.
- 43.
- 44.