Presentation of the Tensor Processing Unit (TPU) at the TensorFlow and Deep Learning: 3rd Event Generative Deep Learning event (https://www.meetup.com/TensorFlow-and-Deep-Learning-Singapore/)

1. 1
Dr HAMADI CHAREF Brahim
Data Storage Institute (DSI)
Agency for Science, Technology and Research (A*STAR)
April 13, 2017

2. 2
 ISCA 2017 Paper
 Motivations
 Photos
 Internals
 Performance
 Summary
 Related Patents
Agenda

3. 3https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk/view
Paper on Google TPUTM

4. 4
 Google Data Center demand / workload
 Neural Networks inference 95%
- Multi-Layer Perceptrons (MLPs)
- Convolutional Neural Networks (CNNs)
- Long Short-Term Memory Units (LSTMs)
 Performance Metrics
- Peak throughput (92 TeraOps/second)
- Power (Watts)
- Cost ($$$)
Google TPU - Motivations

5. 5
 CPU vs GPU vs (FPGA vs) ASIC
CPU - server-class Intel Haswell
GPU - Nvidia K80
ASIC – Tensor Processing Unit
Google TPU - Motivations

6. 6
Google TPU - Photos

7. 7
Google TPU - Photos

8. 8
Google TPU - Photos

9. 9
Google TPU - Internals

10. 10
Google TPU - Internals

11. 11
Google TPU - Internals

12. 12
Google TPU - Performance

13. 13
Google TPU - Performance

14. 14
Google TPU - Performance

15. 15
Google TPU – Performance (roofline)
Roofline: an insightful visual performance model for multicore architectures
Samuel Williams, Andrew Waterman, David Patterson
Communications of the ACM - A Direct Path to Dependable Software: Volume 52 Issue 4, April 2009

16. 16
 TPU is an ASIC for NNets
 BIG Matrix Unit
256x256 8b = 65,536 MACs (32b ACC)
 TPU on average 15X - 30X faster than GPU or CPU
 TOPS/Watt about 30X - 80X higher
 Future TPU could use GDDR5 memory (as GPU)
- triple achieved TOPS
- raise TOPS/Watt to nearly 70X the GPU
- raise TOPS/Watt to nearly 200X the CPU
Google TPU - Summary

17. 17
 Vector Computation Unit in a Neural Network Processor
Gregory Michael Thorson, Christopher Aaron Clark, Dan Luu.
https://www.google.com/patents/US20160342889
 Batch Processing in a Neural Network Processor
Reginald Clifford Young
https://www.google.com/patents/US20160342890
 Neural Network Processor
Jonathan Ross, Norman Paul Jouppi, Andrew Everett Phelps, Reginald
Clifford Young, Thomas Norrie, Gregory Michael Thorson, Dan Luu.
https://www.google.com/patents/US20160342891
System and method for parallelizing convolutional neural networks
Alexander Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton
https://www.google.com/patents/US20140180989
Google TPU - Patents

18. 18
 Computing Convolutions Using a Neural Network Processor
Jonathan Ross, Andrew Everett Phelps.
https://www.google.com/patents/WO2016186811A1
 Prefetching Weights for a Neural Network Processor
Jonathan Ross.
https://www.google.com/patents/US20160342892
 Rotating Data for Neural Network Computations
Jonathan Ross, Gregory Michael Thorson.
http://google.com/patents/US20160342893
Google TPU - Patents