Karthik presented on artificial intelligence hardware. He discussed how AI requires specialized hardware like TPUs due to the large amounts of data processing required. TPUs can process data for AI applications much faster than CPUs or GPUs, with 25 times more MAC operations and 3.5 times more on-chip memory than GPUs. Some applications of AI hardware include datacenters, consumer electronics, and edge devices. The market for AI hardware is expected to reach $109 billion by 2025.

1. VEMANA INSTITUTE OF
TECHNOLOGY
(Affiliated to VTU, Approved by AICTE)
Koramangala, Bengaluru- 560034
“ARTIFICIAL INTELLIGENCE
HARDWARE”
Presented By
Name: Karthik
USN: 1VI17EC033
Under the Guidance of
Dr. Suneeta
Associate. Professor
Department of ECE, Vemana IT

2. Table of Contents
9 June 2021 Department of ECE, Vemana IT 2
• Abstract
• Introduction
• Literature survey
• Technical Aspects
• Applications
• Future Scope
• Conclusion
• References
ARTIFICIAL INTELLIGENCE HARDWARE

3. Abstract
9 June 2021 Department of ECE, Vemana IT 3
• Artificial Intelligence is one of the fastest growing field in tech
sector. Scope of AI is expanded to many fields including
healthcare, transport, security, entertainment, defense and
shopping.
• AI takes a large amount of data, and by processing that data it
recognizes patterns.
• Conventional CPUs are not enough to process large amount of
data for the AI applications. Hence we require a special
hardware called ‘AI accelerators’.
• In this seminar we are discussing about those AI accelerators.
ARTIFICIAL INTELLIGENCE HARDWARE

4. Introduction
9 June 2021 Department of ECE, Vemana IT 4
What is AI, Machine Learning and Deep Learning?
ARTIFICIAL INTELLIGENCE HARDWARE
Artificial Intelligence
Machine Learning
Deep Learning
fx
Fig:. AI

5. Where AI is used?
9 June 2021 Department of ECE, Vemana IT 5
• Astronomy
• Healthcare
• Gaming
• Finance
• Data Security
• Social Media
• Travel
• Automotive Industry
• E commerce
• Apps
ARTIFICIAL INTELLIGENCE HARDWARE

6. Literature survey
9 June 2021 Department of ECE, Vemana IT 6
PAPER NO.1: 2017
TITLE: “In-Datacenter Performance Analysis of a Tensor
Processing 𝐔𝐧𝐢𝐭𝐓𝐌”
AUTHORS: Norman P. Jouppi, Cliff Young, Nishant Patil and
others. Google, Inc., Mountain View, CA USA
DESCRIPTION:
• This paper evaluates an AI hardware called Tensor Processing
Unit (TPU).
• It compares performance of TPU to the Intel Haswell CPU and
Nvidia GPU.
ARTIFICIAL INTELLIGENCE HARDWARE

7. Literature survey
9 June 2021 Department of ECE, Vemana IT 7
PAPER NO.2: 2018
TITLE: “Hardware-Enabled Artificial Intelligence”
AUTHORS: William J. Dally, C. Thomas Gray, John Poulton,
Brucek Khailany and others. NVIDIA
DESCRIPTION:
• This paper discusses the circuit challenges in building deep-
learning hardware both for inference and for training.
• This paper also discusses about GPU system for deep NN.
ARTIFICIAL INTELLIGENCE HARDWARE

8. Literature survey
9 June 2021 Department of ECE, Vemana IT 8
PAPER NO.3: 2018
TITLE: “AI Hardware-Mapping the $100bn IT spending
opportunity”
AUTHORS: Toshiya Hari, Charles Long and others. Goldman S.
DESCRIPTION:
• This paper discusses the scope of AI in hardware industry.
ARTIFICIAL INTELLIGENCE HARDWARE

9. Neural Network
9 June 2021 Department of ECE, Vemana IT 9
ARTIFICIAL INTELLIGENCE HARDWARE
Fig:. Block diagram of neural network

10. Neural Network
9 June 2021 Department of ECE, Vemana IT 10
ARTIFICIAL INTELLIGENCE HARDWARE
0 1
0 1
2×2
1
f(𝒘𝟏𝟏𝒙𝟏 + 𝒘𝟏𝟐𝒙𝟐 + 𝒘𝟏𝟑𝒙𝟑 + ⋯ + ⋯ . ) =
𝒚

11. Neural Network
9 June 2021 Department of ECE, Vemana IT 11
ARTIFICIAL INTELLIGENCE HARDWARE
500×500
X56,675,768,…..

12. CPU
9 June 2021 Department of ECE, Vemana IT 12
ARTIFICIAL INTELLIGENCE HARDWARE
CALCULATED
RESULT
MEMORY
MULTIPLY AND ADD
• Von Neumann Architecture
Fig.: CPU (Central Processing Unit)

13. GPU
9 June 2021 Department of ECE, Vemana IT 13
ARTIFICIAL INTELLIGENCE HARDWARE
RESULT
Fig:. GPU( Graphics Processing Unit)

14. TPU
9 June 2021 Department of ECE, Vemana IT 14
ARTIFICIAL INTELLIGENCE HARDWARE
RESULT
Fig.: TPU (Tensor Processing Unit)

15. TPU
9 June 2021 Department of ECE, Vemana IT 15
ARTIFICIAL INTELLIGENCE HARDWARE
Fig.: TPU Printed Circuit Board

16. TPU
9 June 2021 Department of ECE, Vemana IT 16
ARTIFICIAL INTELLIGENCE HARDWARE
Fig.: TPU Block Diagram

17. Applications
9 June 2021 Department of ECE, Vemana IT 17
ARTIFICIAL INTELLIGENCE HARDWARE
• In Datacenters:
AWS
IBM
Google
Microsoft
Equinix
• In Consumer Electronics:
Fig.: Apple products Fig.: Snapdragon 888 5G

18. Applications
9 June 2021 Department of ECE, Vemana IT 18
ARTIFICIAL INTELLIGENCE HARDWARE
• In Edge AI:
Fig.: Nvidia Jetson Nano Fig.: Google Coral Fig:. Intel Movidius NC stick

19. Future Scope
9 June 2021 Department of ECE, Vemana IT 19
ARTIFICIAL INTELLIGENCE HARDWARE
• As per the goldman sachs Market research, TAM for AI
Hardware will reach $109bn by 2025.

20. Conclusion
9 June 2021 Department of ECE, Vemana IT 20
ARTIFICIAL INTELLIGENCE HARDWARE

21. Conclusion
9 June 2021 Department of ECE, Vemana IT 21
ARTIFICIAL INTELLIGENCE HARDWARE
TPU vs K80 GPU
• 25 times as many MACs than GPU
• 3.5 times more on chip memory than GPU
• Uses 50% less power for the same numbers of operations.
• 15 times as fast as the GPU
Disadvantages:
• Cannot be used in graphic processing.
• Requires optimized softwares.

22. References
9 June 2021 Department of ECE, Vemana IT 22
ARTIFICIAL INTELLIGENCE HARDWARE
1. Jouppi, N. P., Borchers, A., Boyle, R., Cantin, P., Chao, C., Clark, C., … Dean,
J. (2017). “In-Datacenter Performance Analysis of a Tensor Processing Unit.”
Proceedings of the 44th Annual International Symposium on Computer
Architecture - ISCA ’17. doi:10.1145/3079856.3080246 .
2. W. J. Dally, C. T. Gray, J. Poulton, B. Khailany, J. Wilson and L. Dennison,
"Hardware-Enabled Artificial Intelligence," 2018 IEEE Symposium on VLSI
Circuits, 2018, pp. 3-6, doi: 10.1109/VLSIC.2018.8502368.
3. Toshiya Hari, Charles Long, Daiki Takayama, Donald Lu, Ph. D., Mark
Delaney, CFA and Rod Hall. (2018). “AI Hardware-Mapping the $100bn IT
spending opportunity”. Goldman Sachs Global Investment Research.
4. www.cloud.google.com
5. www.nvidia.com
6. www.wikipedia.com
7. www.ark.intel.com

23. 9 June 2021 Department of ECE, Vemana IT 23
ARTIFICIAL INTELLIGENCE HARDWARE
Thank you!