In this deck from SC16, Nvidia CEO Jen-Hsun Huang describes how the company is riding the wave of deep learning in HPC.

1. Jen-Hsun Huang, Founder & CEO | SC’16 | Nov. 14, 2016
THE GREATEST
CHALLENGES CAN’T WAIT

2. 2
Pascal — 5 Miracles Pascal Supercomputers3x Developers in 2 Years
10 of Top 10 HPC
Applications Accelerated
Pascal
16nm FinFET
CoWoS HBM2
NVLink
cuDNN
120,000
2014
400,000
2016
Gaussian
ANSYS Fluent
GROMACS
Simulia Abaqus
NAMD
WRF
VASP
OpenFOAM
LS-DYNA
AMBER
>400HPC Apps
A GREAT YEAR FOR GPU COMPUTING

3. 3
Numerical models to understand
and predict physical and biological
behavior
Based on the laws of physics —
motion, gravity, mass-energy,
thermodynamics, electrostatics
Computational methods like
PDE, FEM, MC, LA
Turbulent Flow
Structural Analysis
Molecular Dynamics
N-body Simulation
COMPUTATIONAL SCIENCE

4. 4
Combinatorial explosion
Incomplete information
No laws-of-physics equations exist
Deep learning extracts multi-
dimensional features from data
Breakthrough for AI
“What’s the next move?” “Is there cancer?”
“What’s happening” “What does she mean?”
DATA SCIENCE

5. 5
DEEP LEARNING
IS A SUPERCOMPUTING CHALLENGE
INFERENCING
RECOGNIZE
CLASSIFY
PREDICT
GENERATE
TRAINING
PIPELINE
MODEL CONFIGURATION
HYPERPARAMETER TUNING
MODEL TRAINING
100’s OF PETAFLOPS TO
EXAFLOPS MACHINES
DATA
PIPELINE
PROCESS
AUGMENT
AUTO LABEL
MANUAL LABEL
CURATE
PETABYTES OF DATA
PETAFLOPS MACHINE
100’s OF GIGAFLOPS
TO TERAFLOPS

6. 6
Future supercomputers designed for
computational and data science
Strong CPU – Variable Precision
Computation – High-Speed Links
4X 5.3 TF FP64
4X 10.6 TF FP32
4X 21.2 TF FP16
640GB/s NVLink
CPU
THE ENGINE FOR AI SUPERCOMPUTING

7. 7
GPU Boosts HPC GPU Boosts AIGPU Boosts AI
ImageNet — Accuracy Speech Recognition — AccuracyProcessor Trends
AI IS THE PATH TO EXASCALE

8. 8
Accelerating Targeted
Drug Development
Reducing Cancer Diagnosis
Error Rate by 85%
Predicting Disease
from Medical Records
AI IS REVOLUTIONIZING HEALTHCARE

9. 9
2014 2016
Higher Ed
Internet
Healthcare
Finance
Automotive
Others
EVERY INDUSTRY HAS AWOKEN TO AI
Organizations Engaged with NVIDIA on Deep Learning
1,549
19,439
Government
Developer Tools

10. 10
GTC, DLI, Inception
One Architecture Everywhere
Advance GPU Deep Learning Accelerate Every Framework
PaddlePaddle
Baidu Deep Learning
GPU DL-as-a-Service
NVIDIA AI COMPUTING PLATFORM

11. 11
NVIDIA Tesla GPU
NVIDIA DGX-1
ANNOUNCING NVIDIA & MICROSOFT
Cognitive Toolkit Optimized for DGX-1 & Azure Cloud
Azure Data Center
NVIDIA GPUDL Toolkit

12. 12
Cortana
Personal Assistant
Skype
Language Translator
Bing
Search Engine
Hololens
Augmented Reality
MICROSOFT COGNITIVE TOOLKIT
Engine Behind Microsoft Products, Now Democratizing AI for All

13. 13
170x Faster
(AlexNet images/sec)
78
13,000
CPU Server DGX-1
170X SPEED-UP OVER COTS SERVER
MICROSOFT COGNITIVE TOOLKIT SUPERCHARGED ON NVIDIA DGX-1
AlexNet training batch size 128, Dual Socket E5-2699v4, 44 cores CNTK 2.0b2 for CPU.
CNTK 2.0b3 (to be released) includes cuDNN 5.1.8, NCCL 1.6.1, NVLink enabled
8x Tesla P100 | 170TF FP16 | NVLink hybrid cube mesh

14. 14
CANDLE
Cancer Distributed Deep
Learning Environment
ANNOUNCING NVIDIA, DOE, NCI BUILD
AI PLATFORM FOR CANCER MOONSHOT

15. 15
Accelerate Discovery
of Cancer Therapies
Automate Analysis
of Treatment Effectiveness
Predict Drug Response
of Cancer Patients
June 2016 NCI Genomic Data Commons = 3PB Data
CANDLE FOR EXASCALE DEEP LEARNING
PRECISION MEDICINE FOR CANCER

16. 16
Fastest AI Supercomputer in TOP500
4.9 Petaflops Peak FP64
19.6 Petaflops Peak FP16
Most Energy Efficient Supercomputer
#1 Green500
9.5 GFLOPS per Watt
Rocket for Cancer Moonshot
CANDLE Development Platform
Common platform with DOE labs — ANL, LLNL,
ORNL, LANL
INTRODUCING DGX SATURNV
124 NVIDIA DGX-1 “Rocket for Cancer Moonshot”

17. 17
AI EnterpriseAI Transportation AI Factory AI Healthcare
PowerAI Toolkit
Minsky
CANDLE
NVIDIA AI COMPUTING FOR EVERY INDUSTRY