NVIDIA CEO and Co-Founder Jen-Hsun Huang presents the keynote at SC11.

1. NVIDIA Confidential
Exascale
An Innovator’s Dilemma
Jen-Hsun Huang, CEO
SC11, Seattle, Washington | Nov. 15, 2011

2. "Generally, disruptive innovations were
technologically straightforward,
consisting of off-the-shelf components
put together in a product architecture
that was often simpler than prior
approaches. They offered less of what
customers in established markets
wanted and so could rarely be initially
employed there. They offered a
different package of attributes valued
only in emerging markets remote from,
and unimportant to, the mainstream.”

3. NVIDIA Confidential

4. A History of Amazing Advances
1988 1998 2009
ZETTAFLOPS
EXAFLOPS
PETAFLOPS
TERAFLOPS
GIGAFLOPS
Cray T3E-1200
0.9 TF
Cray XT-5
“Jaguar”
1.8 PF, 7 MW
Cray Y-MP8
2.7 GF, 0.15 MW

5. Advances with Dennard Scaling
1988 1998 2009
ZETTAFLOPS
EXAFLOPS
PETAFLOPS
TERAFLOPS
GIGAFLOPS
Cray T3E-1200
0.9 TF
Cray XT-5
“Jaguar”
1.8 PF, 7 MW
Cray Y-MP8
2.7 GF, 0.15 MW
½ L every 4 yrs
8X – same P
1.68X per/W CAGR

6. End of Dennard Scaling
1988 1998 2009
ZETTAFLOPS
EXAFLOPS
PETAFLOPS
TERAFLOPS
GIGAFLOPS
Cray T3E-1200
0.9 TF
Cray XT-5
“Jaguar”
1.8 PF, 7 MW
Cray Y-MP8
2.7 GF, 0.15 MW
½ L every 4 yrs
8X @ 4X P
1.19X per/W CAGR

7. Supercomputing is Power Limited
1988 1998 2009
ZETTAFLOPS
PETAFLOPS
TERAFLOPS
GIGAFLOPS
100 PF, 20 MW
2012 2035
EXAFLOPS
Cray T3E-1200
0.9 TF
Cray XT-5
“Jaguar”
1.8 PF, 7 MW
Cray Y-MP8
2.7 GF, 0.15 MW
70 PF, 20 MW
1 EF, 20 MW
CPU-only
“Titan”
6 PF, 8.6 MW
2019 2022

8. CPUs Fast But Complex
Optimized for single-threaded
performance
~50X energy to schedule
instruction than the operation
~20X energy to move data across
chip than the calculation

9. Super Efficient Processors Needed
Many simple processors with
minimal overhead
Locality reduces data movement
energy
Poor single-threaded performance
Innovator’s Dilemma!

10. PRINCIPLE #1
Companies depend on customers
and investors for resources.
Clayton M. Christensen
(1997) The Innovator’s Dilemma:
When New Technologies Cause Great Firms to Fail

11. PRINCIPLE #2
Small markets don’t solve growth
needs of large companies.
Clayton M. Christensen
(1997) The Innovator’s Dilemma:
When New Technologies Cause Great Firms to Fail

13. GPU Computing
“New Market” Disruption

14. Disruptive technologies underperform
established products in mainstream markets.
Cheaper, smaller, and frequently more
convenient.
Clayton M. Christensen
(1997) The Innovator’s Dilemma:
When New Technologies Cause Great Firms to Fail

15. Nagasaki University
Professor Tsuyoshi Hamada’s
Homemade supercomputer

16. GPU Computing
>350,000,000 GPUs with CUDA
>1,000,000 Toolkit Downloads
>120,000 Active Developers
>475 Universities Teaching GPU Computing
100% HPC OEMs offer GPU Clusters

17. World’s First
Whole H1N1 Virus Simulation

18. Lifecycles of fish
in Australia
University of Melbourne
Stars and galaxies
12.5B years ago
University of Groningen
Neural networks in a
self-learning robot
The University of Plymouth
Directives
65x in 2 Days 5.6x in 5 Days 4.7x in 4 Hours

21. The Road to Exascale
70 PF, 20 MW
100 PF, 20 MW
GPU-Accelerated
“Titan”
20 PF, 8.6 MW
2012 2022
PETAFLOPS
1 EF, 20 MW
2019 2035
CPU-only
“Titan”
6 PF, 8.6 MW
1 EF, 20 MW
ZETTAFLOPS

22. GPU-Accelerated
“Titan”
20 PF, 8.6 MW
2012
ZETTAFLOPS
PETAFLOPS
2035
The Road to Exascale
70 PF, 20 MW
100 PF, 20 MW
1 EF, 20 MW
CPU-only
“Titan”
6 PF, 8.6 MW
1 EF, 20 MW
1 EF, 20 MW
2019 2022

23. GPU-Accelerated 1 EF, 20 MW
2012
ZETTAFLOPS
PETAFLOPS
2035
The Road to Exascale
70 PF, 20 MW
100 PF, 20 MW
“Titan”
20 PF, 8.6 MW
1 EF, 20 MW
100 EF, 20 MW
2019 2022
CPU-only
“Titan”
6 PF, 8.6 MW

24. Huge Markets Support GPU
Workstation
5M
Console
20M
PCs
400M
Mobile
1,000M

25. “Teraflops” 5 Watts
ASCI Red @ Sandia Labs
1997 2019

26. DEMO: BATTLEFIELD 3

27. “Tens” of Teraflops 100 Watts
Red Storm @ Sandia Labs
2004 2019

28. DEMO: ASSASSIN’S CREED

29. 1000 Watts
“Hundreds” of Teraflops
Blue Gene @ LLNL
2006 2019

30. DEMO: MAYA

31. DEMO: FUMEFX

32. One Disruption Enables Another

35. ‘Super’ Computing
From Super Computers to Super Phones