This document discusses heterogeneous computing trends and how they can create business value. It notes that the end of CPU frequency scaling around 2005-2007 led to approaches like core replication and exploiting concurrency. Fabrication technology advances have led to parity between CPUs, FPGAs, and GPUs on process nodes, with each expected to increase cores/estate/FLOPS significantly by 2030. Mapping applications to heterogeneous resources optimally is challenging due to expanding options and interconnect limitations. Value can be created through non-functional enhancements, functional enhancements, and business model transformations. A case study shows consistent 10x performance gains and optimal space efficiency through fine-grained use case based optimization across CPUs, FPGAs, and GPUs.

1. Heterogeneous Computing Trends
and
Business Value Creation
Thayaparan Sripavan
MillenniumIT
June 2017

2. End of ‘Free Lunch’
~2005 - 2007
Frequency Scaling

3. End of ‘Free Lunch’
~2005 - 2007
Frequency Scaling Core Replication
NUMA NU I/O

4. Design for ‘concurrency’
~2005 - 2007
Frequency Scaling Core Replication
Exploit
Concurrency
NUMA NU I/O

5. Parallel Processors
Gaming / Graphics Reconfigurable Digital Systems

6. Fabrication Technology
0
10
20
30
40
50
60
70
80
90
100
2006 2008 2011 2013 2016
CPU
FPGA
GPU
ProcessNode(nm)

7. Fabrication Technology
0
10
20
30
40
50
60
70
80
90
100
2006 2008 2011 2013 2016
CPU
FPGA
GPU
“Parity on process node!”
Roadmap visibility till – 2030 (IEEE)
CPU : 35x Cores
FPGA : 20x Estate
GPU : 50x FLOPS
ProcessNode(nm)

8. Evolution
More Cores , Integrated Memory, I/O & Graphics…
Models, Languages, Libs … Hard IP, Soft IP, SoC, High Level Synthesis…
{Scaling / Efficiency}
{Transformation}
2006 – 2016
+ +

9. Interconnects
2006 – 2016
Parity on process technology
increasing interconnect bandwidth
Interconnect latencies
Synchronization overhead
Source : National Instruments
8000x BW ; Latency Wall @ ~300 ns

10. Goldilocks Problem
Dense – Homogeneous
&
Distributed
Distributed
Heterogeneous
&
Distributed
{Extension}
{Transformation}

11. Challenges
• Heterogeneous design space – Continuously Expanding Horizon
• Constrained by interconnect speeds / synchronization – Inefficient Boundaries
• Coarse-grained design partitioning – Compromises in Solutions
• Deal with a mix of tools and methodologies – Complexity / Skill Specificity
Heterogeneous Approach - Differentiator or Killer?

12. Value Creation
Non-Functional
Enhancement
Functional
Enhancement
Paradigm Shifts
Business Model
Transformation
Legacy Heterogeneous
Technology
BusinessModel
ExistingNew
Transformation
Base
Constrained
by Technology
(Prohibitive)
Constrained by
Business Model
(Sub-optimal)
Enhancement

13. Value Creation
Non-Functional
Enhancement
Functional
Enhancement
Paradigm Shift
Business Model
Transformation
Predictably high
performance
Space efficiency
Energy efficiency

14. Case : Real Time Market Data
• Coarse – grained mapping to Processors
• Realtime – FPGA / Rest CPU
• Sub-component on FPGA
• Matching scalability
• Boundary inefficiency
• Use case based optimization
Consistent (>10x) performance at best space
efficiency.
• Best Execution Transparency
• Open Access
• Increased liquidity
• Balanced Performance & Configurability

15. The Convergence
NVLink
Multi-Chip
P2P PCIe
(PCIe 3)
3D FPGAs
Direct Cache Access (PCIe 3)
NVLink (IBM)
Direct Cache Access (PCIe 3)
CAPI (IBM)
FPGA On-Die (Intel)
P2P PCIe
(PCIe 3)
GPU Direct
(P2P PCIe)

16. Goldilocks Problem Gets Complex
Coarse-Grained
{Transformation}
Fine-Grained (Sub-Optimal )
{Extension}
Fine-Grained (Optimal)

17. Tackling the Challenge
“Fine-Grained Modular Designs”
“Dynamic / Late Mapping”“User Scheduled”
“Domain Specific Descriptions”

18. Value Creation
Non-Functional
Enhancement
Functional
Enhancement
Paradigm Shifts
Business Model
Transformation
Brute force solutions in real-
time for complex problems
Sensitive to emerging
data & predictive
analytics
Deep learning,
speculation & statistical
processing

19. Value Creation
Non-Functional
Enhancement
Functional
Enhancement
Paradigm Shifts
Business Model
Transformation
Real time visibility into risk,
margin and portfolio
optimization
Self healing systems
Technology agnostic
Post-Trade Description
Language (PDL)

20. Case : Real Time Risk

21. Thank you
Thayaparan Sripavan
thaya@millenniumit.com
June 2017