In this deck from PASC 2019, Keren Bergman from Columbia University presents: Flexibly Scalable High Performance Architectures with Embedded Photonics. "Computing systems are critically challenged to meet the performance demands of applications particularly driven by the explosive growth in data analytics. Data movement, dominated by energy costs and limited ‘chip-escape’ bandwidth densities, is a key physical layer roadblock to these systems’ scalability. Integrated silicon photonics with deeply embedded optical connectivity is on the cusp of enabling revolutionary data movement and extreme performance capabilities. Beyond alleviating the bandwidth/energy bottlenecks, embedded photonics can enable new disaggregated architectures that leverage the distance independence of optical transmission. We will discuss how the envisioned modular system interconnected by a unified photonic fabric can be flexibly composed to create custom architectures tailored for specific applications." Watch the video: https://wp.me/p3RLHQ-kuh Learn more: https://pasc19.pasc-conference.org/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

1. Rev PA1Rev PA1 1
Keren Bergman
Lightwave Research Lab, Columbia University
New York, NY, USA
Flexibly Scalable High Performance Architectures
with Embedded Photonics

2. Rev PA1Rev PA1 3
Performance/Communications Trends for Top 10 (2010-2018)
Sunway TaihuLight (Nov 2017) B/F = 0.004; Summit HPC (June 2018) B/F = 0.0005  8X decrease

3. Rev PA1Rev PA1 4
Performance and the Data Movement Energy Budget
• GFLOPs/Watt = GFlop/second / Joule/second = GFlop/Joule
• 14 GFLOPs/W (Summit)  72 pJ/FLOP
• Target: 50 GFLOPs/W  20 pJ/FLOP
• Energy per bit total budget (200 bits/FLOP):
14 GFLOPs/W: 72 pJ/FLOP 0.36 pJ/bit
50 GFLOPs/W: 20 pJ/FLOP 0.1 pJ/bit
• Scaling performance under ultra-tight energy budget:
– Raise cache hit rates (expanded caches, more reuse)
– Improve memory access (read, write) energy efficiency
– Improve data movement energy efficiency:
• Novel interconnect technologies and architectures

4. Rev PA1Rev PA1 8
What can photonics bring to the table?
The Photonic Opportunity for Data Movement
Reduce Energy Consumption Eliminate Bandwidth Taper
R. Lucas et al., “Top ten exascale research challenges,” DOE ASCAC subcommittee Report, 2014

5. Rev PA1Rev PA1 9
Maximizing the system‐wide energy efficient data movement benefits of photonics:
Approach 3 Key Concepts:

6. Rev PA1Rev PA1 12
clk
data
clk
data
clk
data
clk
dataTIA
Heater
clk gen
Silicon waveguide
Sensitivity of receiver @
10 Gb/s or 25 Gb/s
Coupler
Loss
Demux Array Penalty
Coupler
Loss
Coupler
Loss
Modulator Array Penalty
Available
Laser
power
Maximum Budget
Extra
Budget
Optical Power
Key Concept #1: Global Link Optimization
Max Bandwidth Density / Min Energy Design

7. Rev PA1Rev PA1 14
Passive Alignment – High Density Optical Fiber IO
• Demonstrated robustness to temperature
and fabrication variations.
• Temperature fluctuations of >80 degrees
have penalty of less than 0.6 dB on the
coupling efficiency.
• Designed and fabricated the funnel for
inverse taper coupling.
• Plug-and-play fiber to waveguide coupling

8. Rev PA1Rev PA1 16
Energy Optimized Performance – 800G
10Gb/s 25Gb/s
Power Penalty 15 dB 13.5 dB
-2.5 dBm -1.5 dBmLaser Power (dB)
0.56 mW 0.71 mWLaser Power (mW)
-17.5 dBm -15 dBmReceiver Sensitivity
10 Gb/s
2.2 pJ/b
25 Gb/s
3.2 pJ/b
• Aggregate bandwidth: 800G per link
• 8 groups of 10G x 10 vs 4 groups of 25G x 8

9. Rev PA1Rev PA1 17
Scaling chip ‘escape’ bandwidth density
~22mm
• 18 NVLink 2.0 ports  9 per long edge top/bottom
• 50GB/s per port (25GB/s each Tx/Rx)
• 1 NVLink ~ 2mm of linear edge
• 50GB/s per 2mm  200Gb/s/mm
NVIDIA NVSwitch
Each fiber ‘pin’ carries
80 s (scalable)Dense WDM Silicon Photonic:
• 250um fiber pitch
• 8 fiber links ~ over 2mm linear edge
• 80 s per fiber link; each  at 10Gb/s = 800 Gb/s per link
• 6.4 Tb/s per 2mm  3.2 Tb/s/mm

10. Rev PA1Rev PA1 18
• OC‐MCM: Optically Connected Multi‐chip Module
– Optical communication among interposers/MCM
– Enables fully flexible and scalable architectures
Approach: Deep Disaggregation
…network of Resources
CMP(s), GPUs
on interposer
Key Concept #2:
Ubiquitous Optically Connected‐MCM

11. Rev PA1Rev PA1 19
Electronic ICs
Photonic IC
(to be packaged)
Silicon Interposer
(100m thick)Silicon Interposer
(100m thick)
Photonic IC
Electronic ICs
Interconnected 2.5D/3D node (MCM)
Active interposer 3D
TI TIAs 8x8 Switch Lasers

12. Rev PA1Rev PA1 20
Current server
Current rack
Disaggregated
rack
Pool and
compose
Key Concept #3: Adaptive, Flexible Connectivity with Bandwidth Steering
Only “Power Up” Needed Optical Links
Inserting photonic switching – directly integrated in MCM – fully flexible connectivity

13. Rev PA1Rev PA1 21
Standard (‘Vanilla’) Fat Tree Architecture - Challenges
1 2 3 4 5 6 7 8
9 10 11 12
13 14 Electronic Packet
Switch (EPS)
Server
Pod 1 Pod 2Spine
Aggregation
ToR
• In a standard Fat Tree, ToR EPSs are only connected to their parent aggregation EPSs
• Inter-pod traffic (defragmentation) needs to traverse (+ hops) through spine to connect between pods
• Spine often oversubscribed, energy consuming large routers
Pre-Publication accepted to SC19: G. Michelogiannakis et al. “Bandwidth Steering for HPC using Silicon Nanophotonics”

14. Rev PA1Rev PA1 23
1 2 3 4 5 6 7 8 A B C D E F G HSiP Switch SiP Switch SiP Switch SiP Switch
Pod 1 Pod 2Spine
Aggregation
ToR 1 2 3 4 5 6 7 8
9 10 11 12
13 14
Electronic Packet
Switch (EPS)
Server
• Embedded SiP switches – flexible network within Fat Tree - direct ToR/Aggregation switch connections across pods
• Bypassing need to travel through congested higher layer links
• Lower latency - by providing a direct path that reduces hop count, and lowers utilization of upper layer
Flexible Fat Tree: Bandwidth steering across Pods

15. Rev PA1Rev PA1 24
Throughput of Upper Layer Links in Standard Fat-Tree Topology
Throughput of Upper Layer Links in Bandwidth-Steered Topology
56 72
25% execution
time
difference
System Experimental Results – Standard Fat Tree vs Flexible Fat Tree
• Operating skeletonized
GTC application with MPI
• Standard Fat Tree:
• all upper layer links used
• runtime = 72 sec
• Flexible PINE Fat Tree:
• direct lower layer connectivity
• only 4 upper layer links utilized
• runtime = 56 sec
Pre-Publication accepted to SC19: G. Michelogiannakis et al. “Bandwidth Steering for HPC using Silicon Nanophotonics”

16. Rev PA1Rev PA1 25
1 2 3 4 5 6 7 8 A B C D E F G H
4x4
SiP Switch 1
4x4
SiP Switch 2
4x4
SiP Switch 3
4x4
SiP Switch 4
Pod 1 Pod 2Spine
Aggregation
ToR 1 2 3 4 5 6 7 8
9 10 11 12
13 14
Electronic Packet
Switch (EPS)
Server
Flexible Fat Tree: Removal of Upper Layer Links
• PINE Flexible Fat Tree – provides direct connectivity for traffic at the lower layer
• Remove spine layer links to reduce energy consumption

17. Rev PA1Rev PA1 26
Throughput of Upper Layer Links in Fat-Tree Topology with Some Upper Links Removed
Throughput of Upper Layer Links in Bandwidth-Steered Topology with
Some Upper Links Removed
115
69% execution time
difference
56
System Results: Standard Fat Tree vs Flexible Fat Tree with Links Removed
• Flexible Fat Tree:
• direct lower layer connectivity
• Can remove upper layer EPSs =
reduced energy consumption
• Standard Fat Tree: all remaining
upper layer links congested
• runtime = 115 sec (vs 72 sec)
• Flexible Fat Tree:
• unaffected by removal of links,
remains at 56 sec runtime

18. Rev PA1Rev PA1 28
Summary: • Data Movement is Critical to any Future Performance Scaling
– Power Consumption
– Bandwidth Density (and Cost)
• Photonics: System‐Wide Tb/s per ‘wire’ and 1 pJ/bit
– Ultra bandwidth dense WDM photonic links
– Energy efficiency and cost via co‐integration 2.5D and 3D platforms
– High bandwidth Optically Connected Memory/GPUs/CPUs
• Deeply disaggregated Architectures
– Optical connectivity for flexibly assembled interconnectivity topologies
• Computer architecture landscape is changing rapidly ‐ Data Analytics, AI
– Optical bandwidth steering, adaptable architectures for scalability
– Ultimate energy efficiency – use only required resources for needed time period