ON2020: future trends in optical networking
A cloud service provider’s perspective
Mark Filer
Optical Network Architecture, Azure Networking
Microsoft’s global datacenter network
United States
United States
Canada
Mexico
Venezuela
Colombia
Peru
Bolivia
Brazil
Argentina
Atlanta Ocean
Algeria
Mali
Niger
Nigeria
Chad
Libya Egypt
Sudan
Ethiopia
Dr Congo
Angola
Zambia
Nambia
South
Africa
Greenland
Svalbard
Sweden
Norway
United
Kingdom
France
Poland
Ukraine
Turkey
Saudi
Arabia
Iran
Kazakistan
India
Russia
Russia
China
Myanmar
(Burma)
Indian Ocean
Indonesia
Australia
Pacific Ocean
Pacific Ocean
Data centerOwned capacity
Future capacity
Leased capacity
Edge site
DCs and network sites not exhaustive
Azure WAN
Backbone
Region
Regional architecture
long haul
DCI
≤100 km
RNGRNG
DC DWDM
PAM4
<100km
PSM4 or
CWDM4
<600m
AOC
<20m
DC ecosystem
mQAM
>100km
mQAM
>100km
DC ecosystem
intra-DC inter-DC inter-region
#ofports(logscale)
Inside DC: technology barriers
Lane speed stalls
Inside DC: integration and controlling cost
Cost effective scaling
beyond 1.6T generation
not solved. All optical
packet switches to the
rescue?
Lane speed stalls
Optics in Package
Optics on Die
PCB
ASIC Optics
Fiber
Switch
PCB
ASIC Optics
Fiber
?
• sources: dedicated coherent transponders
• line system:
• proprietary closed system compatible with
sources
• proprietary NMS for control/monitoring
• expensive and power-hungry!
Yesterday
Inter-DC
to/from
far-end
RNG
• sources: 2-carrier 100G PAM4 (i.e. super-
channel)
• line system
• auto gain config
• auto chromatic dispersion comp
• fixed channel grid in C-band - maximally
lighting multiple fiber pairs day 1
Today
Inter-DC (2)
to/from
far-end
RNG
• sources: single-carrier 400G 16QAM (OIF
400ZR)
• line system
• Same but simplified specifications due to
coherent sources
• Re-use of PAM4 line systems possible
Tomorrow (2020+)
Inter-DC
to/from
far-end
RNG
• 1.6T: super-channel solution?
• Fiber relatively plentiful in DCI – likely still C-
band only
• Architectural optimizations: fast optical circuit
switching?
Beyond (2023+)
Inter-DC (3)
to/from
far-end
RNG
• sources: dedicated coherent transponders
• line system:
• proprietary closed system compatible with
sources
• proprietary NMS for control/monitoring –
internal SDN unaware of optical layer
• expensive and power-hungry!
Yesterday
Inter-region: optical architecture
UI
• sources: ICO with bandwidth-variable
100G/150G/200G QPSK/8QAM/16QAM
• line system
• OLS with full alien-wave support
• colorless / directional / flex-grid (C-band)
• p2p topology – no mesh or CDC
• lighting up segments in 2.4 Tb/s blocks
Today
Inter-region: optical architecture (2)
SwitchFabric
Layer 2/3 coherent line card
switch
chip
MACsec
switch
chip
MACsec
switch
chip
MACsec
switch
chip
MACsec
DSP
CFP2-
ACO
RX
TX
CFP2-
ACO
RX
TX
DSP
CFP2-
ACO
RX
TX
CFP2-
ACO
RX
TX
DSP
CFP2-
ACO
RX
TX
CFP2-
ACO
RX
TX
DSP
CFP2-
ACO
RX
TX
CFP2-
ACO
RX
TX
• sources: ICO with embedded “flex-QAM”
100G-600G, dynamic modulation changes ?
• line system
• C&L band day 1
• further disaggregation possible assuming
standardized data models and APIs
• in-house layer 0 controller
• CDC / mesh ? benefits aren’t clear… layer 3
Tomorrow (2020+)
Inter-region: optical architecture (3)
SwitchFabric
Layer 2/3 coherent line card
switch
chip
MACsec
switch
chip
MACsec
switch
chip
MACsec
switch
chip
MACsec
DSP
+
OBO
RX
TX
RX
TX
DSP
+
OBO
RX
TX
RX
TX
DSP
+
OBO
RX
TX
RX
TX
DSP
+
OBO
RX
TX
RX
TX
• maximally lighting fiber day 1 ? (more DCI-like)
• when does regen everywhere make sense ?
• further photonic integration will enable this
from cost/power standpoint
• benefits of SDM ? tough sell…
Beyond (2023+)
Inter-region: optical architecture (4)
Capacity granularity
• 70% capacity gain possible
100G
150G
99%
43%
200G
100G
100G
150G
100G
150G
200G
OFC 2016, paper M2J.2, Evaluation of Elastic Modulation Gains
in Microsoft’s Optical Backbone in North America, M. Ghobadi
Inter-region: interop
M.Filer, H. Chaouch, X. Wu, Toward
Transport Ecosystem Interoperability
Enabled by Vendor-Diverse Coherent
Optical Sources Over an Open Line
System, OSA JOCN vol 10 (2), 2018
Inter-region: interop (2)
DUT
bulk-
mod
bulk-
mod shaped ASEshaped ASE
bulk
mod
bulk
mod
M.Filer, H. Chaouch, X. Wu, Toward
Transport Ecosystem Interoperability
Enabled by Vendor-Diverse Coherent
Optical Sources Over an Open Line
System, OSA JOCN vol 10 (2), 2018
SiPh: Intra-/Inter-DC 100G Optic Allocation
SiPh parts averaging
<0.11% failure
Real-time poll 02/2018
Note: only 100G deployed optics
shown – not representative of
total DC ecosystem distributions
InP /
LiNbO3
100%
SiPh
0%
InP /
LiNbO3
59%
SiPh
41%
SiPh: Inter-region [100G, 150G, 200G]
Bandwidth growth and drivers
2015 2016 2017 2018
#100GDWDMPorts
Year
mark.filer@microsoft.com

OFC 2018 - ON2020: future trends in optical networking: a cloud service provider's perspective

  • 1.
    ON2020: future trendsin optical networking A cloud service provider’s perspective Mark Filer Optical Network Architecture, Azure Networking
  • 2.
    Microsoft’s global datacenternetwork United States United States Canada Mexico Venezuela Colombia Peru Bolivia Brazil Argentina Atlanta Ocean Algeria Mali Niger Nigeria Chad Libya Egypt Sudan Ethiopia Dr Congo Angola Zambia Nambia South Africa Greenland Svalbard Sweden Norway United Kingdom France Poland Ukraine Turkey Saudi Arabia Iran Kazakistan India Russia Russia China Myanmar (Burma) Indian Ocean Indonesia Australia Pacific Ocean Pacific Ocean Data centerOwned capacity Future capacity Leased capacity Edge site DCs and network sites not exhaustive
  • 3.
  • 4.
  • 5.
    DC ecosystem intra-DC inter-DCinter-region #ofports(logscale)
  • 6.
    Inside DC: technologybarriers Lane speed stalls
  • 7.
    Inside DC: integrationand controlling cost Cost effective scaling beyond 1.6T generation not solved. All optical packet switches to the rescue? Lane speed stalls Optics in Package Optics on Die PCB ASIC Optics Fiber Switch PCB ASIC Optics Fiber ?
  • 8.
    • sources: dedicatedcoherent transponders • line system: • proprietary closed system compatible with sources • proprietary NMS for control/monitoring • expensive and power-hungry! Yesterday Inter-DC to/from far-end RNG
  • 9.
    • sources: 2-carrier100G PAM4 (i.e. super- channel) • line system • auto gain config • auto chromatic dispersion comp • fixed channel grid in C-band - maximally lighting multiple fiber pairs day 1 Today Inter-DC (2) to/from far-end RNG
  • 10.
    • sources: single-carrier400G 16QAM (OIF 400ZR) • line system • Same but simplified specifications due to coherent sources • Re-use of PAM4 line systems possible Tomorrow (2020+) Inter-DC to/from far-end RNG
  • 11.
    • 1.6T: super-channelsolution? • Fiber relatively plentiful in DCI – likely still C- band only • Architectural optimizations: fast optical circuit switching? Beyond (2023+) Inter-DC (3) to/from far-end RNG
  • 12.
    • sources: dedicatedcoherent transponders • line system: • proprietary closed system compatible with sources • proprietary NMS for control/monitoring – internal SDN unaware of optical layer • expensive and power-hungry! Yesterday Inter-region: optical architecture UI
  • 13.
    • sources: ICOwith bandwidth-variable 100G/150G/200G QPSK/8QAM/16QAM • line system • OLS with full alien-wave support • colorless / directional / flex-grid (C-band) • p2p topology – no mesh or CDC • lighting up segments in 2.4 Tb/s blocks Today Inter-region: optical architecture (2) SwitchFabric Layer 2/3 coherent line card switch chip MACsec switch chip MACsec switch chip MACsec switch chip MACsec DSP CFP2- ACO RX TX CFP2- ACO RX TX DSP CFP2- ACO RX TX CFP2- ACO RX TX DSP CFP2- ACO RX TX CFP2- ACO RX TX DSP CFP2- ACO RX TX CFP2- ACO RX TX
  • 14.
    • sources: ICOwith embedded “flex-QAM” 100G-600G, dynamic modulation changes ? • line system • C&L band day 1 • further disaggregation possible assuming standardized data models and APIs • in-house layer 0 controller • CDC / mesh ? benefits aren’t clear… layer 3 Tomorrow (2020+) Inter-region: optical architecture (3) SwitchFabric Layer 2/3 coherent line card switch chip MACsec switch chip MACsec switch chip MACsec switch chip MACsec DSP + OBO RX TX RX TX DSP + OBO RX TX RX TX DSP + OBO RX TX RX TX DSP + OBO RX TX RX TX
  • 15.
    • maximally lightingfiber day 1 ? (more DCI-like) • when does regen everywhere make sense ? • further photonic integration will enable this from cost/power standpoint • benefits of SDM ? tough sell… Beyond (2023+) Inter-region: optical architecture (4)
  • 16.
    Capacity granularity • 70%capacity gain possible 100G 150G 99% 43% 200G 100G 100G 150G 100G 150G 200G OFC 2016, paper M2J.2, Evaluation of Elastic Modulation Gains in Microsoft’s Optical Backbone in North America, M. Ghobadi
  • 17.
    Inter-region: interop M.Filer, H.Chaouch, X. Wu, Toward Transport Ecosystem Interoperability Enabled by Vendor-Diverse Coherent Optical Sources Over an Open Line System, OSA JOCN vol 10 (2), 2018
  • 18.
    Inter-region: interop (2) DUT bulk- mod bulk- modshaped ASEshaped ASE bulk mod bulk mod M.Filer, H. Chaouch, X. Wu, Toward Transport Ecosystem Interoperability Enabled by Vendor-Diverse Coherent Optical Sources Over an Open Line System, OSA JOCN vol 10 (2), 2018
  • 19.
    SiPh: Intra-/Inter-DC 100GOptic Allocation SiPh parts averaging <0.11% failure Real-time poll 02/2018 Note: only 100G deployed optics shown – not representative of total DC ecosystem distributions
  • 20.
  • 21.
    Bandwidth growth anddrivers 2015 2016 2017 2018 #100GDWDMPorts Year
  • 22.