This document discusses Imperial College's network infrastructure and plans to upgrade to 100GbE. It provides facts about the college's student and staff population and network size. It then describes the existing optical network and two use cases where 100GbE will help - the Slough datacenter which is growing beyond 10GbE, and the Science DMZ which is exceeding 40GbE capacity. It poses questions about how a 100GbE upgrade could be implemented across these networks.

1. David Stockdale
ICT Networks Group
100GbE at Imperial College

2. Facts and figures – Imperial

17,000 students

8,000 staff

Main campus – South Kensington, London

Under construction – White City, London

6 other large campuses (hospitals, Silwood Park)

10+ other sites (hospitals, halls, sports grounds)

2 datacentres – Slough & South Ken

Centralised ICT

3. Facts and figures – Network

Over 65,000 unique hosts on wired network

Over 60,000 unique hosts on wireless network

Over 20,000 concurrent wireless clients at peak time

~400 active comms rooms (CWCs)

~20 dark fibre links

~15 Ethernet circuits

40G to Janet via two 2x10G trunks

4. Optical network

All single-mode

CWC → BODF → distribution → core

WDM on dark fibre, mostly passive

Slough DC – 2x 40 channel diverse paths with amplifiers

Coloured optics in our equipment

Transponders for links to other equipment

5. Use case 1 – Slough datacentre

Originally a pair of MLAG distribution switches

… lots of 10GbE in one cage

We’ve grown several times since then

… now 4 cages, across 2 data halls

… and we pay by the core for fibre between cages

MLAG distribution was a problem

… EVPN / VXLAN to the rescue!

6. Use case 1 – Slough datacentre (2)

ICT and hosted servers in 2 cages

HPC split between 2 other cages

Recently taken delivery of 6PB storage for Research Computing

Storage capable of ~ 200Gbit, mostly to HPC

Uh oh!

… enter 100GbE

8. Use case 2 – Science DMZ

Currently a single router connected to border routers at 4x10G

Initially for HEP, soon to be standard service

Bypasses firewall

Avoids upgrading whole path

Cheaper equipment but fewer features

ECMP outbound, multiple subnets inbound = 40G!

9. Use case 2 – Science DMZ (2)

It works… maybe a little too well?

We all know where this is going…

11. Slough revisited

There’s already talk of a further cage

More spines? n² = a lot of fibre

Alternatively… 5 stage Clos

Not forgetting, 100G Science DMZ!

What might it look like…?

13. Any questions?