Management and automation

1. Future Services on Janet14/06/2019
Tech 2Tech: Future Services on Janet 1

2. Management and Automation
14/06/2019 Tech 2 Tech: Future Services on Janet 2

3. Automation
»Ciena MCP
› “Manage, Control, Plan”
› Formerly “Blue Planet MCP” by Cyan
»Point-and-click provisioning around a single region
› Netpath provisioning through the core needs to be
“stitched” manually
› APIs into MCP and the backbone routers could enable
automation of this
»”Zero touch provisioning” for NTEs
14/06/2019 Tech 2 Tech: Future Services on Janet 3

4. Automation: Community
»Several talks in the Networkshop47 archives
› Ansible, python, other methods…
› https://www.jisc.ac.uk/events/networkshop47-09-apr-
2019/programme
»JiscMail NETWORK-AUTOMATION list
› https://www.jiscmail.ac.uk/cgi-
bin/webadmin?A0=NETWORK-AUTOMATION
»#uk_education on networktocode.slack.com
› Join at http://slack.networktocode.com14/06/2019 Tech 2 Tech: Future Services on Janet 4

5. Network Services
14/06/2019 Tech 2 Tech: Future Services on Janet 5

6. Layer 2VPNs
»Netpaths
»Netpath+ services are limited
› 10GE/100GE only
› 10GE relies on a 10x10GE mux
»OTN on the backbone 6500s
»Across the backbone nothing other than 100G is “a
wavelength”
› …and sometimes not even then
14/06/2019 Tech 2 Tech: Future Services on Janet 6

7. Cloud providers
»Dedicated connectivity whenVPNs aren’t enough
»Microsoft Azure ExpressRoute
› In service, several customers
»Amazon Web Service (AWS) Direct Connect
› Not much demand so far
› Looking at potential customers
»Others?
› Let us know
14/06/2019 Tech 2 Tech: Future Services on Janet 7

8. Where next with L2VPNs?
»NOC turnaround is pretty quick.
› Is there any requirement for portal-style provisioning?
› Complexities at either end that usually require human-
to-human contact.
»Virtualised networks?
› Network research
14/06/2019 Tech 2 Tech: Future Services on Janet 8

9. Layer 3VPNs
»Private IP networks
»Like a layer 2VPN, but with BGP peerings
»Can exchange private address space
»Janet routers currently limited to 32 L3VPNs (by license)
»Is there a demand?
› LHCONE; Small Cell project.
»GEANT MD-VPN
14/06/2019 Tech 2 Tech: Future Services on Janet 9

10. Virtualised Services
14/06/2019 Tech 2 Tech: Future Services on Janet 10

11. Network FunctionVirtualisation
»…spoken about NTE options earlier
»Where to virtualise functions?
› Core PoPs
› Openreach exchanges (Need to be wary of space
requirements)
14/06/2019 Tech 2 Tech: Future Services on Janet 11

12. Enhancing Current Services
14/06/2019 Tech 2 Tech: Future Services on Janet 12

13. IPv6
»…are we done yet?
› If not, what can we do?
14/06/2019 Tech 2 Tech: Future Services on Janet 13

14. IPv6
14/06/2019 Tech 2 Tech: Future Services on Janet 14
UK IPv6 deployment stats from APNIC

15. Optimising larger scale data transfers
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 15

16. Supporting larger scale data transfers
» Our end-to-end performance initiative (e2epi) is helping our members make the
most of their Janet connection
› Focused mainly on larger scale data transfers
› Typically scientific data such as synchrotron and cryo-EM (DLS), particle
physics (LHC), astronomy (SKA), climate (CEDA), genomics, etc.
› But approaches can be applied more broadly
» Web site for more info:
› https://www.jisc.ac.uk/rd/projects/janet-end-to-end-performance-initiative
› Pointers to workshops, guidance, etc.
» E2EPI mail list:
› https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=E2EPI
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 16

17. Janet end-to-end performance initiative
» Providing advice to members
› Use Janet for data transfers, not physical media!
› 1TB per hour is ~2Gbit/s; 100TB per day is ~10Gbit/s
› Firewalls designed for thousands of small flows may not cope well with
a small number of very large flows
› Consider your campus architecture – “Science DMZ”, or “Research Data
Transfer Zone” (RDTZ)
› Differentiate your science and general purpose “business” traffic
› Optimise data transfer nodes (DTNs) at your campus edge
› Measure your network characteristics; identify capability and issues
» We have interacted with 40—50 projects or organisations
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 17

18. Science DMZ (aka RDTZ) principles
Four design principles (https://fasterdata.es.net/science-dmz/ ):
1. “A network architecture explicitly designed for high-performance
applications, where the science network is distinct from the
general purpose network”
2. “The use of dedicated systems for data transfer”
3. "Performance measurement and network testing systems that
are regularly used to characterize the network and are available
for troubleshooting” (e.g., perfSONAR)
4. “Security policies and enforcement mechanisms that are tailored
for high performance science environments”
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 18

19. Science DMZ architecture example
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 19
Dark
Fiber
Dark
Fiber
10GE
Dark
Fiber
10GE
10G
Border Router
WAN
Science DMZ
Switch/Routers
Enterprise Border
Router/Firewall
Site / Campus
LAN
Project A DTN
(building A)
Per-project
security
policy
perfSONAR
perfSONAR
Facility B DTN
(building B)
Cluster DTN
(building C)
perfSONARperfSONAR
Cluster
(building C)
fasterdata.es.net

20. Measuring network characteristics
» When investigating network throughput issues, having persistent
network monitoring is really useful
» The Science DMZ model recommends perfSONAR
› https://www.perfsonar.net/
› Measure loss, latency, path, jitter, and (periodic) throughput
› Open source; install as Linux image or via packages
› Jisc is involved in perfSONAR development through the GÉANT project
» Web or CLI management
› Very flexible pscheduler CLI for tests
» Can set up “meshes” between multiple sites
› At-a-glance view of network performance
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 20

21. Jisc perfSONAR nodes and meshes
» We offer two 10G-connected perfSONAR nodes for you to test against:
› London: https://ps-londhx1.ja.net/toolkit/
› Slough: https://ps-slough-10g.ja.net/toolkit/
› You can freely configure perfSONAR tests against either of these
› Smart pscheduler avoids throughput test conflicts
» We provideVM-based hosting of meshes for communities, e.g.:
› UK GridPP - https://ps-dash.dev.ja.net/maddash-
webui/index.cgi?dashboard=UK%20Mesh%20Config
» We offer guidance on running perfSONAR on “small nodes”
› Small form factor Gigabyte Brix platforms, 1GbE, ~ £200 per system
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 21

22. perfSONAR toolkit web interface
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 22

23. Example perfSONAR mesh view - UK GridPP
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 23

24. Performance over time (Durham <> Birmingham)
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 24

25. Using perfSONAR to evaluate Science DMZ
»We did some E2EPI work with Southampton Uni around their
retrieval of experimental data from Diamond Light Source
› Moving 10-40TB of data a few times a year
»Researchers were using physical media
› Attempts to move data via network had been very poor
› Typically 200-300Mbit/s
»We advised on optimising connectivity for an internal filestore
› Led to researcher being able to copy 10TB of data overnight
› Typically able to obtain 2-4Gbit/s using Globus transfer tools
»Also ran a pilot DTN on their campus edge (Science DMZ)
› perfSONAR enables comparison between approaches
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 25

26. Jisc London pS -> Soton internal 10G filestore
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 26
Xmas break

27. Jisc London pS -> Soton external 10G DTN
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 27

28. Jisc’s reference DataTransfer Node (DTN)
» We have deployed a reference DTN in our Slough DC
› Specified with NVMe SSD; can read/write at 10Gbit/s
› Available to member sites for disk-to-disk tests
› Co-located with our Slough perfSONAR system
› Offers a Globus Connect endpoint (as used in Southampton case)
» Also have a second experimental DTN in Slough
› Allows tests of alternative protocols and tools
› e.g.,QUIC,TCP-BBR,WDT, … happy to help members with tests here
» Looking at options to offer a 100G DTN and perfSONAR
› Some good 100GbE material in our recent 100GbE workshop:
› https://www.jisc.ac.uk/events/100-gigabit-ethernet-networking-workshop-04-jul-2018
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 28

29. Working with the GÉANT project
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 29

30. New GÉANT project – GN4-3
» New GÉANT project: January 2019 – December 2022
› Approx €120m, of which €50m for fibre IRU sub-project
» All European national research and education networks (NRENs) take part
› Provides networking between the NRENs, and network services to the
NRENs and their members / customers, such as an-European eduroam
» Jisc is leading the new technologies and service development work
package within the project
» We will be exploring how we can draw on the project outputs to benefit
Janet and our members
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 30

31. GÉANT network technology & service development
»ThreeTasks in the work package
»Task 1: Enabling technologies
› White box (inc. P4 programming)
› QKD, OTFN, (petascale) DTNs, ultra low latency (LoLa)
»Task 2: Orchestration / virtualisation / automation
› Self-service portal (for connectivity services)
› Consensus building on approaches to automation
»Task 3: Network management and monitoring
› perfSONAR, NetMon, NMaaS, and CNaaS development
› WiFiMon, network flow analysis
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 31

32. jisc.ac.uk
Thank you
Tim Chown
tim.chown@jisc.ac.uk
12/06/2019 Tech 2 Tech Durham: Future Services on Janet 32