The document discusses Colt's network evolution towards next generation technologies. It describes moving from separate IP and Ethernet networks running different MPLS protocols to a single converged network using Segment Routing as the unified MPLS control plane. This provides simplified design and operations as well as improved scalability, resiliency and service flexibility. Ethernet services are also evolving from per-pseudowire implementations to EVPN using BGP for unified control and faster convergence. Virtualization of network functions like route reflectors is another focus. Key takeaways include a simplified architecture delivering faster service and lower costs.

1. Evolution Network
Amit Dhamija
Amit Dhamija
Network Engineering
amit.dhamija@colt.net | @AmitDhamijain
SEAMLESS NETWORK EVOLUTION AT COLT
APRICOT 2018
Kathmandu, Nepal

2. Agenda
• Colt’s Legacy Network
• Colt’s VISION for One global converged packet network
• Network design evolution towards NextGen technologies
• Experiences and Key Takeaways

3. COLT’S LEGACY NETWORK
IGP - ISIS
MPLS-LDP
BGP – AS 8220
IGP- OSPF
MPLS – RSVP/TE
T-LDP – PW
IP Network Ethernet Network
Services:
• E P2P
• Etree
• ELAN
Services:
• Internet access
• MPLS L3 VPN’s
(unicast and
multicast)

4. Colt’s VISION: One global converged packet network
Converged Network
Service flexibility
Simplified Design &
Architecture
Resiliency
Scalability
virtualisation
The integration of the network from the service, technology, system and
process point of view to offer innovative services, reduce provisioning
time, automate end-to-end orchestration and offer truly combined network
services.

5. Network design evolution towards NextGen technologies

6. New Network At a Glance
E2E Converged Network
(Ethernet / IP / Multicast Services)
• ISIS – IGP
• MPLS-SR
• RSVP
• BFD etc.
• EVPN – P2P/E-LAN
• IP Services –
L3VPN/Internet access
• M-VPN – NG multicast
VPN’s
ServicesInfrastructure Protocols

7. NextGen MPLS Evolution

8. Legacy Network – MPLS Design
PE-X PE-Y
MPLS-LDP is used as the control
plane protocol for IP Network.
• MPLS applications – L3
VPN’s/6PE/6VPE.
• Shortest IGP Path cost.
• Fast convergence.
• ECMP Paths.
MPLS-LDP
IP Network
MPLS-RSVP is used as control plane protocol
for Ethernet Network.
• MPLS application – VPWS/VPLS
• Traffic engineering with explicit paths.
• Disjoint paths.
• Fast convergence.
PE-X PE-Y
MPLS-RSVP
Ethernet
Network

9. Challenges with existing MPLS Design
• LDP-FRR coverage
issue.
• LDP generates lots of
needless labels and is
inefficient.
MPLS LDP MPLS RSVP
• RSVP traffic engineering LSP’s
scalability issues.
• RSVP traffic engineering ECMP
issues.

10. New Network – MPLS Design (Segment Routing)
Why We Chose Segment Routing
• Simplicity – Single MPLS design as standard.
• MPLS SR supports both modes under one network – (SR-SPT and SR-TE).
• Solves the ECMP for traffic engineering by using (Node SID & Adj-SID) algorithm.
• Simplifies the control plane for COLT’s new network – solves the RSVP scaling
issues/no dedicated MPLS protocols.
• Convergence – 100% protection with any topology.
• Interoperability with existing MPLS Protocols.
• Supports all the existing services.
.
https://datatracker.ietf.org/doc/draft-ietf-spring-segment-routing/

11. Segment Routing Control Plane
COLT
MPLS Segment
Routed Network
PE-
W
PE-N
PE-S
PE-E
• Deployed using IPv4 transport with MPLS data plane.
• Label Allocation – Only loopbacks with Index ID.
• SRGB – 100K.
• Range – 65K label space per node.
router isis COLT
net 49.0001.1120.6409.4142.00
interface Loopback0
passive
circuit-type level-2-only
address-family ipv4 unicast
prefix-sid index 4142 explicit-null

12. Segment Routing – TI LFA Coverage
Prefixes reachable in L2 Critical High Medium Low Total
Priority Priority Priority Priority
All paths protected 0 0 735 1295 2030
Some paths protected 0 0 0 0 0
Unprotected 0 0 0 0 0
Protection coverage 0.00% 0.00% 100.00% 100.00% 100.00%

13. Segment Routing Integration with IP/LDP Network
ISIS / MPLS-SRISIS /
MPLS-LDP
PE
LDP
Mapping
Servers
Mapping
Servers
Mapping
Servers
Mapping
Servers
IP Network
New-Network
PE-
W
PE-N
PE-E
PE-S
AdvertiseSR
labels for
legacy LDP
PE Nodes
• Mapping Severs – In path for our deployment.
• Features working with this deployment: L3 VPN’s/Internet access/Unicast-v4/v6/QOS etc.
• Our use case for Integration: LDP over SR / LDP-SR & SR-LDP.

14. Segment Routing Traffic Engineering – SR-TE
https://tools.ietf.org/html/draft-filsfils-spring-segment-routing-policy-04
PCEP for
tunnel req &
label imposition
BGP-LS for topology info
LSP Setup/ Tear
down(On-
Demand)
MPLS SR
PE-1 PE-2
• Ongoing evaluation in our engineering lab.
• SR-TE features - Disjoint paths with Node/Link and SRLG with static and dynamic paths.
Centralized
statefull
PCE
PCE
PCEP
LSPDB Synch
TED Synch
SR-TED
NSO

15. NextGen Services Evolution

16. Legacy Network – Services Design
PE-X
PE-Y
IRR
VRR
BGP is used as the control plane
protocol for IP-Services.
• Internet access –
IPv4/IPv6(6PE)
• Layer 3 VPN’s – VPNv4 and
VPNv6(6VPE)
• Deployed using RR’s.
PE-X PE-Y
L2-PE
L2-PE
L2-PE
G.8032/ERP G.8032/ERP
BGP/MP-BGP
IP Network
L2-PE
T-LDP
PE-Z
ERP in the access rings and T-LDP sessions in
the core ring for Ethernet Services.
• E-P2P – MPLS-VPWS (Psuedowires)
• ELAN – MPLS-VPLS (Mesh of
Pseudowires)
• E-TREE (Psuedowires based)

17. Challenges with existing Service Design
• Different implementation methods – Layer 3 & Layer 2 VPN’s.
• Complicated provisioning and troubleshooting.
• Layer 2 VPN’s control and data plane are mixed/flooding issues.
• Full mesh of pesudowires required in core network/scalability issues.
• Multihomed customers no support of active-active redundancy.

18. New Network – Services Design (BGP)
IRR
PE-2
PR’s
PE-3
ERR
ERR
• Simplicity - Single BGP protocol as
the standard for all the services in
network.
• IP and MAC distribution by using
“BGP”.
• No use of pseudowires for L2 VPN’s.
• Faster convergence for all BGP based
services.
• Ease of configuration, operations and
manageability.
BGP- Control Plane BGP- Control Plane
BGP Control Plane
AR-1
AR-2 AR-1
AR-2
Why We Chose EVPN Technology

19. Ethernet Services (EVPN) P2P Services – MP-BGP
https://tools.ietf.org/html/rfc8214
COLT
EVPN VPWS
AR-2
AR-1
Single BGP Ethernet
AD Route containing
[RD, RT, ESI,
Label(X),Eth-Tag
ID(AC)]
EVPN NLRI
AFI=25/SAFI=70
Show bgp l2vpn evpn
Route Distinguisher: 1111:1111 (default for vrf VPWS:1111)
*>i[1][0000.0000.0000.0000.0000][111]/120
10.91.126.111 100 0 I
*> [1][0000.0000.0000.0000.0000][222]/120
0.0.0.0 0 i
• Next Gen solution for Ethernet VPN’s.
• EVPN VPWS – Data plane uses MPLS-SR & Control plane – BGP based.
• Simple to deploy and scalable solution.

20. Ethernet Services (EVPN) E-LAN Services – MP-BGP
https://tools.ietf.org/html/rfc7432
COLT
EVPN MPLS
AR-3
AR-1
Four Routes - ELAN
1.Ethernet AD
2.Ethernet MAC
3.ESI Route
4.Inclusive multicast
route
AR-2
EVPN NLRI
AFI=25/SAFI=70
• EVPN-MPLS is similar to L3VPN MPLS.
• Known Unicast – MPLS-SR as transport & BUM – Ingress Replication with MPLS-SR.
• Rapid convergence – non zero ESI for single homed customers.
Show bgp l2vpn evpn
Route Distinguisher: 10.91.126.115:1
*>i[1][10.91.126.115:1][0000.0100.acce.5500.1401][4294967295
]/184
10.91.126.115 100 0 i
*>i[1][000a.5b7e.7300.0000.0000][0]/120
10.91.126.115 100 0 i
*>i[3][0][32][10.91.126.115]/80
10.91.126.115 100 0 I
*>i[2][0][48][0000.0200.0001][0]/104
10.91.126.115 100 0 i

21. IP Services using PW-HE Feature – MP-BGP
Logical
Interface
(PW-HE)
xconnect group xxxxx
p2p xxxxx
interface PW-Ether1
neighbor evpn evi 787 target 200 source 100
PE-2
AR-1
PR’s
PE-3
AR-2
EVPN
VPW
S
EVPNVPWS
MP-BGP
L3-VPN/GRT
PW-HE binds the L2
EVPN VPWS to the
GRT or L3-VPN’s
PW-HE -
VRF
EVPN VPWS
MP-BGP
EVPN VPWS
MP-BGP
• PW-HE using MPLS-SR as transport – Unicast
• PW-HE using MPLS-RSVP as transport - Multicast

22. NFV – Network function Virtualization

23. Colt’s Focus towards Network Function Virtualization
• Virtualization various network infrastructure segments
• RR / PCE Controller/ DDOS controllers etc.
• We deploy on commodity x86 hardware & dedicated VM’s for each network function.
• We run each instance as an independent island.
• Using KVM Hypervisor for our deployment.
x86 HW
KVM-Hypervisor
VNF VNF VNF

24. Example - Route Reflector Virtualization (vRR)
• Next Gen “BGP-RR”
deployed for all the services
by virtualization of control
plane function.
• Virtualized all the RR’s on
KVM Hypervisor.
• Dedicated VM’s for each RR
per Server.
• Centralized architecture
design, each PE forms
sessions with all three RR’s.
IRR-1 IRR-2 IRR-3
VPN-
RR-1
VPN-
RR-2
VPN-
RR-3
E-
RR-1
E-
RR-2
E-
RR-3
MPLS Layer 3
VPN’s RR
Ethernet RR
Server 1 Server 2 Server 3
Internet RR

25. Experiences and Key takeaways

26. Operations: Experiences from our Deployment
• All MPLS Services are fully supported by Segment Routing.
• L2 VPN’s / VPLS / EVPN / L3 VPN’s / IPv6.
• LDP-SR Inter working full multi-vendor support (Cisco/Juniper & Nokia, in our case).
• ISIS doesn’t advertise the merge flags when destination prefix looses the last backup path – Patch!
• Inconsistency in forwarding entries in LC and RP due to TI-LFA bug – Patch!
• No labels assignment for connected prefixes other then loopbacks on MPLS-SR ( SR feature Limitation).
• Native multicast solution for SR not yet available (Tree-SID, BIER etc.)
• Max SID Depth is a problem with SR-TE due to nature of source routing – Binding SID!
• No Interop issue in EVPN between Cisco/Juniper & Nokia.
• EVPN Flow based load balancing no support at this stage.
• Hierarchical EVPN for large networks is still a challenge.

27. Key Takeaways
• Single Infrastructure for all services..
ü Simplified the transport and service architecture.
ü Reduce the cost – opex and capex.
ü Faster Service delivery.
ü Better Orchestrator.
• It’s good to be an early investor in any new technology.
• Benefits Realized with Next Gen MPLS technologies – simplified design,
deployment & operations.
• Next Gen MPLS technologies proved easy to deploy, maintain and use.
• Questions?

28. Thank you
amit.dhamija@colt.net
For your time