Verizon has strict requirements for IP/MPLS-based carrier Ethernet networks providing public Ethernet services. These include supporting large numbers of customers and interfaces through scalable architectures like PBB and MPLS. Services must conform to MEF definitions and provide high reliability through fast convergence and OAM. Extensive testing and certification are required to ensure performance meets standards.

1. Verizon‘s Requirements for IP/MPLS-Based
Carrier Ethernet Networks
Andrew G. Malis & Drew Rexrode
Verizon Communications
andrew.g.malis@verizon.com
charles.a.rexrode@verizon.com
Future-Net 2010

2. Introduction
• Public Ethernet services are exploding in popularity
• External Ethernet interface to the customer does not necessarily
mean ―Ethernet inside‖
– The Internet Engineering Task Force (IETF) has standardized
mechanisms for providing point-to-point and multipoint public Ethernet
services over IP/MPLS-based infrastructures
• This talk discusses Verizon‘s requirements for such solutions,
including functionality, conformance to Metro Ethernet forum (MEF)
service definitions, reliability, scalability, QoS, performance
monitoring, OAM, testing, and certification
2

3. Why Are Ethernet Services
Popular?
• Ubiquity and low cost of Ethernet interfaces in customer equipment,
universal experience with Ethernet in LANs, and perceived simplicity
• Successful marketing of the ―Ethernet‖ brand by vendors, IEEE,
MEF, and others
– Little resemblance with original DIX Ethernet specifications, from
physical layer on up (e.g., today‘s Ethernet is mostly point-to-point or
ring-based rather than CSMA-CD at the physical layer)
– Most everything has changed except for the basic frame format – and
jumbograms (large frames up to 9K bytes) change even that
• Plenty of competition and favorable pricing by service providers
3

4. MEF Carrier Ethernet Service
Definitions
Port-Based VLAN-Based
Connectivity Model
(All to One Bundling) (EVC identified by VLAN ID)
E-Line Ethernet Private Line Ethernet Virtual Private Line
(point-to-point EVC) (EPL) (EVPL)
E-LAN Ethernet Private LAN Ethernet Virtual Private LAN
(multipoint-to-multipoint EVC) (EP-LAN) (EVP-LAN)
E-Tree Ethernet Private Tree Ethernet Virtual Private Tree
(rooted multipoint EVC) (EP-Tree) (EVP-Tree)
• Three service types based on the three Ethernet Virtual Connection (EVC) types
• Two ―UNI Types‖ determine whether services are ‗private‘ or ‗virtual‘
– Port-based (All to One Bundling)  single EVC (transparency, but uses an entire port per service)
– VLAN-based  ‗N‘ EVCs per UNI (not as transparent, but multiple services per port)
• Services are defined by combination of connectivity model and ‗UNI Type‘
• Also Ethernet-based access services to Layer 3 VPNs or dedicated Internet access
4

5. ―Enterprise-Class‖ Ethernet
Limitations
• ―Enterprise-class‖ Ethernet switching has shortcomings as a basic
for public Ethernet services
– Few features for high availability in protocols or equipment
– Scaling limits on MAC addresses, VLAN IDs, and spanning tree
topology limit the size of native Ethernet networks
– Spanning tree routing may take seconds to (occasionally) minutes to re-
converge
• Early Ethernet providers found that enterprise-class Ethernet cannot
naively be deployed for reliable carrier services
5

6. Evolving and Scaling Ethernet
Services
• A typical ―early‖ public Ethernet service provider probably uses
Ethernet switches and Q-in-Q for customer separation
• Typical end user services are
– Ethernet Private LAN (EP-LAN)
– Ethernet Virtual Private LAN (EVP-LAN)
– Ethernet Private Line (EPL)
– Ethernet Virtual Private Line (EVPL)
– Each of these services requires the use of a provider VLAN tag
• As the service becomes successful, the provider will encounter the
usual Ethernet scaling limitations
– MAC address scaling
– VLAN tag scaling (4K customer limit)
– Switching capacity limits
6

7. Typical ―Early‖ Ethernet Service
Network
• Characterized by organic
PE growth driven by customer
PE
location
PE
GigE/LAG
PE • All switches are ―edge
switches‖
PE PE • May be some number of
PE
redundant links
PE
PE
PE
• 802.3ad Link Aggregation may
also be used for resiliency or
PE
PE
PE for additional BW between
PE switches
PE
PE • Flat network with spanning
PE
tree routing
– Network diameter is limited,
often to metro scope
PE – Provider Edge Switch
7

8. Emergence of ―Carrier Ethernet‖
• Limitations in enterprise-class Ethernet have led to the development
of ―Carrier Ethernet‖
• Meant to address unique requirements for carrier Ethernet services,
including Verizon‘s services
– Scaling to support a large number of customers
– Scaling to support large numbers of switches and customer interfaces
– Support both point-to-point (E-Line) and multipoint (E-LAN and E-Tree)
services
– Support for both port-based and VLAN-based services
– Support for QoS other than best-effort to support QoS-based SLAs
– Sub-second outage restoration and routing convergence to support
availability SLAs
– Policing and shaping to support sub-rate services (e.g., 200 Mbps
service on a physical GigE interface)
8

9. IETF Ethernet Services Support
• Point-to-point pseudowires (PWs) to carry layer two frames,
including Ethernet, over IP/MPLS networks
• Extends the MPLS LDP protocol to signal pseudowire establishment
• IETF extended PWs to a multipoint Ethernet service, VPLS (Virtual
Private LAN Service)
• PWs and VPLS extremely popular, implemented by most every
router vendor and in wide use by service providers world-wide
• Verizon uses both point-to-point PWs and VPLS to provide customer
Ethernet services
9

10. IP/MPLS Forum Ethernet Services
Support
• Extended IETF-defined PWs to support non-similar endpoint
interworking
– Supports point-to-point Ethernet-to-Frame Relay, Ethernet-to-ATM, and
ATM-to-Frame Relay interworking over MPLS PWs
– Very useful for multiservice convergence, and to support customers with
a variety of access methods
– Can support applications such as hub location with GigE access, and
low-speed Frame Relay spokes
– Supports interworking of IP packets via ARP Mediation, and bridged
services by interworking native Ethernet with Ethernet frames
encapsulated by FR or ATM
– Can also support VPLS endpoints with FR or ATM-attached customer
equipment
10

11. H-VPLS vs. VPLS+PBB
• VPLS and H-VPLS as originally defined by the IETF cannot meet
Carrier Ethernet service scaling requirements:
– 10s to 100s of thousands of EVCs
– Number of E-LAN bridging instances per edge switch/LER
– Up to millions of customer MAC addresses
• For these reasons, the IETF is now defining the combination of
VPLS in the core with Provider Backbone Bridges (PBB, 802.1ah) at
the edge
11

12. Scalable Network Architecture –
PBB + MPLS
PB PB
N-PE
BEB PBEB BEB
PB N-PE BCB BEB PB
BEB BCB N-PE
PBEB
PBEB
P P
BEB
BEB
/PE
PB
PB PB
• Metro Network Dedicated to Ethernet • Less touch points for cross-metro • MPLS core leveraged across
Service services multiple services (e.g., Ethernet, L3
VPNs)
• Investment Protection • PBB (B-VID) VPLS instance (reduce
PW Meshiness) • Scalable and mature control plane
• Hierarchy with PBB
• Broadcast containment per service • Leverage control plane to ease
• Administrative Traffic Eng. across core (via MMRP/BGP-AD) administration (BGP-Auto Discovery,
TE)
• Operations skill set / OSS Leverage • PBB MAC hiding
12

13. PBB-VPLS— MAC Scaling and
Customer-Addressing Awareness
PE-rs PE-rs
MTU-s MTU-s
MPLS
MPLS
MTU-s
H-VPLS PBB-VPLS MTU-s
No. of MAC addresses/node No. of MAC addresses/node
100,000s
100,000s
1000s Customer MACs
1000s
Backbone MACs
MTU-s PE-rs
0 MTU-s PE-rs
0
• ―Hub‖ PE-rs get visibility of 100,000s of MACs • MAC tables reduced: one B-MAC per
• High customer-addressing awareness node
• No customer-addressing awareness
13

14. PBB-VPLS Benefits — Service/Pseudowire
Scaling and Customer-Service Awareness
B
B
PE-rs PE-rs
B
B
B
B
B B B
MTU-s B
MTU-s MTU-s
H-VPLS
VPLS + PBB
No. of services-PW/node No. of services-PW/node
100,000s 100,000s
Customer services
10,000s 10,000s
Customer PWs
1000s 1000s
Backbone services
0 100s
Backbone PWs
0
MTU-S MTU-S PE-rs PE-rs MTU-S MTU-S PE-rs PE-rs
14

15. OAM Specifications
• The IEEE, ITU-T, and MEF have defined Ethernet OAM (Operations,
Administration and Maintenance) specifications to allow fault
detection and correction. These include:
– Link OAM: IEEE 802.3-2005, Clause 57
• Enables monitoring and troubleshooting of native Ethernet links
– Ethernet Local Management Interface (E-LMI): MEF 16
• Provides EVC status
• Enables automatic configuration of Customer Equipment (CE)
– Connectivity Fault Management (CFM): IEEE 802.1ag
• Enables monitoring and troubleshooting of VLANs within a network
• Supports multiple views (Customer, Service Provider, Operator)
– Service OAM: ITU-T Y.1731
• Extends CFM to include additional FM capabilities
• Performance Monitoring (PM)
15

16. Carrier Ethernet over MPLS
Testing
• Requires documentation and references
– MEF, BBF, IETF, IEEE
• Automation
– Definitive Parameters
16

17. 802.1 q-in-q/ad and 802.1ah - Service
Tunneling Testing Scenario (Example)
BCB1 BCB2
G.8031
Tunnel protection group
PBBN
BEB1 BEB2 BEB3 BEB4
ISID ISID ISID ISID ISID ISID
ISID
ISID BEB5 BEB6
Q-in-Q /82.1ad
ISID ISID ISID
CEs
VLAN 100 VLAN 100 VLAN 100 VLAN 100
0000.c004.0108 0000.c004.0106 0000.c004.0102 0000.c004.0105
0000.c004.0109 0000.c004.0107 0000.c004.0103
0000.c004.0104
17

18. Carrier Ethernet over MPLS
Certification
• ROI • Man Hours
• Time to Market • Resources
18

19. Conclusions
• Verizon‘s Carrier Ethernet services must meet stringent
requirements for:
– Conformance to Metro Ethernet forum (MEF) service definitions
– Scalability to support customer growth
– Reliability, resilience, OAM for troubleshooting and performance
monitoring, to support high service availability
– Standards-based certification
– Pre-deployment and post-deployment testing
19

20. Questions?
Thank you!
andrew.g.malis@verizon.com
charles.a.rexrode@verizon.com
20