Metanoia, Inc.
Critical Systems Thinking™




   Architectural Options for Metro
  Carrier-Ethernet Network Buildout:
       Analysis and Evaluation

                             Metanoia, Inc.
                             Email: experts@metanoia-inc.com
                             Web: http://www.metanoia-inc.com
                             Phone: +1-888-641-0082
     © Copyright 2009        Fax:     +1-888-641-0086
    All Rights Reserved
Metanoia, Inc.
                                                                                 Critical Systems Thinking™




  Workshop Outline
 Overview of Architectural Options


 Applicability of Key Technologies to Access/Metro/Core
          Q-in-Q (PB), MAC-in-MAC (PBB), 802.1Qay (PBB-TE) and PVT, IP/MPLS,
              MPLS-TP


 Basic Architectures and Their Analysis
          Parallel
          Overlay
          Uniform
          Mobile-Backhaul


 Hybrid Architectures and Their Analysis
 ©Copyright 2009             Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                        Operator Best Practices                                 M1-2
Metanoia, Inc.
                                                                                 Critical Systems Thinking™




  Workshop Outline
 Overview of Architectural Options


 Applicability of Key Technologies to Access/Metro/Core
          Q-in-Q (PB), MAC-in-MAC (PBB), 802.1Qay (PBB-TE) and PVT, IP/MPLS,
              MPLS-TP


 Basic Architectures and Their Analysis
          Parallel
          Overlay
          Uniform
          Mobile-Backhaul (presented in detail later)


 Hybrid Architectures and Their Analysis
 ©Copyright 2009             Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                        Operator Best Practices                                 M1-3
Metanoia, Inc.
   Building Carrier Ethernet Networks                                                                Critical Systems Thinking™




   Architectural Choices
                            Carrier Ethernet Network Architectural Choices




Parallel                       Overlay                                      Uniform                 Mobile
Networks                       Networks                                     Networks                Backhaul

  IP/MPLS + Ethernet in
  parallel, spanning same                                                       Ethernet only in
                                 Ethernet over IP /MPLS                                                Ethernet -based
  geographic area                                                               access/metro/core
  Ethernet à L2 services
  IP/MPLS à L3 services           IP/MPLS over Ethernet                         IP/MPLS only in
                                                                                access/metro/core     IP/MPLS-based


         Three basic options to architect a carrier Ethernet network
             Parallel, Overlay, Uniform

         Mobile Backhaul – specially designed for wireless data and mobile voice traffic
         Hybrids – numerous options, based on operator network status and technology
          availability, discussed extensively later in this module
    ©Copyright 2009                 Carrier Ethernet Technology Strategies & Evolving
   All Rights Reserved                            Operator Best Practices                                              M1-4
Metanoia, Inc.
Our Analysis Approach For Each                                                       Critical Systems Thinking™




Basic Architectural Option
 Network Architecture
          Overall architecture/layout of the network


 Logical Structure
          How the devices are logically connected at the service layer


 Physical Realization (of the logical structure)
          What constructs are used at the transport layer (not PHY transport,
              rather the Carrier Ethernet transport) to realize the logical structure


 Protocol Stack
          How tunnelling/interworking occurs in different segments of the
              network

 ©Copyright 2009                 Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                            Operator Best Practices                                 M1-5
Metanoia, Inc.
                                                                                 Critical Systems Thinking™




  Workshop Outline
 Overview of Architectural Options


 Applicability of Key Technologies to Access/Metro/Core
          Q-in-Q (PB), MAC-in-MAC (PBB), 802.1Qay (PBB-TE) and PVT, IP/MPLS,
              MPLS-TP


 Basic Architectures and Their Analysis
          Parallel
          Overlay
          Uniform
          Mobile-Backhaul (presented in detail later)


 Hybrid Architectures and Their Analysis
 ©Copyright 2009             Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                        Operator Best Practices                                 M1-6
Metanoia, Inc.
Applicability of Key Technologies:                                                   Critical Systems Thinking™




Assessment of Suitability
 Reference is to different network segments within the metro
       network itself
          Metro Access (referred to in this workshop as “access”)
          Metro Aggregation (referred to in this workshop as “metro”)
          Metro Core (referred to in this workshop as “core”)
 “Eyeball” assessment of which technology is suitable for which
       particular segment
 Ratings Scale based on inherent properties of technology
             Optimally suited – excellent fit/match for segment
                   Moderately suited – reasonable fit/match for segment
                    Minimally suited – could be used, but would not be effective
                  Not suited – Not recommended for this segment (either
              incapable or overkill for the segment)

 ©Copyright 2009                 Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                            Operator Best Practices                                 M1-7
Metanoia, Inc.
        Applicability of Key Technologies:                                                                                     Critical Systems Thinking™




        A Comparative Rating
                                                   Metro Access                     Metro Aggregation                  Metro Core

                                 Q-in-Q                                                                                  
    Connectionless




                              802.1ad (PB)       Good with small # flows             Scalability strained w/      Unscalable (due to SPT,
                                                    (few hundreds)                 large # flows (thousands)      flooding, convergence)

                              MAC-in-MAC                                                                                 
                             802.1ah (PBB)       Suitable for access, but                                         Unstable and unscalable
                                                                                     Ideally scaled for agg.
                                                      likely overkill                                               (SPT/flooding, mgt.)
                                802.1 Qay                                                                              
                              (PBB-TE) and        Waste, as config./mgt.             Useable, gives TE and         TE/ manageability help
                                  PVT             overhead not worth it                 manageability              long-lived paths, QoS
Connection-




                                                                                                                      
 Oriented




                                 IP/MPLS        Mature tech., suitable, but         Signaling, automatic CP       Long indstry experience,
                                                  needs configuration                   simplify confi.           significant best practices

                                T-MPLS/                                                                               
                                MPLS-TP            Suitable, but config.            Controllability, similarity    Better manageability of
                                                   overhead significant            to SDH/SONET, resilience          aggregated traffic


                     The above assessment/rating is based on the inherent properties/capabilities of each technology
                     (and not its maturity, availability, and standardization status, which are examined in Module 3)
         ©Copyright 2009                             Carrier Ethernet Technology Strategies & Evolving
        All Rights Reserved                                        Operator Best Practices                                                       M1-8
Metanoia, Inc.
                                                                                 Critical Systems Thinking™




  Workshop Outline
 Overview of Architectural Options


 Applicability of Key Technologies to Access/Metro/Core
          Q-in-Q (PB), MAC-in-MAC (PBB), 802.1Qay (PBB-TE) and PVT, IP/MPLS,
              MPLS-TP


 Basic Architectures and Their Analysis
          Parallel
          Overlay
          Uniform
          Mobile-Backhaul (presented in detail later)


 Hybrid Architectures and Their Analysis
 ©Copyright 2009             Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                        Operator Best Practices                                 M1-9
Metanoia, Inc.
                                                                              Critical Systems Thinking™




Parallel Networks: Network Architecture
 Two isolated networks (ships-in-the-night)
          Different fibers, lambdas, or SDH/SONET channels


 Separate systems (for Ethernet and IP/MPLS) or independent
       cards in the same system


 IP/MPLS network can be:
          p2p, mp2mp


 Ethernet network can be PB, PBB, PBB-TE based, and be:
          p2p, mp2mp, p2mp


 ©Copyright 2009          Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                     Operator Best Practices                                M1-10
Metanoia, Inc.
                                                                                  Critical Systems Thinking™




Why Parallel Networks?
If majority of metro services are ...
 Ethernet
          IP/MPLS connections handled by backhauling them from
              spoke(s) to metro hub, where IP routing is done
          IP routing capability not needed in spoke COs



 IP or IP/MPLS
          Mesh IP/MPLS routers by co-locating with Ethernet switches in
              all or some CO/POPs




 ©Copyright 2009              Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                         Operator Best Practices                                M1-11
Metanoia, Inc.
                                                                              Critical Systems Thinking™




Why Parallel Networks?
 Useful when the provider L2 (Ethernet) and L3 (IP/MPLS)
       networks have
          Shared POPs/Cos
          MSPPs with both IP/MPLS and Ethernet capabilities



 Applicable for operators that wish for both technologies to
       co-exist with service aggregation




 ©Copyright 2009          Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                     Operator Best Practices                                M1-12
Metanoia, Inc.
                                                                                                                           Critical Systems Thinking™




Parallel Networks: Network Layout
                                                                                                         IP/MPLS
                      Co-located IP/MPLS                                                                 Linecard
                       Router & Ethernet                                                       IP/MPLS
                            Switch


 IP/MPLS Router                                 IP/MPLS or T-MPLS
                                               or MPLS-TP Network


                                                                                        Ethernet
                                                                                                                    Hybrid system with two
                                                                                                                      types of line cards

                                                                                                             Ethernet Linecard



 Ethernet Switch
                                             PB, PBB, PBB-TE Network




 ©Copyright 2009                           Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                      Operator Best Practices                                                            M1-13
Metanoia, Inc.
                                                                                                                    Critical Systems Thinking™




Parallel Networks: Logical Structure
                                                                                                   IP/MPLS
                      Co-located IP/MPLS                                                           Linecard
                                                                                Logical IP /MPLS
                       Router & Ethernet                                             Link
                            Switch


IP/MPLS Router                                                                 Logical IP /MPLS
                                                                                    Link




                                                                                                              Hybrid system with two
                                                                                                                types of line cards
                                                                               Logical                 Ethernet Linecard
                                                                            Ethernet Link



Ethernet Switch
                                                                                                        Ethernet Switch



                                                                               Logical
                                                                            Ethernet Link



 ©Copyright 2009                           Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                      Operator Best Practices                                                     M1-14
Metanoia, Inc.
 Parallel Networks:                                                                                              Critical Systems Thinking™




 IP/MPLS Network Realization
     Bi-directional                                   Logical Mesh
     logical links                                   IP/MPLS Layer
                                                                                  (a) Logical MPLS Network
               L1                                          L2

                                 L3


                       L4                         L5




                                                                  L3                                mp2p LSPs

                                Physical links can have
                            Ethernet or SONET /SDH framing                                                  L2



(b) LSP Realization of
    Logical Network                                                 L1                         L5
                              p2p LSPs



                                                  L4

  ©Copyright 2009                          Carrier Ethernet Technology Strategies & Evolving
 All Rights Reserved                                     Operator Best Practices                                                  M1-15
Metanoia, Inc.
  Parallel Networks:                                                                                      Critical Systems Thinking™




  Ethernet Network Realization
    Logical Mesh at                              Logical links
    Ethernet Layer                                                              (a) Logical Ethernet Network




(b) E-Line Realization of
     Logical Network
                                                                                              E-Line

                                                                                     (Realization via
                                                                                     Ethernet entities)
                            Physical links with
                             Ethernet framing
   ©Copyright 2009          Carrier Ethernet Technology Strategies & Evolving
  All Rights Reserved                     Operator Best Practices                                                          M1-16
Metanoia, Inc.
                                                                                       Critical Systems Thinking™




Parallel Networks: Protocol Stack
 In parallel networks, the protocol stack is trivial, since each
       network is fully independent of the other
 For Ethernet it is:
                          Ethernet
                           Header                  Varies depending on whether the Ethernet
                          Payload                  network is PB, PBB, or PBB-TE-based



 For IP/MPLS it is:

                         LSP-Label
                                                   May vary if further nesting of LSPs is used
                          VC-Label

                          IP Header

                          Payload

 ©Copyright 2009          Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                     Operator Best Practices                                         M1-17
Metanoia, Inc.
                                                                                                           Critical Systems Thinking™




Parallel Networks: Assessment
Pros                                                                 Cons
 Flexible                                                            Higher CapEX relative to uniform
          IP/Ethernet networks/services can                           network
              grow independently                                              Need 2x fibers, lambdas, or
 Fault isolation                                                                  SDH/SONET channels
          Faults in one n/w, do not affect                                   More systems, cabling; thus,
              service in the other                                                 CO/POP space & power
                                                                              Different mgt. s/w – one for each
 Service independence
                                                                                   network: IP, Ethernet
          L2 svcs. not affected by L3 svcs.
                                                                      Greater OpEX
              and visa-versa
                                                                              More vendors, maintenance
                                                                              Greater number of s/w and mgt.
                                                                                   tools


 Good choice if a provider has either an Ethernet- or IP/MPLS-dominant network,
      and wishes to expand into the other incrementally

 ©Copyright 2009                     Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                Operator Best Practices                                                  M1-18
Metanoia, Inc.
                                                                                Critical Systems Thinking™




Overlay Networks: IP/MPLS over Ethernet
 Metro is built using Ethernet switches
          PB, or PBB, or PBB-TE capable


 Requires transport network
          SDH/SONET, lambda’s or fiber to interconnect switches


 Ethernet layer is transmission network
          Interconnects IP/MPLS routers – using p2p, mp2mp, p2mp constructs


 Ethernet network is oblivious to IP/MPLS layer


 IP/MPLS routers (U-PE, N-PE, PE) connected in logical mesh or
       hub-and-spoke, or some mix of two

 ©Copyright 2009            Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                       Operator Best Practices                                M1-19
Metanoia, Inc.
Why Overlay Networks with                                                         Critical Systems Thinking™




IP/MPLS-over-Ethernet?
If majority of metro services are ...
 Ethernet
          IP/MPLS connections handled by backhauling them from
              spoke(s) to metro hub, where IP routing is done
          IP routing capability not needed in spoke COs



 IP or IP/MPLS
          Mesh IP/MPLS routers by co-locating with Ethernet switches in
              all or some CO/POPs




 ©Copyright 2009              Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                         Operator Best Practices                                M1-20
Metanoia, Inc.
IP/MPLS-over-Ethernet                                                                                        Critical Systems Thinking™




Overlay Network: Network Layout


                                                                                              Co-Located
                                                                                               Systems   IP/MPLS
                Single IP/Ethernet
                    Platform
                                                                                               Ethernet

                                                       PB, PBB, or PBB-TE
                                                        Ethernet Network




        CE




                       Customer                              Co-Located
                      Edge Device                             Systems
 ©Copyright 2009                          Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                                                                                           M1-21
                                     CE                 Operator Best Practices
Metanoia, Inc.
IP/MPLS-over-Ethernet                                                                                 Critical Systems Thinking™




Overlay Network: Logical Structure

                                                                                      Logical Links




             Single IP/Ethernet
                 Platform


                                             PB, PBB, or PBB-TE
                                                  Network




                Logical Mesh




 ©Copyright 2009                  Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                             Operator Best Practices                                                M1-22
Metanoia, Inc.
    IP/MPLS-over-Ethernet                                                                                Critical Systems Thinking™




    Overlay Network: Realization via E-Line
                                                                                  p2p E-Line service
                                                                                (offered by underlying
                                                                                  Ethernet network)




Single IP/Ethernet
    Platform




                                                                              The “transmission network” here is the
                     Logical Links
                                                                               Ethernet network
                                                                              Logical network is thus realized using
                                                                               this network’s entities: E-lines

     ©Copyright 2009                 Carrier Ethernet Technology Strategies & Evolving
    All Rights Reserved                            Operator Best Practices                                                M1-23
Metanoia, Inc.
IP/MPLS-over-Ethernet                                                                     Critical Systems Thinking™




Overlay Network: Realization via E-LAN

Single IP/Ethernet
    Platform                                                 mp2mp Ethernet
                                                                (E-LAN)




                                                               The “transmission network” here is the
                                                                Ethernet network
                                                               Logical network is thus realized using
                                                                this network’s entities: E-LAN

 ©Copyright 2009      Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                 Operator Best Practices                                                M1-24
Metanoia, Inc.
  IP/MPLS-over-Ethernet                                                                                                                      Critical Systems Thinking™




  Overlay Network: Protocol Stack
          CE


                             IP/MPLS                                                                     Co-located
                                                                                                          Systems IP/MPLS
                                               Combined IP/
                                              Ethernet Switch
         IP Service                                                            IP Service
                                                                                                                Ethernet
           CE                                                              Ethernet Svc
                                                                                                                                                         CE
                             Ethernet Svc                             Ethernet Network




  IP Service or                                             PB/PBB               PB/PBB                  PB/PBB
                                                            PBB-TE               PBB-TE                  PBB-TE
IP/MPLS Service
                                           MPLS             MPLS                 MPLS                    MPLS               MPLS
                    Ethernet              Ethernet         Ethernet             Ethernet             Ethernet              Ethernet              Ethernet
                        IP                   IP                IP                  IP                      IP                 IP                    IP
                    Payload               Payload          Payload              Payload              Payload               Payload               Payload




                                                           PB/PBB               PB/PBB                   PB/PBB
 Ethernet Service                                          PBB-TE               PBB-TE                   PBB-TE
                               Ethernet                    Ethernet             Ethernet             Ethernet                         Ethernet
                                Payload                    Payload              Payload              Payload                          Payload


   ©Copyright 2009                                   Carrier Ethernet Technology Strategies & Evolving
  All Rights Reserved                                              Operator Best Practices                                                                    M1-25
Metanoia, Inc.
Overlay Networks IP/MPLS-over                                                                                 Critical Systems Thinking™




-Ethernet: Assessment
Pros                                                                    Cons
      Lower CapEX relative to Parallel n/wks                                Lower flexibility relative to Parallel
             Potentially fewer systems (need only                            network
              one set for underlying txn. network –                                  IP/MPLS & Ethernet networks coupled
              here Ethernet)                                                         Cannot grow independently
             Less fiber – due to common transport
                                                                             Resilience issues
             Simpler mgt – due to fewer systems,
              and hybrid systems (IP+Ethernet)
                                                                                     Faults in Ethernet network may affect
                                                                                      IP/MPLS service
      Lower OpEx
             Potentially, one vendor
             Single s/w and mgt. tools
             Single dept. to manage network




 Good choice if a provider has primarily an Ethernet infrastructure, and wishes to
      expand into IP/MPLS services

 ©Copyright 2009                        Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                   Operator Best Practices                                                  M1-26
Metanoia, Inc.
                                                                                               Critical Systems Thinking™




Overlay Networks: Ethernet-over-IP/MPLS
      Metro is built using IP/MPLS devices
             p2p, mp2mp IP/MPLS, MPLS-TP/T-MPLS network or any mix


      Requires one transport network
             SDH/SONET, lambda’s or fiber to interconnect IP/MPLS routers


      MPLS layer is transmission network
             Interconnects Ethernet switches – using p2p (VPWS) or mp2mp (VPLS) constructs


      VPWS interconnection à
             MPLS network provides a transparent tunnel (network interworking)


      VPLS interconnection à
             MPLS network participates in Ethernet switching via VSI (service interworking)


      Ethernet switches connected as logical mesh or hub-and-spoke or mesh-star
       topology

 ©Copyright 2009                         Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                    Operator Best Practices                                  M1-27
Metanoia, Inc.
Why Overlay Networks with                                                         Critical Systems Thinking™




Ethernet-over-IP/MPLS?
If majority of metro services are ...
 IP/MPLS
          Ethernet connections are backhauled to metro hub, where
              Ethernet switching is done
          Ethernet switching capability not needed in spoke COs/PoPs



 Ethernet
          Mesh Ethernet switches by co-locating with IP/MPLS routers in
              all or some CO/POPs




 ©Copyright 2009              Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                         Operator Best Practices                                M1-28
Metanoia, Inc.
Ethernet-over-IP/MPLS                                                                                          Critical Systems Thinking™




Overlay Network: Network Layout


                                                                                                 Co-Located
                                                                                                  Systems
                                                                                                              Ethernet
                      Single IP/Ethernet
                          Platform                                                                 IP/MPLS

                                                            IP/MPLS or T-MPLS
                                                            or MPLS-TP Network




  IP SVC



                                      Customer Edge                                 Co-Located
            CE
                                          Device                                     Systems
               Ethernet
                 SVC

 ©Copyright 2009                           Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                      Operator Best Practices                                                M1-29
                                CE
Metanoia, Inc.
Ethernet-over-IP/MPLS                                                                                 Critical Systems Thinking™




Overlay Network: Logical Structure

                                                                                          Logical Links




                 Single IP/Ethernet
                     Platform


                                              IP/MPLS or T-MPLS
                                              or MPLS-TP Network




                      Logical Mesh




 ©Copyright 2009                      Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                 Operator Best Practices                                            M1-30
Metanoia, Inc.
Ethernet-over-IP/MPLS                                                                                 Critical Systems Thinking™




Overlay Network: Realization via VPWS
                              p2p PW in IP/MPLS
                                   network


                                                                                            Logical Links




               Single IP/Ethernet
                   Platform




                      Logical Mesh




 ©Copyright 2009                        Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                   Operator Best Practices                                          M1-31
Metanoia, Inc.
Ethernet-over-IP/MPLS                                                                         Critical Systems Thinking™




Overlay Network: Realization via VPLS



        Single IP/Ethernet
                                                                   PW full-mesh (VPLS) with
            Platform                                                  VSI-based routing




 ©Copyright 2009             Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                        Operator Best Practices                                             M1-32
Metanoia, Inc.
Ethernet-over-IP/MPLS                                                                                                                      Critical Systems Thinking™




Overlay Network: Protocol Stack
              CE
                                                                                                                                                         CE

                             IP Service                                                                  Co-located
                                                                                                          Systems         Ethernet


Ethernet SVC                                                                                                IP/MPLS
                                                                         IP Service
                 Ethernet
   CE                                                              IP/MPLS Network                                                                    CE
                   SVC

                                                                        Ethernet SVC



                                                       MPLS                 MPLS                   MPLS
Ethernet                                               Label                Label                  Label
Service                                                PWE3                 PWE3                   PWE3
                                                       Label                Label                  Label
                                       PB/PBB         PB/PBB               PB/PBB                 PB/PBB              PB/PBB
                                       PBB-TE         PBB-TE               PBB-TE                 PBB-TE              PBB-TE
           Ethernet                    Ethernet      Ethernet             Ethernet               Ethernet         Ethernet                Ethernet
           Payload                     Payload       Payload              Payload                Payload          Payload                 Payload




IP Service                                             MPLS                 MPLS                      MPLS
                            Ethernet                                        Label
                                                                                                                               Ethernet
                                                       Label                                          Label
                               IP                       IP                    IP                       IP                         IP
                            Payload                  Payload              Payload                Payload                       Payload

 ©Copyright 2009                                  Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                             Operator Best Practices                                                                     M1-33
Metanoia, Inc.
  Ethernet-over-PB-over-IP/MPLS                                                                                                   Critical Systems Thinking™




  (VPWS) Interworking: Protocol Stack
               CE
                                                                                                                                               CE


                                                                                                     Co-located
                                                                                                      Systems          Ethernet


Ethernet SVC
                                                                                                       IP/MPLS
    CE                                                              IP/MPLS, T-MPLS,                                                         CE
                                                                    MPLS-TP Network


                     Ethernet SVC



                                           One-to-one
                                          Mapping (No
                                          MAC Lookup)   LSP Label    (MPLS)               LSP Label

                                                        VC Label     (PWE3)                VC Label

                                 C-DA                     C-DA                               C-DA                  C-DA

             C-DA                C-SA                     C-SA                               C-SA                  C-SA            C-DA

             C-SA                5-tag                    5-tag                              5-tag                 5-tag           C-SA

             C-tag               C-tag                    C-tag                              C-tag                 C-tag           C-tag

            Payload             Payload                  Payload                           Payload                Payload         Payload

          Customer          Ethernet Switch             IP/MPLS
           Packet               Packet                Router Packet
   ©Copyright 2009                               Carrier Ethernet Technology Strategies & Evolving
  All Rights Reserved                                          Operator Best Practices                                                             M1-34
Metanoia, Inc.
 Ethernet-over-PBB-over-IP/MPLS                                                                                             Critical Systems Thinking™




 (VPWS) Interworking: Protocol Stack
             CE
                                                                                                                                          CE


                                                                                                 Co-located
                                                                                                  Systems        Ethernet


Ethernet SVC
                                                                                                     IP/MPLS
   CE                                                           IP/MPLS, T-MPLS,                                                         CE
                                                                MPLS-TP Network

                 Ethernet SVC         One-to-one
                                    Mapping (no MAC
                                        lookup)   LSP Label      (MPLS)                  LSP Label

                                                     VC Label    (PWE3)                   VC Label

                                B-DA                   B-DA                                B-DA                 B-DA

                                B-SA                   B-SA                                B-SA                 B-SA

                                B-tag                  B-tag                               B-tag                B-tag

                                I-tag                   I-tag                               I-tag               I-tag

                                C-DA                   C-DA                                C-DA                 C-DA
              C-DA              C-SA                   C-SA                                C-SA                 C-SA            C-DA

              C-SA              S-tag                  S-tag                               S-tag                S-tag            C-SA

              C-tag             C-tag                  C-tag                               C-tag                C-tag           C-tag

             Payload        Payload                   Payload                             Payload              Payload         Payload

           Customer      Ethernet Switch             IP/MPLS
 ©Copyright 2009
              Packet         Packet          Carrier Ethernet Technology Strategies & Evolving
                                                   Router Packet
All Rights Reserved                                        Operator Best Practices                                                            M1-35
Metanoia, Inc.
  Ethernet-over-PB-over-IP/MPLS                                                                                                    Critical Systems Thinking™




  (VPLS) Interworking: Protocol Stack
              CE
                                                                                                                                               CE


                                                                                                     Co-located
                                                                                                      Systems          Ethernet


Ethernet SVC
                                                                                                       IP/MPLS
    CE                                                             IP/MPLS, T-MPLS,                                                          CE
                                                                   MPLS-TP Network

                                         Ethernet
                     Ethernet SVC    bridging VPLS
                                        (VSI MAC
                                         Lookup)



                                                       LSP Label    (MPLS)               LSP Label

                                                        VC Label    (PWE3)                VC Label

                                 C-DA                     C-DA                              C-DA                   C-DA

             C-DA                C-SA                     C-SA                              C-SA                   C-SA            C-DA
                                              VPLS
             C-SA                5-tag         VSI        5-tag                             5-tag                  5-tag           C-SA

             C-tag               C-tag                    C-tag                             C-tag                  C-tag           C-tag

            Payload             Payload                 Payload                            Payload                Payload         Payload
          Customer          Ethernet Switch            IP/MPLS
           Packet               Packet               Router Packet
   ©Copyright 2009                               Carrier Ethernet Technology Strategies & Evolving
  All Rights Reserved                                          Operator Best Practices                                                              M1-36
Metanoia, Inc.
 Ethernet-over-PBB-over-IP/MPLS                                                                                               Critical Systems Thinking™




 (VPLS) Interworking: Protocol Stack
               CE
                                                                                                                                             CE


                                                                                                   Co-located
                                                                                                    Systems        Ethernet


Ethernet SVC
                                                                                                       IP/MPLS
    CE                                                             IP/MPLS, T-MPLS,                                                         CE
                                                                   MPLS-TP Network
                                         Ethernet
                   Ethernet SVC      bridging VPLS
                                        (VSI MAC
                                                       LSP Label    (MPLS)                 LSP Label
                                         Lookup)
                                                        VC Label    (PWE3)                  VC Label

                                  B-DA                   B-DA                                 B-DA                B-DA

                                  B-SA                   B-SA                                 B-SA                B-SA

                                  B-tag                  B-tag                                B-tag               B-tag

                                  I-tag      VPLS         I-tag                               I-tag               I-tag

                                  C-DA        VSI        C-DA                                 C-DA                C-DA
                C-DA                                                                                                               C-DA
                                  C-SA                   C-SA                                 C-SA                C-SA
                C-SA              S-tag                  S-tag                                S-tag               S-tag            C-SA

                C-tag             C-tag                  C-tag                                C-tag               C-tag            C-tag

              Payload         Payload                   Payload                             Payload              Payload          Payload

            Customer       Ethernet Switch             IP/MPLS
             Packet            Packet                Router Packet
  ©Copyright 2009                              Carrier Ethernet Technology Strategies & Evolving
 All Rights Reserved                                         Operator Best Practices                                                             M1-37
Metanoia, Inc.
                                                                                   Critical Systems Thinking™




Uniform Networks: Network Architecture
 One technology used throughout (for all three metro segments)
          Access, Aggregation, and Core


 Technology could be either Ethernet or IP/MPLS


 Ethernet-only option
          Access can be 802.1ad (Q-in-Q)
          Metro can be 802.1ah (MAC-in-MAC)
          Core can be 802.1Qay (Provider Backbone Bridging – TE)


 IP/MPLS-only option
          Access uses p2p PWs in hub-and-spoke design
          Metro uses a meshed VPLS design
          Core uses a hierarchically meshed H-VPLS design

 ©Copyright 2009               Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                          Operator Best Practices                                M1-38
Metanoia, Inc.
                                                                                       Critical Systems Thinking™




Uniform Networks: Ethernet-Only
 Network Architecture
          Comprised typically of access, metro, core segments
          Access à PB (802.1ad), metro à PBB (802.1ah), core à PBB/PBB-TE
              (802.1Qay)


 Logical Structure
          Service-Layer is Ethernet, with remote CE’s knowing each other’s MAC address
          Devices at service-layer connected by p2p or p2mp Ethernet tunnels


 Physical Realization (of logical structure)
          Transport layer could be Ethernet or SDH/SONET


 Protocol Stack
          Illustrated ahead ...


 ©Copyright 2009                   Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                              Operator Best Practices                                M1-39
Metanoia, Inc.
     Uniform Networks: Ethernet Only                                                                                                            Critical Systems Thinking™




     Network Layout and Protocol Stack
      Access Network               Aggregation Network                      Core Network                Aggregation Network             Access Network
         (802.1ad)                      (802.1ah)                            (802.1Qay)                      (802.1ah)                     (802.1ad)
                                     Provider Backbone                                                     Provider Backbone           Provider Bridging (PBB)
     Provider Bridging (PBB)                                                     BCB
                                       Bridging (PBB)                                                        Bridging (PBB)
                                                                                                                                                       Last Mile
 Last Mile                         IB-BEB            BCB           B-BEB                      B-BEB            B-BEB         IB-BEB                 PE           CE
CE            PE

                              PB                           B-BEB                                                       BCB
                                                                           PBB – Traffic
                                                                        Engineered (PBB-TE)
                                                                                                                                      PB
CE
               PE                                                                                                                                    PE          CE
                                                                                                                             IB-BEB
                                    IB-BEB           BCB           B-BEB                        B-BEB          B-BEB
        802.1ad/Q-in-Q                  802.1ah                                                                       802.1ah                   802.1ad/Q-in-Q
                                     encapsulation                                BCB
        encapsulation                                                                                              decapsulation                decapsulation
                                             B-DA              B-DA        - Pinned paths                 B-DA
                                                                           - Based only on
                                             B-SA              B-SA          B-DA, B-SA, B-Tag            B-SA
                                                                           - No STP
                                            B-Tag              B-Tag                                      B-Tag
                                                                           - No MAC learning
                                             I-Tag             I-Tag                                      I-Tag
                                                                           Switching based on pre -
                      C-DA                   C-DA              C-DA        configured fwding tables       C-DA                         C-DA

      C-DA            C-SA                   C-SA              C-SA                                       C-SA                         C-SA                 C-DA

      C-SA            S-Tag                 S-Tag              S-Tag                                      S-Tag                        S-Tag                C-SA

      C-Tag           C-Tag                 C-Tag              C-Tag                                      C-Tag                        C-Tag               C-Tag

     Payload         Payload             Payload              Payload                                    Payload                      Payload             Payload


      ©Copyright 2009                                      Carrier Ethernet Technology Strategies & Evolving
     All Rights Reserved                                                 Operator Best Practices                                                                 M1-40
Metanoia, Inc.
                                                                                       Critical Systems Thinking™




Uniform Networks: IP/MPLS-Only
 Network Architecture
          Comprised typically of access, aggregation (metro), core segments
          Two options exist:
          Option 1: Access/Metro (Aggregation) à p2p spoke PWs to backbone (core)
              PEs, Core à H-VPLS mesh between backbone PEs
          Option 2: Access à p2p spoke PWs to aggregation PEs, Metro
              (Aggregation)/Core à H-VPLS mesh between all aggregation PEs

 Logical Structure
          Service-Layer is still Ethernet; remote CE’s/nPE’s learn MAC addresses
          Devices at service-layer connected by p2p MPLS LSPs


 Physical Realization (of logical structure)
          Construct used at transport layer are IP/MPLS p2p PWs in p2p LSPs


 Protocol Stacks
          Illustrated ahead ...
 ©Copyright 2009                   Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                              Operator Best Practices                                M1-41
Metanoia, Inc.
     Uniform Networks: IP/MPLS Only                                                                                                     Critical Systems Thinking™




     Network Layout and Protocol Stack (1)
      Access Network                Aggregation Network                Core Network                           Aggregation Network Access Network
         (p2p PW)                        (p2p PW)                  (PW full mesh, H-VPLS)                          (p2p PW)          (p2p PW)

     Access + Aggregation spanned by spoke PWs                                              Full PW Mesh
                                                                                   P         per service
                     Spoke PW          PE             P             nPE                          nPE              P          PE                             CE
                                                                                                                                               uPE
CE           uPE
                                                                                                                        P
                                P                                                                                                   P


                                                           PE
CE             uPE                                                                                                                              uPE         CE
                                                                                                                             PE
                                       PE             P                                          nPE              P
                                                                    nPE
      VPWS (p2p) PW                                                                                                   Spoke PW              VPWS (p2p PW)
                                                                    VPLS            P            VPLS                                        decapsulation
       encapsulation
                                                                encapsulation                decapsulation



                        LSP                   LSP    Tunnel                LSP                            LSP                       LSP
                       Label                 Label                        Label    Tunnel                Label  Tunnel             Label
                                                     Spoke
                     PW Label               PW Label                   PW Label       Mesh             PW Label Spoke             PW Label
                                                      (PW)                           (VPLS                       (PW)
      C-DA             C-DA                  C-DA                         C-DA     Instance)            C-DA                       C-DA               C-DA
      C-SA             C-SA                  C-SA                         C-SA                           C-SA                      C-SA               C-SA
     S/C-Tag          S/C-Tag               S/C-Tag                     S/C-Tag                        S/C-Tag                    S/C-Tag            S/C-Tag
     Payload          Payload               Payload                     Payload                        Payload                    Payload            Payload

      ©Copyright 2009                                     Carrier Ethernet Technology Strategies & Evolving
     All Rights Reserved                                                Operator Best Practices                                                          M1-42
Metanoia, Inc.
     Uniform Networks: IP/MPLS Only                                                                                                       Critical Systems Thinking™




     Network Layout and Protocol Stack (2)
      Access Network                Aggregation Network                Core Network                       Aggregation Network Access Network
         (p2p PW)                     (PW full mesh)               (PW full mesh, H-VPLS)                   (PW full mesh)       (p2p PW)
                                                         Aggregation and Core spanned by Full PW Mesh
                                                                                    P
                                                                                           PW Full Mesh
                     Spoke PW          nPE          P               PE                            PE              P         nPE                              CE
                                                                                                                                                uPE
CE           uPE
                                                                                                                      P
                                P                                                                                                  P


                                                           PE
CE             uPE                                                                                                                     Spoke uPE             CE
                                                                                                                            nPE         PW
                                       nPE           P                                            PE              P
                                                                    PE
      VPWS (p2p) PW                     VPLS                                                                                  VPLS
                                                                                    P                                                       VPWS (p2p PW)
       encapsulation                encapsulation                                                                         decapsulation
                                                                                                                                             decapsulation



                       LSP                     LSP                         LSP    Tunnel                 LSP                        LSP    Tunnel
                      Label   Tunnel          Label                       Label                         Label                      Label
                                                                                                                                           Spoke
                     PW Label Spoke          PW Label                    PW Label    Mesh              PW Label                   PW Label
                               (PW)                                                                                                         (PW)
                                                                                    (VPLS
      C-DA            C-DA                    C-DA                        C-DA    Instance )            C-DA                       C-DA           C-DA
      C-SA             C-SA                   C-SA                        C-SA                          C-SA                        C-SA                C-SA
     S/C-Tag          S/C-Tag                S/C-Tag                      S/C-Tag                      S/C-Tag                    S/C-Tag             S/C-Tag
     Payload          Payload                Payload                      Payload                      Payload                    Payload             Payload

      ©Copyright 2009                                     Carrier Ethernet Technology Strategies & Evolving
     All Rights Reserved                                                Operator Best Practices                                                            M1-43
Metanoia, Inc.
Uniform Networks: IP/MPLS-Only                                                     Critical Systems Thinking™




Observations
 Trade-offs exist between Option 1 and Option 2 in terms of MAC
       learning and PW scalability

 Option 1: PW full mesh in core, spoke PWs upto core nPEs
          nPEs must learn all customer MAC addresses
          PW mesh confined to core, thus only core PEs must be meshed
          Requires one PW mesh per customer’s service instance


 Option 2: PW full mesh in agg. + core, spoke PWs upto agg. nPEs
          nPEs learn all customer MACs, but fewer than in Option 1
          PW mesh, however, is between nPEs in all aggregation networks (thus
              much larger with more PWs than in Option 1)
          Requires one PW mesh per customer’s service instance



 ©Copyright 2009               Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                          Operator Best Practices                                M1-44
Metanoia, Inc.
                                                                                 Critical Systems Thinking™




  Workshop Outline
 Overview of Architectural Options


 Applicability of Key Technologies to Access/Metro/Core
          Q-in-Q (PB), MAC-in-MAC (PBB), 802.1Qay (PBB-TE) and PVT, IP/MPLS,
              MPLS-TP


 Basic Architectures and Their Analysis
          Overlay
          Parallel
          Uniform
          Mobile-Backhaul (presented in detail later)


 Hybrid Architectures and Their Analysis
 ©Copyright 2009             Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                        Operator Best Practices                                M1-45
Metanoia, Inc.
Our Analysis Approach For                                                      Critical Systems Thinking™




Hybrid Architectures
 Network Overview
          Overall architecture/layout of the network



 Protocol Stack
          Explain tunneling/interworking in different network segments



 Assessment
          Key observations about the architecture
          Benefits and drawbacks of approach




 ©Copyright 2009           Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                      Operator Best Practices                                M1-46
Metanoia, Inc.
                                                                                  Critical Systems Thinking™




Hybrid Architectures: Overview
 Involve a combination of IP/MPLS and Ethernet (PB, PBB, PBB-TE)


 Access network, customer-edge to provider-edge, is common and
       typically one of three Ethernet types (so not shown in figures)
          Vanilla Ethernet (802.1d)
          V-LAN capable Ethernet (802.1q)
          S-VLAN capable Ethernet or Q-in-Q (802.1ad)



 Two principal hybrid architectures
          H-VPLS with PB or PBB Aggregation
          H-VPLS with MPLS Aggregation


 ©Copyright 2009              Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                         Operator Best Practices                                M1-47
Metanoia, Inc.
                                                                                                                     Critical Systems Thinking™




Hybrid Architectures Taxonomy
                                          H-VPLS with PB/PBB Aggregation




                        H-VPLS w/ Homogeneous                                        H-VPLS with Heterogeneous
                           PBB Aggregation                                              PB/PBB Aggregation



          Type I Svc. Interface              Type II Svc. Interface              Modified PB PE           Modified PBB PE
          (S-Tag (B-VID) as delimiter)       (I-SID as delimiter)




                                          H-VPLS with MPLS Aggregation




        PBB-capable uPE                   PBB-capable nPE                                 PBB Migration
       (PBB uPE, for short)              (PBB nPE , for short)


                                                  PB service frames              PBB service frames       PB and PBB service
                                                  over H-VPLS core               over H-VPLS core         frames over H-VPLS core

 ©Copyright 2009                            Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                       Operator Best Practices                                                     M1-48
Metanoia, Inc.
H-VPLS with PBB Aggregation:                                                  Critical Systems Thinking™




Type I Interface -- Overview
 Type I i/f is a B-tagged i/f with B-VID as service delimiter


 Handoff between BCB and N-PE is B-tagged PBB frame


 N-PE itself can be transparent to the frames
          Treats them as 802.1ad frames
          Only need support 802.1ad-style frames




 ©Copyright 2009          Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                     Operator Best Practices                                M1-49
Metanoia, Inc.
H-VPLS with PBB Aggregation:                                                                                                                 Critical Systems Thinking™




Type I i/f – Operation & Protocol Stack
                                                                             PBB

                                 Ethernet                                  MPLS/PW                                     Ethernet


     CE          U-PE                       BCB             N-PE      Raw or Tagged-           N-PE           BCB                        U-PE
                                                                     mode Ethernet PW                                                                    CE

                              PBB
                                                             X             IP/MPLS
                                                                                                 X
                                                                                                                              PBB
                 A1 IB-BEB                                  C1                                  D1                                    IB-BEB B1


                                             S-Tag I/f                                                   S-Tag I/f
                                     (B-VID is Svc. delimiter )                                  (B-VID is Svc. delimiter )
                                                                                                All 3 operational modes
                                                                   Supports VPLS for B-Tag      supported:
                                                                   (need not be PBB-aware)      - Port Mode
                                                                                                - VLAN Mode
                                                                                                - VLAN Bundling Mode

                                                                          LSP-Label

                                                                           VC-Label

                             B-DA                   B-DA                     B-DA                     B-DA                    B-DA

                             B-SA                   B-SA                     B-SA                     B-SA                    B-SA

                             B-Tag                  B-Tag                   B-Tag                     B-Tag                   B-Tag

                             I-Tag                  I-Tag                    I-Tag                    I-Tag                   I-Tag

          C-DA               C-DA                   C-DA                     C-DA                     C-DA                    C-DA                C-DA

          C-SA               C-SA                   C-SA                     C-SA                     C-SA                    C-SA                C-SA

       S/C-Tag           S/C-Tag                  S/C-Tag                  S/C-Tag                   S/C-Tag              S/C-Tag               S/C-Tag

       Payload           Payload                  Payload                  Payload                   Payload              Payload               Payload
 ©Copyright 2009                                      Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                                 Operator Best Practices                                                                   M1-50
Metanoia, Inc.
H-VPLS with PBB Aggregation                                                                        Critical Systems Thinking™




Type I Interface: Assessment
 Application of the overlay model
 Use case: SP has converged to an MPLS core, but prefers Ethernet aggregation
      to connect 802.1ad-based access networks
 A “steady-state” model, once all aggregation networks are 802.1ah-capable



Benefits                                                         Drawbacks
 No new functionality in VPLS PE                                 Needs extra replication in core
 Reduces C-MAC learning and PW                                        (relative to w/o I-SID bundling)
       mesh in core PEs                                           Unknown unicast, brdcast, or
          Due to I-SID bundling, multiple                             mcast in a single svc. instance (I-
              svc. instances map to one bridge                         SID) leads to full brdcast in core
              domain (B-VID)                                              May be addressed by per-I-SID
 Separates service layer (I-SID)                                              flood containment
       from network layer (B-VID)
 ©Copyright 2009                 Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                            Operator Best Practices                                              M1-51
Metanoia, Inc.
H-VPLS with PBB Aggregation:                                                   Critical Systems Thinking™




Type II Interface -- Overview
 Type II i/f: I-tagged i/f with I-SID as service delimiter


 B-Tag is locally significant in PBB cloud, not sent over core


 PE must support B-BEB and VPLS functionality
          Must interpret I-Tag




 ©Copyright 2009           Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                      Operator Best Practices                                M1-52
Metanoia, Inc.
H-VPLS with PBB Aggregation:                                                                                                              Critical Systems Thinking™




Type II i/f – Operation & Protocol Stack
                                          PBB                                   PBB                                          PBB

                                       Ethernet                               MPLS/PW                                       Ethernet


      CE          U-PE                          B-BEB           N-PE                              N-PE              B-BEB               U-PE
                                                                                                                                                       CE

                                   PBB
                                                                    X          IP/MPLS
                                                                                                    X                            PBB
                                                                                           B-BEB D1                              PBB
                  A1                                               C1 B-BEB                                                               B1


                                                       I-tag I/f                                      I-tag I/f (Type II)
                                                      (Type II)                                  (I-SID is svc. delimiter)
                       B-VID locally significant in                                               2 Mapping options
                       PBB, not xported over core                                                 -- I-SID à VPLS
                                                                         Must support B-BEB
                                                                         and VPLS capability      -- I-SID àB-VID à VPLS



                                                             Internal B-VID,
                                      B-BEB removes           enables I-SID LSP-Label
                                       PBB-specific             bundling
                                          B-Tag                              VD-Label

                                 B-DA                                           B-DA                                            B-DA

                                 B-SA                   B-DA                    B-SA                     B-DA                   B-SA

                                 B-Tag                  B-SA                   B-Tag                     B-SA                  B-Tag

                                  I-Tag                 I-Tag                   I-Tag                       I-Tag               I-Tag

           C-DA                  C-DA                   C-DA                    C-DA                     C-DA                   C-DA            C-DA

           C-SA                  C-SA                   C-SA                    C-SA                     C-SA                   C-SA            C-SA

        S/C-Tag                 S/C-Tag               S/C-Tag                 S/C-Tag                   S/C-Tag               S/C-Tag          S/C-Tag

        Payload                 Payload               Payload                 Payload                   Payload               Payload          Payload

 ©Copyright 2009                                        Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                                   Operator Best Practices                                                               M1-53
Metanoia, Inc.
H-VPLS with PBB Aggregation                                                                           Critical Systems Thinking™




Type II Interface: Assessment
 Application of the overlay model
 Use case: SP has converged to MPLS core, but prefers Ethernet aggregation to
      connect 802.1ad-based access networks
 Good for customers with mcast traffic (w/o mcast pruning fn. at VPLE-PE)



Benefits                                                       Drawbacks
      Less replication in core, w/o needing                        N-PE complex with new capability –
       per-I-SID flood containment                                   needs B-comp for I-SID processing


      Supports tightly & loosely-coupled                           Potentially larger # of PWs in core, for
       service domains                                               same # of services (relative to Type I)
                                                                            Increased segregation of svc.
                                                                             instances (by I-SID) over disjoint PW
                                                                             meshes



 ©Copyright 2009               Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                          Operator Best Practices                                                   M1-54
Metanoia, Inc.
H-VPLS with Heterogeneous Access                                               Critical Systems Thinking™




Modified PB PE -- Overview
 Used to interoperate existing 802.1ad (PB) networks with
       new 802.1ah (PBB) aggregation networks


 Both networks connect to an H-VPLS core/backbone


 PE interfacing with PB network must support VPLS and IB-
       BEB functionality
          Mapping of S-VID to I-SID
          I-SID bundling into B-VID and mapping to a VPLS instance




 ©Copyright 2009           Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                      Operator Best Practices                                M1-55
Metanoia, Inc.
H-VPLS with Modified PB PE                                                                                                      Critical Systems Thinking™




Operation & Protocol Stack
                                Q-in-Q                                                     PBB

                              Ethernet                              MPLS/PW                                     Ethernet


    CE          U-PE1                    BCB           N-PE1                            N-PE2           B-BEB               U-PE2
                                                                                                                                            CE

                         Q-in-Q
                                                        X            IP/MPLS              X
                                                                                                                     PBB
                 A1                                    C1                                 D1                                   B1


                                           S-tag I/f                                            I-tag I/f
                                           (Type 1)                                            (Type II)
                                                            Must support   Must support
                                                             IB-BEB and     B-BEB and
                         -- Participates in local I -SID        VPLS          VPLS
                            domain of MPLS core
                         -- Supports I-SID bundling
                         -- 1:1 mapping of S-VID à I-SID            LSP-Label

                                                                     VC-Label

                                                                      B-DA                                          B-DA

                                                                      B-SA                      B-DA                B-SA

                                                                      B-Tag                     B-SA               B-Tag

                                                                      I-Tag                     I-Tag               I-Tag

                         C-DA               C-DA                      C-DA                      C-DA                C-DA

         C-DA            C-SA               C-SA                      C-SA                      C-SA                C-SA             C-DA

         C-SA           S-Tag              S-Tag                      S-Tag                     S-Tag              S-Tag             C-SA

      S/C-Tag           C-Tag              C-Tag                     S/C-Tag                   C/S-Tag            S/C-Tag           S/C-Tag

      Payload           Payload           Payload                    Payload                   Payload            Payload           Payload
 ©Copyright 2009                                Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                           Operator Best Practices                                                            M1-56
Metanoia, Inc.
                                                                                              Critical Systems Thinking™




H-VPLS with Modified PB PE: Assessment
 Overlay with cascade of IP/MPLS and Ethernet networks
 Use case: SP has converged to MPLS core, but has different Ethernet
      aggregation/access networks, which have not converged to a single Ethernet
      technology



Benefits                                                          Drawbacks
 Reduces PW mesh in core                                          N-PE is complex with new
          Via I-SID bundling, where group of                           capability – needs IB-BEB
              I-SIDs is mapped to B-VID and to a                        functionality
              VPLS instance

                                                                   PB PE (N-PE1) needs to learn
 Supports tightly, loosely-coupled,                                    many C-MAC addresses
       and different service domains


 ©Copyright 2009                  Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                             Operator Best Practices                                        M1-57
Metanoia, Inc.
H-VPLS with Heterogeneous Access                                               Critical Systems Thinking™




Modified PBB PE -- Overview
 Used to interoperate existing 802.1ad (PB) networks with
       new 802.1ah (PBB) aggregation networks


 Both networks connect to an H-VPLS core/backbone


 PE interfacing with PBB network must support VPLS and IB-
       BEB functionality
          B-component faces PBB cloud
          I-component faces MPLS core, connects to VPLS forwarder




 ©Copyright 2009           Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                      Operator Best Practices                                M1-58
Metanoia, Inc.
H-VPLS with Modified PBB PE                                                                                                          Critical Systems Thinking™




Operation & Protocol Stack
                                                         Q-in-Q                                                         PBB
                                                                                         I-Comp
                               Ethernet                                MPLS/PW                                         Ethernet

                                                                                                B-Comp
      CE          U-PE1                   PCB            N-PE1                             N-PE2               B-BEB               U-PE2
                                                                                                                                                   CE

                           Q-in-Q
                                                          X             IP/MPLS
                                                                                                X
                                                                                                                            PBB
                   1P                                    1P                                       1P                                  1P


                                             S-tag I/f                                               I-tag I/f
                                             (Type I)                                               (Type II)
                                                            Must support         Must support
                                                           VPLS for B-tag          IB-BEB
                                  -- 1:1 mapping of S-VID (not PBB aware)         and VPLS
                                  to VPLS instance
                                  -- S-VID bundle has no
                                  counterpart in PBB n/w


                                                                                                                           B-DA

                                                                                                       B-DA                B-SA

                                                                      LSP-Label                        B-SA               B-Tag

                                                                       VD-Label                        I-Tag               I-Tag

                           C-DA                  C-DA                    C-DA                          C-DA                C-DA

           C-DA            C-SA                  C-SA                    C-SA                          C-SA                C-SA             C-DA

           C-SA           S-Tag                 S-Tag                    S-Tag                      S-Tag                 S-Tag             C-SA

         C-Tag            C-Tag                 C-Tag                    C-Tag                      C-Tag                 C-Tag            C-Tag

        Payload           Payload               Payload                Payload                      Payload              Payload           Payload

 ©Copyright 2009                                 Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                            Operator Best Practices                                                                  M1-59
Metanoia, Inc.
H-VPLS with Modified PBB PE:                                                                       Critical Systems Thinking™




Assessment
 Overlay with cascade of Ethernet and IP/MPLS networks
 Use case: SP has converged to MPLS core, but has different Ethernet
      aggregation/access networks, which have not converged to a single Ethernet
      technology

Benefits                                                             Drawbacks
      VPLS core can operate without any                                  N-PE2 needs IB-BEB functionality
       modifications
             Maps tags (I-tag) to VPLS instance                          PB PE (N-PE1) must learn C-MAC
                                                                           addresses
      Supports tightly, loosely-coupled, and
       different service domains                                          Absence of S-VID bundling, implies
                                                                           each I-SID maps to independent PW

                                                                           mesh in core à no scaling in PWs
       Viable option to incrementally at PBB
       aggregation networks to existing PB
       networks over IP/MPLS core                                         Need same I-SID domain across all
                                                                           PBB aggregation networks for
                                                                           consistent C-VID grouping

 ©Copyright 2009                     Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                Operator Best Practices                                          M1-60
Metanoia, Inc.
H-VPLS with MPLS Aggregation                                                     Critical Systems Thinking™




PBB uPE -- Overview
 Used to integrate 802.1ah (PBB) functionality into an IP/MPLS
       provider network


 PBB functionality is embedded in uPE to restrict MAC learning as
       close to customer as possible


 PE at the IP/MPLS core is unchanged, and only needs IP/MPLS and
       PW capability


 Bridging over VPLS network need be only 802.1ad capable as
          MAC forwarding is based on B-MAC address space
          Service delimiter is based on B-VLAN ID or B-VID

 ©Copyright 2009             Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                        Operator Best Practices                                M1-61
Metanoia, Inc.
H-VPLS with MPLS Aggregation                                                                                                                            Critical Systems Thinking™




PBB uPE: Operation & Protocol Stack
                                                                                   PBB

                                         MPLS/PW                                 MPLS/PW                                    MPLS/PW


         CE          U-PE                          LSR           N-PE                                N-PE            LSR                       U-PE           CE

                                     MPLS
                                                                  X               IP/MPLS
                                                                                                       X
                            IB-BEB                                                                                              MPLS
                      A1                                         C1                                   D1                                  IB-BEB B1
                                                         MPLS                                               MPLS
                                                          I/f                                                I/f
                                                                                                                       Spoke PWs per
                                                                                                  Full PW mesh per     VPLS instance    -- Port Mode
                                                                                                                                        -- VLAN Mode
                 Must support IB-BEB and                                Must support MPLS/PW VPLS instance
                                                                                                                                        -- VLAN-bundling Mode
                 MPLS/PW encap of C-MAC                                 Operates on B-DA, B-SA,
                                                                                                                                        Maps svcs. to I-SID
                 frames into B- MAC frames                              B-VID only (not I-SID), so
                                                                                                                                        I-SID àB-Tag (bridge domain)
                                                                        need be 802.1ad capable
                                                                                                                                        B-Tag à VPLS instance

                                LSP-Label           LSP-Label                    LSP-Label                 LSP-Label          LSP-Label

                                VC-Label             VC-Label                    VC-Label                  VC-Label            VC-Label

                                     B-DA                B-DA                      B-DA                      B-DA               B-DA

                                     B-SA                B-SA                      B-SA                      B-SA               B-SA

                                     B-Tag               B-Tag                     B-Tag                    B-Tag               B-Tag

                                     I-Tag               I-Tag                     I-Tag                     I-Tag              I-Tag

              C-DA                   C-DA                C-DA                      C-DA                      C-DA               C-DA                   C-DA

              C-SA                   C-SA                C-SA                      C-SA                      C-SA               C-SA                   C-SA

           S/C-Tag               S/C-Tag             S/C-Tag                      S/C-Tag                  S/C-Tag             S/C-Tag                S/C-Tag

           Payload               Payload             Payload                      Payload                  Payload             Payload                Payload



 ©Copyright 2009                                         Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                                    Operator Best Practices                                                                           M1-62
Metanoia, Inc.
H-VPLS with MPLS Aggregation                                                                        Critical Systems Thinking™




PBB uPE: Assessment
 Overlay with IP/MPLS transmission n/w and Ethernet service
 Use case: SP converged to MPLS in core and aggregation networks
 I-SID allocation
         Global across MPLS networksno I-SID translation needed
 I-SID bundling must be consistent across all participating PEs

Benefits                                                       Drawbacks
      PBB functionality at uPE improves                            Customer broadcast/mcast frames
       scalability                                                   must be sent over entire B-VLAN


      Multiple I-SIDs per B-VLAN à fewer                           Greater replication over VPLS PW
       core PWs                                                      mesh
                                                                            As B-VLAN mesh is larger than scope
                                                                             of any particular I-SID
      N-PE only learns uPE MACs à MAC
       addresses scale

 ©Copyright 2009               Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                          Operator Best Practices                                                 M1-63
Metanoia, Inc.
H-VPLS with MPLS Aggregation                                                   Critical Systems Thinking™




PBB nPE -- Overview
 Used to integrate 802.1ah (PBB) functionality into an
       IP/MPLS provider network


 PBB function embedded in nPE, when uPE is not capable


 Bridging over VPLS n/w must be 802.1ah capable, as nPE
          Assigns an I-SID for each service of a customer
          Muxes I-SIDs into a common bridge domain, or B-VLAN
          Maps either I-SIDs or B-VLANs to VPLS instances




 ©Copyright 2009           Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                      Operator Best Practices                                M1-64
Metanoia, Inc.
H-VPLS with MPLS Aggregation                                                                                                       Critical Systems Thinking™




PBB nPE: Operation & Protocol Stack
                                 Q-in-Q                                PBB                                    Q-in-Q

                             MPLS/PW                                 MPLS/PW                                 MPLS/PW


     CE          U-PE                     LSR           N-PE                              N-PE        LSR                      U-PE
                                                                                                                                               CE
                                                                                 IB-BEB
                          MPLS
                                                         X            IP/MPLS
                                                                                           X
                                                                                                                 MPLS
                 A1                                     C1 IB-BEB                          D1                                     B1
                                                MPLS                                             MPLS
                                                 I/f                                              I/f
                                                                                       Full PW mesh per                  PWs per customer
                                                                     Must support    customer or cust. grp               (one service per PW,
                                                                    VPLS and PBB                                         no service muxing)
                                                                                                                         Supports
                                                                                                                         -- Port Mode
                                                                     LSP-Label                                           -- VLAN Mode
                                                                                                                         -- VLAN-bundling Mode
                                                                     VD-Label

                                                                       B-DA

                                                                       B-SA

                        LSP-Label           LSP-Label                  B-Tag                    LSP-Label      LSP-Label
                                              C-DA                                                               C-DA
                        VD-Label            VD-Label                   I-Tag                    VD-Label       VD-Label

                          C-DA                  C-DA                   C-DA                       C-DA           C-DA

          C-DA            C-SA                  C-SA                   C-SA                       C-SA           C-SA                  C-DA

          C-SA           S-Tag                  S-Tag                  S-Tag                     S-Tag           S-Tag                 C-SA

       C-Tag             C-Tag                  C-Tag                 S/C-Tag                    C-Tag           C-Tag                 C-Tag

      Payload           Payload             Payload                   Payload                    Payload        Payload               Payload

 ©Copyright 2009                                 Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                            Operator Best Practices                                                              M1-65
Metanoia, Inc.
H-VPLS with MPLS Aggregation                                                              Critical Systems Thinking™




PBB nPE: Assessment
 Overlay with IP/MPLS txn network and Ethernet service
 Use case: SP has converged to MPLS in core and aggregation n/ws, but
      aggregation n/w is not PBB capable
 Needs integration of PBB functionality into H-VPLS PE



Benefits                                                    Drawbacks
 PBB functionality at nPE                                   Spoke PWs terminate at nPE
       improves VPLS PW scalability                                  nPE must learn all customer MAC
                                                                          addresses

 nPE maps multipe I-SIDs into a B-
       VLAN à fewer core PW meshes                           Per-service spoke PWs à needs
                                                                  significantly more spoke PWs
                                                                     Since uPE is not PBB aware


 ©Copyright 2009            Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                       Operator Best Practices                                          M1-66
Metanoia, Inc.
                                                                                       Critical Systems Thinking™




H-VPLS with PBB Migration
 Provides migration path for operators with investments in IP/MPLS
       (and VPLS) deployments


 Enables incremental induction of PBB into the network


 Expands scalability benefits of PBB over time with deployment


 Three migration scenarios:
          Non-PBB (802.1ad) encapsulated frames over VPLS core
          PBB-encapsulated (802.1ah) frames over VPLS core
          Mixed frames (802.1ad and 802.1ah) over VPLS core
                  Depends on capabilities of different edge devices in network


 ©Copyright 2009                   Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                              Operator Best Practices                                M1-67
Metanoia, Inc.
H-VPLS with PBB Migration: 802.1ad                                            Critical Systems Thinking™




Frames in Core -- Overview
 Existing 802.1ad access/aggregation n/wks
          Unchanged – need no modifications
          nPE’s require no changes, and need no knowledge of PBB


 New MPLS-based access/aggregation networks with PBB
       functionality on uPE
          Need IB-BEB functionality on nPE
          Required to terminate PBB encap. on incoming frames


 Ethernet frames in VPLS/H-VPLS core use 802.1ad format
          VPLS core operation is unchanged


 ©Copyright 2009          Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                     Operator Best Practices                                M1-68
Metanoia, Inc.
H-VPLS w/ PBB Migration 802.1ad                                                                                                           Critical Systems Thinking™




Frames in Core: Operation & Protocols
                                                         Q-in-Q                                                        PBB
                                                                                     I-Comp                                          I-Comp
                                 MPLS/PW                             MPLS/PW                                         MPLS/PW

                                                                                          B-Comp                               B-Comp
      CE          U-PE1                    LSR           N-PE1                                N-PE2            LSR                      U-PE2
                                                                                                                                                        CE

                              MPLS
                                                          X               IP/MPLS
                                                                                                X
                                                                                      IB-BEB D1                          MPLS
                  A1                                      C1                                    MPLS                               IB-BEB B1
                                             MPLS
                                              I/f                                                I/f


                                                          Must support        Must terminate PBB          PWs could be per
                                                                                                                                   Must terminate
                  Spoke PWs per customer                     VPLS               and MPLS/PW               VPLS instance or per
                                                                                                                                     PBB and
                  (one service per PW)                                                                    service
                                                                                                                                     MPLS/PW

                                                                                                    LSP-Label           LSP-Label
                                                                                                                          C-DA
                                                                                                    VC-Label            VC-Label

                                                                                                       B-DA               B-DA

                                                                                                       B-SA               B-SA

                           LSP-Label        LSP-Label                    LSP-Label                    B-Tag              B-Tag
                                              C-DA
                            VC-Label        VC-Label                     VC-Label                      I-Tag              I-Tag

                              C-DA               C-DA                      C-DA                        C-DA               C-DA

           C-DA               C-SA               C-SA                      C-SA                        C-SA               C-SA                  C-DA

           C-SA              S-Tag               S-Tag                    S-Tag                       S-Tag              S-Tag                  C-SA

         C-Tag               C-Tag               C-Tag                   S/C-Tag                      C-Tag              C-Tag                  C-Tag

        Payload             Payload          Payload                     Payload                      Payload            Payload              Payload

 ©Copyright 2009                                 Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                            Operator Best Practices                                                                       M1-69
Metanoia, Inc.
H-VPLS with PBB Migration: 802.1ad                                                         Critical Systems Thinking™




Frames in Core: Assessment
 Overlay with IP/MPLS txn network and Ethernet service
 Use Case: SP has converged to MPLS in core and aggregation n/ws, but all
      access/aggregation networks are not PBB capable
 Ideal when SP has single PBB-capable aggregation/access network



Benefits                                                    Drawbacks
 Requires no change to existing                             nPE1 must learn all customer
       access/aggregation n/wk (Q-in-Q)                           MAC addresses (as spoke PWs
                                                                  terminate at nPE1)
                                                                     PBB benefit cannot be leveraged
 No change to existing VPLS nPEs
       (those not facing 802.1ah n/wks)
                                                             nPEs need full mesh of PWs



 ©Copyright 2009            Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                       Operator Best Practices                                           M1-70
Metanoia, Inc.
H-VPLS with PBB Migration: 802.1ah                                              Critical Systems Thinking™




Frames in Core -- Overview
 Existing 802.1ad access/aggregation n/wks
          Unchanged – need no modifications


 nPE’s must be upgraded for PBB-based processing (IB-BEB)
          All Ethernet svc frames over VPLS core are PBB-encapsulated


 New MPLS-based access/aggregation networks have PBB
       functionality on uPE
          Need IB-BEB functionality on remote nPE
          PBB encap. of incoming frames terminated at far-end nPE


 Ethernet frames in VPLS/H-VPLS core use 802.1ah format
          Thus, VPLS core operation is modified


 ©Copyright 2009            Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                       Operator Best Practices                                M1-71
Metanoia, Inc.
H-VPLS w/ PBB Migration 802.1ah                                                                                                                Critical Systems Thinking™




Frames in Core: Operation & Protocols
                                     Q-in-Q                                                           PBB
                                                                 I-Comp                                                                   I-Comp
                                 MPLS/PW                                        MPLS/PW                                   MPLS/PW

                                                                      B-Comp                                                        B-Comp
      CE          U-PE1                       LSR           N-PE1                                 N-PE2             LSR                      U-PE2
                                                                                                                                                             CE

                              MPLS
                                                             X                  IP/MPLS
                                                                                                     X
                                                                                                                              MPLS
                  A1                                        C1 IB-BEB                               D1 MPLS                             IB-BEB B1
                                                MPLS
                                                 I/f                                                    I/f


                                                         Must terminate PBB                Must support        PWs per VPLS             Must terminate
                  Spoke PWs per customer                   and MPLS/PW                      only VPLS          instance                   PBB and
                  (one service per PW)                                                                                                    MPLS/PW

                                                                               LSP-Label                  LSP-Label          LSP-Label
                                                                                                                               C-DA
                                                                               VC-Label                   VC-Label           VC-Label

                                                                                 B-DA                       B-DA               B-DA

                                                                                 B-SA                       B-SA               B-SA

                           LSP-Label           LSP-Label                        B-Tag                      B-Tag              B-Tag
                                                C-DA
                            VC-Label           VC-Label                          I-Tag                      I-Tag              I-Tag

                              C-DA                  C-DA                         C-DA                       C-DA               C-DA

           C-DA               C-SA                  C-SA                         C-SA                       C-SA               C-SA                  C-DA

           C-SA              S-Tag                  S-Tag                       S-Tag                      S-Tag              S-Tag                  C-SA

         C-Tag               C-Tag                  C-Tag                       C-Tag                      C-Tag              C-Tag                  C-Tag

        Payload             Payload             Payload                        Payload                    Payload             Payload              Payload

 ©Copyright 2009                                    Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                               Operator Best Practices                                                                         M1-72
Metanoia, Inc.
H-VPLS with PBB Migration: 802.1ah                                                             Critical Systems Thinking™




Frames in Core: Assessment
 Overlay with IP/MPLS txn network and Ethernet service
 Use Case: SP has converged to MPLS in core and aggregation n/ws, and
      wishes to have the benefit of PBB scaling in the VPLS core
 Useful for the operator whose MPLS access/aggregation networks are being
      upgraded to support PBB IB-BEB at uPEs

Benefits                                                         Drawbacks
 Reduces MAC learning at nPEs                                    Must upgrate nPEs of all existing
          Due to PBB at nPEs, thus giving                             MPLS access/aggregation n/wks
              MAC address scalability                                     nPE needs IB-BEB functions


 Lowers PW mesh counts in core
          Multiple customer instances can
              be bound to a single B-VLAN and
              VPLS instance

 ©Copyright 2009                 Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                            Operator Best Practices                                          M1-73
Metanoia, Inc.
H-VPLS with PBB Migration: Mixed                                                   Critical Systems Thinking™




Frames in Core -- Overview
 Existing 802.1ad access/aggregation n/wks unchanged
          Exchange Ethernet frames with 802.1ad format over PWs in core


 nPE’s must be upgraded for PBB-based processing (IB-BEB)
          All Ethernet svc frames over VPLS core are PBB-encapsulated



 New MPLS-based access/aggregation networks have PBB functionality on
       uPE
          Exchange PBB-encap frames over VPLS core


 Interworking b/ween PBB-capable and PB-capable requires that nPE of
       PBB network has IB-BEB functionality


 Ethernet frames in VPLS/H-VPLS core use 802.1ah/802.1ad format

 ©Copyright 2009               Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                          Operator Best Practices                                M1-74
Metanoia, Inc.
     H-VPLS w/ PBB Migration Mixed                                                                                            Critical Systems Thinking™




     Frames in Core: Operation
                                                     PBB                                                        Q-in-Q

                              Q-in-Q                                                            PBB
                                                          I-Comp                                                              I-Comp
                             MPLS/PW                                    MPLS/PW                               MPLS/PW

                                                               B-Comp                                                    B-Comp
CE         U-PE1                        LSR         N-PE1                                  N-PE2        LSR                       U-PE2
                                                                                                                                                 CE

                           MPLS
                                                      X      B-BEB      IP/MPLS
                                                                                               X
                                                                                                                     MPLS
            A1                                       C1      IB-BEB                           D1 MPLS
                                             MPLS                                                                           IB-BEB B1
                                              I/f                                                 I/f

Spoke PWs per customer                 -- Must terminate PBB and MPLS/PW             Must support     PWs per VPLS
(one service per PW)                                                                                                        Must terminate
                                                                                      only VPLS       instance                PBB and
                                       -- Needs IB-BEB and B-BEB capability                                                   MPLS/PW

                                       -- Should be aware of remote PEs’
                                       capabilities – via static config . or
                                       extended VPLS control plane

     Non-PBB Capable                                                                                     PBB- Capable
     Existing Access Networks                                                                            New Access Networks



                 Protocol stack is a combination of those
                 shown earlier for 802.1ad and 802.1ah in core

      ©Copyright 2009                             Carrier Ethernet Technology Strategies & Evolving
     All Rights Reserved                                        Operator Best Practices                                                        M1-75
Metanoia, Inc.
H-VPLS with PBB Migration: 802.1ah                                                                        Critical Systems Thinking™




Frames in Core: Assessment
 Overlay with IP/MPLS txn network and Ethernet service
 Use Case: SP has converged to MPLS in core and aggregation n/ws, has a mix
      of PB- and PBB-capable access/aggregation networks



Benefits                                                           Drawbacks
      Core can serve both PB-encapsulated                              Must upgrate nPEs of PBB-capable
       and PBB-encapsulated frames                                       access/aggregation n/wks. nPE needs
                                                                                IB-BEB to interface with PBB network
                                                                                B-BEB to interface with PB network
      Lowers PW mesh counts in core
             For PBB-capable aggregation networks
                                                                        PE must be aware of remote peer’s
                                                                         capability
                                                                                Requires either static config. or VPLS
                                                                                 control plane extensions


 ©Copyright 2009                   Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                              Operator Best Practices                                                   M1-76
Metanoia, Inc.
                                                                                 Critical Systems Thinking™




  Workshop Outline
 Overview of Architectural Options


 Applicability of Key Technologies to Access/Metro/Core
          Q-in-Q (PB), MAC-in-MAC (PBB), 802.1Qay (PBB-TE) and PVT, IP/MPLS,
              MPLS-TP


 Basic Architectures and Their Analysis
          Overlay
          Parallel
          Uniform
          Mobile-Backhaul
 Hybrid Architectures and Their Analysis


 ©Copyright 2009             Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                        Operator Best Practices                                M1-77
Metanoia, Inc.
                                                                               Critical Systems Thinking™




Mobile Backhaul Architectures
 Mobile backhaul architectures derive from the basic and
       hybrid architectures presented earlier


 We examine them separately due to their unique needs:
          Interface with the core network
          Timing and synchronization requirements
          Evolution requirements – from TDM or ATM to IP/MPLS and/or
              Ethernet




 ©Copyright 2009           Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                      Operator Best Practices                                M1-78
Metanoia, Inc.
                                                                                   Critical Systems Thinking™




Evolution of Cellular Technology: 2G to 4G
 2G
          GSM (TDM, 56-114 kbps)


 2.5G
          GPRS, EDGE (TDM, 236-473 kbps)
          CDMA 1XRTT (HDLC, TDM, 144 kbps)


 3G
          WCDMA/UMTS (3GPP) (R99, R4) (ATM, 384 kbps uplink, 2Mbps downlink)
          UMTS (3GPP) (R5 (HSDPA), R6 (HSUPA))( IP, 2-3 Mbps)
          EV-DO (3GPP2) (Rev0, RevA, RevB, RevC) (IP, 1.8 Mbps uplink, 3.1Mbps
              downlink)


 4G (LTE)
          3GPP (R7/R8) (IP, >50 Mbps uplink, >100 Mbps downlink)
          WiMAX (802.16e, 802.16m) (IP, 50-100 Mbps)
 ©Copyright 2009               Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                          Operator Best Practices                                M1-79
Metanoia, Inc.
Evolution of Cellular Technology                                                           Critical Systems Thinking™




and Backhaul Types
         Network             Speed                             Interface

       GSM/GPRS            56-114 Kbps                             TDM

            EDGE         236 – 473 Kbps                            TDM

     3G (UMTS/          384 Kbps Uplink
                                                                   ATM
   WCDMA) R3, R4       384 Kbps Downlink
   3G, R5 (HSDPA),      384 Kbps Uplink
                                                             IP/Ethernet
     R6 (HSUPA)        14.4 Kbps Downlink
                         500 Mbps Uplink
   LTE R8 (20 Mhz)                                           IP/Ethernet
                       >100 Mbps Downlink

      CDMA1X-RTT            100 Kbps                               TDM            Legend

      CDMA EV-DO          1.8 Mbps Uplink                                             2G
                                                             IP/Ethernet
        Rev A/B       1.8 to 5 Mbps Downlink                                          2.5G
   WiMAX (10 Mhz)           50 Mbps                          IP/Ethernet              3G
                                                                                      4G
                                                           Backhaul Types
 ©Copyright 2009              Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                         Operator Best Practices                                         M1-80
Metanoia, Inc.
                                                                                     Critical Systems Thinking™




Mobile Backhaul Components
 Backhaul network – defined as the network that connects
          Base Transceiver Station (BTS, or Base Station) to Base Station Controller
              (BSC) in 3GPP2 – GSM-based cellular networks
          Node-B to Radio Network Controller (RNC) in 3GPP – CDMA-based cellular
              networks

 Traditional backhaul networks have used ...
          E1/T1 leased lines
          SONET/SDH TDM channels (for higher rate aggregation)


 Mobile transport infrastructure has hitherto been ...
          Microwave links
          Optical fiber with SDH/SONET


 Evolution to packet-based wireless services creates a push for the
       transport itself to be packet-based: Ethernet or IP/MPLS or a combination

 ©Copyright 2009                 Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                            Operator Best Practices                                M1-81
Metanoia, Inc.
                                                                                                                    Critical Systems Thinking™




 Traditional Backhaul Evolution
2G BTS

                                                                                                              BSC

         TDM
         TI/EI         Cellsite       SONET/SDH                                               SONET/SDH
                       Gateway         XConnect                                                XConnect E1


                              T1/E1/STM                           SDH/SONET
                                                                   Network
                                                                                                             ATM                   RNC
     ATM

           nxE1

                                                                                                          ATM
3G BTS                                                                                                   Switch


                                                                Separate transmission facilities for different
                                                                technologies (TDM and packets)


  ©Copyright 2009                         Carrier Ethernet Technology Strategies & Evolving
 All Rights Reserved                                    Operator Best Practices                                                      M1-82
Metanoia, Inc.
                                                                                        Critical Systems Thinking™




Mobile Backhaul Challenges
 Exponential growth in mobile subscribers
          3 Billion subscribers (2007)
          Expected to touch 5 Billion by 2013!
          Leads to massive increase in traffic volume

 Shift in mobile traffic patterns
          High-speed data, including multimedia traffic (video, VoIP, IMS)
          Bandwidth insensitive but QoS sensitive applications!
          With increasing speed, revenue-per-bit is decreasing
                  Result is that traffic and revenue are decoupled !


 These trends lead to massive increase in traffic volume

 Backhaul accounts for 30% of OpEX (Per Yankee Group 2005)

 Traditional backhaul unsustainable with such traffic growth
 ©Copyright 2009                    Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                               Operator Best Practices                                M1-83
Metanoia, Inc.
Key Performance Requirements                                                      Critical Systems Thinking™




for Mobile Backhaul
 Delay – budget for entire RAN is 100ms
          Backhaul segment delay must be < 3-4 ms



 Loss – Target BER is
          2G networks = 10-7
          3G networks = 10-5 to 10-4



 Synchronization – Frequency and Time accuracy
          2G: 50 ppb freq. accuracy at radio interface
          3G: 50 ppb freq. accuracy and 2.5 µs time accuracy for TDD
          WiMAX: 8 ppm freq. for FDD/TDD, and 5-25 µs time accuracy for TDD


 ©Copyright 2009              Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                         Operator Best Practices                                M1-84
Metanoia, Inc.
                                                                                   Critical Systems Thinking™




Backhaul Strategy Fundamentals
 Future-Proof
          Should support existing legacy (TDM) infrastructure
          Evolving new packet-based and IP services using diverse and
              coexistent technologies


 Scalable
          Grow b/w to support next-gen. wireless access technology LTE/4G
          Ethernet interfaces versus nxT1/E1


 Cost Effective – reduce OpEx

 Must meet timing and synchronization targets

 Simplify provisioning & planning – advanced OAM,
       troubleshooting
 ©Copyright 2009               Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                          Operator Best Practices                                M1-85
Metanoia, Inc.
                                                                                       Critical Systems Thinking™




Timing and Synchronization
 ITU-T G.8261 specifies timing & synchronization in packet
       networks

 Synchronization needed for
              Radio framing accuracy, hand-off control, backhaul transport
              reliability

 Three Methods
          Synch. Ethernet: Similar to SONET/SDH -- embed clock in PHY layer
                  Requires changes in PHY chip


          IEEE 1588 (Precision Time Protocol)
                  Distributed protocol: specifies how real-time PTP clocks synchronize


          IEEE 802.1as
                  Aimed at adapting IEEE 1588 to Carrier Ethernet
 ©Copyright 2009                   Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                              Operator Best Practices                                M1-86
Metanoia, Inc.
                                                                                    Critical Systems Thinking™




Clock Distribution Methods Used
 Physical Layer Clock
          Via synchronous TDM i/fs e.g. PDH/SDH/SONET
          Via Sync. Ethernet as per G.8261/G.8262


 GPS-receiver based synchronization

 Clock distribution over packet network
          IEEE 1588v2 – being looked at in ITU-T Q13/SG15, who are developing
              a telecom profile for 1588 v2
          NTP – IETF currently working on NTP v4


 Adaptive and Differential clock synchronization




 ©Copyright 2009                Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                           Operator Best Practices                                M1-87
Metanoia, Inc.
                                                                               Critical Systems Thinking™




Packet Transport for Mobile Backhaul
 Packet-based transport can provide high bandwidth at lower
       cost (than TDM transport)


 Ideal choice for LTE and 4G technologies


 Challenges ...
          Support legacy traffic (TDM and ATM ) via circuit emulation
          Meet timing and synchronization requirements
          Provide QoS and protection switching
          Furnish advanced OAM capabilities


 ©Copyright 2009           Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                      Operator Best Practices                                M1-88
Metanoia, Inc.
                                                                                                        Critical Systems Thinking™




 Pseudowires (PW) for Legacy Transport
                                          PW
                                     PSN Tunnel


                                                                                         BSC
                 AC                                                            AC
                                 Carrier Ethernet                                                  Wireless
 CE                                  Network                                                        Core
                         PE                                         PE
(BTS)




                AC: Attachment Ckt         CE : Customer Edge (BTS)
                PE: Provider Edge          BSC: Base Station Controller

       Encapsulation                                                   PSN Tunnels
          
                                                                               May be IP/MPLS, T-MPLS/MPLS-TP, or
               Structure-Agnostic TDM-over-IP
                                                                                PB/PBB/PBB-TE based
               (SAToIP) (RFC 4553)
              Structure-Aware TDM Circuit
                                                                        PW Signaling
               Emulation (CESoPSN) (RFC 5086)
              ATMoPSN (RFC 4717)

  ©Copyright 2009                    Carrier Ethernet Technology Strategies & Evolving
 All Rights Reserved                               Operator Best Practices                                               M1-89
Metanoia, Inc.
                                                                                 Critical Systems Thinking™




PBB/PBB-TE for Mobile Backhaul
 Connection-oriented p2p tunnels b/ween BTS (BS) -- BSC (NC)
          Provides MEF-style EVPL/EVP-Tree between RAN BS and RAN NC



 IEEE 802.1ag OAM enables carrier-grade OAM tools


 Delay/loss ensured via admission control & 802.1Q PCP


 Supporting Legacy TDM over PBB/PBB-TE
          IETF has draft on PW over PBB-TE
          802.1ah supports encap. of non-Ethernet frames (via short I-TAG TCI)




 ©Copyright 2009             Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                        Operator Best Practices                                M1-90
Metanoia, Inc.
                                                                                             Critical Systems Thinking™




PBB/PBB-TE for TDM Encapsulation
                       IEEE 802.1ah PBB Short & Long I-Tag


          DEI                     I-SID
                                                                                    C-DA   C-SA
          Etc.             (Service Identifier)


                        Long I-TAG Tag Control Information


          DEI                     I-SID
          Etc.             (Service Identifier)

                           Short I-Tag
    (Only I-SID used for encapsulation of multiple protocols)

     EtherType used to indicate encapsulation of Ethernet or multiple protocols



We show I-SID with C-DA and C-SA but without Ethernet encapsulation

 ©Copyright 2009                Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                           Operator Best Practices                                         M1-91
Metanoia, Inc.
                                                                                                                Critical Systems Thinking™




MEF Services for Mobile Backhaul
         RNC                                                                             RNC

                      BSC                                                                               BSC




                             Service
                            Multiplexing                                                                        mp2mp EVC

         Metro Ethernet
       EVC           EVC                                                                      Metro Ethernet




                                                                                                               BTS
      BTS              BTS                                                              BTS
                                                                                                     BTS
EVPL Service for Backhaul using                                           EP-LAN Service for Backhaul using
   Metro Ethernet Networks                                                    Metro Ethernet Networks


 Services muxed at RNC UNI                                          Needed when inter-BS communication
                                                                  is permitted like in LTE/802.16m (WiMAX)
 ©Copyright 2009                    Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                               Operator Best Practices                                                        M1-92
Metanoia, Inc.
                                                                                Critical Systems Thinking™




MEF Services for Mobile Backhaul
                            RNC

                                                    BSC




                                                             Service
                                                            Multiplexing

                            Metro Ethernet
                          EVC                         EVC




                         BS/           BS/             BS/
                         BTS                           BTS
                                       BTS

                      EP-Tree Service for Backhaul using
                           Metro Ethernet Networks
 ©Copyright 2009            Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                       Operator Best Practices                                M1-93
Metanoia, Inc.
                                                                                Critical Systems Thinking™




IP/MPLS for Mobile Backhaul
 MPLS PW/VPLS can provide effective backhaul
          Traditional circuit-based services are offered using PWs


 PW signaling – uses either BGP or LDP


 Protection switching
          Achieved via MPLS Fast Reroute (FRR) on LSP tunnel


 OAM provided by
          LSP Ping, VCCV
          BFD
 ©Copyright 2009            Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                       Operator Best Practices                                M1-94
Metanoia, Inc.
                                                                                      Critical Systems Thinking™




IP/MPLS and PBB/PBB-TE Caveats
 Poor support for P2MP LSPs in MPLS
          Such support is desirable:
                  For Clock Distribution from BSC to BTSs

                  When BSs allowed to communicate with each other and cooperate
                      as in WiMAX/LTE


 PW over PBB-TE is not fully developed




 ©Copyright 2009                  Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                             Operator Best Practices                                M1-95
Metanoia, Inc.
                                                                                     Critical Systems Thinking™




Migration Strategies for Service Providers
 HSDPA/3G Data Off-load
          Separation of transport
                  GSM/UMTS voice sent over SONET/SDH

                  Data traffic transported over PSN tunnel



 Packet-based Backhaul
          TDM/ATM/Ethernet/IP all transported over PSN tunnel via PWs



 Converged Transport
          Single packet-switched infrastructure for wireline, broadband
              and mobile services

 ©Copyright 2009                 Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                            Operator Best Practices                                M1-96
Metanoia, Inc.
                                                                                                                     Critical Systems Thinking™




  HSDPA/3G Data Off-Load
2G BTS



         TDM
         TI/EI

                                                                                        Multi-Service
                  Cellsite          SONET/SDH                                 SONET/SDH Aggregation
                  Gateway            XConnect                                  XConnect   Router           2G BSC

                             TI/EI/STM                  SDH/SONET
                                                         Network
                                                                                                        3G RNC   Wireless Core
    ATM
           nxE1

                               Ethernet
                                                    Carrier Ethernet
3G BTS                                                  Network
                                         Ethernet                                 Ethernet
                                          Switch                                   Switch




   ©Copyright 2009                             Carrier Ethernet Technology Strategies & Evolving
  All Rights Reserved                                        Operator Best Practices                                                  M1-97
Metanoia, Inc.
                                                                                             Critical Systems Thinking™




Packet-Based Backhaul
2G BTS



                                                                                               BSC

            TDM
                      Cellsite
                      Gateway
                                                   PE                                   PE

            ATM                                                   Carrier Ethernet
                             TI/EI/STM                            Network (PSN)                 RNC/GW




            Ethernet

3G/4G                            IP/ATM/Ethernet/TDM are all transported over the PSN
 BTS                             Tunnel using PWs


 ©Copyright 2009                    Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                               Operator Best Practices                                     M1-98
Metanoia, Inc.
                                                                                                                Critical Systems Thinking™




Converged Packet-Based Transport


                                                                                           Converged Service
                                                                                            Delivery Platform
              3G/4G Base
              Station           Aggregation
                                                                  Converged
                                 Network
                                                                  Metro Core




                                                                                   Internet
                      Access
                      Network




                                   Common packet-switched infrastructure for both wireline
                                   and wireless services
 ©Copyright 2009                       Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                                  Operator Best Practices                                                     M1-99
Metanoia, Inc.
                                                                                   Critical Systems Thinking™




Backhaul References
      MEF White Paper, “Carrier Ethernet Access for Mobile Backhaul Networks”, Feb
       2008.

      MEF Technical Specification Draft 2, “Mobile Backhaul Implementation Agreement-
       Phase 1”, Feb 2008

      Nortel Networks White Paper, “Mobile Network Evolves with Carrier Ethernet”, 2008.

      ITU-T G.8261, “Timing and Synchronization Aspects in Packet Networks”, Sept 2007.

      IP/MPLS Forum White Paper, “Use of MPLS Technology in Mobile Backhaul
       Networks”, Feb 2008

      Kireeti Kompella and Mallik Tatipamula, “IP/MPLS in Next Gen Mobile Backhaul
       Networks”, MPLS 2007.




 ©Copyright 2009               Carrier Ethernet Technology Strategies & Evolving
All Rights Reserved                          Operator Best Practices                               M1-100

Architectural Options for Metro Carrier-Ethernet Network Buildout: Analysis & Evaluation

  • 1.
    Metanoia, Inc. Critical SystemsThinking™ Architectural Options for Metro Carrier-Ethernet Network Buildout: Analysis and Evaluation Metanoia, Inc. Email: experts@metanoia-inc.com Web: http://www.metanoia-inc.com Phone: +1-888-641-0082 © Copyright 2009 Fax: +1-888-641-0086 All Rights Reserved
  • 2.
    Metanoia, Inc. Critical Systems Thinking™ Workshop Outline  Overview of Architectural Options  Applicability of Key Technologies to Access/Metro/Core  Q-in-Q (PB), MAC-in-MAC (PBB), 802.1Qay (PBB-TE) and PVT, IP/MPLS, MPLS-TP  Basic Architectures and Their Analysis  Parallel  Overlay  Uniform  Mobile-Backhaul  Hybrid Architectures and Their Analysis ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-2
  • 3.
    Metanoia, Inc. Critical Systems Thinking™ Workshop Outline  Overview of Architectural Options  Applicability of Key Technologies to Access/Metro/Core  Q-in-Q (PB), MAC-in-MAC (PBB), 802.1Qay (PBB-TE) and PVT, IP/MPLS, MPLS-TP  Basic Architectures and Their Analysis  Parallel  Overlay  Uniform  Mobile-Backhaul (presented in detail later)  Hybrid Architectures and Their Analysis ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-3
  • 4.
    Metanoia, Inc. Building Carrier Ethernet Networks Critical Systems Thinking™ Architectural Choices Carrier Ethernet Network Architectural Choices Parallel Overlay Uniform Mobile Networks Networks Networks Backhaul IP/MPLS + Ethernet in parallel, spanning same Ethernet only in Ethernet over IP /MPLS Ethernet -based geographic area access/metro/core Ethernet à L2 services IP/MPLS à L3 services IP/MPLS over Ethernet IP/MPLS only in access/metro/core IP/MPLS-based  Three basic options to architect a carrier Ethernet network  Parallel, Overlay, Uniform  Mobile Backhaul – specially designed for wireless data and mobile voice traffic  Hybrids – numerous options, based on operator network status and technology availability, discussed extensively later in this module ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-4
  • 5.
    Metanoia, Inc. Our AnalysisApproach For Each Critical Systems Thinking™ Basic Architectural Option  Network Architecture  Overall architecture/layout of the network  Logical Structure  How the devices are logically connected at the service layer  Physical Realization (of the logical structure)  What constructs are used at the transport layer (not PHY transport, rather the Carrier Ethernet transport) to realize the logical structure  Protocol Stack  How tunnelling/interworking occurs in different segments of the network ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-5
  • 6.
    Metanoia, Inc. Critical Systems Thinking™ Workshop Outline  Overview of Architectural Options  Applicability of Key Technologies to Access/Metro/Core  Q-in-Q (PB), MAC-in-MAC (PBB), 802.1Qay (PBB-TE) and PVT, IP/MPLS, MPLS-TP  Basic Architectures and Their Analysis  Parallel  Overlay  Uniform  Mobile-Backhaul (presented in detail later)  Hybrid Architectures and Their Analysis ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-6
  • 7.
    Metanoia, Inc. Applicability ofKey Technologies: Critical Systems Thinking™ Assessment of Suitability  Reference is to different network segments within the metro network itself  Metro Access (referred to in this workshop as “access”)  Metro Aggregation (referred to in this workshop as “metro”)  Metro Core (referred to in this workshop as “core”)  “Eyeball” assessment of which technology is suitable for which particular segment  Ratings Scale based on inherent properties of technology     Optimally suited – excellent fit/match for segment    Moderately suited – reasonable fit/match for segment   Minimally suited – could be used, but would not be effective   Not suited – Not recommended for this segment (either incapable or overkill for the segment) ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-7
  • 8.
    Metanoia, Inc. Applicability of Key Technologies: Critical Systems Thinking™ A Comparative Rating Metro Access Metro Aggregation Metro Core Q-in-Q    Connectionless 802.1ad (PB) Good with small # flows Scalability strained w/ Unscalable (due to SPT, (few hundreds) large # flows (thousands) flooding, convergence) MAC-in-MAC    802.1ah (PBB) Suitable for access, but Unstable and unscalable Ideally scaled for agg. likely overkill (SPT/flooding, mgt.) 802.1 Qay    (PBB-TE) and Waste, as config./mgt. Useable, gives TE and TE/ manageability help PVT overhead not worth it manageability long-lived paths, QoS Connection-    Oriented IP/MPLS Mature tech., suitable, but Signaling, automatic CP Long indstry experience, needs configuration simplify confi. significant best practices T-MPLS/    MPLS-TP Suitable, but config. Controllability, similarity Better manageability of overhead significant to SDH/SONET, resilience aggregated traffic The above assessment/rating is based on the inherent properties/capabilities of each technology (and not its maturity, availability, and standardization status, which are examined in Module 3) ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-8
  • 9.
    Metanoia, Inc. Critical Systems Thinking™ Workshop Outline  Overview of Architectural Options  Applicability of Key Technologies to Access/Metro/Core  Q-in-Q (PB), MAC-in-MAC (PBB), 802.1Qay (PBB-TE) and PVT, IP/MPLS, MPLS-TP  Basic Architectures and Their Analysis  Parallel  Overlay  Uniform  Mobile-Backhaul (presented in detail later)  Hybrid Architectures and Their Analysis ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-9
  • 10.
    Metanoia, Inc. Critical Systems Thinking™ Parallel Networks: Network Architecture  Two isolated networks (ships-in-the-night)  Different fibers, lambdas, or SDH/SONET channels  Separate systems (for Ethernet and IP/MPLS) or independent cards in the same system  IP/MPLS network can be:  p2p, mp2mp  Ethernet network can be PB, PBB, PBB-TE based, and be:  p2p, mp2mp, p2mp ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-10
  • 11.
    Metanoia, Inc. Critical Systems Thinking™ Why Parallel Networks? If majority of metro services are ...  Ethernet  IP/MPLS connections handled by backhauling them from spoke(s) to metro hub, where IP routing is done  IP routing capability not needed in spoke COs  IP or IP/MPLS  Mesh IP/MPLS routers by co-locating with Ethernet switches in all or some CO/POPs ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-11
  • 12.
    Metanoia, Inc. Critical Systems Thinking™ Why Parallel Networks?  Useful when the provider L2 (Ethernet) and L3 (IP/MPLS) networks have  Shared POPs/Cos  MSPPs with both IP/MPLS and Ethernet capabilities  Applicable for operators that wish for both technologies to co-exist with service aggregation ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-12
  • 13.
    Metanoia, Inc. Critical Systems Thinking™ Parallel Networks: Network Layout IP/MPLS Co-located IP/MPLS Linecard Router & Ethernet IP/MPLS Switch IP/MPLS Router IP/MPLS or T-MPLS or MPLS-TP Network Ethernet Hybrid system with two types of line cards Ethernet Linecard Ethernet Switch PB, PBB, PBB-TE Network ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-13
  • 14.
    Metanoia, Inc. Critical Systems Thinking™ Parallel Networks: Logical Structure IP/MPLS Co-located IP/MPLS Linecard Logical IP /MPLS Router & Ethernet Link Switch IP/MPLS Router Logical IP /MPLS Link Hybrid system with two types of line cards Logical Ethernet Linecard Ethernet Link Ethernet Switch Ethernet Switch Logical Ethernet Link ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-14
  • 15.
    Metanoia, Inc. ParallelNetworks: Critical Systems Thinking™ IP/MPLS Network Realization Bi-directional Logical Mesh logical links IP/MPLS Layer (a) Logical MPLS Network L1 L2 L3 L4 L5 L3 mp2p LSPs Physical links can have Ethernet or SONET /SDH framing L2 (b) LSP Realization of Logical Network L1 L5 p2p LSPs L4 ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-15
  • 16.
    Metanoia, Inc. Parallel Networks: Critical Systems Thinking™ Ethernet Network Realization Logical Mesh at Logical links Ethernet Layer (a) Logical Ethernet Network (b) E-Line Realization of Logical Network E-Line (Realization via Ethernet entities) Physical links with Ethernet framing ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-16
  • 17.
    Metanoia, Inc. Critical Systems Thinking™ Parallel Networks: Protocol Stack  In parallel networks, the protocol stack is trivial, since each network is fully independent of the other  For Ethernet it is: Ethernet Header Varies depending on whether the Ethernet Payload network is PB, PBB, or PBB-TE-based  For IP/MPLS it is: LSP-Label May vary if further nesting of LSPs is used VC-Label IP Header Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-17
  • 18.
    Metanoia, Inc. Critical Systems Thinking™ Parallel Networks: Assessment Pros Cons  Flexible  Higher CapEX relative to uniform  IP/Ethernet networks/services can network grow independently  Need 2x fibers, lambdas, or  Fault isolation SDH/SONET channels  Faults in one n/w, do not affect  More systems, cabling; thus, service in the other CO/POP space & power  Different mgt. s/w – one for each  Service independence network: IP, Ethernet  L2 svcs. not affected by L3 svcs.  Greater OpEX and visa-versa  More vendors, maintenance  Greater number of s/w and mgt. tools  Good choice if a provider has either an Ethernet- or IP/MPLS-dominant network, and wishes to expand into the other incrementally ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-18
  • 19.
    Metanoia, Inc. Critical Systems Thinking™ Overlay Networks: IP/MPLS over Ethernet  Metro is built using Ethernet switches  PB, or PBB, or PBB-TE capable  Requires transport network  SDH/SONET, lambda’s or fiber to interconnect switches  Ethernet layer is transmission network  Interconnects IP/MPLS routers – using p2p, mp2mp, p2mp constructs  Ethernet network is oblivious to IP/MPLS layer  IP/MPLS routers (U-PE, N-PE, PE) connected in logical mesh or hub-and-spoke, or some mix of two ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-19
  • 20.
    Metanoia, Inc. Why OverlayNetworks with Critical Systems Thinking™ IP/MPLS-over-Ethernet? If majority of metro services are ...  Ethernet  IP/MPLS connections handled by backhauling them from spoke(s) to metro hub, where IP routing is done  IP routing capability not needed in spoke COs  IP or IP/MPLS  Mesh IP/MPLS routers by co-locating with Ethernet switches in all or some CO/POPs ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-20
  • 21.
    Metanoia, Inc. IP/MPLS-over-Ethernet Critical Systems Thinking™ Overlay Network: Network Layout Co-Located Systems IP/MPLS Single IP/Ethernet Platform Ethernet PB, PBB, or PBB-TE Ethernet Network CE Customer Co-Located Edge Device Systems ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved M1-21 CE Operator Best Practices
  • 22.
    Metanoia, Inc. IP/MPLS-over-Ethernet Critical Systems Thinking™ Overlay Network: Logical Structure Logical Links Single IP/Ethernet Platform PB, PBB, or PBB-TE Network Logical Mesh ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-22
  • 23.
    Metanoia, Inc. IP/MPLS-over-Ethernet Critical Systems Thinking™ Overlay Network: Realization via E-Line p2p E-Line service (offered by underlying Ethernet network) Single IP/Ethernet Platform  The “transmission network” here is the Logical Links Ethernet network  Logical network is thus realized using this network’s entities: E-lines ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-23
  • 24.
    Metanoia, Inc. IP/MPLS-over-Ethernet Critical Systems Thinking™ Overlay Network: Realization via E-LAN Single IP/Ethernet Platform mp2mp Ethernet (E-LAN)  The “transmission network” here is the Ethernet network  Logical network is thus realized using this network’s entities: E-LAN ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-24
  • 25.
    Metanoia, Inc. IP/MPLS-over-Ethernet Critical Systems Thinking™ Overlay Network: Protocol Stack CE IP/MPLS Co-located Systems IP/MPLS Combined IP/ Ethernet Switch IP Service IP Service Ethernet CE Ethernet Svc CE Ethernet Svc Ethernet Network IP Service or PB/PBB PB/PBB PB/PBB PBB-TE PBB-TE PBB-TE IP/MPLS Service MPLS MPLS MPLS MPLS MPLS Ethernet Ethernet Ethernet Ethernet Ethernet Ethernet Ethernet IP IP IP IP IP IP IP Payload Payload Payload Payload Payload Payload Payload PB/PBB PB/PBB PB/PBB Ethernet Service PBB-TE PBB-TE PBB-TE Ethernet Ethernet Ethernet Ethernet Ethernet Payload Payload Payload Payload Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-25
  • 26.
    Metanoia, Inc. Overlay NetworksIP/MPLS-over Critical Systems Thinking™ -Ethernet: Assessment Pros Cons  Lower CapEX relative to Parallel n/wks  Lower flexibility relative to Parallel  Potentially fewer systems (need only network one set for underlying txn. network –  IP/MPLS & Ethernet networks coupled here Ethernet)  Cannot grow independently  Less fiber – due to common transport  Resilience issues  Simpler mgt – due to fewer systems, and hybrid systems (IP+Ethernet)  Faults in Ethernet network may affect IP/MPLS service  Lower OpEx  Potentially, one vendor  Single s/w and mgt. tools  Single dept. to manage network  Good choice if a provider has primarily an Ethernet infrastructure, and wishes to expand into IP/MPLS services ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-26
  • 27.
    Metanoia, Inc. Critical Systems Thinking™ Overlay Networks: Ethernet-over-IP/MPLS  Metro is built using IP/MPLS devices  p2p, mp2mp IP/MPLS, MPLS-TP/T-MPLS network or any mix  Requires one transport network  SDH/SONET, lambda’s or fiber to interconnect IP/MPLS routers  MPLS layer is transmission network  Interconnects Ethernet switches – using p2p (VPWS) or mp2mp (VPLS) constructs  VPWS interconnection à  MPLS network provides a transparent tunnel (network interworking)  VPLS interconnection à  MPLS network participates in Ethernet switching via VSI (service interworking)  Ethernet switches connected as logical mesh or hub-and-spoke or mesh-star topology ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-27
  • 28.
    Metanoia, Inc. Why OverlayNetworks with Critical Systems Thinking™ Ethernet-over-IP/MPLS? If majority of metro services are ...  IP/MPLS  Ethernet connections are backhauled to metro hub, where Ethernet switching is done  Ethernet switching capability not needed in spoke COs/PoPs  Ethernet  Mesh Ethernet switches by co-locating with IP/MPLS routers in all or some CO/POPs ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-28
  • 29.
    Metanoia, Inc. Ethernet-over-IP/MPLS Critical Systems Thinking™ Overlay Network: Network Layout Co-Located Systems Ethernet Single IP/Ethernet Platform IP/MPLS IP/MPLS or T-MPLS or MPLS-TP Network IP SVC Customer Edge Co-Located CE Device Systems Ethernet SVC ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-29 CE
  • 30.
    Metanoia, Inc. Ethernet-over-IP/MPLS Critical Systems Thinking™ Overlay Network: Logical Structure Logical Links Single IP/Ethernet Platform IP/MPLS or T-MPLS or MPLS-TP Network Logical Mesh ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-30
  • 31.
    Metanoia, Inc. Ethernet-over-IP/MPLS Critical Systems Thinking™ Overlay Network: Realization via VPWS p2p PW in IP/MPLS network Logical Links Single IP/Ethernet Platform Logical Mesh ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-31
  • 32.
    Metanoia, Inc. Ethernet-over-IP/MPLS Critical Systems Thinking™ Overlay Network: Realization via VPLS Single IP/Ethernet PW full-mesh (VPLS) with Platform VSI-based routing ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-32
  • 33.
    Metanoia, Inc. Ethernet-over-IP/MPLS Critical Systems Thinking™ Overlay Network: Protocol Stack CE CE IP Service Co-located Systems Ethernet Ethernet SVC IP/MPLS IP Service Ethernet CE IP/MPLS Network CE SVC Ethernet SVC MPLS MPLS MPLS Ethernet Label Label Label Service PWE3 PWE3 PWE3 Label Label Label PB/PBB PB/PBB PB/PBB PB/PBB PB/PBB PBB-TE PBB-TE PBB-TE PBB-TE PBB-TE Ethernet Ethernet Ethernet Ethernet Ethernet Ethernet Ethernet Payload Payload Payload Payload Payload Payload Payload IP Service MPLS MPLS MPLS Ethernet Label Ethernet Label Label IP IP IP IP IP Payload Payload Payload Payload Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-33
  • 34.
    Metanoia, Inc. Ethernet-over-PB-over-IP/MPLS Critical Systems Thinking™ (VPWS) Interworking: Protocol Stack CE CE Co-located Systems Ethernet Ethernet SVC IP/MPLS CE IP/MPLS, T-MPLS, CE MPLS-TP Network Ethernet SVC One-to-one Mapping (No MAC Lookup) LSP Label (MPLS) LSP Label VC Label (PWE3) VC Label C-DA C-DA C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-DA C-SA 5-tag 5-tag 5-tag 5-tag C-SA C-tag C-tag C-tag C-tag C-tag C-tag Payload Payload Payload Payload Payload Payload Customer Ethernet Switch IP/MPLS Packet Packet Router Packet ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-34
  • 35.
    Metanoia, Inc. Ethernet-over-PBB-over-IP/MPLS Critical Systems Thinking™ (VPWS) Interworking: Protocol Stack CE CE Co-located Systems Ethernet Ethernet SVC IP/MPLS CE IP/MPLS, T-MPLS, CE MPLS-TP Network Ethernet SVC One-to-one Mapping (no MAC lookup) LSP Label (MPLS) LSP Label VC Label (PWE3) VC Label B-DA B-DA B-DA B-DA B-SA B-SA B-SA B-SA B-tag B-tag B-tag B-tag I-tag I-tag I-tag I-tag C-DA C-DA C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-DA C-SA S-tag S-tag S-tag S-tag C-SA C-tag C-tag C-tag C-tag C-tag C-tag Payload Payload Payload Payload Payload Payload Customer Ethernet Switch IP/MPLS ©Copyright 2009 Packet Packet Carrier Ethernet Technology Strategies & Evolving Router Packet All Rights Reserved Operator Best Practices M1-35
  • 36.
    Metanoia, Inc. Ethernet-over-PB-over-IP/MPLS Critical Systems Thinking™ (VPLS) Interworking: Protocol Stack CE CE Co-located Systems Ethernet Ethernet SVC IP/MPLS CE IP/MPLS, T-MPLS, CE MPLS-TP Network Ethernet Ethernet SVC bridging VPLS (VSI MAC Lookup) LSP Label (MPLS) LSP Label VC Label (PWE3) VC Label C-DA C-DA C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-DA VPLS C-SA 5-tag VSI 5-tag 5-tag 5-tag C-SA C-tag C-tag C-tag C-tag C-tag C-tag Payload Payload Payload Payload Payload Payload Customer Ethernet Switch IP/MPLS Packet Packet Router Packet ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-36
  • 37.
    Metanoia, Inc. Ethernet-over-PBB-over-IP/MPLS Critical Systems Thinking™ (VPLS) Interworking: Protocol Stack CE CE Co-located Systems Ethernet Ethernet SVC IP/MPLS CE IP/MPLS, T-MPLS, CE MPLS-TP Network Ethernet Ethernet SVC bridging VPLS (VSI MAC LSP Label (MPLS) LSP Label Lookup) VC Label (PWE3) VC Label B-DA B-DA B-DA B-DA B-SA B-SA B-SA B-SA B-tag B-tag B-tag B-tag I-tag VPLS I-tag I-tag I-tag C-DA VSI C-DA C-DA C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-SA S-tag S-tag S-tag S-tag C-SA C-tag C-tag C-tag C-tag C-tag C-tag Payload Payload Payload Payload Payload Payload Customer Ethernet Switch IP/MPLS Packet Packet Router Packet ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-37
  • 38.
    Metanoia, Inc. Critical Systems Thinking™ Uniform Networks: Network Architecture  One technology used throughout (for all three metro segments)  Access, Aggregation, and Core  Technology could be either Ethernet or IP/MPLS  Ethernet-only option  Access can be 802.1ad (Q-in-Q)  Metro can be 802.1ah (MAC-in-MAC)  Core can be 802.1Qay (Provider Backbone Bridging – TE)  IP/MPLS-only option  Access uses p2p PWs in hub-and-spoke design  Metro uses a meshed VPLS design  Core uses a hierarchically meshed H-VPLS design ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-38
  • 39.
    Metanoia, Inc. Critical Systems Thinking™ Uniform Networks: Ethernet-Only  Network Architecture  Comprised typically of access, metro, core segments  Access à PB (802.1ad), metro à PBB (802.1ah), core à PBB/PBB-TE (802.1Qay)  Logical Structure  Service-Layer is Ethernet, with remote CE’s knowing each other’s MAC address  Devices at service-layer connected by p2p or p2mp Ethernet tunnels  Physical Realization (of logical structure)  Transport layer could be Ethernet or SDH/SONET  Protocol Stack  Illustrated ahead ... ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-39
  • 40.
    Metanoia, Inc. Uniform Networks: Ethernet Only Critical Systems Thinking™ Network Layout and Protocol Stack Access Network Aggregation Network Core Network Aggregation Network Access Network (802.1ad) (802.1ah) (802.1Qay) (802.1ah) (802.1ad) Provider Backbone Provider Backbone Provider Bridging (PBB) Provider Bridging (PBB) BCB Bridging (PBB) Bridging (PBB) Last Mile Last Mile IB-BEB BCB B-BEB B-BEB B-BEB IB-BEB PE CE CE PE PB B-BEB BCB PBB – Traffic Engineered (PBB-TE) PB CE PE PE CE IB-BEB IB-BEB BCB B-BEB B-BEB B-BEB 802.1ad/Q-in-Q 802.1ah 802.1ah 802.1ad/Q-in-Q encapsulation BCB encapsulation decapsulation decapsulation B-DA B-DA - Pinned paths B-DA - Based only on B-SA B-SA B-DA, B-SA, B-Tag B-SA - No STP B-Tag B-Tag B-Tag - No MAC learning I-Tag I-Tag I-Tag Switching based on pre - C-DA C-DA C-DA configured fwding tables C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-SA C-DA C-SA S-Tag S-Tag S-Tag S-Tag S-Tag C-SA C-Tag C-Tag C-Tag C-Tag C-Tag C-Tag C-Tag Payload Payload Payload Payload Payload Payload Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-40
  • 41.
    Metanoia, Inc. Critical Systems Thinking™ Uniform Networks: IP/MPLS-Only  Network Architecture  Comprised typically of access, aggregation (metro), core segments  Two options exist:  Option 1: Access/Metro (Aggregation) à p2p spoke PWs to backbone (core) PEs, Core à H-VPLS mesh between backbone PEs  Option 2: Access à p2p spoke PWs to aggregation PEs, Metro (Aggregation)/Core à H-VPLS mesh between all aggregation PEs  Logical Structure  Service-Layer is still Ethernet; remote CE’s/nPE’s learn MAC addresses  Devices at service-layer connected by p2p MPLS LSPs  Physical Realization (of logical structure)  Construct used at transport layer are IP/MPLS p2p PWs in p2p LSPs  Protocol Stacks  Illustrated ahead ... ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-41
  • 42.
    Metanoia, Inc. Uniform Networks: IP/MPLS Only Critical Systems Thinking™ Network Layout and Protocol Stack (1) Access Network Aggregation Network Core Network Aggregation Network Access Network (p2p PW) (p2p PW) (PW full mesh, H-VPLS) (p2p PW) (p2p PW) Access + Aggregation spanned by spoke PWs Full PW Mesh P per service Spoke PW PE P nPE nPE P PE CE uPE CE uPE P P P PE CE uPE uPE CE PE PE P nPE P nPE VPWS (p2p) PW Spoke PW VPWS (p2p PW) VPLS P VPLS decapsulation encapsulation encapsulation decapsulation LSP LSP Tunnel LSP LSP LSP Label Label Label Tunnel Label Tunnel Label Spoke PW Label PW Label PW Label Mesh PW Label Spoke PW Label (PW) (VPLS (PW) C-DA C-DA C-DA C-DA Instance) C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-SA C-SA C-SA S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag Payload Payload Payload Payload Payload Payload Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-42
  • 43.
    Metanoia, Inc. Uniform Networks: IP/MPLS Only Critical Systems Thinking™ Network Layout and Protocol Stack (2) Access Network Aggregation Network Core Network Aggregation Network Access Network (p2p PW) (PW full mesh) (PW full mesh, H-VPLS) (PW full mesh) (p2p PW) Aggregation and Core spanned by Full PW Mesh P PW Full Mesh Spoke PW nPE P PE PE P nPE CE uPE CE uPE P P P PE CE uPE Spoke uPE CE nPE PW nPE P PE P PE VPWS (p2p) PW VPLS VPLS P VPWS (p2p PW) encapsulation encapsulation decapsulation decapsulation LSP LSP LSP Tunnel LSP LSP Tunnel Label Tunnel Label Label Label Label Spoke PW Label Spoke PW Label PW Label Mesh PW Label PW Label (PW) (PW) (VPLS C-DA C-DA C-DA C-DA Instance ) C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-SA C-SA C-SA S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag Payload Payload Payload Payload Payload Payload Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-43
  • 44.
    Metanoia, Inc. Uniform Networks:IP/MPLS-Only Critical Systems Thinking™ Observations  Trade-offs exist between Option 1 and Option 2 in terms of MAC learning and PW scalability  Option 1: PW full mesh in core, spoke PWs upto core nPEs  nPEs must learn all customer MAC addresses  PW mesh confined to core, thus only core PEs must be meshed  Requires one PW mesh per customer’s service instance  Option 2: PW full mesh in agg. + core, spoke PWs upto agg. nPEs  nPEs learn all customer MACs, but fewer than in Option 1  PW mesh, however, is between nPEs in all aggregation networks (thus much larger with more PWs than in Option 1)  Requires one PW mesh per customer’s service instance ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-44
  • 45.
    Metanoia, Inc. Critical Systems Thinking™ Workshop Outline  Overview of Architectural Options  Applicability of Key Technologies to Access/Metro/Core  Q-in-Q (PB), MAC-in-MAC (PBB), 802.1Qay (PBB-TE) and PVT, IP/MPLS, MPLS-TP  Basic Architectures and Their Analysis  Overlay  Parallel  Uniform  Mobile-Backhaul (presented in detail later)  Hybrid Architectures and Their Analysis ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-45
  • 46.
    Metanoia, Inc. Our AnalysisApproach For Critical Systems Thinking™ Hybrid Architectures  Network Overview  Overall architecture/layout of the network  Protocol Stack  Explain tunneling/interworking in different network segments  Assessment  Key observations about the architecture  Benefits and drawbacks of approach ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-46
  • 47.
    Metanoia, Inc. Critical Systems Thinking™ Hybrid Architectures: Overview  Involve a combination of IP/MPLS and Ethernet (PB, PBB, PBB-TE)  Access network, customer-edge to provider-edge, is common and typically one of three Ethernet types (so not shown in figures)  Vanilla Ethernet (802.1d)  V-LAN capable Ethernet (802.1q)  S-VLAN capable Ethernet or Q-in-Q (802.1ad)  Two principal hybrid architectures  H-VPLS with PB or PBB Aggregation  H-VPLS with MPLS Aggregation ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-47
  • 48.
    Metanoia, Inc. Critical Systems Thinking™ Hybrid Architectures Taxonomy H-VPLS with PB/PBB Aggregation H-VPLS w/ Homogeneous H-VPLS with Heterogeneous PBB Aggregation PB/PBB Aggregation Type I Svc. Interface Type II Svc. Interface Modified PB PE Modified PBB PE (S-Tag (B-VID) as delimiter) (I-SID as delimiter) H-VPLS with MPLS Aggregation PBB-capable uPE PBB-capable nPE PBB Migration (PBB uPE, for short) (PBB nPE , for short) PB service frames PBB service frames PB and PBB service over H-VPLS core over H-VPLS core frames over H-VPLS core ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-48
  • 49.
    Metanoia, Inc. H-VPLS withPBB Aggregation: Critical Systems Thinking™ Type I Interface -- Overview  Type I i/f is a B-tagged i/f with B-VID as service delimiter  Handoff between BCB and N-PE is B-tagged PBB frame  N-PE itself can be transparent to the frames  Treats them as 802.1ad frames  Only need support 802.1ad-style frames ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-49
  • 50.
    Metanoia, Inc. H-VPLS withPBB Aggregation: Critical Systems Thinking™ Type I i/f – Operation & Protocol Stack PBB Ethernet MPLS/PW Ethernet CE U-PE BCB N-PE Raw or Tagged- N-PE BCB U-PE mode Ethernet PW CE PBB X IP/MPLS X PBB A1 IB-BEB C1 D1 IB-BEB B1 S-Tag I/f S-Tag I/f (B-VID is Svc. delimiter ) (B-VID is Svc. delimiter ) All 3 operational modes Supports VPLS for B-Tag supported: (need not be PBB-aware) - Port Mode - VLAN Mode - VLAN Bundling Mode LSP-Label VC-Label B-DA B-DA B-DA B-DA B-DA B-SA B-SA B-SA B-SA B-SA B-Tag B-Tag B-Tag B-Tag B-Tag I-Tag I-Tag I-Tag I-Tag I-Tag C-DA C-DA C-DA C-DA C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-SA C-SA C-SA S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag Payload Payload Payload Payload Payload Payload Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-50
  • 51.
    Metanoia, Inc. H-VPLS withPBB Aggregation Critical Systems Thinking™ Type I Interface: Assessment  Application of the overlay model  Use case: SP has converged to an MPLS core, but prefers Ethernet aggregation to connect 802.1ad-based access networks  A “steady-state” model, once all aggregation networks are 802.1ah-capable Benefits Drawbacks  No new functionality in VPLS PE  Needs extra replication in core  Reduces C-MAC learning and PW (relative to w/o I-SID bundling) mesh in core PEs  Unknown unicast, brdcast, or  Due to I-SID bundling, multiple mcast in a single svc. instance (I- svc. instances map to one bridge SID) leads to full brdcast in core domain (B-VID)  May be addressed by per-I-SID  Separates service layer (I-SID) flood containment from network layer (B-VID) ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-51
  • 52.
    Metanoia, Inc. H-VPLS withPBB Aggregation: Critical Systems Thinking™ Type II Interface -- Overview  Type II i/f: I-tagged i/f with I-SID as service delimiter  B-Tag is locally significant in PBB cloud, not sent over core  PE must support B-BEB and VPLS functionality  Must interpret I-Tag ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-52
  • 53.
    Metanoia, Inc. H-VPLS withPBB Aggregation: Critical Systems Thinking™ Type II i/f – Operation & Protocol Stack PBB PBB PBB Ethernet MPLS/PW Ethernet CE U-PE B-BEB N-PE N-PE B-BEB U-PE CE PBB X IP/MPLS X PBB B-BEB D1 PBB A1 C1 B-BEB B1 I-tag I/f I-tag I/f (Type II) (Type II) (I-SID is svc. delimiter) B-VID locally significant in 2 Mapping options PBB, not xported over core -- I-SID à VPLS Must support B-BEB and VPLS capability -- I-SID àB-VID à VPLS Internal B-VID, B-BEB removes enables I-SID LSP-Label PBB-specific bundling B-Tag VD-Label B-DA B-DA B-DA B-SA B-DA B-SA B-DA B-SA B-Tag B-SA B-Tag B-SA B-Tag I-Tag I-Tag I-Tag I-Tag I-Tag C-DA C-DA C-DA C-DA C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-SA C-SA C-SA S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag Payload Payload Payload Payload Payload Payload Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-53
  • 54.
    Metanoia, Inc. H-VPLS withPBB Aggregation Critical Systems Thinking™ Type II Interface: Assessment  Application of the overlay model  Use case: SP has converged to MPLS core, but prefers Ethernet aggregation to connect 802.1ad-based access networks  Good for customers with mcast traffic (w/o mcast pruning fn. at VPLE-PE) Benefits Drawbacks  Less replication in core, w/o needing  N-PE complex with new capability – per-I-SID flood containment needs B-comp for I-SID processing  Supports tightly & loosely-coupled  Potentially larger # of PWs in core, for service domains same # of services (relative to Type I)  Increased segregation of svc. instances (by I-SID) over disjoint PW meshes ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-54
  • 55.
    Metanoia, Inc. H-VPLS withHeterogeneous Access Critical Systems Thinking™ Modified PB PE -- Overview  Used to interoperate existing 802.1ad (PB) networks with new 802.1ah (PBB) aggregation networks  Both networks connect to an H-VPLS core/backbone  PE interfacing with PB network must support VPLS and IB- BEB functionality  Mapping of S-VID to I-SID  I-SID bundling into B-VID and mapping to a VPLS instance ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-55
  • 56.
    Metanoia, Inc. H-VPLS withModified PB PE Critical Systems Thinking™ Operation & Protocol Stack Q-in-Q PBB Ethernet MPLS/PW Ethernet CE U-PE1 BCB N-PE1 N-PE2 B-BEB U-PE2 CE Q-in-Q X IP/MPLS X PBB A1 C1 D1 B1 S-tag I/f I-tag I/f (Type 1) (Type II) Must support Must support IB-BEB and B-BEB and -- Participates in local I -SID VPLS VPLS domain of MPLS core -- Supports I-SID bundling -- 1:1 mapping of S-VID à I-SID LSP-Label VC-Label B-DA B-DA B-SA B-DA B-SA B-Tag B-SA B-Tag I-Tag I-Tag I-Tag C-DA C-DA C-DA C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-SA C-DA C-SA S-Tag S-Tag S-Tag S-Tag S-Tag C-SA S/C-Tag C-Tag C-Tag S/C-Tag C/S-Tag S/C-Tag S/C-Tag Payload Payload Payload Payload Payload Payload Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-56
  • 57.
    Metanoia, Inc. Critical Systems Thinking™ H-VPLS with Modified PB PE: Assessment  Overlay with cascade of IP/MPLS and Ethernet networks  Use case: SP has converged to MPLS core, but has different Ethernet aggregation/access networks, which have not converged to a single Ethernet technology Benefits Drawbacks  Reduces PW mesh in core  N-PE is complex with new  Via I-SID bundling, where group of capability – needs IB-BEB I-SIDs is mapped to B-VID and to a functionality VPLS instance  PB PE (N-PE1) needs to learn  Supports tightly, loosely-coupled, many C-MAC addresses and different service domains ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-57
  • 58.
    Metanoia, Inc. H-VPLS withHeterogeneous Access Critical Systems Thinking™ Modified PBB PE -- Overview  Used to interoperate existing 802.1ad (PB) networks with new 802.1ah (PBB) aggregation networks  Both networks connect to an H-VPLS core/backbone  PE interfacing with PBB network must support VPLS and IB- BEB functionality  B-component faces PBB cloud  I-component faces MPLS core, connects to VPLS forwarder ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-58
  • 59.
    Metanoia, Inc. H-VPLS withModified PBB PE Critical Systems Thinking™ Operation & Protocol Stack Q-in-Q PBB I-Comp Ethernet MPLS/PW Ethernet B-Comp CE U-PE1 PCB N-PE1 N-PE2 B-BEB U-PE2 CE Q-in-Q X IP/MPLS X PBB 1P 1P 1P 1P S-tag I/f I-tag I/f (Type I) (Type II) Must support Must support VPLS for B-tag IB-BEB -- 1:1 mapping of S-VID (not PBB aware) and VPLS to VPLS instance -- S-VID bundle has no counterpart in PBB n/w B-DA B-DA B-SA LSP-Label B-SA B-Tag VD-Label I-Tag I-Tag C-DA C-DA C-DA C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-SA C-DA C-SA S-Tag S-Tag S-Tag S-Tag S-Tag C-SA C-Tag C-Tag C-Tag C-Tag C-Tag C-Tag C-Tag Payload Payload Payload Payload Payload Payload Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-59
  • 60.
    Metanoia, Inc. H-VPLS withModified PBB PE: Critical Systems Thinking™ Assessment  Overlay with cascade of Ethernet and IP/MPLS networks  Use case: SP has converged to MPLS core, but has different Ethernet aggregation/access networks, which have not converged to a single Ethernet technology Benefits Drawbacks  VPLS core can operate without any  N-PE2 needs IB-BEB functionality modifications  Maps tags (I-tag) to VPLS instance  PB PE (N-PE1) must learn C-MAC addresses  Supports tightly, loosely-coupled, and different service domains  Absence of S-VID bundling, implies each I-SID maps to independent PW  mesh in core à no scaling in PWs Viable option to incrementally at PBB aggregation networks to existing PB networks over IP/MPLS core  Need same I-SID domain across all PBB aggregation networks for consistent C-VID grouping ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-60
  • 61.
    Metanoia, Inc. H-VPLS withMPLS Aggregation Critical Systems Thinking™ PBB uPE -- Overview  Used to integrate 802.1ah (PBB) functionality into an IP/MPLS provider network  PBB functionality is embedded in uPE to restrict MAC learning as close to customer as possible  PE at the IP/MPLS core is unchanged, and only needs IP/MPLS and PW capability  Bridging over VPLS network need be only 802.1ad capable as  MAC forwarding is based on B-MAC address space  Service delimiter is based on B-VLAN ID or B-VID ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-61
  • 62.
    Metanoia, Inc. H-VPLS withMPLS Aggregation Critical Systems Thinking™ PBB uPE: Operation & Protocol Stack PBB MPLS/PW MPLS/PW MPLS/PW CE U-PE LSR N-PE N-PE LSR U-PE CE MPLS X IP/MPLS X IB-BEB MPLS A1 C1 D1 IB-BEB B1 MPLS MPLS I/f I/f Spoke PWs per Full PW mesh per VPLS instance -- Port Mode -- VLAN Mode Must support IB-BEB and Must support MPLS/PW VPLS instance -- VLAN-bundling Mode MPLS/PW encap of C-MAC Operates on B-DA, B-SA, Maps svcs. to I-SID frames into B- MAC frames B-VID only (not I-SID), so I-SID àB-Tag (bridge domain) need be 802.1ad capable B-Tag à VPLS instance LSP-Label LSP-Label LSP-Label LSP-Label LSP-Label VC-Label VC-Label VC-Label VC-Label VC-Label B-DA B-DA B-DA B-DA B-DA B-SA B-SA B-SA B-SA B-SA B-Tag B-Tag B-Tag B-Tag B-Tag I-Tag I-Tag I-Tag I-Tag I-Tag C-DA C-DA C-DA C-DA C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-SA C-SA C-SA S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag Payload Payload Payload Payload Payload Payload Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-62
  • 63.
    Metanoia, Inc. H-VPLS withMPLS Aggregation Critical Systems Thinking™ PBB uPE: Assessment  Overlay with IP/MPLS transmission n/w and Ethernet service  Use case: SP converged to MPLS in core and aggregation networks  I-SID allocation  Global across MPLS networksno I-SID translation needed  I-SID bundling must be consistent across all participating PEs Benefits Drawbacks  PBB functionality at uPE improves  Customer broadcast/mcast frames scalability must be sent over entire B-VLAN  Multiple I-SIDs per B-VLAN à fewer  Greater replication over VPLS PW core PWs mesh  As B-VLAN mesh is larger than scope of any particular I-SID  N-PE only learns uPE MACs à MAC addresses scale ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-63
  • 64.
    Metanoia, Inc. H-VPLS withMPLS Aggregation Critical Systems Thinking™ PBB nPE -- Overview  Used to integrate 802.1ah (PBB) functionality into an IP/MPLS provider network  PBB function embedded in nPE, when uPE is not capable  Bridging over VPLS n/w must be 802.1ah capable, as nPE  Assigns an I-SID for each service of a customer  Muxes I-SIDs into a common bridge domain, or B-VLAN  Maps either I-SIDs or B-VLANs to VPLS instances ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-64
  • 65.
    Metanoia, Inc. H-VPLS withMPLS Aggregation Critical Systems Thinking™ PBB nPE: Operation & Protocol Stack Q-in-Q PBB Q-in-Q MPLS/PW MPLS/PW MPLS/PW CE U-PE LSR N-PE N-PE LSR U-PE CE IB-BEB MPLS X IP/MPLS X MPLS A1 C1 IB-BEB D1 B1 MPLS MPLS I/f I/f Full PW mesh per PWs per customer Must support customer or cust. grp (one service per PW, VPLS and PBB no service muxing) Supports -- Port Mode LSP-Label -- VLAN Mode -- VLAN-bundling Mode VD-Label B-DA B-SA LSP-Label LSP-Label B-Tag LSP-Label LSP-Label C-DA C-DA VD-Label VD-Label I-Tag VD-Label VD-Label C-DA C-DA C-DA C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-SA C-DA C-SA S-Tag S-Tag S-Tag S-Tag S-Tag C-SA C-Tag C-Tag C-Tag S/C-Tag C-Tag C-Tag C-Tag Payload Payload Payload Payload Payload Payload Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-65
  • 66.
    Metanoia, Inc. H-VPLS withMPLS Aggregation Critical Systems Thinking™ PBB nPE: Assessment  Overlay with IP/MPLS txn network and Ethernet service  Use case: SP has converged to MPLS in core and aggregation n/ws, but aggregation n/w is not PBB capable  Needs integration of PBB functionality into H-VPLS PE Benefits Drawbacks  PBB functionality at nPE  Spoke PWs terminate at nPE improves VPLS PW scalability  nPE must learn all customer MAC addresses  nPE maps multipe I-SIDs into a B- VLAN à fewer core PW meshes  Per-service spoke PWs à needs significantly more spoke PWs  Since uPE is not PBB aware ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-66
  • 67.
    Metanoia, Inc. Critical Systems Thinking™ H-VPLS with PBB Migration  Provides migration path for operators with investments in IP/MPLS (and VPLS) deployments  Enables incremental induction of PBB into the network  Expands scalability benefits of PBB over time with deployment  Three migration scenarios:  Non-PBB (802.1ad) encapsulated frames over VPLS core  PBB-encapsulated (802.1ah) frames over VPLS core  Mixed frames (802.1ad and 802.1ah) over VPLS core  Depends on capabilities of different edge devices in network ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-67
  • 68.
    Metanoia, Inc. H-VPLS withPBB Migration: 802.1ad Critical Systems Thinking™ Frames in Core -- Overview  Existing 802.1ad access/aggregation n/wks  Unchanged – need no modifications  nPE’s require no changes, and need no knowledge of PBB  New MPLS-based access/aggregation networks with PBB functionality on uPE  Need IB-BEB functionality on nPE  Required to terminate PBB encap. on incoming frames  Ethernet frames in VPLS/H-VPLS core use 802.1ad format  VPLS core operation is unchanged ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-68
  • 69.
    Metanoia, Inc. H-VPLS w/PBB Migration 802.1ad Critical Systems Thinking™ Frames in Core: Operation & Protocols Q-in-Q PBB I-Comp I-Comp MPLS/PW MPLS/PW MPLS/PW B-Comp B-Comp CE U-PE1 LSR N-PE1 N-PE2 LSR U-PE2 CE MPLS X IP/MPLS X IB-BEB D1 MPLS A1 C1 MPLS IB-BEB B1 MPLS I/f I/f Must support Must terminate PBB PWs could be per Must terminate Spoke PWs per customer VPLS and MPLS/PW VPLS instance or per PBB and (one service per PW) service MPLS/PW LSP-Label LSP-Label C-DA VC-Label VC-Label B-DA B-DA B-SA B-SA LSP-Label LSP-Label LSP-Label B-Tag B-Tag C-DA VC-Label VC-Label VC-Label I-Tag I-Tag C-DA C-DA C-DA C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-SA C-DA C-SA S-Tag S-Tag S-Tag S-Tag S-Tag C-SA C-Tag C-Tag C-Tag S/C-Tag C-Tag C-Tag C-Tag Payload Payload Payload Payload Payload Payload Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-69
  • 70.
    Metanoia, Inc. H-VPLS withPBB Migration: 802.1ad Critical Systems Thinking™ Frames in Core: Assessment  Overlay with IP/MPLS txn network and Ethernet service  Use Case: SP has converged to MPLS in core and aggregation n/ws, but all access/aggregation networks are not PBB capable  Ideal when SP has single PBB-capable aggregation/access network Benefits Drawbacks  Requires no change to existing  nPE1 must learn all customer access/aggregation n/wk (Q-in-Q) MAC addresses (as spoke PWs terminate at nPE1)  PBB benefit cannot be leveraged  No change to existing VPLS nPEs (those not facing 802.1ah n/wks)  nPEs need full mesh of PWs ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-70
  • 71.
    Metanoia, Inc. H-VPLS withPBB Migration: 802.1ah Critical Systems Thinking™ Frames in Core -- Overview  Existing 802.1ad access/aggregation n/wks  Unchanged – need no modifications  nPE’s must be upgraded for PBB-based processing (IB-BEB)  All Ethernet svc frames over VPLS core are PBB-encapsulated  New MPLS-based access/aggregation networks have PBB functionality on uPE  Need IB-BEB functionality on remote nPE  PBB encap. of incoming frames terminated at far-end nPE  Ethernet frames in VPLS/H-VPLS core use 802.1ah format  Thus, VPLS core operation is modified ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-71
  • 72.
    Metanoia, Inc. H-VPLS w/PBB Migration 802.1ah Critical Systems Thinking™ Frames in Core: Operation & Protocols Q-in-Q PBB I-Comp I-Comp MPLS/PW MPLS/PW MPLS/PW B-Comp B-Comp CE U-PE1 LSR N-PE1 N-PE2 LSR U-PE2 CE MPLS X IP/MPLS X MPLS A1 C1 IB-BEB D1 MPLS IB-BEB B1 MPLS I/f I/f Must terminate PBB Must support PWs per VPLS Must terminate Spoke PWs per customer and MPLS/PW only VPLS instance PBB and (one service per PW) MPLS/PW LSP-Label LSP-Label LSP-Label C-DA VC-Label VC-Label VC-Label B-DA B-DA B-DA B-SA B-SA B-SA LSP-Label LSP-Label B-Tag B-Tag B-Tag C-DA VC-Label VC-Label I-Tag I-Tag I-Tag C-DA C-DA C-DA C-DA C-DA C-DA C-SA C-SA C-SA C-SA C-SA C-DA C-SA S-Tag S-Tag S-Tag S-Tag S-Tag C-SA C-Tag C-Tag C-Tag C-Tag C-Tag C-Tag C-Tag Payload Payload Payload Payload Payload Payload Payload ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-72
  • 73.
    Metanoia, Inc. H-VPLS withPBB Migration: 802.1ah Critical Systems Thinking™ Frames in Core: Assessment  Overlay with IP/MPLS txn network and Ethernet service  Use Case: SP has converged to MPLS in core and aggregation n/ws, and wishes to have the benefit of PBB scaling in the VPLS core  Useful for the operator whose MPLS access/aggregation networks are being upgraded to support PBB IB-BEB at uPEs Benefits Drawbacks  Reduces MAC learning at nPEs  Must upgrate nPEs of all existing  Due to PBB at nPEs, thus giving MPLS access/aggregation n/wks MAC address scalability  nPE needs IB-BEB functions  Lowers PW mesh counts in core  Multiple customer instances can be bound to a single B-VLAN and VPLS instance ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-73
  • 74.
    Metanoia, Inc. H-VPLS withPBB Migration: Mixed Critical Systems Thinking™ Frames in Core -- Overview  Existing 802.1ad access/aggregation n/wks unchanged  Exchange Ethernet frames with 802.1ad format over PWs in core  nPE’s must be upgraded for PBB-based processing (IB-BEB)  All Ethernet svc frames over VPLS core are PBB-encapsulated  New MPLS-based access/aggregation networks have PBB functionality on uPE  Exchange PBB-encap frames over VPLS core  Interworking b/ween PBB-capable and PB-capable requires that nPE of PBB network has IB-BEB functionality  Ethernet frames in VPLS/H-VPLS core use 802.1ah/802.1ad format ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-74
  • 75.
    Metanoia, Inc. H-VPLS w/ PBB Migration Mixed Critical Systems Thinking™ Frames in Core: Operation PBB Q-in-Q Q-in-Q PBB I-Comp I-Comp MPLS/PW MPLS/PW MPLS/PW B-Comp B-Comp CE U-PE1 LSR N-PE1 N-PE2 LSR U-PE2 CE MPLS X B-BEB IP/MPLS X MPLS A1 C1 IB-BEB D1 MPLS MPLS IB-BEB B1 I/f I/f Spoke PWs per customer -- Must terminate PBB and MPLS/PW Must support PWs per VPLS (one service per PW) Must terminate only VPLS instance PBB and -- Needs IB-BEB and B-BEB capability MPLS/PW -- Should be aware of remote PEs’ capabilities – via static config . or extended VPLS control plane Non-PBB Capable PBB- Capable Existing Access Networks New Access Networks Protocol stack is a combination of those shown earlier for 802.1ad and 802.1ah in core ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-75
  • 76.
    Metanoia, Inc. H-VPLS withPBB Migration: 802.1ah Critical Systems Thinking™ Frames in Core: Assessment  Overlay with IP/MPLS txn network and Ethernet service  Use Case: SP has converged to MPLS in core and aggregation n/ws, has a mix of PB- and PBB-capable access/aggregation networks Benefits Drawbacks  Core can serve both PB-encapsulated  Must upgrate nPEs of PBB-capable and PBB-encapsulated frames access/aggregation n/wks. nPE needs  IB-BEB to interface with PBB network  B-BEB to interface with PB network  Lowers PW mesh counts in core  For PBB-capable aggregation networks  PE must be aware of remote peer’s capability  Requires either static config. or VPLS control plane extensions ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-76
  • 77.
    Metanoia, Inc. Critical Systems Thinking™ Workshop Outline  Overview of Architectural Options  Applicability of Key Technologies to Access/Metro/Core  Q-in-Q (PB), MAC-in-MAC (PBB), 802.1Qay (PBB-TE) and PVT, IP/MPLS, MPLS-TP  Basic Architectures and Their Analysis  Overlay  Parallel  Uniform  Mobile-Backhaul  Hybrid Architectures and Their Analysis ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-77
  • 78.
    Metanoia, Inc. Critical Systems Thinking™ Mobile Backhaul Architectures  Mobile backhaul architectures derive from the basic and hybrid architectures presented earlier  We examine them separately due to their unique needs:  Interface with the core network  Timing and synchronization requirements  Evolution requirements – from TDM or ATM to IP/MPLS and/or Ethernet ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-78
  • 79.
    Metanoia, Inc. Critical Systems Thinking™ Evolution of Cellular Technology: 2G to 4G  2G  GSM (TDM, 56-114 kbps)  2.5G  GPRS, EDGE (TDM, 236-473 kbps)  CDMA 1XRTT (HDLC, TDM, 144 kbps)  3G  WCDMA/UMTS (3GPP) (R99, R4) (ATM, 384 kbps uplink, 2Mbps downlink)  UMTS (3GPP) (R5 (HSDPA), R6 (HSUPA))( IP, 2-3 Mbps)  EV-DO (3GPP2) (Rev0, RevA, RevB, RevC) (IP, 1.8 Mbps uplink, 3.1Mbps downlink)  4G (LTE)  3GPP (R7/R8) (IP, >50 Mbps uplink, >100 Mbps downlink)  WiMAX (802.16e, 802.16m) (IP, 50-100 Mbps) ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-79
  • 80.
    Metanoia, Inc. Evolution ofCellular Technology Critical Systems Thinking™ and Backhaul Types Network Speed Interface GSM/GPRS 56-114 Kbps TDM EDGE 236 – 473 Kbps TDM 3G (UMTS/ 384 Kbps Uplink ATM WCDMA) R3, R4 384 Kbps Downlink 3G, R5 (HSDPA), 384 Kbps Uplink IP/Ethernet R6 (HSUPA) 14.4 Kbps Downlink 500 Mbps Uplink LTE R8 (20 Mhz) IP/Ethernet >100 Mbps Downlink CDMA1X-RTT 100 Kbps TDM Legend CDMA EV-DO 1.8 Mbps Uplink 2G IP/Ethernet Rev A/B 1.8 to 5 Mbps Downlink 2.5G WiMAX (10 Mhz) 50 Mbps IP/Ethernet 3G 4G Backhaul Types ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-80
  • 81.
    Metanoia, Inc. Critical Systems Thinking™ Mobile Backhaul Components  Backhaul network – defined as the network that connects  Base Transceiver Station (BTS, or Base Station) to Base Station Controller (BSC) in 3GPP2 – GSM-based cellular networks  Node-B to Radio Network Controller (RNC) in 3GPP – CDMA-based cellular networks  Traditional backhaul networks have used ...  E1/T1 leased lines  SONET/SDH TDM channels (for higher rate aggregation)  Mobile transport infrastructure has hitherto been ...  Microwave links  Optical fiber with SDH/SONET  Evolution to packet-based wireless services creates a push for the transport itself to be packet-based: Ethernet or IP/MPLS or a combination ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-81
  • 82.
    Metanoia, Inc. Critical Systems Thinking™ Traditional Backhaul Evolution 2G BTS BSC TDM TI/EI Cellsite SONET/SDH SONET/SDH Gateway XConnect XConnect E1 T1/E1/STM SDH/SONET Network ATM RNC ATM nxE1 ATM 3G BTS Switch Separate transmission facilities for different technologies (TDM and packets) ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-82
  • 83.
    Metanoia, Inc. Critical Systems Thinking™ Mobile Backhaul Challenges  Exponential growth in mobile subscribers  3 Billion subscribers (2007)  Expected to touch 5 Billion by 2013!  Leads to massive increase in traffic volume  Shift in mobile traffic patterns  High-speed data, including multimedia traffic (video, VoIP, IMS)  Bandwidth insensitive but QoS sensitive applications!  With increasing speed, revenue-per-bit is decreasing  Result is that traffic and revenue are decoupled !  These trends lead to massive increase in traffic volume  Backhaul accounts for 30% of OpEX (Per Yankee Group 2005)  Traditional backhaul unsustainable with such traffic growth ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-83
  • 84.
    Metanoia, Inc. Key PerformanceRequirements Critical Systems Thinking™ for Mobile Backhaul  Delay – budget for entire RAN is 100ms  Backhaul segment delay must be < 3-4 ms  Loss – Target BER is  2G networks = 10-7  3G networks = 10-5 to 10-4  Synchronization – Frequency and Time accuracy  2G: 50 ppb freq. accuracy at radio interface  3G: 50 ppb freq. accuracy and 2.5 µs time accuracy for TDD  WiMAX: 8 ppm freq. for FDD/TDD, and 5-25 µs time accuracy for TDD ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-84
  • 85.
    Metanoia, Inc. Critical Systems Thinking™ Backhaul Strategy Fundamentals  Future-Proof  Should support existing legacy (TDM) infrastructure  Evolving new packet-based and IP services using diverse and coexistent technologies  Scalable  Grow b/w to support next-gen. wireless access technology LTE/4G  Ethernet interfaces versus nxT1/E1  Cost Effective – reduce OpEx  Must meet timing and synchronization targets  Simplify provisioning & planning – advanced OAM, troubleshooting ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-85
  • 86.
    Metanoia, Inc. Critical Systems Thinking™ Timing and Synchronization  ITU-T G.8261 specifies timing & synchronization in packet networks  Synchronization needed for  Radio framing accuracy, hand-off control, backhaul transport reliability  Three Methods  Synch. Ethernet: Similar to SONET/SDH -- embed clock in PHY layer  Requires changes in PHY chip  IEEE 1588 (Precision Time Protocol)  Distributed protocol: specifies how real-time PTP clocks synchronize  IEEE 802.1as  Aimed at adapting IEEE 1588 to Carrier Ethernet ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-86
  • 87.
    Metanoia, Inc. Critical Systems Thinking™ Clock Distribution Methods Used  Physical Layer Clock  Via synchronous TDM i/fs e.g. PDH/SDH/SONET  Via Sync. Ethernet as per G.8261/G.8262  GPS-receiver based synchronization  Clock distribution over packet network  IEEE 1588v2 – being looked at in ITU-T Q13/SG15, who are developing a telecom profile for 1588 v2  NTP – IETF currently working on NTP v4  Adaptive and Differential clock synchronization ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-87
  • 88.
    Metanoia, Inc. Critical Systems Thinking™ Packet Transport for Mobile Backhaul  Packet-based transport can provide high bandwidth at lower cost (than TDM transport)  Ideal choice for LTE and 4G technologies  Challenges ...  Support legacy traffic (TDM and ATM ) via circuit emulation  Meet timing and synchronization requirements  Provide QoS and protection switching  Furnish advanced OAM capabilities ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-88
  • 89.
    Metanoia, Inc. Critical Systems Thinking™ Pseudowires (PW) for Legacy Transport PW PSN Tunnel BSC AC AC Carrier Ethernet Wireless CE Network Core PE PE (BTS) AC: Attachment Ckt CE : Customer Edge (BTS) PE: Provider Edge BSC: Base Station Controller  Encapsulation  PSN Tunnels   May be IP/MPLS, T-MPLS/MPLS-TP, or Structure-Agnostic TDM-over-IP PB/PBB/PBB-TE based (SAToIP) (RFC 4553)  Structure-Aware TDM Circuit  PW Signaling Emulation (CESoPSN) (RFC 5086)  ATMoPSN (RFC 4717) ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-89
  • 90.
    Metanoia, Inc. Critical Systems Thinking™ PBB/PBB-TE for Mobile Backhaul  Connection-oriented p2p tunnels b/ween BTS (BS) -- BSC (NC)  Provides MEF-style EVPL/EVP-Tree between RAN BS and RAN NC  IEEE 802.1ag OAM enables carrier-grade OAM tools  Delay/loss ensured via admission control & 802.1Q PCP  Supporting Legacy TDM over PBB/PBB-TE  IETF has draft on PW over PBB-TE  802.1ah supports encap. of non-Ethernet frames (via short I-TAG TCI) ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-90
  • 91.
    Metanoia, Inc. Critical Systems Thinking™ PBB/PBB-TE for TDM Encapsulation IEEE 802.1ah PBB Short & Long I-Tag DEI I-SID C-DA C-SA Etc. (Service Identifier) Long I-TAG Tag Control Information DEI I-SID Etc. (Service Identifier) Short I-Tag (Only I-SID used for encapsulation of multiple protocols) EtherType used to indicate encapsulation of Ethernet or multiple protocols We show I-SID with C-DA and C-SA but without Ethernet encapsulation ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-91
  • 92.
    Metanoia, Inc. Critical Systems Thinking™ MEF Services for Mobile Backhaul RNC RNC BSC BSC Service Multiplexing mp2mp EVC Metro Ethernet EVC EVC Metro Ethernet BTS BTS BTS BTS BTS EVPL Service for Backhaul using EP-LAN Service for Backhaul using Metro Ethernet Networks Metro Ethernet Networks Services muxed at RNC UNI Needed when inter-BS communication is permitted like in LTE/802.16m (WiMAX) ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-92
  • 93.
    Metanoia, Inc. Critical Systems Thinking™ MEF Services for Mobile Backhaul RNC BSC Service Multiplexing Metro Ethernet EVC EVC BS/ BS/ BS/ BTS BTS BTS EP-Tree Service for Backhaul using Metro Ethernet Networks ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-93
  • 94.
    Metanoia, Inc. Critical Systems Thinking™ IP/MPLS for Mobile Backhaul  MPLS PW/VPLS can provide effective backhaul  Traditional circuit-based services are offered using PWs  PW signaling – uses either BGP or LDP  Protection switching  Achieved via MPLS Fast Reroute (FRR) on LSP tunnel  OAM provided by  LSP Ping, VCCV  BFD ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-94
  • 95.
    Metanoia, Inc. Critical Systems Thinking™ IP/MPLS and PBB/PBB-TE Caveats  Poor support for P2MP LSPs in MPLS  Such support is desirable:  For Clock Distribution from BSC to BTSs  When BSs allowed to communicate with each other and cooperate as in WiMAX/LTE  PW over PBB-TE is not fully developed ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-95
  • 96.
    Metanoia, Inc. Critical Systems Thinking™ Migration Strategies for Service Providers  HSDPA/3G Data Off-load  Separation of transport  GSM/UMTS voice sent over SONET/SDH  Data traffic transported over PSN tunnel  Packet-based Backhaul  TDM/ATM/Ethernet/IP all transported over PSN tunnel via PWs  Converged Transport  Single packet-switched infrastructure for wireline, broadband and mobile services ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-96
  • 97.
    Metanoia, Inc. Critical Systems Thinking™ HSDPA/3G Data Off-Load 2G BTS TDM TI/EI Multi-Service Cellsite SONET/SDH SONET/SDH Aggregation Gateway XConnect XConnect Router 2G BSC TI/EI/STM SDH/SONET Network 3G RNC Wireless Core ATM nxE1 Ethernet Carrier Ethernet 3G BTS Network Ethernet Ethernet Switch Switch ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-97
  • 98.
    Metanoia, Inc. Critical Systems Thinking™ Packet-Based Backhaul 2G BTS BSC TDM Cellsite Gateway PE PE ATM Carrier Ethernet TI/EI/STM Network (PSN) RNC/GW Ethernet 3G/4G IP/ATM/Ethernet/TDM are all transported over the PSN BTS Tunnel using PWs ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-98
  • 99.
    Metanoia, Inc. Critical Systems Thinking™ Converged Packet-Based Transport Converged Service Delivery Platform 3G/4G Base Station Aggregation Converged Network Metro Core Internet Access Network Common packet-switched infrastructure for both wireline and wireless services ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-99
  • 100.
    Metanoia, Inc. Critical Systems Thinking™ Backhaul References  MEF White Paper, “Carrier Ethernet Access for Mobile Backhaul Networks”, Feb 2008.  MEF Technical Specification Draft 2, “Mobile Backhaul Implementation Agreement- Phase 1”, Feb 2008  Nortel Networks White Paper, “Mobile Network Evolves with Carrier Ethernet”, 2008.  ITU-T G.8261, “Timing and Synchronization Aspects in Packet Networks”, Sept 2007.  IP/MPLS Forum White Paper, “Use of MPLS Technology in Mobile Backhaul Networks”, Feb 2008  Kireeti Kompella and Mallik Tatipamula, “IP/MPLS in Next Gen Mobile Backhaul Networks”, MPLS 2007. ©Copyright 2009 Carrier Ethernet Technology Strategies & Evolving All Rights Reserved Operator Best Practices M1-100