Carrier-Grade Ethernet
for Power Utilities
Ensuring Reliable Communications in
the Smart Grid




                          Carrier-Grade Ethernet for Utilities Slide 1
Agenda

• Towards a Smarter Grid: Utility Networks in Transition
• Carrier-Grade Ethernet Mechanisms
• Timing over Packet Synchronization
• Choosing the Right Packet Network
• Security
• RAD’s Carrier-Grade Ethernet Solutions for Power Utilities




                                                               Carrier-Grade Ethernet for Utilities Slide 2
Towards a Smarter Grid




                         Carrier-Grade Ethernet for Utilities Slide 3
Utility Networks in Transition

• Ethernet, IP, MPLS replace SDH/SONET; higher throughput and lower
  OpEx for:
    Upgrades to Smart Grid
    New applications: Substation automation, IEC 61850 IEDs, WASA
    synchrophasors, IP video surveillance
• $3.2 billion spent on utilities telecom equipment in the US – transport
  networks second largest category (UTC, 2011)
• RAD survey: 43% of utilities begin migration with backbone comm
  network; 28% with SCADA system




                                                               Carrier-Grade Ethernet for Utilities Slide 4
Migration Performance Requirements

• Need service assurance for mission critical apps in a
  PSN environment:
      Low end-to-end delay
      High availability (99.99% - 99.999% )
      SDH/SONET-level resiliency
• Requirements vary depending on application:
      Teleprotection: < 10ms symmetrical delay, minimal
      “jitter”
      SCADA: May tolerate 1s latency
• Cyber security:
      More interconnected devices = more entry points to
      protect
      Need scalable protection tools



                                                           Carrier-Grade Ethernet for Utilities Slide 5
Traffic Types and Transmission over PSN

• Ethernet and IP data and signals from SA IEDs → sent natively over
  packet
• TDM traffic, analog voice, serial SCADA, Teleprotection → require
  “packetization” or pseudowire emulation (PWE):




       Synchronous bit stream is segmented; headers are added to form packet
       Packets are forwarded to destination over the PSN network
       At destination, headers are removed, original bit stream is reconstructed,
       timing regenerated
       Common pseudowire emulation standards: CESoPSN, SAToP, TDMoIP

                                                                   Carrier-Grade Ethernet for Utilities Slide 6
Carrier-Grade Ethernet Mechanisms




                                    Carrier-Grade Ethernet for Utilities Slide 7
Ethernet is Now Carrier-Grade

• Ethernet is no longer a “Best Effort” technology
       Engineered by industry to provide SLA-based
       performance guarantees, reliability schemes, service
       management tools
• Full set of standards developed by the IEEE, ITU-T,
  MEF
• Metro Ethernet Forum = a consortium of carriers,
  service providers, equipment vendors promoting
  Carrier Ethernet adoption
• Extensive deployments of premium Ethernet
  services with double digit growth worldwide

Can now provide deterministic quality of service, predictable latency and jitter
 performance for mission-critical substation applications: SCADA, IEC 61850
                                GOOSE, etc

                                                                    Carrier-Grade Ethernet for Utilities Slide 8
Extensive Traffic Management and
Quality of Service Toolset

• Classification of incoming traffic into flows, e.g., IEC 60870-5-
  104, DNP3 per DSCP, 61850 GOOSE traffic per PCP, VLAN-ID
• Metering and policing to regulate traffic with different bandwidth
  profiles
• Advanced scheduling and queue management to ensure minimal
  latency and jitter
• Shaping to smooth out bursts and avoid buffer overruns in
  subsequent network elements
• Packet editing and marking to signal proper handling instructions for
  subsequent network elements




                                                                Carrier-Grade Ethernet for Utilities Slide 9
Performance Monitoring and Testing

• A wealth of carrier-grade Ethernet tools to remotely test, monitor
  and troubleshoot the operation of communications links
• Utility network operators anticipate service degradation ahead of
  time, as well as cut down truck-rolls and on-site technician calls

                                                  Fault
  Connectivity                      Fault                          Performance
                 Stress Testing                Propagation
  Verification                    Detection                         Monitoring
                                               & Isolation




                                                             Carrier-Grade Ethernet for Utilities Slide 10
Communications Channel Resiliency


• Hardware redundancy:
     No single point of failure (NSPF) design with redundant, hot-swappable
     power supplies
     Redundant control plane and switch fabric cards
• Link redundancy:
     1+1 protection topology with automatic switchover between links
     Link aggregation group (LAG) per IEEE 802.3-2005 LACP (link
     aggregation control protocol) for Ethernet-based services
• Path protection:
     Ethernet Linear protection Switching (G.8031) , AKA “EVC (Ethernet
     Virtual Connection) protection”
     Ethernet Ring Protection Switching (G.8032 ERPS) to provide Five Nines
     (99.999%) availability


                                                                   Carrier-Grade Ethernet for Utilities Slide 11
Timing over Packet (ToP)
Synchronization
• PSNs require complementary clock transfer solutions with a high
  level of precision
• ~1ms highest accuracy level required today for utility
  applications
     Upcoming Smart Grid apps will require 1μs and better
     Time of Day (ToD) required for synchrophasors and to avoid
     cascading blackouts
• GPS is accurate but costly, susceptible to jamming
• ITU-T Synchronous Ethernet (Sync-E)
     Uses the Ethernet physical layer to accurately distribute frequency
• IETF NTP and IEEE 1588-2008 Precision Time Protocol (PTP)
     Exchange timestamp information to deliver frequency and ToD
• Many network operators use Sync-E for frequency distribution
  and PTP for time synchronization

                                                                  Carrier-Grade Ethernet for Utilities Slide 12
Choosing the Right Packet Network




                                    Carrier-Grade Ethernet for Utilities Slide 13
Comparing Technology Options

Utility network operators need to choose which technology to employ:
• IP
• MPLS (Multi-Protocol Label Switching)
• Carrier-grade Ethernet


 Protocol          OAM/APS                  Security
 IP                No standard end-to-end   Strong (IPsec)
                   mechanisms
 MPLS              Recently developed for   No built-in security
                   MPLS-TP
 Ethernet          Carrier-grade            Several security
                                            mechanisms defined
                                            (802.1X, MACsec)



                                                                   Carrier-Grade Ethernet for Utilities Slide 14
Option 1: Combining Ethernet Access
with MPLS Core

                                 Ethernet GOOSE delivered directly = No
 Lower cost per port
                                 tunneling/conversion over MPLS core
 Richer OAM, PM tools
 Advanced protection
 mechanisms




                        Maintain existing
                        access media installed
                        base
                                                           Carrier-Grade Ethernet for Utilities Slide 15
Option 2: Replace SDH/SONET with
     Ethernet Operational Network
                                                 Scalable IP/MPLS for IT/enterprise network
                                                     VoIP, IP video, internet connectivity, billing

                  IT/Enterprise                                                                  Substation
                     Center              IT/Enterprise                                 Multiservice        RTU
                                      IP/MPLS Network                                    Access     RS-232
                                                                                           Node
                                                                                                     Teleprotection
                  VoIP                                        Operational                           X.21
                                                            Ethernet Network                                      PBX
                                                                                                          E1

                                                    Ethernet
                                                     Service
                                                  Aggregation
                                                    Platform
                                                                 1 GbE / 10 GbE
                                                                      Ring

                                                                 Ethernet Service
                                                                  Aggregation
                                                                    Platform
 Simplified architecture         Operational
                                                                                                 Substation
  and management                    Center                                             Multiservice
                                                                                                    RS-232
                                                                                                           RTU
                                                                                         Access
 Increased security                       NMS                                             Node
                                                                                                     Teleprotection
 Lower latency                                   Replace SDH/SONET                                 X.21
                                       SCADA
 Assured QoS                                                                                                    PBX
                                                  operational network with                                E1
 Ongoing performance
  monitoring
                                                  Carrier-Grade L2 Ethernet
                                                   SCADA, Teleprotection,
                                                    Automation
                                                                                          Carrier-Grade Ethernet for Utilities Slide 16
Security




           Carrier-Grade Ethernet for Utilities Slide 17
Layered Security Approach

• Utility networks require “Defense in Depth” implementing multiple
  levels of protection:




• Should be deployed in the network without adding too many
  dedicated security appliances on top of the communications
  infrastructure


                                                          Carrier-Grade Ethernet for Utilities Slide 18
RAD’s Carrier-Grade Ethernet
Solutions for Power Utilities




                                Carrier-Grade Ethernet for Utilities Slide 19
Full Portfolio for Power Utility Networks




                                                           IPmux
       Megaplex                     ETX-A
                                                       TDM Pseudowire
   Multiservice Access         Carrier Ethernet
                                                          Gateway
       Platforms                  over Fiber




                ETX-5300A                     RICi-GE             RADiFlow
                Multiservice           Ethernet over NG-SDH     Service-Aware
                Aggregation                                    Secure Ethernet
                 Platform                                         Switches



                                                                    Carrier-Grade Ethernet for Utilities Slide 20
Ensuring a Smooth Migration

• Hybrid SDH/SONET and PSN access solutions
     Allow utilities to choose the migration path
     that best suits their needs
     Deterministic QoS for NGN services and
     advanced grid applications over TDM or packet
     transport
     Service continuity for legacy equipment

• Multi-standard ToP synchronization, including 1588
  Grandmaster functionality in the same communications
  device
• Ethernet switches with cyber security and authentication
  protocols




                                                             Carrier-Grade Ethernet for Utilities Slide 21
Conclusion

• Ethernet has been engineered and standardized and now includes
  critical “must haves” for NG utility networks:
     Robust clock accuracy
     Traffic management and hard/hierarchical QoS
     Resiliency
     On-going performance monitoring
     Support for legacy services and traffic
     Security
• A combination of carrier-grade Ethernet in the access/aggregation
  with an MPLS core can address the needs of the Smart Grid
• RAD offers a wide selection of utility-grade solutions to ensure high
  performance, cost-effective and easy migration

Download comprehensive Carrier-Grade Ethernet for Power Utilities
Solution Paper


                                                             Carrier-Grade Ethernet for Utilities Slide 22
Thank You
For Your
Attention

                www.rad.com

            Carrier-Grade Ethernet for Utilities Slide 23

Carrier ethernet-for-power-utilities-presentation

  • 1.
    Carrier-Grade Ethernet for PowerUtilities Ensuring Reliable Communications in the Smart Grid Carrier-Grade Ethernet for Utilities Slide 1
  • 2.
    Agenda • Towards aSmarter Grid: Utility Networks in Transition • Carrier-Grade Ethernet Mechanisms • Timing over Packet Synchronization • Choosing the Right Packet Network • Security • RAD’s Carrier-Grade Ethernet Solutions for Power Utilities Carrier-Grade Ethernet for Utilities Slide 2
  • 3.
    Towards a SmarterGrid Carrier-Grade Ethernet for Utilities Slide 3
  • 4.
    Utility Networks inTransition • Ethernet, IP, MPLS replace SDH/SONET; higher throughput and lower OpEx for: Upgrades to Smart Grid New applications: Substation automation, IEC 61850 IEDs, WASA synchrophasors, IP video surveillance • $3.2 billion spent on utilities telecom equipment in the US – transport networks second largest category (UTC, 2011) • RAD survey: 43% of utilities begin migration with backbone comm network; 28% with SCADA system Carrier-Grade Ethernet for Utilities Slide 4
  • 5.
    Migration Performance Requirements •Need service assurance for mission critical apps in a PSN environment: Low end-to-end delay High availability (99.99% - 99.999% ) SDH/SONET-level resiliency • Requirements vary depending on application: Teleprotection: < 10ms symmetrical delay, minimal “jitter” SCADA: May tolerate 1s latency • Cyber security: More interconnected devices = more entry points to protect Need scalable protection tools Carrier-Grade Ethernet for Utilities Slide 5
  • 6.
    Traffic Types andTransmission over PSN • Ethernet and IP data and signals from SA IEDs → sent natively over packet • TDM traffic, analog voice, serial SCADA, Teleprotection → require “packetization” or pseudowire emulation (PWE): Synchronous bit stream is segmented; headers are added to form packet Packets are forwarded to destination over the PSN network At destination, headers are removed, original bit stream is reconstructed, timing regenerated Common pseudowire emulation standards: CESoPSN, SAToP, TDMoIP Carrier-Grade Ethernet for Utilities Slide 6
  • 7.
    Carrier-Grade Ethernet Mechanisms Carrier-Grade Ethernet for Utilities Slide 7
  • 8.
    Ethernet is NowCarrier-Grade • Ethernet is no longer a “Best Effort” technology Engineered by industry to provide SLA-based performance guarantees, reliability schemes, service management tools • Full set of standards developed by the IEEE, ITU-T, MEF • Metro Ethernet Forum = a consortium of carriers, service providers, equipment vendors promoting Carrier Ethernet adoption • Extensive deployments of premium Ethernet services with double digit growth worldwide Can now provide deterministic quality of service, predictable latency and jitter performance for mission-critical substation applications: SCADA, IEC 61850 GOOSE, etc Carrier-Grade Ethernet for Utilities Slide 8
  • 9.
    Extensive Traffic Managementand Quality of Service Toolset • Classification of incoming traffic into flows, e.g., IEC 60870-5- 104, DNP3 per DSCP, 61850 GOOSE traffic per PCP, VLAN-ID • Metering and policing to regulate traffic with different bandwidth profiles • Advanced scheduling and queue management to ensure minimal latency and jitter • Shaping to smooth out bursts and avoid buffer overruns in subsequent network elements • Packet editing and marking to signal proper handling instructions for subsequent network elements Carrier-Grade Ethernet for Utilities Slide 9
  • 10.
    Performance Monitoring andTesting • A wealth of carrier-grade Ethernet tools to remotely test, monitor and troubleshoot the operation of communications links • Utility network operators anticipate service degradation ahead of time, as well as cut down truck-rolls and on-site technician calls Fault Connectivity Fault Performance Stress Testing Propagation Verification Detection Monitoring & Isolation Carrier-Grade Ethernet for Utilities Slide 10
  • 11.
    Communications Channel Resiliency •Hardware redundancy: No single point of failure (NSPF) design with redundant, hot-swappable power supplies Redundant control plane and switch fabric cards • Link redundancy: 1+1 protection topology with automatic switchover between links Link aggregation group (LAG) per IEEE 802.3-2005 LACP (link aggregation control protocol) for Ethernet-based services • Path protection: Ethernet Linear protection Switching (G.8031) , AKA “EVC (Ethernet Virtual Connection) protection” Ethernet Ring Protection Switching (G.8032 ERPS) to provide Five Nines (99.999%) availability Carrier-Grade Ethernet for Utilities Slide 11
  • 12.
    Timing over Packet(ToP) Synchronization • PSNs require complementary clock transfer solutions with a high level of precision • ~1ms highest accuracy level required today for utility applications Upcoming Smart Grid apps will require 1μs and better Time of Day (ToD) required for synchrophasors and to avoid cascading blackouts • GPS is accurate but costly, susceptible to jamming • ITU-T Synchronous Ethernet (Sync-E) Uses the Ethernet physical layer to accurately distribute frequency • IETF NTP and IEEE 1588-2008 Precision Time Protocol (PTP) Exchange timestamp information to deliver frequency and ToD • Many network operators use Sync-E for frequency distribution and PTP for time synchronization Carrier-Grade Ethernet for Utilities Slide 12
  • 13.
    Choosing the RightPacket Network Carrier-Grade Ethernet for Utilities Slide 13
  • 14.
    Comparing Technology Options Utilitynetwork operators need to choose which technology to employ: • IP • MPLS (Multi-Protocol Label Switching) • Carrier-grade Ethernet Protocol OAM/APS Security IP No standard end-to-end Strong (IPsec) mechanisms MPLS Recently developed for No built-in security MPLS-TP Ethernet Carrier-grade Several security mechanisms defined (802.1X, MACsec) Carrier-Grade Ethernet for Utilities Slide 14
  • 15.
    Option 1: CombiningEthernet Access with MPLS Core Ethernet GOOSE delivered directly = No Lower cost per port tunneling/conversion over MPLS core Richer OAM, PM tools Advanced protection mechanisms Maintain existing access media installed base Carrier-Grade Ethernet for Utilities Slide 15
  • 16.
    Option 2: ReplaceSDH/SONET with Ethernet Operational Network Scalable IP/MPLS for IT/enterprise network  VoIP, IP video, internet connectivity, billing IT/Enterprise Substation Center IT/Enterprise Multiservice RTU IP/MPLS Network Access RS-232 Node Teleprotection VoIP Operational X.21 Ethernet Network PBX E1 Ethernet Service Aggregation Platform 1 GbE / 10 GbE Ring Ethernet Service Aggregation Platform  Simplified architecture Operational Substation and management Center Multiservice RS-232 RTU Access  Increased security NMS Node Teleprotection  Lower latency Replace SDH/SONET X.21 SCADA  Assured QoS PBX operational network with E1  Ongoing performance monitoring Carrier-Grade L2 Ethernet  SCADA, Teleprotection, Automation Carrier-Grade Ethernet for Utilities Slide 16
  • 17.
    Security Carrier-Grade Ethernet for Utilities Slide 17
  • 18.
    Layered Security Approach •Utility networks require “Defense in Depth” implementing multiple levels of protection: • Should be deployed in the network without adding too many dedicated security appliances on top of the communications infrastructure Carrier-Grade Ethernet for Utilities Slide 18
  • 19.
    RAD’s Carrier-Grade Ethernet Solutionsfor Power Utilities Carrier-Grade Ethernet for Utilities Slide 19
  • 20.
    Full Portfolio forPower Utility Networks IPmux Megaplex ETX-A TDM Pseudowire Multiservice Access Carrier Ethernet Gateway Platforms over Fiber ETX-5300A RICi-GE RADiFlow Multiservice Ethernet over NG-SDH Service-Aware Aggregation Secure Ethernet Platform Switches Carrier-Grade Ethernet for Utilities Slide 20
  • 21.
    Ensuring a SmoothMigration • Hybrid SDH/SONET and PSN access solutions Allow utilities to choose the migration path that best suits their needs Deterministic QoS for NGN services and advanced grid applications over TDM or packet transport Service continuity for legacy equipment • Multi-standard ToP synchronization, including 1588 Grandmaster functionality in the same communications device • Ethernet switches with cyber security and authentication protocols Carrier-Grade Ethernet for Utilities Slide 21
  • 22.
    Conclusion • Ethernet hasbeen engineered and standardized and now includes critical “must haves” for NG utility networks: Robust clock accuracy Traffic management and hard/hierarchical QoS Resiliency On-going performance monitoring Support for legacy services and traffic Security • A combination of carrier-grade Ethernet in the access/aggregation with an MPLS core can address the needs of the Smart Grid • RAD offers a wide selection of utility-grade solutions to ensure high performance, cost-effective and easy migration Download comprehensive Carrier-Grade Ethernet for Power Utilities Solution Paper Carrier-Grade Ethernet for Utilities Slide 22
  • 23.
    Thank You For Your Attention www.rad.com Carrier-Grade Ethernet for Utilities Slide 23