SEAMLESS INTERWORKING OF SDN AND IP
1
UMESH KRISHNASWAMY, PINGPING LIN, JONATHAN HART, TETSUYA MURAKAMI
MASAYOSHI KOBAYASHI, ALI AL-SHABIBI, K. C. WANG, KUNIHIRO ISHIGURO
HOW CAN WE SEAMLESSLY PEER BETWEEN SDN
AND IP NETWORKS?
2
IP
IP
IP
IP
IP
IP
IP
IP
IP
IP
IP
IP
SDNSDN
SDN
IP ROUTING IN SDN
3
SDNIP
IP
IP
IP
NOS
BGP
Daemon
RIB RoutingRIB
Sync
BGP routing updates
Considerations
Entire SDN AS appears as a single big router to external peers
SDN
PROACTIVE FLOW INSTALLATION
4
Considerations
Proactive flow installer creates IP prefix based flow entries
IP
IP
NOS
RIB Match Action
Add Prefix ...
Match Action
Add MAC ...Match Action
Add MAC ...
Match Action
Add MAC ...
Proactive Flow Installer
BGP route update Match Action
Add Prefix ...
SDN
FLOW ENTRY COMPUTATION
5
Match Action
Prefix X Rewrite destination to MAC A, output 1
Prefix Y Rewrite destination to MAC B, output 2
Prefix Z Rewrite destination to MAC C, output 2
IP
MAC based forwarding in SDN Core
Prefix Y
Prefix Z
MAC A
MAC B
MAC C
1
1
2
2
Prefix based lookup at
the first hop switch
Flow Table
Y
Prefix X
Flow Table
Match Action
MAC_B output = 1
MAC_C output = 2
SDN
AFTER BGP REROUTE
6
Match Action
Prefix X Rewrite destination to MAC A, output 1
Prefix Y Rewrite destination to MAC B, output 2
Prefix Z Rewrite destination to MAC C, output 2
IP
Prefix Z
MAC A
MAC B
MAC C
1
1
2
2
Prefix based lookup at
the first hop switch
Flow Table
Considerations
Reduce churn within SDN core when BGP routes flap
Y
Prefix X
Flow Table
Match Action
MAC_B output = 1
MAC_C output = 2
Prefix YPrefix Y
Match Action
Prefix X Rewrite destination to MAC A, output 1
Prefix Y Rewrite destination to MAC C, output 2
Prefix Z Rewrite destination to MAC C, output 2
MAC based forwarding in SDN Core
CURRENT IMPLEMENTATION
Proactive Flow
Installer
Prepopulate flows based
on BGP updates
ZebOS
BGPD
RIB
RIB
pusher
External BGP
peers
Prefix, Nexthop
BGP Route
RIB
RIB
Syncer
ONOS
Flow Manager
Topology
Discovery
Openflow
RELATED WORK - ROUTEFLOW
Routeflow SDNIP
Emulates distributed IP control plane on
centralized controller
Native application on SDN OS
Treat each Openflow switch as an IP router Treat entire SDN AS as a single big router
Topology discovery done by IGP Topology discovery done by SDN OS
DEMONSTRATION OF SDN-IP ON ONOS
192.168.20.1/24
AS4
AS2 172.16.20.1/24
AS3172.16.30.1/24 172.16.40.1/24
172.16.10.1/24
192.168.10.1/24
192.168.30.1/24
192.168.40.1/24
192.168.50.1/24
SDN AS emulated in Mininet
Quagga BGPd
LAX
CHI
IAH
NYC
ATL
SLC
BGP
ONOS
BGPD
Routing GUI
Host
SDN AS1
DEPLOYMENT IN GOOGLE PROJECT CARDIGAN
Wellington
Internet Exchange
SDN
REANNZ
WIX
Pica8 3290 Pica8 3780
Research and Education
Advanced Network
NZ
ONOS
SDN-IP
Timeframe: May – July 2013
Demonstrate that Openflow/SDN can peer with production IP networks
DEPLOYMENT IN GOOGLE PROJECT TREEHOUSE
Timeframe: June – August 2013
Demonstrate that Openflow/SDN software and hardware is ready for WAN applications
REANNZ
NOX
Routeflow
ES.Net
NOX
Routeflow
Stanford
ONOS
SDN-IP
 RIB update speed is very slow due REST API
 Implement a high performance RIB syncer
 Completely transition from Floodlight
 Currently using Forwarding and StaticFlowPusher
 Implement Proxy Arp to resolve MACs of BGP peers
 Add new ONOS Flow API to program edge switch
 Code cleanup:
 Rewrite proactive flow installer from Python to Java
 Use ONOS Flow API to program core switch
NEAR-TERM IMPROVEMENTS TO SDN-IP
SDN-IP MEMORY SCALING (UNOPTIMIZED)
 Limits of running on a single instance of ONOS:
 No scale-out of control plane
 No fault tolerance of the control plane
 Scale-out provided by multi instance ONOS:
 Flow programming and monitoring is scaled out
 Topology discovery is scaled out
 High availability:
 ONOS provides HA for installed flow paths
 BGP uses graceful restart or non-stop routing
 Restarting SDN-IP re-syncs RIB and applies changes
 Single instance of SDN-IP should suffice
 Heavy lifting in BGPD and ONOS
SDN-IP ON MULTI-INSTANCE ONOS
SDN-IP ON MULTI-INSTANCE ONOS
SDN-IP
Prefix, Nexthop
Instance 1
FM
Instance 2
FM
Instance 3
FM
BGPD
RIB RIB
 Limits of single BGP process:
 Limited scaling of BGP router process as peers grow
• Maintaining BGP sessions with neighbors
• Processing incoming updates, sending updates to peers
• Updating the IP RIB with BGP entries
 Use case: private IP peering like MPLS L3VPNs
 Multiple BGP processes:
 Partition VRFs handled by each BGP process
 Use a route reflector to consolidate RIB
 High availability:
 BGP uses graceful-restart or non-stop routing
SCALING OUT BGP PROCESSING
SCALING OUT BGP PROCESSING
SDN-IP
Prefix, Nexthop
Instance 1
FM
Instance 2
FM
Instance 3
FM
BGPD
Route
ReflectorRoute
Reflector
BGPD
BGPD
RIB
RIB
ROADMAP
Area Q3 2013 Q4 2013
Features High performance RIB syncer
Use ONOS proxy-arp and flow API for
IP prefix match and rewrite
Runs on multi-instance ONOS
Policy based routing within SDN: make
use of multiple internal paths
Traffic engineering within SDN: API for
applications to control internal path
selection
Scale and
performance
Target 10K routes in RIB
Target 50 RIB updates/sec
Target 100 peers
Target 100K routes in RIB
Release Release 0.1 to open source Release 0.2 to open source
Deployment Deploy in Google Project CARDIGAN
Deploy in Google Project TREEHOUSE
Deploy on Internet2 100G network
Roadmap for open source BGPD owned by IP Infusion
PLEASE JOIN US
Learn Collaborate Contribute
Try out your innovative ideas
with our tools
Improve our tools and
platforms
Stay informed about SDN
Users and contributors
Keep track of latest SDN
research and innovations
Demonstrate early stage SDN
ideas with ON.LAB
Co-develop platforms and
use cases
Organizations
www.onlab.us

SDN-IP Peering using BGP

  • 1.
    SEAMLESS INTERWORKING OFSDN AND IP 1 UMESH KRISHNASWAMY, PINGPING LIN, JONATHAN HART, TETSUYA MURAKAMI MASAYOSHI KOBAYASHI, ALI AL-SHABIBI, K. C. WANG, KUNIHIRO ISHIGURO
  • 2.
    HOW CAN WESEAMLESSLY PEER BETWEEN SDN AND IP NETWORKS? 2 IP IP IP IP IP IP IP IP IP IP IP IP SDNSDN SDN
  • 3.
    IP ROUTING INSDN 3 SDNIP IP IP IP NOS BGP Daemon RIB RoutingRIB Sync BGP routing updates Considerations Entire SDN AS appears as a single big router to external peers
  • 4.
    SDN PROACTIVE FLOW INSTALLATION 4 Considerations Proactiveflow installer creates IP prefix based flow entries IP IP NOS RIB Match Action Add Prefix ... Match Action Add MAC ...Match Action Add MAC ... Match Action Add MAC ... Proactive Flow Installer BGP route update Match Action Add Prefix ...
  • 5.
    SDN FLOW ENTRY COMPUTATION 5 MatchAction Prefix X Rewrite destination to MAC A, output 1 Prefix Y Rewrite destination to MAC B, output 2 Prefix Z Rewrite destination to MAC C, output 2 IP MAC based forwarding in SDN Core Prefix Y Prefix Z MAC A MAC B MAC C 1 1 2 2 Prefix based lookup at the first hop switch Flow Table Y Prefix X Flow Table Match Action MAC_B output = 1 MAC_C output = 2
  • 6.
    SDN AFTER BGP REROUTE 6 MatchAction Prefix X Rewrite destination to MAC A, output 1 Prefix Y Rewrite destination to MAC B, output 2 Prefix Z Rewrite destination to MAC C, output 2 IP Prefix Z MAC A MAC B MAC C 1 1 2 2 Prefix based lookup at the first hop switch Flow Table Considerations Reduce churn within SDN core when BGP routes flap Y Prefix X Flow Table Match Action MAC_B output = 1 MAC_C output = 2 Prefix YPrefix Y Match Action Prefix X Rewrite destination to MAC A, output 1 Prefix Y Rewrite destination to MAC C, output 2 Prefix Z Rewrite destination to MAC C, output 2 MAC based forwarding in SDN Core
  • 7.
    CURRENT IMPLEMENTATION Proactive Flow Installer Prepopulateflows based on BGP updates ZebOS BGPD RIB RIB pusher External BGP peers Prefix, Nexthop BGP Route RIB RIB Syncer ONOS Flow Manager Topology Discovery Openflow
  • 8.
    RELATED WORK -ROUTEFLOW Routeflow SDNIP Emulates distributed IP control plane on centralized controller Native application on SDN OS Treat each Openflow switch as an IP router Treat entire SDN AS as a single big router Topology discovery done by IGP Topology discovery done by SDN OS
  • 9.
    DEMONSTRATION OF SDN-IPON ONOS 192.168.20.1/24 AS4 AS2 172.16.20.1/24 AS3172.16.30.1/24 172.16.40.1/24 172.16.10.1/24 192.168.10.1/24 192.168.30.1/24 192.168.40.1/24 192.168.50.1/24 SDN AS emulated in Mininet Quagga BGPd LAX CHI IAH NYC ATL SLC BGP ONOS BGPD Routing GUI Host SDN AS1
  • 10.
    DEPLOYMENT IN GOOGLEPROJECT CARDIGAN Wellington Internet Exchange SDN REANNZ WIX Pica8 3290 Pica8 3780 Research and Education Advanced Network NZ ONOS SDN-IP Timeframe: May – July 2013 Demonstrate that Openflow/SDN can peer with production IP networks
  • 11.
    DEPLOYMENT IN GOOGLEPROJECT TREEHOUSE Timeframe: June – August 2013 Demonstrate that Openflow/SDN software and hardware is ready for WAN applications REANNZ NOX Routeflow ES.Net NOX Routeflow Stanford ONOS SDN-IP
  • 12.
     RIB updatespeed is very slow due REST API  Implement a high performance RIB syncer  Completely transition from Floodlight  Currently using Forwarding and StaticFlowPusher  Implement Proxy Arp to resolve MACs of BGP peers  Add new ONOS Flow API to program edge switch  Code cleanup:  Rewrite proactive flow installer from Python to Java  Use ONOS Flow API to program core switch NEAR-TERM IMPROVEMENTS TO SDN-IP
  • 13.
    SDN-IP MEMORY SCALING(UNOPTIMIZED)
  • 14.
     Limits ofrunning on a single instance of ONOS:  No scale-out of control plane  No fault tolerance of the control plane  Scale-out provided by multi instance ONOS:  Flow programming and monitoring is scaled out  Topology discovery is scaled out  High availability:  ONOS provides HA for installed flow paths  BGP uses graceful restart or non-stop routing  Restarting SDN-IP re-syncs RIB and applies changes  Single instance of SDN-IP should suffice  Heavy lifting in BGPD and ONOS SDN-IP ON MULTI-INSTANCE ONOS
  • 15.
    SDN-IP ON MULTI-INSTANCEONOS SDN-IP Prefix, Nexthop Instance 1 FM Instance 2 FM Instance 3 FM BGPD RIB RIB
  • 16.
     Limits ofsingle BGP process:  Limited scaling of BGP router process as peers grow • Maintaining BGP sessions with neighbors • Processing incoming updates, sending updates to peers • Updating the IP RIB with BGP entries  Use case: private IP peering like MPLS L3VPNs  Multiple BGP processes:  Partition VRFs handled by each BGP process  Use a route reflector to consolidate RIB  High availability:  BGP uses graceful-restart or non-stop routing SCALING OUT BGP PROCESSING
  • 17.
    SCALING OUT BGPPROCESSING SDN-IP Prefix, Nexthop Instance 1 FM Instance 2 FM Instance 3 FM BGPD Route ReflectorRoute Reflector BGPD BGPD RIB RIB
  • 18.
    ROADMAP Area Q3 2013Q4 2013 Features High performance RIB syncer Use ONOS proxy-arp and flow API for IP prefix match and rewrite Runs on multi-instance ONOS Policy based routing within SDN: make use of multiple internal paths Traffic engineering within SDN: API for applications to control internal path selection Scale and performance Target 10K routes in RIB Target 50 RIB updates/sec Target 100 peers Target 100K routes in RIB Release Release 0.1 to open source Release 0.2 to open source Deployment Deploy in Google Project CARDIGAN Deploy in Google Project TREEHOUSE Deploy on Internet2 100G network Roadmap for open source BGPD owned by IP Infusion
  • 19.
    PLEASE JOIN US LearnCollaborate Contribute Try out your innovative ideas with our tools Improve our tools and platforms Stay informed about SDN Users and contributors Keep track of latest SDN research and innovations Demonstrate early stage SDN ideas with ON.LAB Co-develop platforms and use cases Organizations
  • 20.

Editor's Notes

  • #2 I want to take a minute to acknowledge my other collaborators without whom this project would not have been possible. The current dev-team comprises of Pingping and Jono from ON Lab and Tetsuya from IP Infusion. Masa and Ali from ON Lab, KC from Clemson University and Ishi from IP Infusion were heavily involved in the first phase of the project. This project would not have reached this stage without their efforts.
  • #3 An SDN needs to exist in an Internet of predominantly IP networks at least for now. SDN and IP networks need to be able to exchange routing information and forward traffic to and from each other. How to implement seamless peering between SDN and IP networks? That is the motivation of this work.
  • #4 The protocol of choice used in the Internet is BGP. We need the SDN to be able to speak BGP with its IP neighbors to exchange routing information (represented in this route information base). Since the SDN control plane is logically centralized; let us treat the BGP function as also logically centralized. The entire SDN AS appears as a single big router to the outside. A logically centralized BGP daemon handles routing updates with all peers. In the simplest scale such as public IP peering, it is a single BGP process for the entire network.  With chose this approach for simplicity and correctness, Since all the external paths are selected by BGP, we are guaranteed not to break BGP semantics or create routing loops because BGP is computing the best paths. Other benefits are centralized monitoring, code upgrade.
  • #5 Once the RIB is available, the SDN control plane is used to program the forwarding tables or flow entries in the case of Openflow switches.  Flow entry programming in SDN is often associated with reactive flows triggered by packet ins. But this is not appropriate for programming the forwarding tables from BGP updates. BGP updates need a proactive flow installer. BGP routing information is in the form of prefix and nexthop. This is nicely summarized information that we want to carry forward when we program the flows. The proactive flow installer creates flow entries that use prefix match conditions and are installed or deleted at the time of BGP update.
  • #6 Now let us look a bit more closely at how flow entries are computed. We do prefix based lookup at the first hop switch and we do MAC based forwarding in the SDN core.  The flow table in the first-hop switch looks like this. Note it has prefixes as match conditions and actions that rewrite the destination mac. The flow table in the SDN core looks like this. It has destination mac addresses as match condition and action is to forward to a port.  Let us walk through the example of packet destined to Y. Packet reaches first hop switch, lookup by prefix. Action is rewrite destination mac to be B, send it out of port 2. The next switch knows that to reach B, it has go out of port 1. And so on.
  • #7 Let us see what happens during a BGP reroute. Suppose, Prefix Y becomes reachable from router C. BGP update comes to the SDN and it only has to update the first hop switch. The core remains unchanged. This reduces the churn within the SDN when BGP routes flap.
  • #8 SDN-IP is an application on ONOS. It uses ONOS services for topology discovery and flow programming.  The BGP speaker is from IPInfusion’sZebOS. For folks who may not have heard of ZebOS but heard of open source routing software like Quagga, IshiIshuguro who founded IPI also wrote Quagga. ZebOS is well supported and maintained and seen a lot of production deployment. We are using just the BGPD portion of ZebOS that is also modified to run standalone.  The bits that run on top of ONOS are the RIB syncer to sync the RIB from BGPD and the proactive flow installer.
  • #9 Many of you may be familiar with Routeflow. Routeflow is one of the first implementations of IP routing on Openflow switches. This is a comparison of SDN-IP and Routeflow with the intention of contrasting the two approaches. Routefow emulates 1-1 the IP control plane on a centralized controller while SDN-IP takes parts of the IP control plane and integrates it with the SDN OS. Routeflow turns each Openflow switch into a separate router, while SDN-IP treats the entire SDN AS as a big router. Topology discovery is done additionally at layer 3 by IGP. SDN-IP relies on SDN OS to do topology discovery.
  • #10 You can see a demonstration of SDN-IP during lunch. We show peering between an SDN and IP networks.
  • #20 I would like to end with a different message. In this project, we have had to good fortune of many outside organizations helping us and we in ON.Lab would like to extend that same courtesy to you. You are our community. If you would like to collaborate, contribute or learn with us, we are there to help.