Published on

Interop Tokyo 2014 SDI (Software Defined Infrustructure) ShowCase Seminoar Presentation. The presentation covers Neutron API models (L2/L3 and Advanced Network services), Neutron Icehouse Update and Juno topics.

Published in: Technology


  1. 1. Software Defined Network in OpenStack ~OpenStack Neutron~ Icehouse Update and Towards Juno Akihiro Motoki (@ritchey98) Jun 12th, 2014
  2. 2. Who am I? • Akihiro Motoki / 元木 顕弘 – @ritchey98, amotoki on freenode IRC – NEC Knowledge Discovery Research Laboratories – Long experiences in developing network equipments/applications • IP routers, wide-area Ethernet • Hardware spam filter appliance, FPGA-based IDS – Recently working on OpenFlow, OpenStack and related testbeds • OpenStack Developer – Neutron and Horizon Core Developer – I18N (Internationalizationn) team sub-lead • Linux JM (Japanese manuals) Project Maintainer – Translating manpages from Linux documentation project (LDP), GNU tools and Linux networking tools 2013/12/12 Okinawa OpenDays 2013 2
  3. 3. Agenda • Concept of Neutron API abstraction – Layer2/Layer3 abstraction – Advanced network services and Service Insertion • Icehouse Update • Topics in “Juno” (next release) • Recent topics in the development community
  4. 4. ???Neutron OpenStack Project と Neutron Compute Storage Network Nova Swift (Objects) Glance (Images) Cinder (Block) Identity Keystone *-as-a-Service Capability OpenStack Service
  5. 5. Neutron Goal #1 • Provide Network-as-a-Service (NaaS) – To allow tenants to control their own private networks • Ability to create “multi-tier” networks • Control IP addressing (IP address overlapping) – Neutron API for operating logical networks • Separate logical operations and backend provisioning • Backend technologies are provisioned/configured by Neutron plugins/drivers.
  6. 6. Neutron Goal #2 • Support multiple emerging network technologies – New requirements never before in cloud data centers • multi tenancy, remote data center, VM mobility, advanced network services, … – Nova provides only legacy basic technology (VLAN + iptables) – Needs new architecture/framework to support emerging network technologies • SDN/OpenFlow-based network • Overlay tunneling (VXLAN, NVGRE, STT, …) • L2 Fabric (Fabric Path, QFabric, …) Neutron provides common abstract network API independent from specific technologies/vendors
  7. 7. Neutron Basic Network Model TenantA-VM1 TenantA-VM3 TenantA-VM2 External Net Tenant-A Net1 Tenant-A Net2 L2 virtual network virtual port virtual server virtual interface (VIF) virtual subnet SNAT for external traffic Floating IP • Tenant can create multiple (Layer2) networks • Subnet defines L3 subnet information (CIDR, gateway etc) – Multiple subnets can be associated to a network • E.g., IPv4 subnet + IPv6 subnet. Multiple IPv4 address pools – Support IP address overlapping among tenants • Network have multiple ports (similar to physical L2SW ports) – Virtual NICs of VMs or router interfaces are associated with neutron ports. Neutron
  8. 8. Network API • Networking API is RESTful API. • Only “logical” information are visible in APIs for regular users. • In admin API, some physical/backend information such VLAN ID can be specified. GET http://localhost:9696/v2.0/networks/20b49737- c718-4f10-8edc-7fe9f8ee7cfd { "network": { "admin_state_up": true, "id": "20b49737-c718-4f10-8edc-7fe9f8ee7cfd", "name": "net1", "router:external": false, "shared": false, "status": "ACTIVE", "subnets": [ "644f729f-1c86-48b3-91a2-761e29398e81" ], "tenant_id": "be11ce3cabc74ee68d4f22253ffbb5e8" } } GET http://localhost:9696/v2.0/networks/20b49737- c718-4f10-8edc-7fe9f8ee7cfd { "network": { "admin_state_up": true, "id": "20b49737-c718-4f10-8edc-7fe9f8ee7cfd", "name": "net1", "router:external": false, "shared": false, "status": "ACTIVE", "subnets": [ "644f729f-1c86-48b3-91a2-761e29398e81" ], "tenant_id": "be11ce3cabc74ee68d4f22253ffbb5e8", "provider:network_type": "vlan", "provider:physical_network": "physnet_1", "provider:segmentation_id": 201 } }Regular user Admin user
  9. 9. Advanced Network Service • Network Service – LBaaS, FWaaS, VPNaaS and so on. – Provide additional network features rather than simple data transfer (L2/L3) • Would like to insert such network services on demand. – Use network services in “Cloud” style (through API and on demand) – It fits into SDN.
  10. 10. Service deployment pattern – Firewall Network 1 External Network Network 2 Network 1 FW on network FW on Port (security group?) FW on Router
  11. 11. Service deployment pattern – load balancer Network 1 Network 1 Network 1 Network 2 Direct Return 2-arm LB 1-arm LB - terminated - Router
  12. 12. Network Service Insertion Network 1 Network 2 External Network VPN Connection Service Insertion
  13. 13. Understanding Neutron API Virtual Network Virtual Router VM VM VM Compute APIL2/L3 API Horizon, Heat or Orchestration LB Service VM Process Based LB (HAProxy) HW Appliance Advanced network service API  OpenStack provides “Abstraction” and “Integration”.  NOT Virtualization (e.g., OpenStack deos not develop hypervisors)  Neutron has various layers of API abstraction  L2/L3 API or compute API are primitive APIs.  Advanced network service APIs are higher layer API. They consumes primitive APIs internally. LBaaS FWaaS VPNaaS
  14. 14. Nova Neutron Internal Architecture • A plugin per feature API • There are multiple plugins (Core plugin + Service plugins * N) – ML2 plugin for Layer2 API, LBaaS plugin for LBaaS API • The traditional model (one plugin for all APIs) is supported. • By switching a plugin implementation , various network technologies can be used. • Neutron also can be used as standalone, though it is a part of OpenStack. – Can/May we avoid vendor lock-in by adopting Neutron API? Neutron Server Compute Node VM VM vSwitch HW Switch L2/L3 PluginFWaaS Plugin LBaaS Plugin Agent L2 APIL3 API FWaaS API LBaaS API Network Controller OpenFlow Controller など Neutron API L3 Agent HAProxy Agent Compute API XXX Driver
  15. 15. Neutron SDN controller with OpenStack - centralized controller model - Nova OpenFlow Plugin Nova Compute Virtual Switch VM VM Nova Compute Virtual Switch VM VM HW Switch CLI / Dashboard (Horizon) / Orchestration Tool OpenFlow Controller Network OFC API Neutron API OpenFlow Protocol Nova API agent agent - NEC OpenFlow/Trema - OpenDaylight - NSX ….
  16. 16. Neutron SDN controllers with OpenStack - Controller on every compute node - Nova ML2 Plugin (OpenFlow Agent) Nova Compute Virtual Switch VM VM Nova Compute Virtual Switch VM VM HW Switch CLI / Dashboard (Horizon) / Orchestration Tool In OpenFlow agent mech driver in ML2, OpenFlow controllers are deployed per compute node and each OF controller controls OVS on each node. Neutron API OpenFlow Protocol Nova API agent agent OpenFlow Controller OpenFlow Controller Ryu
  17. 17. Neutron ML2 Plugin • ML2 = Modular Layer 2 – New framework for Layer2 network functions to avoid duplicated/copy-and-paste efforts across plugins. – Default reference implementation in Neutron. OVS / Linux Bridge plugins will be removed in the near future. – We can use multiple network technologies at the same time. – Two types of drivers: Type Driver and Mechanism Driver • Type Driver – Manages logical network resources depending on “network type” (e.g., VLAN-ID/VXLAN VNI. – “network type” is specified by “provider:network_type” attribute of the provider network extension when creating a network. – Supported network types: VLAN, VXLAN, GRE, Local, Flat • Mechanism Driver – Configures/provisions network devices. – Multiple mechanism drivers can be enabled (ordered list). Each mechanism driver configures corresponding network devices if the driver supports a specified network type. – Only one mechanism driver can handle plugging VM VIF to a network (bind_port). – Open vSwitch agent driver, Linux Bridge agent driver, OpenFlow Agent driver, … – Cisco Nexus 1000V driver, Arista driver, ….. – L2 population driver • Install MAC addresses of endpoints into OVS/Linux bridges to reduce broadcast in tunneling. Features of OVS/Linux Bridge Plugin are now frozen. All new features are added to ML2 plugin. Let’s use ML2 plugin.
  18. 18. Icehouse Update • Focused on improvements in performance and stability – Multiple API worker / RPC worker – Improvements in dhcp-agent, l3-agent, OVS agent … – Scenario test / API test coverage in tempest – Third party testing (vendor driver testing) • nova event callback – Notify nova of port status changes when fixed_ips/floatingips change – It ensure a VM is launched after network connection is available. • Deprecate OVS Plugin and Linux Bridge plugin – Migration to ML2 plugin is recommended. • Not so many feature additions – We added many feature additions in the past releases. We improved the quality of the whole neutron codebase and now are ready for next feature additions :-) Key New Features During Icehouse cycle the team focused on stability and testing of the Neutron codebase. Many of the existing plugins and drivers were revised to address known performance and stability issues. Icehouse Release Notes
  19. 19. Multiple API/RPC Worker Neutron server Message queue OVS agent Dhcp agent l3- agent OVS agent Dhcp agent l3- agent OVS agent Dhcp agent l3- agent DB RPC worker Message queue OVS agent Dhcp agent l3- agent OVS agent Dhcp agent l3- agent OVS agent Dhcp agent l3- agent DB RPC worker RPC worker API worker API worker API worker API request RPC message
  20. 20. Compute host VM Nova event callbacks • Mechanism to inform nova of state changes in Neutron ports • Nova waits VM launching until network connection becomes ready – Previously Nova launched VMs without checking network connection status. As a result, IP address allocations by DHCP sometimes fail with timeout. nova-compute VM OVS VIF Neutron Server OVS agent (1) Create Port (2) Create Tap device Network (3) Setup NW connection (4) Report “port is ready” (5) Notify Nova (6) VM Launch Dhcp server Wait notification from neutron before launching VM
  21. 21. Vendor Plugin/Driver Testing • Vendor plugin/driver scenario/API testing (so-called “Third Party Testing” is now one of requirements for new vendor/driver addition. It also applies to all existing plugins/drivers. • All third party CI system should run scenario/API tests for every patch proposal to Gerrit. • Test cases of scenario/API tests are developed by OpenStack community (“tempest” projecct). • Previously there is no way to check vendor plugins/drivers are tested.
  22. 22. Neutron Icehouse update detail (1/2) • Main Focuses are “Stabilization” and “Test Coverage” (rather than new features) – Scenario and API test coverage in tempest – Enable parallel testing on tempest • Nova-Network Feature Parity – Nova-network feature parity towards deprecating nova-network – A lot of progresses but the completion will be Juno (or later) • Nova Integration : nova-event-callback – Notify nova of port status changes when fixed_ips/floatingips change – It ensure a VM is launched after network connection is available. • ML2 (Modular Layer2) plugin – Add support for vnic type request to be managed by ML2 mechanism drivers • Allows nova to request vnic type for a port : normal, direct (pass-through), macvtap – Implement the binding:profile port attribute in ML2 – Extensible port attribute for plugin to provide details to VIF driver – Migration from OVS / linux bridge plugin to ML2 • Provide a tool to migration DB data from deprecated plugins to ML2. • L3 : Operational Status for Floating IPs • Advanced Network Services (LBaaS, VPNaaS, FWaaS) – No major features – Some vendor drivers: Radware/Embrane (LBaaS), Cisco (VPNaaS)
  23. 23. Neutron Icehouse update detail (2/2) • New ML2 mechanism drivers – OpenDayLight plugin (It works with Hydrogen ODL release) – OpenFlow agent (ryu) driver (OpenFlow controller per compute node) – Big Switch, Mellanox, Brocade • New plugins : IBM SDN-VE, Nuage, OneConvergence • Existing plugins: – BigSwtich (async ope, secgroup, resync with backend) – Vmware NSX (renamed from Nicira NVP plugin), Plumgrid (provider network) – NEC plugin : packet filter extension support for PFC • Items deferred to Juno …. – IPv6 related attributes to control IPv6 RA and address assignments (ipv6-two-attributes) • API layer is defined but backend implementation is not merged. – Layer 3 HA • L3-agent HA • Distributed Virtual Router (with OVS) – Advanced Network Service “flavor” support – Advanced service in VM – and so on
  24. 24. Neutron topics in “Juno” • Migration from “nova-network” – OpenStack has two network implementation: nova-network and neutron. – The community seems to have a consensus that both are merged into Neutrn. – Feature parity to nova-network – Clarify the migration scenarios from nova-network in the production environments. • Migration of Network/Port/Security Group/… from nova to neutron. • Exploring a migration way without stopping tenant data plane traffic. One of the current goals is to avoid breaking TCP connection. • Requires stoping API service during the migration. – Tracking the progress in Neutron weekly meeting. • Distributed Virtual Router (DVR) • L3-agent HA • IPv6 full support • Baremetal (Ironic) Integration – Multi Tenant isolation with baremetal instance
  25. 25. Neutron topics in “Juno” • Distributed Virtual Router (DVR) with Open vSwitch – In the current OSS implementation, all traffic across logical routers needs to be transferred via L3-agent nodes. It can be a bottleneck in both performance and HA perspectives. – DVR implements distributed virtual routers using OVS bridge. The routing functionality is distributed over all compute nodes. – It is expected to be an alternative of nova-network “multi_host” feature (which is a common solution to achieve network HA in nova-network).
  26. 26. Neutron topics in “Juno” Network Services • Flavor Framework – A mechanism to provide multiple service levels or implementations in advanced network services such as LBaaS. – Operators can choose backend implementations according to the service level of a LBaaS resource (e.g., physical appliance for “gold” load balancer and virtual appliance for “bronze” load balancer). • Service Insertion / Service Chaining • Traffic Steering – Allows to redirect traffic which matches a defined policy to a specific logical port. It can be used to implement service chaining or insert “L2- transparent” appliances. • Group Policy Framework
  27. 27. Neutron topics in “Juno” Network Services • Service VM – VM which provides network services; “virtual appliance” – Service VM itself is related to multiple projects such as neutron and nova, so it will be a separate project on stackforge. – We decided to gather requirements for neutron for service VMs and implement them into Neutron. Neutron team is also interested in these efforts and continues to support the effort. • Disabling Security Group / Anti-spoofing rule • Trunk VLAN support for VM port • Traffic Steering • …. –
  28. 28. Recent topics in developer community • NFV Sub-team • Cross-Project design summit session • Ops-Dev Meet-up • Blueprint Spec Review
  29. 29. Backup slides
  30. 30. Neutron Functional modules • Neutron Server – Manage logical networks – Neutron plugin is a part of neutron server (now) • Agent – Service processes other than neutron-server. They can be located on a different host from neutrn-server. – Some plugins uses agents and some do not. It depends on plugin implementations. • Various agents – Plugin Agent • Mainly configures and monitos virtual switches. • Runs on compute nodes and network nodes. – DHCP Agent • Manages DHCP servers per logica network – L3 Agent • Manages logical routers. • Network namespace and itpables are used to create logical routers in l3-agent. – Metadata Agent • Proxies metadata access from neutron network to the nova metadata service.
  31. 31. Nova(CC) Neutron Server Compute Node Compute Node Data Network eth1 eth1 eth0eth0eth0 OVS (br-int) OVS (br-int) Nova Compute Nova Compute Plugin Agent Plugin Agent Management Network Neutron modules and deployment Controller Node eth0 eth1 OVS (br-int) DHCP agent L3 agent Plugin Agent OVS (br-ex) Network Node eth2 External network metadata agt
  32. 32. Neutron Nova Neutron Plugin Nova Compute Virtual Switch VM VM Nova Compute Virtual Switch VM VM HW Switch CLI / Dashboard (Horizon) / Orchestration Tool Physical Network Neutron API Nova API agent agent
  33. 33. LBaaS • Added in Grizzly – Define LBaaS API (tenant API) – HAProxy reference implementation • Havana – Service Type Framework • Allow to specify a backend driver. It allows operators to enable mutiple load- balancer drivers and choose it when creating a load-balancer insntace. – New vendor LBaaS drivers and plugins • Icehouse (planned) – Multiple VIPs per pool, moving VIP across LB pools – Layer7 support – SSL termination – Routed Service Insertion (LVS based) ? – [OLD[ Slide at the beginning of Icehouse dev-cycle
  34. 34. FWaaS • Added in Havana – Horizon, Heat support – FWaaS API (rule, policy, firewall resource) – L3-agent based impl, some vendor plugin – Limitations • 1 firewall per tenant (with l3-agent) • Firewall applies to all routers • Icehouse (planned) – Service Insertion/Service Type Framework – Zone – Apply Operation – Counters – [OLD[ Slide at the beginning of Icehouse dev-cycle
  35. 35. VPNaaS • Added in Havana – IPSec VPN (L3 VPN) is supported – Horizon, Heat support • Icehouse (planned) – Service Type Framework • Support multiple typs of VPNs at the same time – SSL-VPN – IPSec Certification Support – MPLS/BGP – (L2-VPN?) – House [OLD[ Slide at the beginning of Icehouse dev-cycle