ONIC 2016
Contrail 3.x update & Contrail + Something
Daisuke Nakajima | Systems Engineer
Contrail Re-cap
Physical IP Fabric
(no changes)
CONTRAIL Overview
CONTRAIL CONTROLLER
ORCHESTRATOR
Host O/SvRouter
Network / Storage
orchestration
Gateway
…
Internet / WAN
or Legacy Env.
(Config, Control, Analytics, Svr Mgmt)
(Windows, Linux ….) on BMS
TOR
Compute
orchestration
Virtual Network
Blue
Virtual Network
Red
FW
Logical View
…
Centralized
PolicyDefinition
Distributed
PolicyEnforcement
BGP
BGP XMPPOVSDB
Integration with VM,Container and BMS
Green Virtual
Network
(RT = G)
VM
1
VM
2
Blue Virtual
Network
(RT = B)
LOGICALPHYSICAL
Bare Metal Server Integration (using
vRouter) achieved through:
 L2 / L3 Gateway
 Containers Docker / LXC
 Smart NIC
VN Green
(RT = G)
VM
1
VM
2
… VLAN
VN Blue
(RT = B)
VM
3
VM
3
VRF
(RT=B)
IP Fabric
C1 C2
C1 C2 C3
Containers
C3
VM
NFV traffic management
load balancing as
Virtual Service
load balancing in
vRouter
Mechanisms
ECMP
Flow Tables
Consistent Hashing
Challenges
Scale and performance
Stickiness
Symmetry
Grow Scalability with simple way
Blue
Network
Yellow
Network
NFV1
NFV2
NFV3 Add additional NFVs
to expand its scaling
OpenStack Survey Apr 2016
% of users of OpenStack Network (Neutron) per driver
[ Source: https://www.openstack.org/assets/survey/April-2016-User-Survey-Report.pdf]
Contrail 3.x Update
vRouter DPDK
vRouter Overview (Today)
vRouter
Kernel Space
User Space
QEMU Layer
Kernel Space
User Space
Guest VM
tap-xyz(vif)
vHOST
tap-xyz(vif)
VIRTIO
Nova Agent
vRouter Host Agent
Application
VM
DPDK vRouter Overview
Kernel Space
User Space
QEMU Layer
Kernel Space
User Space
Application VM
DPDK
Guest VM
Nova Agent
vRouter Host Agent
vRouter DPDK
eth0
VIF: TAP
eth1
VIF: TAP
DPDK vRouter Architecture
VM (Virtual Machine)
VIRTIO Ring
VIRTIO
Frontend
User Space vHost (libvirt 1.2.7)
 vHost-Net : Kernel Space (Before QEUMU 2.1)
 vHost-User: User Space vHost (QEMU 2.1)
vRouter (User-Space)
VRFWD hugetlbfs (DPDK Ring)
User-Space
Qemu Uvhost client
Kernel Space
Virtio ring
Mmap’ed memory in VRFWD from hugetlbfs
Uvhost Server
Unix Socket
(Message exchanged
once VM isUP)
1 2 3 4
NIC Queues (1,2..N)
DPDK NIC
DPDK vRouter
1 2 3 4
DPDK lcores
Lcores to NIC Queue
Mapping 1-1
Poll
vRouter
Forwarding
netlink
pkt0
VRF
Config
Policy
Tables
vRouter Agent
(vnswad)
Uvhost Server: Assigns lcore to virtio interfaces based
on Unix Socket Message communications
TCP Connection
(routes/nexthops/
interfaces/flows
Created by DPDK EAL
(Environment Abstraction Layer)
Created by DPDK EAL
(Environment Abstraction Layer)
VIRTIO
Bandend
Host
Compute Node
QEMU 2.2
Process Per VM
Host Process per VM
DPDK 2.0 Libraries
Guest
VNF Health Check
Contrail-Controller
Contrail Health Check (Service Liveliness)
VM-HC-01
4.4.4.5/24
(eth0)
vRouter
Hypervisor
02:a6:8f:d7:ed:f7
00:00:5e:00:01:00
192.168.1.236/24 (eth0/bond0)
IP FIB Test-VN
4.4.4.5 – interface tapx
169.254.169.254 - Link Local
ac:16:2d:9f:fa:9d
vhost0
L2-receive
L2-receive
MAC FIB Test-VN
02:a6:8f:d7:ed:f7 – interface tapx
ff:ff:ff:ff:ff:ff – L2 Composite
VMI:tapxyz-00
contrail-vrouter-agent-health-check.py
# contrail-vrouter-agent-health-check.py -m
PING/HTTP -d 169.254.32.0 -t 5 -r 1 -i 180
HC: Health Check (Flow)
HC:HealthCheck(Flow)
contrail-vrouter-agent
Health
Check Failed
Purge Route VRF
(local)
Contrail-Controller
XMPP
Purge Route VRF (Global)
Controller
1. Create Health Check PING or HTTP
• Local-IP, URI, IP:Port
• Delay, Timeout, Retries
2. Attach the HC to SI V2 or Contrail Port (VMI)
3. When Health Check failed the route will be purged from the VRF
Note: A REST API, Heat or GUI can be used to create a Health Check object and properties. The same Health Check can apply to multiple VM interfaces
and a VM Interface can be associated to multiple Health Check Objects
Summary: Determines the liveliness of a service provided by
a VM by checking if its operationally up or down. vRouter
agent uses ping and HTTP URL to the link local address to
check the liveliness of the VMI. If the health check determines
the service is not operational it removes the routes for the VM
disabling forwarding of packets to the VM
Health Check Objects are:
• Enabled
• Monitor-type # Health Check protocol type to be used
(HTTP/PING)
• Delay # delay between to health check attempts
• Timeout #timeout for single health check attempt
• max-retries #number of retries to attempt before declaring a failure
• url-path # url string for HTTP, destination IP for all other cases
Contrail Health Check (Contrail GUI)
Create Health Check
Apply Health Check to Service Instance
Contrail Health Check via VMI
Contrail Analytics
Unified Network management
Virtual-network, vPort, Underlay Switch,
Gateway router, Physical / Virtual
correlation by LLDP and SNMP.
Contrail Port Mirroring (Virtual Machine Interface)
VN: Green
172.16.10.0/24
VN: Red
192.168.10.0/24
VN: Red
172.20.0.0/24
VN: Analyzer
192.168.100.0/24
Shared VN
Green-VM
172.16.10.252/24
Red-VM
192.168.10.252/24
Red-VM
172.20.0.3/24
Analyzer-VM
192.168.100.252/24
Tenant: Admin Tenant: Demo
Steps:
• Create Overlay Topology via Heat, GUI or CLI
• Collect all VM Ports VMI (CLI or GUI)
• Use “add-mirror.py” script to start mirroring each VMI
traffic to Analyzer IP address “192.168.100.252”
Mirror Packets
Mirror
Packets
Mirror Packets
Contrail Interface Base Port Mirroring (VMI)
Port IP: 172.20.0.3
Contrail + Something
Private Cloud + Colocation
Gateway solution
Green Virtual
Network
(RT = G)
VM1 VM2
Blue Virtual
Network
(RT = B)
VM1 VM2
LOGICAL
(PolicyDefinition)
PHYSICAL
(PolicyEnforcement)
Colocation Servers
…
VLAN
VRF
(RT=B)
IP Fabric
VM1 VM2 VM2VM1
VRF
(RT=A)
VLAN
Private Cloud User Colocation
Colocation network connects Private Cloud by GW
router. A switch located user Colocation connects GW
router via VLAN. GW router creates VRFs
corresponded its VLANs/ports.
Private Cloud + Colocation
ToR Switch (VXLAN) solution
Green Virtual
Network
(RT = G)
VM1 VM2
Blue Virtual
Network
(RT = B)
VM1 VM2
LOGICAL
(PolicyDefinition)
PHYSICAL
(PolicyEnforcement)
Colocation Servers
…
VLAN
IP Fabric
VM1 VM2 VM2VM1
VLAN
Private Cloud User Colocation
Private cloud and Colocation server are integrated via
ToR Switch (QFX5100).
A switch connecting Colocation servers connects to
ToR Switch by VLANs or ports. ToR Switch is
configured by Contrail
※Note: Consider ToR Switch redundancy.
Private Cloud + AWS
IPsec connect
Green Virtual
Network
(RT = G)
VM1 VM2
Blue Virtual
Network
(RT = B)
VM1 VM2
LOGICAL
(PolicyDefinition)
PHYSICAL
(PolicyEnforcement)
… vNW
IP Fabric
VM1 VM2
Private Cloud AWS
vSRX on Contrail connects AWS by IPsec. In this case,
virtual-network on Contrail must be different from AWS.
VM3 VM3
Internet
vSRX
IPsec TunnelVM3
VM1 VM2 VM3
Private Cloud + AWS
Direct connect
Green Virtual
Network
(RT = G)
VM1 VM2
Blue Virtual
Network
(RT = B)
VM1 VM2
LOGICAL
(PolicyDefinition)
PHYSICAL
(PolicyEnforcement)
… vNW
IP Fabric
VM1 VM2
Private Cloud AWS
vSRX on Contrail connects AWS by IPsec. In this case,
virtual-network on Contrail must be different from AWS.
VM3 VM3
Internet
vSRX
VM3
VM1 VM2 VM3
Direct Connect
DEMO Slide
Demo 環境
GW Router
Data / Control
Switch
Management
Switch
PrivateCloudColocation
Colocation
Switch
LOGICALtPHYSICAL
Private Cloud
Network
VM1 VM2
Colocation
Network
仮想ネットワークの作成
仮想ルータの作成
仮想マシンの作成
通信確認(VM – VM)
既存環境の接続
既存環境の接続
既存環境の接続
通信確認(VM – 既存環境)
Thank youThank you

ONIC Japan 2016 - Contrail アップデート

  • 1.
    ONIC 2016 Contrail 3.xupdate & Contrail + Something Daisuke Nakajima | Systems Engineer
  • 2.
  • 3.
    Physical IP Fabric (nochanges) CONTRAIL Overview CONTRAIL CONTROLLER ORCHESTRATOR Host O/SvRouter Network / Storage orchestration Gateway … Internet / WAN or Legacy Env. (Config, Control, Analytics, Svr Mgmt) (Windows, Linux ….) on BMS TOR Compute orchestration Virtual Network Blue Virtual Network Red FW Logical View … Centralized PolicyDefinition Distributed PolicyEnforcement BGP BGP XMPPOVSDB
  • 4.
    Integration with VM,Containerand BMS Green Virtual Network (RT = G) VM 1 VM 2 Blue Virtual Network (RT = B) LOGICALPHYSICAL Bare Metal Server Integration (using vRouter) achieved through:  L2 / L3 Gateway  Containers Docker / LXC  Smart NIC VN Green (RT = G) VM 1 VM 2 … VLAN VN Blue (RT = B) VM 3 VM 3 VRF (RT=B) IP Fabric C1 C2 C1 C2 C3 Containers C3
  • 5.
    VM NFV traffic management loadbalancing as Virtual Service load balancing in vRouter Mechanisms ECMP Flow Tables Consistent Hashing Challenges Scale and performance Stickiness Symmetry
  • 6.
    Grow Scalability withsimple way Blue Network Yellow Network NFV1 NFV2 NFV3 Add additional NFVs to expand its scaling
  • 7.
    OpenStack Survey Apr2016 % of users of OpenStack Network (Neutron) per driver [ Source: https://www.openstack.org/assets/survey/April-2016-User-Survey-Report.pdf]
  • 8.
  • 9.
  • 10.
    vRouter Overview (Today) vRouter KernelSpace User Space QEMU Layer Kernel Space User Space Guest VM tap-xyz(vif) vHOST tap-xyz(vif) VIRTIO Nova Agent vRouter Host Agent Application VM
  • 11.
    DPDK vRouter Overview KernelSpace User Space QEMU Layer Kernel Space User Space Application VM DPDK Guest VM Nova Agent vRouter Host Agent vRouter DPDK eth0 VIF: TAP eth1 VIF: TAP
  • 12.
    DPDK vRouter Architecture VM(Virtual Machine) VIRTIO Ring VIRTIO Frontend User Space vHost (libvirt 1.2.7)  vHost-Net : Kernel Space (Before QEUMU 2.1)  vHost-User: User Space vHost (QEMU 2.1) vRouter (User-Space) VRFWD hugetlbfs (DPDK Ring) User-Space Qemu Uvhost client Kernel Space Virtio ring Mmap’ed memory in VRFWD from hugetlbfs Uvhost Server Unix Socket (Message exchanged once VM isUP) 1 2 3 4 NIC Queues (1,2..N) DPDK NIC DPDK vRouter 1 2 3 4 DPDK lcores Lcores to NIC Queue Mapping 1-1 Poll vRouter Forwarding netlink pkt0 VRF Config Policy Tables vRouter Agent (vnswad) Uvhost Server: Assigns lcore to virtio interfaces based on Unix Socket Message communications TCP Connection (routes/nexthops/ interfaces/flows Created by DPDK EAL (Environment Abstraction Layer) Created by DPDK EAL (Environment Abstraction Layer) VIRTIO Bandend Host Compute Node QEMU 2.2 Process Per VM Host Process per VM DPDK 2.0 Libraries Guest
  • 13.
  • 14.
    Contrail-Controller Contrail Health Check(Service Liveliness) VM-HC-01 4.4.4.5/24 (eth0) vRouter Hypervisor 02:a6:8f:d7:ed:f7 00:00:5e:00:01:00 192.168.1.236/24 (eth0/bond0) IP FIB Test-VN 4.4.4.5 – interface tapx 169.254.169.254 - Link Local ac:16:2d:9f:fa:9d vhost0 L2-receive L2-receive MAC FIB Test-VN 02:a6:8f:d7:ed:f7 – interface tapx ff:ff:ff:ff:ff:ff – L2 Composite VMI:tapxyz-00 contrail-vrouter-agent-health-check.py # contrail-vrouter-agent-health-check.py -m PING/HTTP -d 169.254.32.0 -t 5 -r 1 -i 180 HC: Health Check (Flow) HC:HealthCheck(Flow) contrail-vrouter-agent Health Check Failed Purge Route VRF (local) Contrail-Controller XMPP Purge Route VRF (Global) Controller 1. Create Health Check PING or HTTP • Local-IP, URI, IP:Port • Delay, Timeout, Retries 2. Attach the HC to SI V2 or Contrail Port (VMI) 3. When Health Check failed the route will be purged from the VRF Note: A REST API, Heat or GUI can be used to create a Health Check object and properties. The same Health Check can apply to multiple VM interfaces and a VM Interface can be associated to multiple Health Check Objects Summary: Determines the liveliness of a service provided by a VM by checking if its operationally up or down. vRouter agent uses ping and HTTP URL to the link local address to check the liveliness of the VMI. If the health check determines the service is not operational it removes the routes for the VM disabling forwarding of packets to the VM Health Check Objects are: • Enabled • Monitor-type # Health Check protocol type to be used (HTTP/PING) • Delay # delay between to health check attempts • Timeout #timeout for single health check attempt • max-retries #number of retries to attempt before declaring a failure • url-path # url string for HTTP, destination IP for all other cases
  • 15.
    Contrail Health Check(Contrail GUI) Create Health Check Apply Health Check to Service Instance
  • 16.
  • 17.
  • 18.
    Unified Network management Virtual-network,vPort, Underlay Switch, Gateway router, Physical / Virtual correlation by LLDP and SNMP.
  • 19.
    Contrail Port Mirroring(Virtual Machine Interface) VN: Green 172.16.10.0/24 VN: Red 192.168.10.0/24 VN: Red 172.20.0.0/24 VN: Analyzer 192.168.100.0/24 Shared VN Green-VM 172.16.10.252/24 Red-VM 192.168.10.252/24 Red-VM 172.20.0.3/24 Analyzer-VM 192.168.100.252/24 Tenant: Admin Tenant: Demo Steps: • Create Overlay Topology via Heat, GUI or CLI • Collect all VM Ports VMI (CLI or GUI) • Use “add-mirror.py” script to start mirroring each VMI traffic to Analyzer IP address “192.168.100.252” Mirror Packets Mirror Packets Mirror Packets
  • 20.
    Contrail Interface BasePort Mirroring (VMI) Port IP: 172.20.0.3
  • 21.
  • 22.
    Private Cloud +Colocation Gateway solution Green Virtual Network (RT = G) VM1 VM2 Blue Virtual Network (RT = B) VM1 VM2 LOGICAL (PolicyDefinition) PHYSICAL (PolicyEnforcement) Colocation Servers … VLAN VRF (RT=B) IP Fabric VM1 VM2 VM2VM1 VRF (RT=A) VLAN Private Cloud User Colocation Colocation network connects Private Cloud by GW router. A switch located user Colocation connects GW router via VLAN. GW router creates VRFs corresponded its VLANs/ports.
  • 23.
    Private Cloud +Colocation ToR Switch (VXLAN) solution Green Virtual Network (RT = G) VM1 VM2 Blue Virtual Network (RT = B) VM1 VM2 LOGICAL (PolicyDefinition) PHYSICAL (PolicyEnforcement) Colocation Servers … VLAN IP Fabric VM1 VM2 VM2VM1 VLAN Private Cloud User Colocation Private cloud and Colocation server are integrated via ToR Switch (QFX5100). A switch connecting Colocation servers connects to ToR Switch by VLANs or ports. ToR Switch is configured by Contrail ※Note: Consider ToR Switch redundancy.
  • 24.
    Private Cloud +AWS IPsec connect Green Virtual Network (RT = G) VM1 VM2 Blue Virtual Network (RT = B) VM1 VM2 LOGICAL (PolicyDefinition) PHYSICAL (PolicyEnforcement) … vNW IP Fabric VM1 VM2 Private Cloud AWS vSRX on Contrail connects AWS by IPsec. In this case, virtual-network on Contrail must be different from AWS. VM3 VM3 Internet vSRX IPsec TunnelVM3 VM1 VM2 VM3
  • 25.
    Private Cloud +AWS Direct connect Green Virtual Network (RT = G) VM1 VM2 Blue Virtual Network (RT = B) VM1 VM2 LOGICAL (PolicyDefinition) PHYSICAL (PolicyEnforcement) … vNW IP Fabric VM1 VM2 Private Cloud AWS vSRX on Contrail connects AWS by IPsec. In this case, virtual-network on Contrail must be different from AWS. VM3 VM3 Internet vSRX VM3 VM1 VM2 VM3 Direct Connect
  • 26.
  • 27.
    Demo 環境 GW Router Data/ Control Switch Management Switch PrivateCloudColocation Colocation Switch LOGICALtPHYSICAL Private Cloud Network VM1 VM2 Colocation Network
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.