In the OpenStack ecosystem, containers were introduced as first class citizens recently with the project Magnum and the networking for containers has also evolved since then. Project Kuryr makes networking available to containers through Neutron. This all brings together how Neutron networking benefits containers like it does virtual machines. However, to make Neutron, Kuryr and Magnum cover all the use cases for containers, nested containers inside Nova VMs require networking to work as seamlessly as it works for virtual machines or bare metal containers. In this session, we will talk about Magnum, Kuryr, Neutron integration and how the problem of nested container networking has been solved in the OpenStack community, it's architecture, the design, current status and next steps.

1. Magnum, Kuryr, Neutron Integration
Networking for Nested Containers
Fawad Khaliq - @fawadkhaliq
Antoni Segura – @celebdor
Gal Sagie - @GalSagie

2. Copyright © PLUMgrid, Inc. 2011-2016
Introduction
Speakers
Sr. Software Engineer
PLUMgrid
Khaliq
Fawad
2
Senior Engineer
Midokura
Segura
Antoni
Architect
Huawei
Sagie
Gal

3. Copyright © PLUMgrid, Inc. 2011-2016
• Magnum, Neutron
• Kuryr
• Nested Containers and Networking Problem
• Nested Containers Networking – Solution/Design
• Capabilities and considerations
• Current Status
• Next Steps
• Q&A
Agenda
3

4. Magnum
Container-as-a-service in OpenStack
4

5. Copyright © PLUMgrid, Inc. 2011-2016
Magnum
Container-as-a-service in OpenStack
5
Docker Swarm (Bay)
Nova Instance
Container
Container
Container
Nova Instance
Container
Container
Container

6. Copyright © PLUMgrid, Inc. 2011-2016
Magnum
Container-as-a-service in OpenStack
6
Kubernetes (Bay)
Nova Instance
Pod
Container
Container
Nova Instance
Pod
Container
Container

7. Copyright © PLUMgrid, Inc. 2011-20167

8. Neutron
Networking in OpenStack
8

9. Copyright © PLUMgrid, Inc. 2011-2016
• Provides “network as a service”
• Provides rich network topologies
• Technology agnostic; pluggable networking backends
• Extensible
• Offers advanced services like LBaas, VPNaas, FWaas etc
Neutron
9

10. Kuryr
Container Networking in OpenStack
10

11. Copyright © PLUMgrid, Inc. 2011-2016
Kuryr
11
Neutron as the
production-ready
networking
abstraction
containers need

12. Copyright © PLUMgrid, Inc. 2011-2016
VM/Container Networking: Similar Concepts
12
Docker C1 Docker C2 Docker C3
libNetwork
Endpoint Endpoint EndpointEndpoint
Frontend
Network
Backend
Network
Network Sandbox Network Sandbox Network Sandbox
VM2
192.168.1.7
192.168.5.2
VM1
Tenant A Net1
192.168.1.0/0
Tenant A Net2
192.168.5.0/0
192.168.1.5
Neutron

13. Copyright © PLUMgrid, Inc. 2011-2016
• Open source
• Part of OpenStack Big-Tent
• Brings the Neutron networking model to containers
• Aims to support different Container Runtimes (docker, rkt, etc)
• E.g. Kubernetes, Mesos, Docker Swarm
• Weekly IRC meetings
• Working together with OpenStack community
• Neutron, Magnum, Kolla
Kuryr Project Overview
13

14. Copyright © PLUMgrid, Inc. 2011-2016
Kuryr Components
14
Configuration
ManagementKuryr libNetwork
Network Plugin
K8S CNI Driver
Keystone
Authentication
& Neutron
Client
Interface
Generic
VIF Binding
Kuryr libNetwork
IPAM Plugin

15. Problems with current Nested Containers
Why do we need to consider this as a special scenario?
15

16. Copyright © PLUMgrid, Inc. 2011-2016
• Two Separate networking infrastructures
• Hard to enforce network policy (N-tier applications)
• Security and Isolation
• Performance and unneeded overhead
Problems with Current Nested Containers Networking
16

17. Copyright © PLUMgrid, Inc. 2011-2016
Problems with Current Nested Containers Networking
17
Docker 0
OVS
VXLAN Overlay
VM
Docker 0
Neutron Plugin
VXLAN Overlay
VM
SDN Overlay
Neutron
Overlay

18. Copyright © PLUMgrid, Inc. 2011-2016
Problems with Current Nested Containers Networking
18
Neutron Networks
VMVM VM
Tenant A Net1
192.168.1.0/0

19. Copyright © PLUMgrid, Inc. 2011-2016
Problems with Current Nested Containers Networking
19
Container Networks
VMVM VM
Backend
Network
10.2.0.0/24
Endpoint Endpoint Endpoint Endpoint Endpoint Endpoint Endpoint
Frontend Network
10.1.0.0/24

20. Nested Container Networking Solution
Design for the nested container networking in OpenStack
20

21. Copyright © PLUMgrid, Inc. 2011-2016
• Nested/baremetal container to nested/baremetal container same/
different hosts
• Nested/baremetal container to virtual machine communication
• Nested/baremetal container to baremetal communication
• Container networking as a first class entity in Neutron
• Consistent policy enforcement across containers, VMs, bare metal
• Enable advanced networking services like FWaas, LBaas, VPNaas
etc
Nested Container Networking Use Cases
21

22. Copyright © PLUMgrid, Inc. 2011-2016
Nested Container Networking Design
Magnum, Kuryr, Neutron Integration
22
VLAN:100 VLAN:200 VLAN:400 VLAN:100

23. Copyright © PLUMgrid, Inc. 2011-2016
Neutron Trunk Ports
23
Nova Instance
port-1
port-0
port-2
network-1
network-0
network-2
Port combined into one vif
by turning port-0 into trunk
and other ports into supports
of the trunk

24. Copyright © PLUMgrid, Inc. 2011-201624

25. Capabilities and Considerations
25

26. Copyright © PLUMgrid, Inc. 2011-2016
• Neutron resources spec approved and patches under review
• Trunk
• Subport
• Subports bring isolation to container-in-VM use cases
• Port forwarding can take us further
• Vendors can implement new segmentation types
• Tagged traffic that does not match a sub port, is considered of the
trunk port
Capabilities and Considerations
26

27. Copyright © PLUMgrid, Inc. 2011-2016
• Limitations
• Policy is applied at the Host level
• Initial only VLAN tags for segmentation type
• Tags are unique per “trunk port” scope
• VM users can alter subport traffic
• Logging of VM actions is dependent on integration
• Can't work with current OVS
Capabilities and Considerations
27

28. Current Status
28

29. Copyright © PLUMgrid, Inc. 2011-2016
• Trunk Port Extension spec approved and code in progress
• Binding profile workaround to proceed in parallel
• Nested Container networking spec approved in Kuryr
• Docker Swarm Integration completed
• Kubernetes in progress
• Mesos in design stages
Current Status
29

30. Next Steps
30

31. Copyright © PLUMgrid, Inc. 2011-2016
• Follow up on the Neutron Trunk port implementation
• Finish COE baremetal integration
• Policy translation
• Make Neutron resources available through native APIs
• Magnum deployment prototype of worker VM with Kuryr agent
• Magnum administrator VM that communicates with Neutron
Next Steps
31

32. Questions
32

33. Join us at #openstack-kuryr
THANK YOU!
irc: #openstack-kuryr @ freenode