Download as PDF, PPTX
Uploaded byJakub Pavlik
Kubernetes SDN performance and architecture
The document discusses Kubernetes SDN performance and architecture. It provides an overview of Calico and OpenContrail, two common SDN solutions for Kubernetes. Calico uses standard protocols and has no overhead but lacks L2 capabilities. OpenContrail provides advanced networking features through an overlay but has more overhead and complexity. Both solutions were tested on a 100 node cluster and their performance and production considerations are examined. The presentation concludes with a comparison of Calico and OpenContrail and examples of multi-cloud architectures using them.
More Related Content
Similar to Kubernetes SDN performance and architecture
Kubernetes SDN performance and architecture
- 1. Copyright © 2016Mirantis, Inc. All rights reserved www.mirantis.com Kubernetes SDN Performance and Architecture Jakub Pavlik Marek Celoud
- 2. Copyright © 2016Mirantis, Inc. All rights reserved Presentation Agenda 1. Overlay vs Non-Overlay 2. Calico 3. OpenContrail 4. Connection/comparison 5. Q&A
- 3. Copyright © 2016Mirantis, Inc. All rights reserved About us Marek Celoud mceloud@mirantis.com @MCeloud Jakub Pavlík jpavlik@mirantis.com @JakubPav
- 4. Copyright © 2016Mirantis, Inc. All rights reserved Networking in Kubernetes ● Networking in containers used to be an issue ● Kubernetes solved the biggest problems of port mapping ● Different approaches for different use cases ● Overlay vs. Non-overlay ● Multitenancy and security ● Performance and scaling ● Multiple plugins similar like OpenStack Neutron
- 5. Copyright © 2016Mirantis, Inc. All rights reserved Network solutions in Kubernetes SDNs: ● Calico ● OpenContrail ● Romana ● Weave ● Contiv ● OpenVSwitch ● ...
- 6. Copyright © 2016Mirantis, Inc. All rights reserved Overlay vs. Non-overlay Common Overlay concerns: ● Loose benefit of simplicity ● Loose performance ● Difficult to maintain and troubleshoot Overlay benefits: ● Multitenancy, Security, Micro-segmentation ● L2, L3, EVPN, L3VPN capability ● Analytics From performance perspective not using an overlay, it is still necessary to use an internal bridge to demux the container virtual-ethernet interface pairs. “The key aspect to consider is operational complexity!” Pedro Marques
- 7. Copyright © 2016Mirantis, Inc. All rights reserved Test environment ● Run various functional and performance tests ● Calico bare metal ● OpenContrail bare metal ● OpenContrail running on Kubernetes with Calico ● OpenContrail and Kubernetes next together ● Calico in OpenStack with OpenContrail ● OpenContrail Kubernetes in OpenStack with OpenContrail ● 100 nodes with 32GB RAM with 8 CPUs and 2x 10Gb links
- 8. Copyright © 2016Mirantis, Inc. All rights reserved Calico
- 9. Copyright © 2016Mirantis, Inc. All rights reserved Calico Overview ● CNI network plugin ● BIRD routing daemon ● Etcd ● Confd ● Felix ● Pure L3
- 10. Copyright © 2016Mirantis, Inc. All rights reserved Calico
- 11. Copyright © 2016Mirantis, Inc. All rights reserved Calico Calico Pros: ● No overhead ● Reduce Complexity ● Using standard protocols Cons: ● Underlay depended ● No L2
- 12. Copyright © 2016Mirantis, Inc. All rights reserved Calico with k8s ● Using CNI ● Calico 0.22.0 version with kubernetes 1.4 ● Kubernetes Policy for security
- 13. Copyright © 2016Mirantis, Inc. All rights reserved Production consideration for Calico ● Use separate etcd cluster for Calico ● Use at least etcd v3 ● Disable BGP full mesh peering ● Do not run Calico in k8s manifests, but as separated systemd/docker
- 14. Copyright © 2016Mirantis, Inc. All rights reserved OpenContrail
- 15. Copyright © 2016Mirantis, Inc. All rights reserved OpenContrail Overview ● Overlay SDN ● Control, config, analytics, database, agent ● Multiple encapsulations (MPLSoverGRE/UDP, VXLAN) ● Uses (usually) physical gateways
- 16. Copyright © 2016Mirantis, Inc. All rights reserved OpenContrail overview
- 17. Copyright © 2016Mirantis, Inc. All rights reserved OpenContrail overview OpenContrail Pros: ● Underlay agnostic ● Advanced networking features ● Uses physical gateways Cons: ● Overhead ● Complex
- 18. Copyright © 2016Mirantis, Inc. All rights reserved OpenContrail with s8s ● Network manager which provides bridge between Contrail and k8s ● Using ECMP instead of kube-proxy (iptables) balancing ● Networks created based on labels in manifests ● Security and Multi-tenancy done by policy ● Contrail 3.0.3 supports Kubernetes 1.4
- 19. Copyright © 2016Mirantis, Inc. All rights reserved Production consideration for OpenContrail ● Separate Cassandra cluster for analytics ● Use physical routers as gateways
- 20. Copyright © 2016Mirantis, Inc. All rights reserved Comparison
- 21. Copyright © 2016Mirantis, Inc. All rights reserved Performance
- 22. Copyright © 2016Mirantis, Inc. All rights reserved Why not both?
- 23. Copyright © 2016Mirantis, Inc. All rights reserved Multi-cloud examples ● Connection Baremetal, VMs, container ● Run k8s on top of OpenStack with same Contrail (VM sub-interfaces)
- 24. Copyright © 2016Mirantis, Inc. All rights reserved Kubernetes production findings ● build own binaries (Mirantis Downstream) instead of reusing existing docker containers with unknown origin ● use single or high available cluster setup ● run ETCD control services in systemd not only in manifests and docker ● cleanup from mixing bash, salt, and unrelated features for production ● manage native SSL cert by Salt or external cert entity ● pull images from private docker registry with authentication
- 25. Copyright © 2016Mirantis, Inc. All rights reserved Calico vs OpenContrail comparison
- 26. Copyright © 2016Mirantis, Inc. All rights reserved MCP
- 27. Copyright © 2016Mirantis, Inc. All rights reserved Q&A Thank you for your time
- 28. Copyright © 2016Mirantis, Inc. All rights reserved Backup Slides