Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Simplifying open stack and kubernetes networking with romana

675 views

Published on

Romana, the open source project by Pani Networks, brings stunning simplicity to the usually so complex networking in OpenStack and Kubernetes. Using only native L3 routing and no overlays, along with automated distributed application of network policies and security rules, it provides operators with easy to understand and manage networking, while allowing network hardware to operate at its best and with full efficiency.

These slides were used during the OpenStack meetup in Auckland in May 2016, hosted by Catalyst IT.

Published in: Software
  • Be the first to comment

Simplifying open stack and kubernetes networking with romana

  1. 1. Simplifying the network stack with Romana Pani Networks OpenStack Meetup, Auckland, May 2016
  2. 2. romana.io Simplifying the network stack with Romana @romanaproject Agenda ● “Cloud native”, why does it matter? ● A better network for cloud native architectures ● Demos
  3. 3. romana.io Simplifying the network stack with Romana @romanaproject About us ● Team background: – Data center networks – Low-level traffic management ● Created L2 overlay network startup – Bought by Cisco ● OpenStack networking ● There's got to be a better way – Time is right
  4. 4. What is 'cloud native'?
  5. 5. romana.io Simplifying the network stack with Romana @romanaproject The past: Enterprise networking ● Full control ● Applications need L2 and L3 – May need hard-wired IP addresses – Broadcasts ● Servers are pets, not cattle: “Careful!” – VM migration ● Complex!
  6. 6. romana.io Simplifying the network stack with Romana @romanaproject Cloud native applications ● Automate all the things! – Infrastructure as code – Cattle, not pets: “Meh... just kill it.” – Workloads come and go quickly – Build for resiliance ● IP is all you need – No hardcoded IP addresses, discovery – No special network requirements – Basic IP connectivity
  7. 7. The problem
  8. 8. romana.io Simplifying the network stack with Romana @romanaproject We have a mismatch ● Building cloud native applications… ● … on top of enterprise networking – SDN controllers use overlay L2 domains – VLAN, VXLAN, OVS, etc. ● Complexity and brittleness – Lose benefits of simplicity – Lose performance (encap, blinded hardware) – Difficult to maintain and trouble shoot
  9. 9. romana.io Simplifying the network stack with Romana @romanaproject The price you pay: Complexity VXLAN Decap VXLAN Decap VXLAN Encap VXLAN Encap 2 Top of Rack Round Trips East/West Traffic Per Instance Security
  10. 10. romana.io Simplifying the network stack with Romana @romanaproject The price you pay: Performance Router Endpoint A Endpoint B Router L2 overlay A L2 overlay B VRouter
  11. 11. romana.io Simplifying the network stack with Romana @romanaproject Why do we do this to ourselves? ● We don't need any L2 features ● Except maybe traffic segmentation – Multi tenancy – Tiers and policies
  12. 12. The solution
  13. 13. romana.io Simplifying the network stack with Romana @romanaproject Networking the way it was intended ● Use native L3 capabilities ● No overlays ● De-emphasize IP address ranges ● Still provide segmentation, multi tenancy ● Simple, clear and scalable network setup
  14. 14. romana.io Simplifying the network stack with Romana @romanaproject Truly cloud native networking ● Project Romana ● Open source ● Apache 2.0 license ● Mostly written in Go ● Kubernetes and OpenStack
  15. 15. romana.io Simplifying the network stack with Romana @romanaproject Truly cloud native networking ● Use only IP routing – No overlays – All workload addresses are 'real' – Simplicity! ● Use smart addressing – Encode tenant or segment in IP address – Assign “virtual” addresses with host prefixes – Massive (!) collapse of route table ● Routes are static – No route updates, no broadcasts for new endpoint
  16. 16. romana.io Simplifying the network stack with Romana @romanaproject Romana Architecture ● On each host: Agent – Configures routes – Connects endpoint interfaces – Sets policy implementations ● Controller: Cooperating microservices – Each service with RESTful interface – Specialized for different tasks ● Environment: Different integration points – APIs, drivers for various parts of OpenStack or Kubernetes
  17. 17. romana.io Simplifying the network stack with Romana @romanaproject Romana Architecture Host A Host B Host C Agent Agent Agent Tenant Topology IPAM Root Environment (OpenStack or Kubernetes) Policy
  18. 18. Beautifully simple networking
  19. 19. romana.io Simplifying the network stack with Romana @romanaproject Routing and route aggregation Host A eth0: 192.168.8.11 Host B eth0: 192.168.8.22 Host C eth0: 192.168.8.33
  20. 20. romana.io Simplifying the network stack with Romana @romanaproject Routing and route aggregation Host A eth0: 192.168.8.11 romana-gw: 10.0.0.1/16 Host B eth0: 192.168.8.22 romana-gw: 10.1.0.1/16 Host C eth0: 192.168.8.33 romana-gw: 10.2.0.1/16
  21. 21. romana.io Simplifying the network stack with Romana @romanaproject Routing and route aggregation Host A eth0: 192.168.8.11 romana-gw: 10.0.0.1/16 10.0.0.5 10.0.1.7 10.0.1.19 10.0.5.3 Host B eth0: 192.168.8.22 romana-gw: 10.1.0.1/16 10.1.3.52 10.1.9.2 Host C eth0: 192.168.8.33 romana-gw: 10.2.0.1/16 10.2.0.16 10.2.3.81 10.2.4.6
  22. 22. romana.io Simplifying the network stack with Romana @romanaproject Routing and route aggregation Host A eth0: 192.168.8.11 romana-gw: 10.0.0.1/16 10.0.0.5 10.0.1.7 10.0.1.19 10.0.5.3 Host B eth0: 192.168.8.22 romana-gw: 10.1.0.1/16 10.1.3.52 10.1.9.2 Host C eth0: 192.168.8.33 romana-gw: 10.2.0.1/16 10.2.0.16 10.2.3.81 10.2.4.6
  23. 23. romana.io Simplifying the network stack with Romana @romanaproject Routing and route aggregation Host A eth0: 192.168.8.11 romana-gw: 10.0.0.1/16 10.0.0.5 10.0.1.7 10.0.1.19 10.0.5.3 Host B eth0: 192.168.8.22 romana-gw: 10.1.0.1/16 10.1.3.52 10.1.9.2 Host C eth0: 192.168.8.33 romana-gw: 10.2.0.1/16 10.2.0.16 10.2.3.81 10.2.4.6
  24. 24. romana.io Simplifying the network stack with Romana @romanaproject Routing and route aggregation Host A eth0: 192.168.8.11 romana-gw: 10.0.0.1/16 10.0.0.5 10.0.1.7 10.0.1.19 10.0.5.3 Routes: 10.1/16 → 192.168.8.22 10.2/16 → 192.168.8.33 Host B eth0: 192.168.8.22 romana-gw: 10.1.0.1/16 10.1.3.52 10.1.9.2 Routes: 10.0/16 → 192.168.8.11 10.2/16 → 192.168.8.33 Host C eth0: 192.168.8.33 romana-gw: 10.2.0.1/16 10.2.0.16 10.2.3.81 10.2.4.6 Routes: 10.0/16 → 192.168.8.11 10.1/16 → 192.168.8.22
  25. 25. romana.io Simplifying the network stack with Romana @romanaproject Larger network: L2 under ToR Host B1 Host B2 Host B3 Host B4 Host A1 ToR A ToR B spine network 192.168.1.200 192.168.2.200 192.168.1.1 Host A2 192.168.1.2 Host A3 192.168.1.3 Host A4 192.168.1.4 192.168.2.1 192.168.2.2 192.168.2.3 192.168.2.4 Rack A Rack B
  26. 26. romana.io Simplifying the network stack with Romana @romanaproject Larger network: L2 under ToR Host B1 Host B2 Host B3 Host B4 Host A1 ToR A ToR B spine network 192.168.1.200 192.168.2.200 192.168.1.1 Host A2 192.168.1.2 Host A3 192.168.1.3 Host A4 192.168.1.4 10.68/14 10.72/14 10.76/14 10.80/14 192.168.2.1 192.168.2.2 192.168.2.3 192.168.2.4 10.132/14 10.136/14 10.140/14 10.144/14 Rack A Rack B 10.64/10 10.128/10
  27. 27. romana.io Simplifying the network stack with Romana @romanaproject Larger network: L2 under ToR Host B1 Host B2 Host B3 Host B4 Host A1 ToR A ToR B spine network 192.168.1.200 192.168.2.200 192.168.1.1 Host A2 192.168.1.2 Host A3 192.168.1.3 Host A4 192.168.1.4 10.68/14 10.72/14 10.76/14 10.80/14 192.168.2.1 192.168.2.2 192.168.2.3 192.168.2.4 10.132/14 10.136/14 10.140/14 10.144/14 Rack A Rack B 10.64/10 10.128/10 Host A2 Routes 0.0.0.0      192.168.1.200→ 10.68/14     192.168.1.1→ 10.76/14     192.168.1.3→ 10.80/14     192.168.1.4→
  28. 28. romana.io Simplifying the network stack with Romana @romanaproject Larger network: L2 under ToR Host B1 Host B2 Host B3 Host B4 Host A1 ToR A ToR B spine network 192.168.1.200 192.168.2.200 192.168.1.1 Host A2 192.168.1.2 Host A3 192.168.1.3 Host A4 192.168.1.4 10.68/14 10.72/14 10.76/14 10.80/14 192.168.2.1 192.168.2.2 192.168.2.3 192.168.2.4 10.132/14 10.136/14 10.140/14 10.144/14 Rack A Rack B 10.64/10 10.128/10 ToR A Routes 10.128/10    192.168.2.200→ 10.68/14     192.168.1.1→ 10.72/14     192.168.1.2→ 10.76/14     192.168.1.3→ 10.80/14     192.168.1.4→ Host A2 Routes 0.0.0.0      192.168.1.200→ 10.68/14     192.168.1.1→ 10.76/14     192.168.1.3→ 10.80/14     192.168.1.4→
  29. 29. romana.io Simplifying the network stack with Romana @romanaproject Larger network: Full L3 Host B1 Host B2 Host B3 Host B4 Host A1 ToR A ToR B spine network 192.168.1.200 192.168.2.200 192.168.1.1 Host A2 192.168.1.2 Host A3 192.168.1.3 Host A4 192.168.1.4 10.68/14 10.72/14 10.76/14 10.80/14 192.168.2.1 192.168.2.2 192.168.2.3 192.168.2.4 10.132/14 10.136/14 10.140/14 10.144/14 Rack A Rack B 10.64/10 10.128/10 ToR A Routes 10.128/10    192.168.2.200→ 10.68/14     192.168.1.1→ 10.72/14     192.168.1.2→ 10.76/14     192.168.1.3→ 10.80/14     192.168.1.4→ Host Routes 0.0.0.0      192.168.1.200→
  30. 30. Scalable distributed firewall and traffic policies
  31. 31. romana.io Simplifying the network stack with Romana @romanaproject Romana: Traffic segmentation ● Tenant traffic separated: – Tenants don't get whole CIDR prefix or L2 domain – But fully isolated from other tenants' traffic ● Tenants can define segments: – Like tiers, provide isolation and policies ● Use segment and tenant bits in IP addresses: – Apply policies (iptables) based on that – Segments can stretch across hosts
  32. 32. romana.io Simplifying the network stack with Romana @romanaproject Semantic and topological addressing 3 1 3 0 2 9 2 8 2 7 2 6 2 5 2 4 2 3 2 2 2 1 2 0 1 9 1 8 1 7 1 6 1 5 1 4 1 3 1 2 1 1 1 0 9 8 7 6 5 4 3 2 1 0 0 0 0 0 1 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 1 10 Network prefix bits The network prefix. In this example, we are using the 10/8 address space. 6 Host ID Segment ID We currently store tenant ID in upper bits of segment ID. 4 67 Endpoint ID Widths are configurable, don't have to use byte boundaries.
  33. 33. romana.io Simplifying the network stack with Romana @romanaproject Semantic and topological addressing 3 1 3 0 2 9 2 8 2 7 2 6 2 5 2 4 2 3 2 2 2 1 2 0 1 9 1 8 1 7 1 6 1 5 1 4 1 3 1 2 1 1 1 0 9 8 7 6 5 4 3 2 1 0 0 0 0 0 1 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 1 10 Network prefix bits The network prefix. In this example, we are using the 10/8 address space. 6 Host ID Segment ID We currently store tenant ID in upper bits of segment ID. 4 67 Endpoint ID Widths are configurable, don't have to use byte boundaries. Encode the tenant ID
  34. 34. romana.io Simplifying the network stack with Romana @romanaproject Host BHost A Allowing traffic within tenant 10.0.0.5 10.1.0.12 iptables: check src/dst addrs “tenant/segment bits must match” Src: 10.0.0.5 Dst: 10.1.0.12 Same tenant/segment bits
  35. 35. romana.io Simplifying the network stack with Romana @romanaproject Host BHost A Isolating tenant traffic: Default 10.0.0.5 10.1.128.9 iptables: check src/dst addrs “tenant/segment bits must match” Src: 10.0.0.5 Dst: 10.1.128.9 Different tenant/segment bits Different tenant
  36. 36. romana.io Simplifying the network stack with Romana @romanaproject Host BHost A Apply network policy between segments (full isolation as default) 10.0.0.5 10.1.1.9 iptables: Does policy chain exist? Otherwise: DROP Src: 10.0.0.5 Dst: 10.1.1.9 Same tenant, different segment policy-chain: From segment 0? Protocol TCP? To port 80?
  37. 37. Demo 1: Kubernetes + Romana cluster on top of Catalyst OpenStack cloud
  38. 38. romana.io Simplifying the network stack with Romana @romanaproject Baking layered cakes ● Kubernetes on OpenStack? Why? – On demand clusters – Full tenant isolation ● Not all workloads fit into containers – Seamless connection between pods and VMs ● Really nice with fully routed networking – No double encapsulation – Logical, efficient packet forwarding
  39. 39. romana.io Simplifying the network stack with Romana @romanaproject Demo 1 - Overview
  40. 40. romana.io Simplifying the network stack with Romana @romanaproject Demo 1 - Overview bar-1 bar-2foo Jump host with public IP address
  41. 41. romana.io Simplifying the network stack with Romana @romanaproject
  42. 42. romana.io Simplifying the network stack with Romana @romanaproject Demo 1 - Overview bar-1 bar-2foo
  43. 43. romana.io Simplifying the network stack with Romana @romanaproject Demo 1 - Overview bar-1 bar-2foo Install OpenStack command line tools
  44. 44. romana.io Simplifying the network stack with Romana @romanaproject Demo 1 - Overview bar-1 bar-2foo $ neutron port-update e925b70e-031e-4ef7-a27c-583b4b775290 --allowed-address-pairs type=dict list=true mac_address=fa:16:3e:e1:df:59,ip_address=10.0.0.0/8
  45. 45. romana.io Simplifying the network stack with Romana @romanaproject Demo 1 - Overview bar-1 bar-2foo $ git clone https://github.com/romana/romana $ cd romana/romana-install $ ./romana-setup -p static -i my-inventory -s kubernetes install
  46. 46. romana.io Simplifying the network stack with Romana @romanaproject Demo 1 - Overview bar-1 bar-2foo Romana installer
  47. 47. romana.io Simplifying the network stack with Romana @romanaproject Demo 1 - Overview bar-1 bar-2foo Kubernetes + Romana Romana cluster address range: 10/8
  48. 48. romana.io Simplifying the network stack with Romana @romanaproject Demo 1 - Overview bar-1 bar-2foo Kubernetes + Romana Pods with containers. Pods have Romana IP addresses.
  49. 49. romana.io Simplifying the network stack with Romana @romanaproject Demo 1 - What you will see ● Creation of pods ● Network configuration ● Application of network policies
  50. 50. Demo 2: Mixing containers with legacy workloads
  51. 51. romana.io Simplifying the network stack with Romana @romanaproject Demo 2 - Overview bar-1 bar-2foo Kubernetes + Romana
  52. 52. romana.io Simplifying the network stack with Romana @romanaproject Demo 2 - Overview bar-1 bar-2foo Kubernetes + Romana vm-workload Legacy application in VM
  53. 53. romana.io Simplifying the network stack with Romana @romanaproject Demo 2 - Overview bar-1 bar-2foo Kubernetes + Romana vm-workload Direct connection: - No gateway - No encap/decap - No NAT
  54. 54. romana.io Simplifying the network stack with Romana @romanaproject Demo 2 - What you will see ● Creation of pods ● Contact pod from VM ● See the packet route
  55. 55. Demo 3: Romana + Kubernetes cluster on top of Romana + OpenStack cluster
  56. 56. romana.io Simplifying the network stack with Romana @romanaproject Demo 3 - Overview HW1 HW2 HW3 HW4
  57. 57. romana.io Simplifying the network stack with Romana @romanaproject Demo 3 - Overview HW1 HW2 HW3 HW4 $ ./romana-setup -p static -i hw-inventory -s devstack install
  58. 58. romana.io Simplifying the network stack with Romana @romanaproject Demo 3 - Overview HW1 HW2 HW3 HW4 OpenStack + Romana Romana cluster 1 address range: 10/8
  59. 59. romana.io Simplifying the network stack with Romana @romanaproject Demo 3 - Overview VM2 VM3VM1 HW1 HW2 HW3 HW4 OpenStack + Romana OpenStack VMs VMs have IP addresses of Romana cluster 1
  60. 60. romana.io Simplifying the network stack with Romana @romanaproject Demo 3 - Overview VM2 VM3VM1 HW1 HW2 HW3 HW4 OpenStack + Romana $ ./romana-setup -p static -i vm-inventory -s kubernetes install
  61. 61. romana.io Simplifying the network stack with Romana @romanaproject Demo 3 - Overview VM2 VM3 Kubernetes + Romana VM1 HW1 HW2 HW3 HW4 OpenStack + Romana Romana cluster 2 address range: 172.16/12
  62. 62. romana.io Simplifying the network stack with Romana @romanaproject Demo 3 - Overview VM2 VM3 Kubernetes + Romana VM1 HW1 HW2 HW3 HW4 OpenStack + Romana Pods with containers. Pods have IP addresses of Romana cluster 2
  63. 63. romana.io Simplifying the network stack with Romana @romanaproject OpenStack + Romana Kubernetes + Romana Demo 3 - Overview VM2 VM3VM1 HW1 HW2 HW3 HW4
  64. 64. romana.io Simplifying the network stack with Romana @romanaproject OpenStack + Romana Kubernetes + Romana Demo 3 - Overview VM2 VM3VM1 HW1 HW2 HW3 HW4 Remember this one? 2 Top of Rack Round Trips East/West Traffic Per Instance Security Without pure L3 network layered clusters would be even more complex.
  65. 65. romana.io Simplifying the network stack with Romana @romanaproject OpenStack + Romana Kubernetes + Romana Demo 3 - Overview VM2 VM3VM1 HW1 HW2 HW3 HW4 But with Romana, networking even in layered clusters becomes really easy...
  66. 66. romana.io Simplifying the network stack with Romana @romanaproject Demo 3 - What you will see ● Creation of pods ● Pods and VMs with fully routable addresses ● Ease of use showcase: Trouble shooting
  67. 67. romana.io Simplifying the network stack with Romana @romanaproject Conclusion ● Cloud native architectures simplify things ● Need cloud native networking to enjoy benefits ● Romana: – Cloud native without compromises – Native network performance – Mostly static config: Solid network – Very easy to work with and understand ● Easy to try: – Simple installers for Kubernetes and OpenStack
  68. 68. romana.io Simplifying the network stack with Romana @romanaproject Thank you! ● Romana Links – http://romana.io - Project home – http://romana.io/blog - Blog – https://github.com/romana/romana - Sources ● Contact – @romanaproject - Twitter – info@romana.io - Email – https://romana.slack.com/ - Slack channel

×