Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cilium - Network security for microservices

1,736 views

Published on

Cilium is open source software for providing and transparently securing network connectivity and loadbalancing between application containers and services deployed using Linux container management platforms like Docker and Kubernetes.

A new Linux kernel technology called eBPF is at the foundation of Cilium, which enables the dynamic insertion of BPF bytecode into the Linux kernel. Cilium generates eBPF programs for each individual application container to provide networking, security, loadbalancing and visibility.

Published in: Software

Cilium - Network security for microservices

  1. 1. Cilium L7 Aware Network Security for Microservices using BPF & XDP Thomas Graf Covalent
  2. 2. BPF - The Superpowers inside Linux
  3. 3. BPF is revolutionizing…
  4. 4. BPF is revolutionizing… • Tracing / Profiling
  5. 5. BPF Revolution #1: Tracing / Profiling bcc – Tools for BPF-based Linux IO analysis, networking, monitoring, and more
  6. 6. BPF is revolutionizing… • Tracing / Profiling • Networking
  7. 7. BPF Revolution #2: XDP - DDoS mitigation Metric iptables / ipset XDP DDoS rate [packets/s] 11.6M 11.6M Drop rate [packets/s] 7.1M 11.6M Time to load rules [time] 3 min 20 sec 31 sec Latency under load [ms] 2.3ms 0.1ms Throughput under DDoS [Gbit/s] 0.014 6.5 Requests/s under DDoS [kReq/s] 0.28 82.8 Sender: Send 64B packets as fast as possible è Receiver: Drop as fast as possible Source: Daniel Borkmann’s presentation yesterday: http://schd.ws/hosted_files/ossna2017/da/BPFandXDP.pdf
  8. 8. Facebook published BPF/XDP numbers for L3/L4 LB at Netdev 2.1 ECMP L7 LBL3/L4 LB App
  9. 9. Source: https://www.netdevconf.org/2.1/slides/apr6/zhou-netdev-xdp-2017.pdf Facebook published BPF/XDP numbers for L3/L4 LB at Netdev 2.1 BPF/XDP throughput IPVS throughput
  10. 10. BPF Revolution #2: XDP Running BPF programs at the Network driver level https://netdevconf.org/2.1/slides/apr7/miller-XDP-MythBusters.pdf Netdev 2.1 Keynote by David S. Miller
  11. 11. BPF Revolution #2: XDP Running BPF programs at the Network driver level https://netdevconf.org/2.1/slides/apr7/miller-XDP-MythBusters.pdf Netdev 2.1 Keynote by David S. Miller
  12. 12. BPF is revolutionizing… • Tracing / Profiling • Networking • Security
  13. 13. Source: https://lwn.net/Articles/703876/ BPF Revolution #3: Security
  14. 14. Network Security for Microservices using BPF
  15. 15. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low Evolution of Application Design & Delivery Frequency
  16. 16. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low 3-Tier App Monthly Moderate Evolution of Application Design & Delivery Frequency
  17. 17. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low Distributed Microservices 10-100 x’s / day Extreme 3-Tier App Monthly Moderate Evolution of Application Design & Delivery Frequency
  18. 18. Network Security has barely evolved $ iptables -A INPUT -p tcp -s 15.15.15.3 --dport 80 -m conntrack --ctstate NEW -j ACCEPT The world still runs on iptables matching IPs and ports:
  19. 19. Your HTTP ports be like …
  20. 20. Network Security for Microservices Gordon the intern has a brilliant idea…
  21. 21. Gordon wants to build a service to tweet out all job offerings. We’re Hiring! Tweet Service
  22. 22. GET /healthz GET /jobs/{id} GET /applicants/{job-id} POST /jobs API GET /jobs/{id} Jobs API Service Tweet Service The Jobs API service has all the data Gordon needs.
  23. 23. GET /healthz GET /jobs/{id} GET /applicants/{job-id} POST /jobs API GET /jobs/331 GET /jobs/{id} Jobs API Service Tweet Service Gordon uses the GET /jobs/ API call
  24. 24. GET /healthz GET /jobs/{id} GET /applicants/{job-id} POST /jobs API GET /jobs/331 GET /jobs/{id} TLS Jobs API Service Tweet Service Developer etiquette. Super simple stuff. Gordon uses mutual TLS Auth Good thinking Gordon
  25. 25. L3/L4 GET /healthz GET /jobs/{id} GET /applicants/{job-id} POST /jobs API GET /jobs/331 The security team has L3/L4 network security in place for all services GET /jobs/{id} Jobs API Service Tweet Service TLS iptables -s 10.1.1.1 -p tcp --dport 80 -j ACCEPT
  26. 26. Jobs API Service L3/L4 GET /healthz GET /jobs/{id} GET /applicants/{job-id} POST /jobs API exposed exposed exposed GET /jobs/331 Large parts of the API are still exposed unnecessarily Tweet Service GET /jobs/{id} TLS iptables -s 10.1.1.1 -p tcp --dport 80 -j ACCEPT
  27. 27. Not exactly least privilege Security
  28. 28. GET /healthz GET /jobs/{id} GET /applicants/{job-id} POST /jobs API GET /jobs/331 Back to the drawing board… GET /jobs/{id} TLS Jobs API Service Tweet Service
  29. 29. L3/L4 GET /healthz GET /jobs/{id} GET /applicants/{job-id} POST /jobs API GET /jobs/331 Least privilege security for microservices GET /jobs/{id} FROM “TurtleTweets” ALLOW “GET /jobs/” TLS Jobs API Service Tweet Service
  30. 30. We demand a demo
  31. 31. Kubernetes Integration
  32. 32. Kubernetes Integration NetworkPolicy Standard Resources L3, L4 policy (ingress only in k8s 1.7)
  33. 33. Kubernetes Integration NetworkPolicy Services Standard Resources L3, L4 policy (ingress only in k8s 1.7) ClusterIP, NodePort, LoadBalancer
  34. 34. Kubernetes Integration NetworkPolicy Services Standard Resources L3, L4 policy (ingress only in k8s 1.7) Pods Pod Labels to specify policy on ClusterIP, NodePort, LoadBalancer
  35. 35. Kubernetes Integration NetworkPolicy Services Standard Resources L3, L4 policy (ingress only in k8s 1.7) Nodes Pods Pod Labels to specify policy on ClusterIP, NodePort, LoadBalancer NodeIP to Node CIDR mapping
  36. 36. Kubernetes Integration NetworkPolicy CiliumNetworkPolicy Services Standard Resources Custom Resource Definitions (CRD) L3, L4 policy (ingress only in k8s 1.7) L3 (Labels/CIDR), L4, L7 (ingress & egress) Nodes Pods Pod Labels to specify policy on ClusterIP, NodePort, LoadBalancer NodeIP to Node CIDR mapping
  37. 37. Should I encapsulate or not? Node 1 Node 2 Node 3 Mode I: Overlay
  38. 38. Should I encapsulate or not? Node 1 Node 2 Node 3 Mode I: Overlay Name NodeIP Node CIDR Node 1 192.168.10.1 10.0.1.0/24 Node 2 192.168.10.8 10.0.2.0/24 Node 3 192.168.10.9 10.0.3.0/24 Kubernetes Node resources table: Installation Run the kube-controller- manager with the --allocate- node-cidrs option
  39. 39. Should I encapsulate or not? Mode I: Overlay Mode II: Native Routing Node 1 Node 2 Node 3 L3 Network Use case: • Run your own routing daemon • Use the cloud provider’s router Use case: • Simple • “Just works” on Kubernetes Node 1 Node 2 Node 3
  40. 40. L3 Policy (Labels Based) Metadata Allow from pods Pods the policy applies to… From Pod To Pod
  41. 41. L3 Policy (CIDR) Metadata Allow to IP 8.8.8.8/32 Pods the policy applies to… To CIDR From Pod
  42. 42. L4 Policy Metadata Policy applies to pods … Allow incoming on port 80 Pod To Port
  43. 43. L7 Policy – Only allow “GET /v1/” L4 Policy Rule 1: Allow “GET /v/1” Rule 2: Allow PUT If header is set Allowed API Calls
  44. 44. How are these policies enforced?
  45. 45. How are these policies enforced? • L3 & L4: BPF in the kernel
  46. 46. How are these policies enforced? • L3 & L4: BPF in the kernel • L7: Sidecar proxy or KProxy / BPF
  47. 47. Node 2Node 1 ServiceService HTTP Request What is a sidecar proxy?
  48. 48. Node 1 Service Sidecar Proxy What is a sidecar proxy? Node 2 Service Sidecar Proxy
  49. 49. Node 1 Service Sidecar Proxy What is a sidecar proxy? Node 2 Service Sidecar Proxy
  50. 50. Node 2Node 1 ServiceService HTTP RequestSidecar Proxy Sidecar Proxy What is a sidecar proxy?
  51. 51. Node 2Node 1 ServiceService HTTP RequestSidecar Proxy Sidecar Proxy What is a sidecar proxy? Provides L7 functionality • Routing / Load balancing • Retries • Circuit breaking • Metrics More info? Google is your friend “sidecar” / “services mesh”
  52. 52. Node 2Node 1 Service Operating System Service Network Sidecar Proxy Sidecar Proxy Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP • 3x Socket memory requirement • 3x TCP/IP stack traversals • 3x Context switches • Complexity Networking Path with a Sidecar Network
  53. 53. Can we turn the sidecar into a racecar?
  54. 54. Node 2Node 1 Task Operating System Kernel Proxy Task Network Socket KProxy with BPF TCP/IP Socket TCP/IP KProxy with BPF kTLS kTLS Sidecar Proxy Sidecar Proxy Network
  55. 55. Socket Redirect Task Socket Socket Task TCP/IP TCP/IP Loopback
  56. 56. Socket Redirect Task Socket Socket Task TCP/IP TCP/IP Loopback
  57. 57. Socket Redirect – Performance?
  58. 58. Node 2Node 1 Service Operating System Service Network Sidecar Proxy Sidecar Proxy Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP The Before and After Network
  59. 59. Node 1 Node 2 Service Operating System Service Network Socket TCP/IP The Before and After KProxy Socket TCP/IP KProxy Network
  60. 60. Cilium Summary • Kubernetes, Mesos, Docker • CNI / libnetwork • Networking: Overlay or Native Routing • Network Security (ingress/egress) • L3 (Identity or CIDR), L4 • L7: HTTP (0.11), gRPC (0.12), Mongo (0.12) • Load Balancing (XDP / BPF) • Dependencies: kvstore (etcd / consul)
  61. 61. @ciliumproject http://github.com/cilium/cilium Thank You! Questions? Tutorial / Getting Started: http://cilium.io/try

×