Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cilium: Kernel Native Security & DDOS Mitigation for Microservices with BPF

749 views

Published on

We have introduced Cilium at DockerCon US 2017 this year. Cilium provides application-aware network connectivity, security, and load-balancing for containers. This talk will follow up on the introduction and deep dive into recent kernel developments that address two fundamental questions: How can I provide application-aware security and routing efficiently without overhead embedded into every service? How can container hosts protect themselves from internal and external DDoS attacks? The solutions include:

kproxy: a kernel-based socket proxy which allows for application-aware routing and security enforcement with minimal overhead.

XDP: A lightning-fast packet processing datapath using BPF. The technology is intended for DDoS mitigation, load-balancing, and forwarding.

This talk will deep dive into these exciting technologies and show how Cilium makes BPF and these kernel features available on Linux for your Docker containers.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Cilium: Kernel Native Security & DDOS Mitigation for Microservices with BPF

  1. 1. Cilium – Kernel Native Security & DDOS Mitigation for Microservices with BPF Cynthia Thomas Technology Evangelist, Covalent @_techcet_
  2. 2. Gordon is back! And he got a job at Lego.
  3. 3. Robot Competition ● Local marketing launches an Robot competition for Danish residents ● Upcoming deadline means tons of uploaded media
  4. 4. Robot Competition User Data Mongo DB Data store Inventory Image Upload Service Web Front- End Kafka Broker Kafka Broker Kafka Broker Image processing Image processing Image processing ZookeeperZookeeperZookeeper End- User Admin task Audit
  5. 5. Gordon’s Task List ● Deploy Cilium HTTP-aware security for microservices ● Isolate Kafka resources for old & new services ● DDOS mitigation via XDP/BPF
  6. 6. Gordon’s 1st Task Update HTTP Security for Microservices
  7. 7. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low Evolution of Application Design & Delivery Frequency
  8. 8. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low 3-Tier App Monthly Moderate Evolution of Application Design & Delivery Frequency
  9. 9. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low Distributed Microservices 10-100 x’s / day Extreme 3-Tier App Monthly Moderate Evolution of Application Design & Delivery Frequency
  10. 10. The world still runs on iptables matching IPs and ports: $ iptables -A INPUT -p tcp -s 15.15.15.3 --dport 80 -m conntrack --ctstate NEW -j ACCEPT
  11. 11. Robot Competition User Data Mongo DB Data store Inventory Image Upload Service Web Front- End Kafka Broker Image processing Zookeeper End- User Admin task Audit Example: HTTP calls
  12. 12. API Web Front-End Security for Microservices Image Upload Service GET /image POST /image GET /image/flagged PUT /image/id
  13. 13. API POST /image Web Front-End Security for Microservices Image Upload Service GET /image GET /image/flagged PUT /image/id POST /image
  14. 14. L3/L4 API POST /image Web Front-End Security for Microservices iptables -s 10.1.1.1 -p tcp --dport 80 -j ACCEPT Image Upload Service GET /image POST /image GET /image/flagged PUT /image/id
  15. 15. L3/L4 API POST /image Web Front-End Security for Microservices iptables -s 10.1.1.1 -p tcp --dport 80 -j ACCEPT exposed exposed exposed Image Upload Service GET /image POST /image GET /image/flagged PUT /image/id
  16. 16. Network Security has not evolved for microservices… …until now.
  17. 17. API POST /image Web Front-End Cilium API-Aware Security Image Upload Service GET /image GET /image/flagged PUT /image/id POST /image
  18. 18. L7 API POST /image Web Front-End Cilium API-Aware Security Image Upload Service GET /image POST /image GET /image/flagged PUT /image/id FROM Web Front-End ALLOW POST /image
  19. 19. Under the Hood: BPF Linux Superpowers Unleashed
  20. 20. BPF instruction set struct bpf_insn prog[] = { BPF_MOV64_REG(BPF_REG_6, BPF_REG_1), BPF_LD_ABS(BPF_B, ETH_HLEN + offsetof(struct iphdr, protocol) /* R0 = ip->proto */), BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_0, -4), /* *(u32 *)(fp - 4) = r0 */ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), /* r2 = fp - 4 */ BPF_LD_MAP_FD(BPF_REG_1, map_fd), BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 2), BPF_MOV64_IMM(BPF_REG_1, 1), /* r1 = 1 */ BPF_RAW_INSN(BPF_STX | BPF_XADD | BPF_DW, BPF_REG_0, BPF_REG_1, 0, 0), /* xadd r0 += r1 */ BPF_MOV64_IMM(BPF_REG_0, 0), /* r0 = 0 */ BPF_EXIT_INSN(), }; Reference: https://git.kernel.org/pub/scm/linux/kernel/git/ast/bpf.git/tree/samples/bpf/sock_example.c
  21. 21. Cilium Cilium Agent
  22. 22. User space Kernel space Cilium Architecture Cilium Agent
  23. 23. API Calls User space Kernel space Plugins Cilium Architecture Cilium Agent
  24. 24. BPF API Calls BPF Code & Maps User space Kernel space Plugins Cilium Architecture Cilium Agent
  25. 25. BPF API Calls BPF Code & Maps User space Kernel space Plugins Cilium Architecture Cilium Agent Extremely scalable, highly customized network filtering, load- balancing, and monitoring
  26. 26. BPF BPF API Calls BPF Code & Maps User space Kernel space Plugins Cilium Architecture Cilium Agent
  27. 27. BPF BPF API Calls BPF Code & Maps Packets User space Kernel space Plugins Cilium Architecture Cilium Agent
  28. 28. BPF BPF BPF API Calls BPF Code & Maps Packets User space Kernel space Plugins Cilium Architecture Cilium Agent
  29. 29. BPF BPF BPF API Calls BPF Code & Maps Packets User space Kernel space Plugins Cilium Architecture Cilium Agent
  30. 30. BPF BPF BPF API Calls BPF Code & Maps Packets User space Kernel space CLI, Monitoring, Policies Plugins Cilium Architecture Cilium Agent
  31. 31. Gordon’s 2nd Task Isolate Kafka Topics
  32. 32. Kafka? Used for building real-time pipelines and streaming apps. - Horizontally scalable - Fault-tolerant - “Wicked fast” Defined by its own protocol. 1/3 of all Fortune 500 companies use Kafka[1]
  33. 33. Kafka Concepts Topic 1 Producers Kafka Broker Topic 1 Topic N Topic 1 Consumer Group A Topic 1 Consumer Group B
  34. 34. Security for Microservices Gordon worries about open Kafka topics The API is too open! How do I bring on new services?
  35. 35. Robot Competition User Data Mongo DB Data store Inventory Image Upload Service Web Front- End Kafka Broker Kafka Broker Kafka Broker Image processing Image processing Image processing ZookeeperZookeeperZookeeper End- User Admin task Audit Example: Kafka calls
  36. 36. Kafka. API Image processing Security for Microservices Kafka Broker Fetch inventory Fetch image Fetch userid Fetch analytics Produce inventory Produce image Produce userid Produce analytics
  37. 37. Kafka. API Fetch image Image processing Security for Microservices Kafka Broker Fetch inventory Fetch image Produce userid Fetch userid
  38. 38. L3/L4 Kafka. API Fetch image Image processing Security for Microservices iptables -s 10.1.1.1 -p tcp --dport 9092 -j ACCEPT Kafka Broker Fetch inventory Fetch image Produce userid Fetch userid
  39. 39. L3/L4 Kafka. API Fetch image Image processing Security for Microservices iptables -s 10.1.1.1 -p tcp --dport 9092 -j ACCEPT exposed exposed exposed Kafka Broker Fetch inventory Fetch image Produce userid Fetch userid
  40. 40. Kafka. API Fetch image Image processing Security for Microservices Kafka Broker Fetch inventory Fetch image Produce userid Fetch userid
  41. 41. L7 Kafka. API Fetch image Image processing Security for Microservices Kafka Broker Fetch inventory Fetch image Produce userid Fetch userid FROM Image processing ALLOW Fetch image
  42. 42. Demo! Get real with Gordon
  43. 43. Demo: Kafka API Filtering app1- producer app1- consumer app2- producer app2- consumer kafka
  44. 44. What just happened?
  45. 45. L7 Kafka. API Produce imageprocessing app1- producer Demo: Kafka API Filtering Kafka Broker Produce imageprocessing Produce userdata FROM app1-producer ALLOW Produce imageprocessing
  46. 46. BPF BPF BPF API Calls BPF Code & Maps Packets User space Kernel space CLI, Monitoring, Policies Plugins Sidecar Proxy Cilium Agent Sidecar Proxy
  47. 47. BPF BPF BPF API Calls BPF Code & Maps Packets User space Kernel space CLI, Monitoring, Policies Plugins In-Kernel Kafka Parsing Cilium Agent kernel proxy BPF
  48. 48. Gordon’s 3rd Task DDOS Mitigation
  49. 49. Robot Competition ● Local marketing launches competition for Danish residents only ● DDOS anticipated from other countries and anti-Lego activists
  50. 50. Robot Competition User Data Mongo DB Data store Inventory Image Upload Service Web Front- End Kafka Broker Image processing Zookeeper End- User Admin task Audit Example: DDOS attack!
  51. 51. BPF BPF BPF API Calls BPF Code & Maps Packets User space Kernel space CLI, Monitoring, Policies Plugins XDP/BPF handled in NIC driver Cilium Agent XDP
  52. 52. Source: https://www.netdevconf.org/2.1/slides/apr6/zhou-netdev-xdp-2017.pdf Facebook published BPF/XDP numbers for L3/L4 LB at Netdev 2.1 BPF/XDP throughput IPVS throughput
  53. 53. BPF with XDP Setup pktgen attack: ~11.6 Mpps randomly in 10.0.0.0/8 legit traffic: netperf tests on 10.192.1.0/24 Blacklist 16M rules All /32s in 10.0.0.0/8
  54. 54. BPF with XDP for DDoS mitigation Metric iptables / ipset XDP DDoS rate [packets/s] 11.6M 11.6M Drop rate [packets/s] 7.1M 11.6M Time to load rules [time] 3 min 20 sec 31 sec Latency under load [ms] 2.3ms 0.1ms Throughput under DDoS [Gbit/s] 0.014 6.5 Requests/s under DDoS [kReq/s] 0.28 82.8 Sender: Send 64B packets as fast as possible è Receiver: Drop as fast as possible Source: Daniel Borkmann’s presentation: http://schd.ws/hosted_files/ossna2017/da/BPFandXDP.pdf
  55. 55. ● Cilium deployment for microservices successfully secured HTTP traffic ● Kafka resources were isolated to protect existing services from new ones ● Mitigated DDOS attacks via XDP/BPF Gordon’s Summary
  56. 56. Cilium Project Status • Cilium v0.12 release in October • Docker, Kubernetes, and Mesos integration • Looking for feedback and contributions
  57. 57. Take Action! • Getting Started Using Docker: docs.cilium.io/ • Join our Slack community! • Check out the project website for more details: https://www.cilium.io/ Please ★ us on GitHub
  58. 58. Thank You! github.com/cilium/cilium cilium.io @ciliumproject

×