Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP

5,923 views

Published on

This talk will start with a deep dive and hands on examples of BPF, possibly the most promising low level technology to address challenges in application and network security, tracing, and visibility. We will discuss how BPF evolved from a simple bytecode language to filter raw sockets for tcpdump to the a JITable virtual machine capable of universally extending and instrumenting both the Linux kernel and user space applications. The introduction is followed by a concrete example of how the Cilium open source project applies BPF to solve networking, security, and load balancing for highly distributed applications. We will discuss and demonstrate how Cilium with the help of BPF can be combined with distributed system orchestration such as Docker to simplify security, operations, and troubleshooting of distributed applications.

Published in: Software
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP

  1. 1. CILIUM: NETWORK AND APPLICATION SECURITY WITH BPF AND XDP Thomas Graf Co-founder & CTO Covalent
  2. 2. Who is this guy?
  3. 3. Helped build the biggest monolith ever … Who is this guy?
  4. 4. Helped build the biggest monolith ever … Who is this guy?
  5. 5. Time to rethink the kernel
  6. 6. syscalls syscalls Net IOBlock IO Time to rethink the kernel
  7. 7. Time to rethink the kernel From monolith to “microkernel” with BPF syscalls syscalls BPF BPF BPF BPF Net IOBlock IO
  8. 8. Time to rethink the kernel From monolith to “microkernel” with BPF syscalls syscalls BPF BPF BPF BPF BPF BPF Security Networking Net IOBlock IO
  9. 9. BPF is revolutionizing… • Tracing / Profiling
  10. 10. BPF is revolutionizing… • Tracing / Profiling Container Performance Analysis Brendan Gregg Wed 1:30pm “Black Belt”
  11. 11. BPF is revolutionizing… • Tracing / Profiling • Networking Container Performance Analysis Brendan Gregg Wed 1:30pm “Black Belt”
  12. 12. BPF is revolutionizing… • Tracing / Profiling • Networking • Security Container Performance Analysis Brendan Gregg Wed 1:30pm “Black Belt”
  13. 13. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low Delivery Frequency
  14. 14. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low 3-Tier App Monthly Moderate Delivery Frequency
  15. 15. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low Distributed Microservices 10-100 x’s / day Extreme 3-Tier App Monthly Moderate Delivery Frequency
  16. 16. Network Security has not evolved $ iptables -A INPUT -p tcp -s 15.15.15.3 --dport 80 -m conntrack --ctstate NEW -j ACCEPT The world still runs on iptables matching IPs and ports:
  17. 17. Your HTTP ports be like …
  18. 18. Network Security for Microservices Example Gordon is looking for a job…
  19. 19. Gordon Job Postings Example: Security for Microservices
  20. 20. GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API Gordon Job Postings Example: Security for Microservices
  21. 21. GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API GET /jobs/331 Gordon Job Postings Example: Security for Microservices
  22. 22. L3/L4 GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API iptables -s 10.1.1.1 -p tcp --dport 80 -j ACCEPT GET /jobs/331 Gordon Job Postings Example: Security for Microservices
  23. 23. L3/L4 GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API exposed exposed exposed GET /jobs/331 Gordon Job Postings Example: Security for Microservices iptables -s 10.1.1.1 -p tcp --dport 80 -j ACCEPT
  24. 24. Not exactly least privilege Security team is not amused
  25. 25. GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API GET /jobs/331 Gordon Job Postings Example: Security for Microservices
  26. 26. L3/L4 GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API FROM Gordon ALLOW GET /jobs/.* GET /jobs/331 Gordon Job Postings Example: Security for Microservices
  27. 27. We demand a demo
  28. 28. BPF - The Superpowers inside Linux
  29. 29. SANDBOX BPF GET /foo BPF: Transparent redirection into proxy
  30. 30. SANDBOX BPF Proxy GET /foo redirect rules sk BPF: Transparent redirection into proxy
  31. 31. SANDBOX BPF Proxy GET /foo redirect rules sk Shared State • Orig Dest IP • Identity BPF: Transparent redirection into proxy
  32. 32. SANDBOX BPF Proxy GET /foo redirect reinject rules sk sk Shared State • Orig Dest IP • Identity BPF: Transparent redirection into proxy
  33. 33. SANDBOX BPF Proxy GET /foo rules BPF: Transparent redirection into proxy sk sk 403 Access Denied
  34. 34. So what is BPF exactly?
  35. 35. .insns = { BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8), BPF_ST_MEM(BPF_DW, BPF_REG_2, 0, 0), BPF_LD_MAP_FD(BPF_REG_1, 0), BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem), BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -152), BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0), BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 2), BPF_LDX_MEM(BPF_DW, BPF_REG_3, BPF_REG_1, 0), BPF_ST_MEM(BPF_DW, BPF_REG_3, 0, 42), BPF_EXIT_INSN(), } What is BPF? Learn more about BPF: docs.cilium.io
  36. 36. BPF: Toolchain – from user to kernel USER SPACE SOURCE CODE [C] </>
  37. 37. BPF: LLVM compiles program code to bytecode USER SPACE SOURCE CODE [C] </> BYTE CODE [BPF] </>
  38. 38. BPF: Bytecode is loaded and verified into kernel USER SPACE KERNELVERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </>
  39. 39. BPF: Bytecode runs inside safe kernel sandbox USER SPACE KERNELVERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> SANDBOX BPF
  40. 40. BPF: Program is attached to event (packet-in) USER SPACE KERNELVERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> SANDBOX BPF
  41. 41. BPF: Program can redirect to netns & sockets USER SPACE KERNELVERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> SANDBOX BPF
  42. 42. BPF – An opportunity to rethink security policy enforcement
  43. 43. Status Quo: Policy Enforcement connect()
  44. 44. Status Quo: Policy Enforcement connect() TCP
  45. 45. Status Quo: Policy Enforcement connect() TCP Network packets
  46. 46. Status Quo: Policy Enforcement connect() TCP Network packets veth veth namespace boundary
  47. 47. Status Quo: Policy Enforcement connect() TCP Network packets iptables veth veth namespace boundary
  48. 48. Status Quo: Policy Enforcement connect() drop TCP Network packets iptables veth veth namespace boundary
  49. 49. Status Quo: Policy Enforcement connect() drop TCP Network packets ETIMEDOUT iptables veth veth namespace boundary
  50. 50. Status Quo: Policy Enforcement connect() drop TCP Network packets ETIMEDOUT/ ECONNREFUSED iptables RST veth veth namespace boundary
  51. 51. Can we do better? connect()
  52. 52. BPF: Leverage user space tool chain USER SPACE KERNEL connect() VERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </>
  53. 53. BPF: Attach program to connect() syscall (LSM) USER SPACE KERNEL connect() VERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> BPF LSM Hook
  54. 54. BPF: Return EACCESS – No packets created at all USER SPACE KERNEL connect() EACCESS VERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> BPF LSM Hook
  55. 55. XDP/BPF – The software loadbalancer of the future
  56. 56. WHAT IF I TOLD YOU XDP allows for 10x IPVS performance
  57. 57. Source: https://www.netdevconf.org/2.1/slides/apr6/zhou-netdev-xdp-2017.pdf FB moves from IPVS to BPF/XDP for L3/L4 LB XDP throughput IPVS throughput Source:
  58. 58. Regular BPF mode BPF Driver Software Stack
  59. 59. XDP [Express Data Path] mode BPF Driver Run BPF Program inside network driver with access to DMA buffer Software Stack
  60. 60. XDP [Express Data Path] mode BPF Driver Can drop millions of packets per Second while under DDoS Software Stack drop
  61. 61. XDP [Express Data Path] mode BPF Driver Can pass packets to network stack Software Stack drop Stack
  62. 62. XDP [Express Data Path] mode BPF Driver Can perform loadbalancing and transmit out the wire again Software Stack drop LB & TX Stack
  63. 63. How can I use BPF with Docker?
  64. 64. Cilium Architecture Cilium Agent
  65. 65. Cilium Architecture Cilium Agent Plugins
  66. 66. Cilium Architecture BPF Cilium Agent Plugins
  67. 67. Cilium Architecture BPF BPF Cilium Agent Plugins
  68. 68. Cilium Architecture BPF BPF Cilium Agent Plugins
  69. 69. Cilium Architecture BPF BPF BPF Cilium Agent Plugins
  70. 70. Cilium Architecture BPF BPF BPF Cilium Agent Plugins
  71. 71. Cilium Architecture BPF BPF BPF Cilium Agent CLI Monitor Policy Plugins
  72. 72. Project Status • Initial release two weeks ago • Docker & Kubernetes integration • Looking for feedback and contributions
  73. 73. Getting Started • Play with our vagrant box: $ git clone https://github.com/cilium/cilium $ cd cilium/examples/getting-started $ vagrant up
  74. 74. Summary
  75. 75. Summary • Never underestimate the Jedi
  76. 76. Summary • Never underestimate the Jedi • Traditional L3/L4 network policies are insufficient for microservices. Least privilege requires HTTP / API / Function awareness.
  77. 77. Summary • BPF/XDP will drive the future of software based networking on Linux. • Never underestimate the Jedi • Traditional L3/L4 network policies are insufficient for microservices. Least privilege requires HTTP / API / Function awareness.
  78. 78. Summary • Never underestimate the Jedi • Traditional L3/L4 network policies are insufficient for microservices. Least privilege requires HTTP / API / Function awareness. • BPF/XDP will drive the future of software based networking on Linux. • Cilium brings BPF/XDP and L7 policies to containers and microservices.
  79. 79. Thank You! github.com/cilium/cilium http://cilium.io/ @ciliumproject Want to chat? DM me! @tgraf__ Don’t forget to vote and grab a shirt on the way out!
  80. 80. 75 140 205 240 325 365 370 365 410 412 425 445 450 460 460 490 495 505 515 525 545 565 0 100 200 300 400 500 600 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 BPF redirect() performance [GBit per core] Intel Xeon 3.5Ghz Sandy Bridge, 24 Cores, (1 TCP GSO flow per core, netperf -t TCP_SENDFILE, 10K policies)

×