The document discusses how Cilium can accelerate Envoy and Istio by using eBPF/XDP to provide transparent acceleration of network traffic between Kubernetes pods and sidecars without any changes required to applications or Envoy. Cilium also provides features like service mesh datapath, network security policies, load balancing, and visibility/tracing capabilities. BPF/XDP in Cilium allows for transparent TCP/IP acceleration during the data phase of communications between pods and sidecars.

1. Cilium
Accelerating Envoy and Istio with
Cilium and the Linux Kernel
Thomas Graf, Covalent

2. BPF - The
Superpowers
inside Linux

3. The Rise of BPF [and XDP]

4. Source: https://www.netdevconf.org/2.1/slides/apr6/zhou-netdev-xdp-2017.pdf
Facebook published BPF numbers
for L3/L4 LB at NetDev 2.1
BPF/XDP performance
IPVS performance

5. BPF/XDP: DDoS mitigation
Metric iptables / ipset BPF / XDP
DDoS rate [packets/s] 11.6M 11.6M
Drop rate [packets/s] 7.1M 11.6M
Latency under load [ms] 2.3ms 0.1ms
Requests/s under DDoS [Requests/s] 280 82’800
Sender: Send 64B packets as fast as possible
Receiver: Drop as fast as possible
Source: http://schd.ws/hosted_files/ossna2017/da/BPFandXDP.pdf

6. BPF: “dtrace for Linux”
Source: http://www.brendangregg.com/blog/

8. What is your favorite
iptables memory?

9. What is your favorite
iptables memory?

10. Kernel developers are
saying goodbye to iptables
https://lwn.net/Articles/747504/

11. Early performance benchmark
Source: https://bit.ly/2Ks9gJH

12. So many more examples….
• Cloudflare DDoS mitigation [https://www.netdevconf.org/2.1/slides/apr6/bertin_Netdev-XDP.pdf]
• BCC [https://github.com/iovisor/bcc]
• Systemtap [https://sourceware.org/git/gitweb.cgi?p=systemtap.git;a=summary]
• Weave Scope [https://github.com/weaveworks/scope]
• Suricata [http://suricata.readthedocs.io/en/latest/capture-hardware/ebpf-xdp.html]
• systemd [http://0pointer.net/blog/ip-accounting-and-access-lists-with-systemd.html]
• gobpf [https://github.com/iovisor/gobpf]
• ply [https://github.com/wkz/ply]
• bpfps [https://github.com/genuinetools/bpfps]
• Perf [https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/perf]
• bpftrace [https://github.com/ajor/bpftrace]
• Open vSwitch [http://www.openvswitch.org//support/ovscon2016/7/1120-tu.pdf]
• PCP [https://github.com/performancecopilot/pcp]
• …

13. BPF toolchain

14. How does BPF apply to
Envoy and Service Mesh?

15. Kubernetes Pod
Service
Kubernetes Pod
Service Mesh / Sidecar Architecture
Network
Service
SidecarSidecar

16. Kubernetes Pod
Service
Kubernetes Pod
Operating
System
Socket Socket
Sidecar Injection
Network
Service
Socket Socket Socket Socket

17. Kubernetes Pod
Service
Kubernetes Pod
Operating
System
Socket
TCP/IP
Socket
TCP/IP
Sidecar Injection
Network
Service
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP

18. Kubernetes Pod
Service
Kubernetes Pod
Sidecar Injection
Network
Service
Operating
System
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Ethernet Ethernet Ethernet Ethernet EthernetEthernet

19. Kubernetes Pod
Service
Kubernetes Pod
Operating
System
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Sidecar Injection (Non-transparent)
Network
Service
Ethernet Ethernet
Loopback
Ethernet
eth0
Ethernet Ethernet
Loopback
Ethernet
eth0

20. Kubernetes Pod
Service
Kubernetes Pod
Operating
System
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Sidecar Injection (Transparent)
Network
Service
Ethernet Ethernet
Loopback
Ethernet
eth0
Ethernet Ethernet
Loopback
Ethernet
eth0
iptables iptables iptablesiptables

21. Why use TCP and Ethernet
in a single-node, lossless
environment?

22. Kubernetes Pod
Service
Kubernetes Pod
Operating
System
Socket Socket Socket
TCP/IP
Socket
TCP/IP
Socket Socket
Transparent Sidecar Injection with Cilium
Network
Service
Ethernet
eth0
Ethernet
eth0
CiliumCilium

23. Sidecar Injection Performance

24. Socket
TCP/IP
Socket
TCP/IP
Service A Service B
Ethernet Ethernet
Loopback
TCP Handshake Phase
Socket Socket
Service A Process B
Data Phase
Cilium
Transparent acceleration during data phase
• Transparent acceleration, behavior stays the same
• No changes to application or Envoy needed
1 2

25. Socket
TCP/IP
Socket
TCP/IP
Process A Process B
Ethernet Ethernet
Cilium
But wait…
Works with any local socket communication
TCP Handshake Phase
Socket Socket
Process A Process B
Data Phase
Cilium
vethveth

26. How do I get this?

27. http://github.com/cilium/cilium
Cilium
Sidecar Accel:
- Kernel: 4.16
- Cilium: 1.1/1.2
Cilium:
- Kernel: 4.9

28. Cilium in a Nutshell
• Current Release: 1.0.1
• Highly efficient BPF datapath
• Fully Distributed
• Service Mesh datapath
• CNI and CMM plugin
• Network Security on both Packet and API level
• Identity Based
• IP/CIDR as fallback
• API Aware (HTTP, gRPC, Kafka, [more coming soon])
• Distributed and Scalable Load Balancing
• Simplified Networking Model
• Overlay
• Direct Routing
• Delegation to other plugin (1.1)
• Visibility / Tracing

29. It gets better…

30. It gets better…
… and a little bit scary

31. Service A Service B
Mutual TLS
TLS in Service Mesh
Clear Text
Encrypted
Sidecar Sidecar
Key

32. Service A
Accessing external services via
sidecar using TLS
GitHub
API
Clear Text
Encrypted
Sidecar
Encrypt
Decrypt

33. Service A
Option #1: Inject Root CA
GitHub
API
Clear Text
Encrypted
Sidecar
Inject new root CA
Real root CA
Encrypt
Decrypt
Encrypt Decrypt

34. kTLS
Kubernetes Pod
App
Kubernetes Pod
Operating
System
Socket Socket
Network
App
Socket Socket Socket Socket
SSL SSL
… … … … … …
Ethernet Ethernet
Loopback
Ethernet
eth0
Ethernet Ethernet
Loopback
Ethernet
eth0
Symmetric encryption
offloaded to kernel
~4% CPU gain

35. Kubernetes Pod
App
Kubernetes Pod
Operating
System
Socket Socket Socket
TCP/IP
Socket
TCP/IP
Socket Socket
kTLS + Cilium + Envoy
Network
App
Ethernet
eth0
Ethernet
eth0
iptablesiptables
CiliumCilium
SSL SSL

36. Service A
Option #2: kTLS
GitHub
API
Clear Text
Encrypted
Sidecar
Deferred kTLS
encryption
Decrypt

37. Sidecar for TLS encrypted connections
without CA injection & decryption

38. What date is today?

39. @ciliumproject
http://github.com/cilium/cilium
Thank You!
Questions?
Getting Started:
http://cilium.io/