This document discusses network security and policies in Kubernetes. It provides 3 use cases of applying network policies to control communication between pods. It also discusses limitations of current network policy implementation and promising technologies being developed, such as Cilium, that can provide more flexible policy enforcement that is not dependent on IP addresses. The key takeaway is that while network policies can whitelist communication, the implementation has limitations and security needs to keep pace with the dynamic nature of containers and Kubernetes.

1. 共享 x 安全 = NetworkPolicy ?
施嘉峻 (ccshih)
iiiccshih@gmail.com
Presented at iThome Kubernetes Day, Sep 20, 2017

2. About Me
@ ccshih
# Kubernetes
# OpenShift
+ Red Hat Certificate of Expertise in
Platform-as-a-Service (OpenShift Admin)
2

3. Agenda
• Share x Security @ k8s 1.7 (p. 4)
• Network Policies (p. 6)
• Use cases
• How to
• Limitations
• Adoption Advices (p. 18)
• Looking Forward (p. 22)
• Takeaway (p. 25)
3
3 cases
1 advice
2 promising tech

4. Share x Security @ k8s 1.7
Network
Policies
(Beta from 1.6)
RBAC
Namespace
(GA from 1.7)
Virtual cluster
Multiple teams or projects
Scope of control
Role-Based Access Control
Attach permissions to roles
Bind roles to users
Control pods incoming connection
Based on Labels
Project A Project B
Namespace Namespace
Pod Pod
Admin
Role
User
Role
?
4
*Also matters: Resource Quota, Pod Security Policies
允入

5. Use Labels to Organize Objects
5
tier=frontend
(key-value pairs)
tier=backend
tier=database
env=dev
env=qa
env=staging
env=prod
release=stable
release=canary
Pod
Labels …
env=prod
tier=frontend
release=stable
env=prod
Namespace

6. First Scenario
6
Pod
Pod Pod
Pod Pod
Pod
tier=frontend tier=backend tier=database
Namespace b
PodPod
PodPod
Namespace a

7. Without Network Policy (k8s default)
7
Pod
Pod Pod
Pod Pod
Pod
tier=frontend tier=backend tier=database
Namespace b
PodPod
PodPod
Namespace a
Pod accepts any connection from any namespace

8. Network Policy Use Case (1)
8
Pod
Pod Pod
Pod Pod
Pod
tier=frontend tier=backend tier=database
Namespace b
PodPod
PodPod
Namespace a

9. apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-backend-access-database
namespace: b
spec:
podSelector:
matchLabels:
tier: database
ingress:
- from:
- podSelector:
matchLabels:
tier: backend
ports:
- protocol: TCP
port: 3306
Network Policy Example (1)
9
This policy applies to …
Allows connection from …
to this port
Pod
Pod Pod
Pod Pod
Pod
tier=frontend tier=backend tier=database
Namespace b
PodPod
PodPod
Namespace a

10. Policy:
Network Policy Rules
• If no policy applied to a pod
• Allow connection from any source
• If any policy applied to a pod
• Whitelist: Deny all incoming connections, unless source satisfies at least one
policies
10
PodPod
Pod
PodPod
Pod
No Policy
applied
Policy:

11. Network Policy Use Case (2)
11
Pod
Pod Pod
Pod
Pod
tier=frontend tier=backend tier=database
PodPod
PodPod
release=canary
release=stable
Namespace b

12. Network Policy Example (2)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-canary-backend-access-database
namespace: b
spec:
podSelector:
matchLabels:
tier: database
release: stable
ingress:
- from:
- podSelector:
matchLabels:
tier: backend
ports:
- protocol: TCP
port: 3306
12
This policy applies to …
Allows connection from …
to this port
Pod
Pod Pod
Pod
Pod
tier=frontend tier=backend tier=database
PodPod
PodPod
release=canary
release=stable
Namespace b
(AND)
in the same selector

13. Network Policy Non-Use Case
13
Pod
Pod Pod
Pod
Pod
tier=frontend tier=backend tier=database
PodPod
PodPod
Namespace
b-plus
Namespace
b

14. Network Policy Non-Example
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
namespace: b
…
…
ingress:
- from:
- podSelector:
matchLabels:
tier: backend
- namespaceSelector:
matchLabels:
name: b-plus
ports:
- protocol: TCP
port: 3306
14
Allows connection from …(OR)
Pod
Pod Pod
Pod
Pod
tier=frontend tier=backend tier=database
PodPod
PodPod
Namespace
b-plus
Namespace
b
Need (AND),
but NOT supported yet*
* https://github.com/kubernetes/kubernetes/issues/50451
in different selector

15. Network Policy Limitation in K8s 1.7
• A selector can not match both Namespace labels and Pod labels
• No Egress Policy
• What if your pods are compromised?
• Prerequisite: Need support of Container Network Providers
15
Pod
(Ingress) (Egress)

16. Network Providers who Support Network Policy
16
Calico cilium ROMANA WeaveNet
&

17. Note:
• Not standard k8s Network Policy
• Run with calicoctl, instead of kubectl
• Calico also supports Egress policy (not
shown here)
Calico’s Policy Definition
17
Pod
Pod Pod
Pod
Pod
tier=frontend tier=backend tier=database
PodPod
PodPod
Namespace
b-plus
Namespace
bapiVersion: v1
kind: policy
metadata:
name: allow-b-plus-backend-access-b-database
spec:
selector: >
calico/k8s_ns == 'b' &&
tier == 'database'
ingress:
- action: allow
source:
selector: >
calico/k8s_ns == 'b-plus' &&
tier == 'backend'
protocol: tcp
destination:
ports: [3306]
# calicoctl apply -f <FILENAME>

18. Adoption Advices
18

19. Adoption Advices
• (skipped) Use special labels for network policy
• Rethink IP-based packet filter
19

20. Dynamic Nature of K8s
20
Source: https://www.slideshare.net/gmccance/cern-data-centre-evolution
K8s is primarily
designed for
(Static)
(Dynamic)
(change IP)

21. Rethink IP-based packet filter
• Pod up/down triggers firewall rule adjustment
• Translate application-level policy to network-level policy
• Given that Firewall can recognize pod IP
21
IP
Source Labels
Destination
Labels
IP IP
IP
IP
IP IP
IP
IP-based
Firewall rules
Application-level
Policy
change
frequently

22. Looking Forward
22

23. Looking forward (1)
• Cilium
• Not use IP to define filter
• Flexible and Efficient
23
IP
Source Labels
Destination
Labels
IP IP
IP
IP
IP IP
IP
eBPF
(Packet Filter)
Application-level
Policy
change
frequently
(Require Linux Kernel 4.8 ↑)
id
id

24. Looking forward (2)
24
• Istio
• Each service pod equips with a sidecar container (envoy), which enforces policy
Source: https://github.com/istio/auth

25. Takeaway
25

26. Takeaway
• By default, pod can talk to each other at network level
• Network Policy
• whitelists pod incoming connection
• needs container network provider’s support
• can not apply to some cross-namespace scenario
• Adoption
• security needs as agile as containers
• Looking forward
• Flavored policy enforcement
26

27. Contact
• iiiccshih@gmail.com
27

28. References
• Network Policy
• Official document
• Recipes by ahmetb
• Calico
• Getting started with k8s
28