1.
Security Rationale
for Istio
Rowan Baker
@controlplaneio

2.
Talk Structure
● Secure Kubernetes multi-tenancy and Istio
● Istio security
○ Secure installation and configuration
○ Threats and issues

3.
Securing Multi-tenanted Kubernetes Clusters
● Secure administration
○ Cloud IaaS & PaaS services
○ Kubernetes Platform
● CI/CD pipeline security
● Runtime security
○ This is where Istio fits in

4.
A slice of the Kubernetes runtime threat model
What can go wrong?
Compromised microservice
attempts to:
● Eavesdrop
● Impersonate
● Escalate privilege
● Pivot & access other
services
● Initiate an outbound
connection
API
db
webadmin
API
db
webadmin
Compromised
Service

5.
A slice of the Kubernetes runtime threat model
What are we going to do
about that?
● Authn & Authz
● Encryption in transit
● L3/4 Firewalling
API
db
webadmin
API
db
webadmin
Mitigated threat

6.
A slice of the Kubernetes runtime threat model
What are we going to do
about that?
● Authn & Authz
● Encryption in transit
● L3/4 Firewalling
API
db
webadmin
API
db
webadmin

7.
A slice of the Kubernetes runtime threat model
API
db
webadmin
API
db
webadmin
What are we going to do
about that?
● Authn & Authz
● Encryption in transit
● L3/4 Firewalling

8.
A slice of the Kubernetes runtime threat model
What are we going to do
about that?
● Authn & Authz
● Encryption in transit
● L3/4 Firewalling
Kubernetes v1.7+
Network Policy
API
db
webadmin
API
db
webadmin

9.
Satisfying requirements at scale
Security requirements for every application team:
● Authentication
● Authorisation
● Encryption
Against a backdrop of:
● Different developer teams working with
different languages
● Security vs Speed dichotomy
● Migration of legacy applications.

10.
Satisfying requirements at scale
● Option 1: In-house or 3rd party
libraries
○ In all the languages your company
uses
○ And maintain them for all your
services and teams
● Option 2: Sidecars
○ Repeatable
○ Abstracted from Devs
○ Could end up maintaining a large
number of sidecars over time

11.
Istio

12.
Security controls provided by Istio
● Mutual TLS - Encryption &
Authentication
● L7 Authorisation
● Rate limiting
● Whitelisting/Denials
● Egress control
Compromised services
have a blast radius
defined by Istio policy
API
db
webadmin
API
db
webadmin

13.
Threats not mitigated by Istio
Amongst others:
● Injection Attacks
● Container breakout
API
db
webadmin
API
db
webadmin
Worker Node

14.
Istio Threats & Issues

15.
Threat: Insecure Control Plane Configuration
● Secure the control plane:
○ Enable control plane mutual TLS
○ Protect Citadel
○ Don’t write authorization policies for Istio control plane
components

16.
Threat: User Misconfiguration
YAML, lots of YAML:
● Ingress Gateway (Gateways, Virtual
Services)
● Authentication (Mesh Policy,
Destination Rules)
● Authorisation (ServiceRole,
ServiceRoleBinding)
● Rate Limits (QuotaSpec
QuotaSpecBinding etc)
● Denials (Denier)
● Egress (Gateway,Service Entry, Virtual
Service)

17.
Threat: User Misconfiguration
● Avoid manual configuration
● Regularly apply config defined in git
○ Regular CI server job
○ GitOps

18.
Threat: Compromised workload attacks Istio sidecar
API
db
admin
API
db
web service
K8s
Network
Policy
K8s
Network
Policy

19.
Threat: Compromised workload attacks Istio sidecar
● Access to in-mesh
services subject to Istio
RBAC & Auth
● Attack other services
● Circumvent Istio egress
control
● Can mitigate with a
Kubernetes Network
Policy
O
API
db
admin
API
db
web service
K8s
Network
Policy
K8s
Network
Policy

20.
Threat: Compromised workload attacks Istio sidecar
● Defence in depth: use dedicated Egress Gateway, K8S
Network Policy & IaaS FW rules
API
db
web service
K8s Network
Policy + IaaS FW
Worker Node (Egress)
Egress Gateway
Worker Node

21.
Threat: Init Containers Run Off-mesh
● Init container for application runs before the Istio init container
○ Unconstrained by istio security
○ Use K8s network policy
App Init
Istio init
App Init
(completed)
Pod Initialising Pod Ready
Application

22.
Issue: PodSecurityPolicy blocks Istio init & sidecar
● I want my Pods to comply to a
restrictive Pod Security Policy
○ Non-privileged
○ Drop linux capabilities
Kubernetes API
Authentication &
Authorisation
Admission Controllers
Mutating Validating
apiVersion: apps/v1
kind: Deployment
...
securityContext:
privileged: false
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL Pod Security Policy
...
kind: PodSecurityPolicy
...
spec:
privileged: false
allowPrivilegeEscalation: false
requiredDropCapabilities:
- ALL

23.
Issue: PodSecurityPolicy blocks Istio init & sidecar
Kubernetes API
Authentication &
Authorisation
Admission Controllers
Mutating Validating
apiVersion: apps/v1
kind: Deployment
….
securityContext:
privileged: false
allowPrivilegeEscalation: false
capabilities:
drop:
- all Pod Security Policy
...
kind: PodSecurityPolicy
...
spec:
privileged: false
allowPrivilegeEscalation: false
requiredDropCapabilities:
- ALL
istio-sidecar-
injector
template: |-
initContainers:
...
securityContext:
capabilities:
add:
- NET_ADMIN
privileged: true
containers:
- name: istio-proxy
...
securityContext:
readOnlyRootFilesystem: true
capabilities:
add:
- NET_ADMIN

24.
Issue: PodSecurityPolicy blocks Istio init & sidecar
● Must relax Pod Security Policy to run Istio
● Might be fixed by:
○ CNI Plugin
○ Sub-pod isolation proposals

25.
Threat: Misconfigured app container could run as
privileged or use NET_ADMIN to exit the mesh
● Due to relaxed PodSecurityPolicy
● Workarounds (weak!)
○ Templating YAML
○ Reviews & process (e.g Kubesec.io review)
○ IDS
● Istio CNI Plugin intended to mitigate this long term

26.
Issue: Some security features are in Alpha

27.
Pending Improvements
● TLS health-checking: coming in v1.1
● CNI Plugin: optional in v1.1
● Hardening, robustification, v2 workload attestation, plugable CA
adapters: all on the way

28.
In Conclusion
● Istio is exciting
● Security functionality at scale for microservices
○ Authentication & Authorisation
○ Encryption in transit
● Still maturing
○ Some improvements required
○ Some features still in alpha

29.
Thank you!
Rowan Baker
@controlplaneio

30.
Appendix

31.
Security controls provided by Istio
● Compromised services only
have the blast radius of their
Istio policy
○ RBAC L7 policy is
granular: HTTP verb
and path
○ Tight configuration
increases cluster
compromise
resilience
API
db
webadmin
API
db
webadmin

32.
Security controls provided by Istio
● Mutual TLS - Encryption &
Authentication
● L7 Authorisation
● Rate limiting
● Whitelisting/Blacklisting
● Egress control
+
● Kubernetes Network
Policy
API
db
webadmin
API
db
webadmin

33.
Bugs and Issues
● Improper namespace labelling
○

34.
A slice of the Kubernetes runtime threat model
What are we going to do about
that?
● Authn & Authz
● Encryption in transit
● L3/4 Firewalling
Kubernetes v1.7>
Network Policy
API
db
webadmin
API
db
webadmin

35.
Threat Modelling Runtime Security
Threat Modelling
1. What are we building?
2. What can go wrong?
3. What are we going to do about
that?
4. Did we do a good enough job?
API
db
webadmin
API
db
webadmin

36.
Threat: PodSecurityPolicy blocks Istio init & sidecar
● I want my Pods to comply to a restrictive Pod Security Policy
○ Non-privileged
○ Drop linux capabilities
● Istio init & sidecar runs as root and requires NET_ADMIN
● Threat: Misconfigured app container could run as root or use
NET_ADMIN to exit the mesh
● Workaround - by templating YAML, using Kubesec or IDS
● Istio CNI Plugin intended to mitigate this long term

37.
Threat: PodSecurityPolicy blocks Istio init & sidecar
● I want my Pods to comply to a restrictive Pod Security Policy
○ Non-privileged
○ Drop linux capabilities

38.
Threat: Compromised workload attacks Istio sidecar
● Compromised services could attack Istio sidecar proxy to exit
the mesh
○ access to in-mesh services subject to Istio RBAC & Auth
○ circumvent egress control
○ Mitigate using a combination of egress gateway, K8s
network policy and infrastructure firewall rules

39.
Limiting Compromised Workloads
● Compromised services only have the blast radius of their Istio
RBAC policy
○ RBAC L7 policy is granular: HTTP verb and path
○ Tight configuration increases cluster compromise resilience

40.
Reassess the Threat Model

41.
Risk: Accidental deployments outside of the mesh
● Improper namespace labelling
○ failure to trigger automatic sidecar injection

42.
A slice of the Kubernetes runtime threat model
Worker node 1 Worker node 2
App1
Service A
App1
Service B
App2
Service A
App2
Service B
App2
Service B
App2
Service A
App1
Service B
App1
Service A
Worker node 1 Worker node 2
App1
Service A
App1
Service B
App2
Service A
App2
Service B
App2
Service B
App2
Service A
+
App1
Service B
App1
Service A
L3/L4 Network Policy - Kubernetes v1.7