Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Control Plane: Security Rationale for Istio (DevSecOps - London Gathering, January 2019)

399 views

Published on

Security Rationale For Istio
An introduction to Istio security, looking at how Istio helps to keeps your security team happy by satisfying Kubernetes security requirements for multi-tenancy, and your developers happy by reducing implementation effort. Istio is still an evolving technology, and outstanding issues and impending improvements will be discussed.

Published in: Technology
  • Be the first to comment

Control Plane: Security Rationale for Istio (DevSecOps - London Gathering, January 2019)

  1. 1. Security Rationale for Istio Rowan Baker @controlplaneio
  2. 2. Talk Structure ● Secure Kubernetes multi-tenancy and Istio ● Istio security ○ Secure installation and configuration ○ Threats and issues
  3. 3. Securing Multi-tenanted Kubernetes Clusters ● Secure administration ○ Cloud IaaS & PaaS services ○ Kubernetes Platform ● CI/CD pipeline security ● Runtime security ○ This is where Istio fits in
  4. 4. A slice of the Kubernetes runtime threat model What can go wrong? Compromised microservice attempts to: ● Eavesdrop ● Impersonate ● Escalate privilege ● Pivot & access other services ● Initiate an outbound connection API db webadmin API db webadmin Compromised Service
  5. 5. A slice of the Kubernetes runtime threat model What are we going to do about that? ● Authn & Authz ● Encryption in transit ● L3/4 Firewalling API db webadmin API db webadmin Mitigated threat
  6. 6. A slice of the Kubernetes runtime threat model What are we going to do about that? ● Authn & Authz ● Encryption in transit ● L3/4 Firewalling API db webadmin API db webadmin
  7. 7. A slice of the Kubernetes runtime threat model API db webadmin API db webadmin What are we going to do about that? ● Authn & Authz ● Encryption in transit ● L3/4 Firewalling
  8. 8. A slice of the Kubernetes runtime threat model What are we going to do about that? ● Authn & Authz ● Encryption in transit ● L3/4 Firewalling Kubernetes v1.7+ Network Policy API db webadmin API db webadmin
  9. 9. Satisfying requirements at scale Security requirements for every application team: ● Authentication ● Authorisation ● Encryption Against a backdrop of: ● Different developer teams working with different languages ● Security vs Speed dichotomy ● Migration of legacy applications.
  10. 10. Satisfying requirements at scale ● Option 1: In-house or 3rd party libraries ○ In all the languages your company uses ○ And maintain them for all your services and teams ● Option 2: Sidecars ○ Repeatable ○ Abstracted from Devs ○ Could end up maintaining a large number of sidecars over time
  11. 11. Istio
  12. 12. Security controls provided by Istio ● Mutual TLS - Encryption & Authentication ● L7 Authorisation ● Rate limiting ● Whitelisting/Denials ● Egress control Compromised services have a blast radius defined by Istio policy API db webadmin API db webadmin
  13. 13. Threats not mitigated by Istio Amongst others: ● Injection Attacks ● Container breakout API db webadmin API db webadmin Worker Node
  14. 14. Istio Threats & Issues
  15. 15. Threat: Insecure Control Plane Configuration ● Secure the control plane: ○ Enable control plane mutual TLS ○ Protect Citadel ○ Don’t write authorization policies for Istio control plane components
  16. 16. Threat: User Misconfiguration YAML, lots of YAML: ● Ingress Gateway (Gateways, Virtual Services) ● Authentication (Mesh Policy, Destination Rules) ● Authorisation (ServiceRole, ServiceRoleBinding) ● Rate Limits (QuotaSpec QuotaSpecBinding etc) ● Denials (Denier) ● Egress (Gateway,Service Entry, Virtual Service)
  17. 17. Threat: User Misconfiguration ● Avoid manual configuration ● Regularly apply config defined in git ○ Regular CI server job ○ GitOps
  18. 18. Threat: Compromised workload attacks Istio sidecar API db admin API db web service K8s Network Policy K8s Network Policy
  19. 19. Threat: Compromised workload attacks Istio sidecar ● Access to in-mesh services subject to Istio RBAC & Auth ● Attack other services ● Circumvent Istio egress control ● Can mitigate with a Kubernetes Network Policy O API db admin API db web service K8s Network Policy K8s Network Policy
  20. 20. Threat: Compromised workload attacks Istio sidecar ● Defence in depth: use dedicated Egress Gateway, K8S Network Policy & IaaS FW rules API db web service K8s Network Policy + IaaS FW Worker Node (Egress) Egress Gateway Worker Node
  21. 21. Threat: Init Containers Run Off-mesh ● Init container for application runs before the Istio init container ○ Unconstrained by istio security ○ Use K8s network policy App Init Istio init App Init (completed) Pod Initialising Pod Ready Application
  22. 22. Issue: PodSecurityPolicy blocks Istio init & sidecar ● I want my Pods to comply to a restrictive Pod Security Policy ○ Non-privileged ○ Drop linux capabilities Kubernetes API Authentication & Authorisation Admission Controllers Mutating Validating apiVersion: apps/v1 kind: Deployment ... securityContext: privileged: false allowPrivilegeEscalation: false capabilities: drop: - ALL Pod Security Policy ... kind: PodSecurityPolicy ... spec: privileged: false allowPrivilegeEscalation: false requiredDropCapabilities: - ALL
  23. 23. Issue: PodSecurityPolicy blocks Istio init & sidecar Kubernetes API Authentication & Authorisation Admission Controllers Mutating Validating apiVersion: apps/v1 kind: Deployment …. securityContext: privileged: false allowPrivilegeEscalation: false capabilities: drop: - all Pod Security Policy ... kind: PodSecurityPolicy ... spec: privileged: false allowPrivilegeEscalation: false requiredDropCapabilities: - ALL istio-sidecar- injector template: |- initContainers: ... securityContext: capabilities: add: - NET_ADMIN privileged: true containers: - name: istio-proxy ... securityContext: readOnlyRootFilesystem: true capabilities: add: - NET_ADMIN
  24. 24. Issue: PodSecurityPolicy blocks Istio init & sidecar ● Must relax Pod Security Policy to run Istio ● Might be fixed by: ○ CNI Plugin ○ Sub-pod isolation proposals
  25. 25. Threat: Misconfigured app container could run as privileged or use NET_ADMIN to exit the mesh ● Due to relaxed PodSecurityPolicy ● Workarounds (weak!) ○ Templating YAML ○ Reviews & process (e.g Kubesec.io review) ○ IDS ● Istio CNI Plugin intended to mitigate this long term
  26. 26. Issue: Some security features are in Alpha
  27. 27. Pending Improvements ● TLS health-checking: coming in v1.1 ● CNI Plugin: optional in v1.1 ● Hardening, robustification, v2 workload attestation, plugable CA adapters: all on the way
  28. 28. In Conclusion ● Istio is exciting ● Security functionality at scale for microservices ○ Authentication & Authorisation ○ Encryption in transit ● Still maturing ○ Some improvements required ○ Some features still in alpha
  29. 29. Thank you! Rowan Baker @controlplaneio
  30. 30. Appendix
  31. 31. Security controls provided by Istio ● Compromised services only have the blast radius of their Istio policy ○ RBAC L7 policy is granular: HTTP verb and path ○ Tight configuration increases cluster compromise resilience API db webadmin API db webadmin
  32. 32. Security controls provided by Istio ● Mutual TLS - Encryption & Authentication ● L7 Authorisation ● Rate limiting ● Whitelisting/Blacklisting ● Egress control + ● Kubernetes Network Policy API db webadmin API db webadmin
  33. 33. Bugs and Issues ● Improper namespace labelling ○
  34. 34. A slice of the Kubernetes runtime threat model What are we going to do about that? ● Authn & Authz ● Encryption in transit ● L3/4 Firewalling Kubernetes v1.7> Network Policy API db webadmin API db webadmin
  35. 35. Threat Modelling Runtime Security Threat Modelling 1. What are we building? 2. What can go wrong? 3. What are we going to do about that? 4. Did we do a good enough job? API db webadmin API db webadmin
  36. 36. Threat: PodSecurityPolicy blocks Istio init & sidecar ● I want my Pods to comply to a restrictive Pod Security Policy ○ Non-privileged ○ Drop linux capabilities ● Istio init & sidecar runs as root and requires NET_ADMIN ● Threat: Misconfigured app container could run as root or use NET_ADMIN to exit the mesh ● Workaround - by templating YAML, using Kubesec or IDS ● Istio CNI Plugin intended to mitigate this long term
  37. 37. Threat: PodSecurityPolicy blocks Istio init & sidecar ● I want my Pods to comply to a restrictive Pod Security Policy ○ Non-privileged ○ Drop linux capabilities
  38. 38. Threat: Compromised workload attacks Istio sidecar ● Compromised services could attack Istio sidecar proxy to exit the mesh ○ access to in-mesh services subject to Istio RBAC & Auth ○ circumvent egress control ○ Mitigate using a combination of egress gateway, K8s network policy and infrastructure firewall rules
  39. 39. Limiting Compromised Workloads ● Compromised services only have the blast radius of their Istio RBAC policy ○ RBAC L7 policy is granular: HTTP verb and path ○ Tight configuration increases cluster compromise resilience
  40. 40. Reassess the Threat Model
  41. 41. Risk: Accidental deployments outside of the mesh ● Improper namespace labelling ○ failure to trigger automatic sidecar injection
  42. 42. A slice of the Kubernetes runtime threat model Worker node 1 Worker node 2 App1 Service A App1 Service B App2 Service A App2 Service B App2 Service B App2 Service A App1 Service B App1 Service A Worker node 1 Worker node 2 App1 Service A App1 Service B App2 Service A App2 Service B App2 Service B App2 Service A + App1 Service B App1 Service A L3/L4 Network Policy - Kubernetes v1.7

×