Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

K8s security best practices

974 views

Published on

K8s security best practices

Published in: Software

K8s security best practices

  1. 1. K8S security – Best practices By: Sharon Vendrov
  2. 2. 0 500 1000 1500 2000 2500 Mac Os X Windows 7 Windows XP Windows 8.1 Windows 10 CVE Sum CVE Sum 2 Total Number Of Vulnerabilities in 2017 – Source: CVEdetails.com
  3. 3. 3
  4. 4. 4
  5. 5. Sharon Vendrov Sr. DevOps Engineer About Me 5 Storm-runner functional
  6. 6.  Infrastructure protection  K8s internal security  Authentication & Authorization options  Network  Secrets  Container runtime Security  Some other security tools and considerations 6 Agenda
  7. 7. Infrastructure protection 7  Limit SSH access to your cluster  Use hardened images for your cluster ( )  Encrypt your storage volume  Avoid from exposing your cluster to the internet  Limit the access to the K8s API (consider to use bastion machine)  Create dedicated cluster for each environment (Prod, Stg, Dev)  Separate sensitive pods into different nodes
  8. 8. Kubernetes internal security 8  Use minimal base docker image  Don’t use arbitrary base images  Separate sensitive workloads across instances (using anti-affinity, taints and tolerations)  Use namespaces for isolation  Enforce resource quota (CPU, Memory, Storage) Image Name node:latest ubuntu:latest alpine:latest scratch Image Size 670MB~ 110MB~ 4.1MB~ 0
  9. 9. Secure kubelet 9  curl -sk https://<nodeIP>:10250/run/<namespace>/<pod-name>/<container-name> -d "cmd=ls -la /“  Protect kubelet by enable authentication and authorization: start the apiserver with --kubelet-client-certificate and --kubelet-client-key flags /usr/local/bin/kubelet --anonymous-auth=false --authorization-mode=Webhook --allow-privileged=true --kubeconfig=/var/lib/kubelet/kubeconfig --client-ca-file=/var/lib/kubernetes/ca.pem • Enable kubelet certification rotation (1.8 beta)
  10. 10. Authentication & Authorization
  11. 11. 11
  12. 12. 12
  13. 13. Authentication 13  Static password/token file  Client certificates x509  Proxy + headers  OpenID Connect  Custom (Web hook) password,user,uid,”group1,group2,group3”
  14. 14. Authentication 14  Service accounts  Default service account have full permissions over the cluster, use custom SA instead  Set “automountServiceAccountToken : false” in your pod spec – when possible
  15. 15. Authorization 15  ABAC  Difficult to manage and understand  Requires ssh and root filesystem access on the master  For permission changes to take effect the cluster API server must be restarted {"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": { "user": "bob", "namespace": "projectSpaceX", "resource": "pods", "readonly": true } }
  16. 16. Authorization 16  RBAC (stable 1.8) Service Account User Role binding Role
  17. 17. 17 kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: read-pods namespace: default subjects: - kind: User name: Bob apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: namespace: default name: pod-reader rules: - apiGroups: [""] group resources: ["pods"] verbs: ["get", "watch", "list"]
  18. 18. Authorization 18  Custom (Web hook)  Node Restrict kubelet to perform R/W operation only to his bound pods --authorization-mode=Node,RBAC --admission-control=NodeRestriction
  19. 19. Network
  20. 20. 20
  21. 21. Netwok 21  Limit the access to cloud provider metadata (http://169.254.169.254/latest/meta-data/)
  22. 22. 22 $ curl -s 169.254.169.254/latest/meta-data/iam/security- credentials/kubernetes-worker-iam-policy { "Code" : "Success", "LastUpdated" : "2017-12-25T00:00:00Z", "Type" : "AWS-HMAC", "AccessKeyId" : "MyAccessKeyID", "SecretAccessKey" : "MySecretAccessKey", "Token" : "MySessionToken", "Expiration" : "2017-12-25T04:00:00Z" } @bradgeesaman
  23. 23. 23 # Place credentials in ENV vars $ export AWS_REGION=us-east-1 $ export AWS_ACCESS_KEY_ID=MyAccessKeyID $ export AWS_SECRET_ACCESS_KEY=MySecretAccessKey $ export AWS_SESSION_TOKEN=MySessionToken $ aws ec2 … @bradgeesaman
  24. 24. The solution 24 • For AWS use kube2iam or kiam (using docker proxy for requests to the metadata) • For GCE use k8s-metadata-proxy • Limit egress with network policy
  25. 25. 25  Use network policy (GA from 1.7) https://goo.gl/HRtn5B  Egress rules are beta from 1.8 kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: access-nginx spec: podSelector: matchLabels: run: nginx ingress: - from: - podSelector: matchLabels: access: "true" • Istio
  26. 26. Network policy guidelines 26  Label your workloads properly  Isolate workloads from each other  Restrict income traffic to the kube-system (except kube-dns)  Consider limit egress to the internet
  27. 27. “The definition of Secret— something you tell everybody to tell nobody.” – The universe
  28. 28. Treat your secrets with respect 28  Don’t store your secrets on Git, it will remain in history even If you delete it.  Create dedicated secrets for dev and prod environments  Secrets are stored at etcd as base64 (almost like plain text)  encrypt your secrets (K8S encryption –alpha 1.7)  Use Vault as you secret management (starting from Vault 0.8.3)
  29. 29. Security Context A security context defines privilege and access control settings for a Pod or Container 29  Discretionary Access Control: Permission to access an object, like a file, is based on user ID (UID) and group ID (GID).  Security Enhanced Linux (SELinux): Objects are assigned security labels.  Running as privileged or unprivileged.  Linux Capabilities: Give a process some privileges, but not all the privileges of the root user.  AppArmor: Use program profiles to restrict the capabilities of individual programs.  Seccomp: Limit a process’s access to open file descriptors.  AllowPrivilegeEscalation: Controls whether a process can gain more privileges than its parent process.
  30. 30. Example: RunasNonRoot 30 apiVersion: v1 kind: Pod metadata: name: security-context-demo spec: containers: - name: sec-ctx-demo image: gcr.io/google-samples/node-hello:1.0 securityContext: runAsNonRoot : true
  31. 31. 31
  32. 32. Example: readOnlyRootFilesystem 32 apiVersion: v1 kind: Pod metadata: name: security-context-demo spec: containers: - name: sec-ctx-demo image: gcr.io/google-samples/node-hello:1.0 securityContext: runAsNonRoot : false readOnlyRootFilesystem : true
  33. 33. 33
  34. 34. 34
  35. 35. Other security tools and considerations 35  Scan your docker images for vulnerabilities, (Clair CoreOS /Quay.io, Docker Security Scanning, aqua, Twistlock).  Use kube-bench (aqua security) or kubernetes-auto-analyzer (nccgroup) to execute CIS Kubernetes Benchmark  Enforce cluster wide security policy w/podSecurityPolicy  Use only trusted private docker registry  Always tag your images avoid from using “latest”  Audit events and store them on external storage (beta 1.8)  Consider using kubeaudit to audit security issue
  36. 36. 36
  37. 37. Other security considerations 37  Specify an image with its digest (SHA256)  Keep up with K8S stable releases  Implement monitoring and set alerts  Don’t run “kubectl create –f <some unknown URL to some unknown yamls>  Keep updated with new security vulnerabilities from the google group “kubernetes-announces” https://groups.google.com/forum/#!forum/kubernetes-announce
  38. 38. 38
  39. 39. Thanks and credit 39  My Wife   All K8s contributors  Hacking and Hardening Kubernetes Clusters by Example [I] - Brad Geesaman - https://goo.gl/komeXN  Running containers securely with Google Container Engine, Alex Mohr and Jessica Frazelle - https://goo.gl/AFhTyp  Shipping in Pirate-Infested Waters: Practical Attack and Defense in Kubernetes [A] - Greg Castle - https://goo.gl/WFDrrv  Compliance and Identity Management in Kubernetes [I] Marc Boorshtein - https://goo.gl/Jf7Rkh  Securing K8s Microservices with Calico Network Policies, Vadim Solvey - https://goo.gl/rWGGXM
  40. 40. Thank You.

×