The document discusses securing a Kubernetes cluster from multiple layers of risk. It covers securing the infrastructure layer by limiting access and exposure, the control plane layer by enabling TLS and RBAC, the workload layer using pod security policies and network policies, the container runtime layer with tools like Kata Containers, the user misconfiguration layer by avoiding defaults and validating configurations, and useful security tools. The presenter then provides contact information for potential job opportunities.

1. Secure your K8s cluster
from multi-layers
Jiantang Hao

2. Agend
• About me
• K8s bug report
• Why kubernetes make security difficult
• Infrastructure Layer
• K8s Control Plane Layer
• K8s Workload Layer
• K8s Container Runtime Layer
• User Misconfiguration Layer
• Useful Tool&Documents
• References

3. About me
• Jiantang Hao
• Platform Engineer at Yahoo! JAPAN
• Focus on CaaS(Container as a Service)
• Like Challengeable/Interesting tech

4. K8s bug report
• https://k8s.devstats.cncf.io/d/39/issues-opened-closed-by-
sig?orgId=1&var-period=w&var-sig_name=All&var-
kind_name=bug

5. Details of kubernetes in CVE
• https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=kubernetes

6. Why kubernetes make security difficult
• Traffic is everywhere. Containers can be dynamically
deployed across hosts or even clouds.
• Increased attack surface. Each container and have a
different attack surface and vulnerabilities which can be
exploited.
• Old security tool. Old models and tools for security will not
be able to keep up in a constantly changing container
environment.

7. Infrastructure Layer
• Turn on audit log
• Never expose a port, which doesn't need exposure
• Host the cluster in a private subnet or VPC if possible
• Limit SSH to kubernetes nodes, use “kubectl” more
• Limit the access to kube-api

8. K8s Control Plane Layer
• Enable RBAC, at least make --anonymous-auth false.
• Enable TLS among component connection
• Encrypting Secret Data at Rest
• Turn on K8s audit logging
• Reserve Compute Resources for System Daemons
• Admission Controllers

9. Admission Controllers
• Enable by set flag in Kubernetes API server
• Admission controllers may be “validating”, “mutating”, or
both
• This admission controller calls any validating webhooks
which match the request. Matching webhooks are called in
parallel
• Caution about using mutating webhooks

10. Admission Controllers

11. ValidatingAdmissionWebhook

12. ValidatingAdmissionWebhook

13. K8s Workload Layer
• Run Containers as a Non-Root User
• Run a Cluster-wide Pod Security Policy
• Create and Define Cluster Network Policies
• Use namespace for isolation
• Controlling which nodes pods may access
• Controlling the capabilities by setting Resource Quota
• Security Context

14. Pod Security Policy

15. Deploy Pod Security Policy Via RBAC
• First, a Role or ClusterRole needs to grant access
to use the desired policies.

16. Lessons about using PSP
• If you only want to grant usage for pods being run in the
namespace, you have to create RoleBinding per namespace
• The above lead to a lot of individual rules for different use-
cases and it is difficult to make it maintainable in the long
term.
• RBAC Authorization is based on a whitelist, it is hard to set
Blacklist-based PSP
• Open Policy Agent could solve Blacklist-based problems

17. K8s Container Runtime Layer

18. Kata Containers

19. Kata Containers

20. Multi-tenant K8s Cluster

21. User Misconfiguration Layer
• One recent study found that 70‒75% of companies have at
least one serious cloud security misconfiguration
Image from https://compliancex.com/embarrassing-6bn-fat-finger-trade-another-blow-to-top-firm/

22. User Misconfiguration Layer
• Donʼt specify default values unnecessarily
• Simple, minimal configuration will make errors less likely
• Put object descriptions in annotations, to allow better
introspection
• Specify the latest stable API version
• Check the configuration on CICD pipeline
• https://kubesec.io/

23. Useful Tool&Documents
• (CIS) Benchmark for Kubernetes
• (CIS) Benchmark for Docker
• aquasecurity/kube-bench
• aquasecurity/kube-hunter
• Sysdig Inspect
• Shopify/kubeaudit
• coreos/clair
• Open Policy Agent

24. References• https://www.cncf.io/blog/2019/01/14/9-kubernetes-security-best-practices-everyone-must-follow/
• https://neuvector.com/container-security/kubernetes-security-guide/
• https://www.darkreading.com/vulnerabilities---threats/security-at-the-speed-of-devops-maturity-orchestration-
and-detection/a/d-id/1333583
• https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=kubernetes
• https://github.com/freach/kubernetes-security-best-practice
• https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#imagepolicywebhook
• https://kubernetes.io/docs/tasks/debug-application-cluster/audit/
• https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/s
• https://kubernetes-security.info/#securing-the-cluster
• https://medium.com/devopslinks/kubernetes-security-are-your-container-doors-open-2c4b99c8d786
• https://kubernetes.io/docs/concepts/configuration/overview/
• https://kubedex.com/kubernetes-container-runtimes/
• https://katacontainers.io/
• https://katacontainers.io/media/uploads/katacontainers/uploads/katacontainers/kata-containers-on-boarding-
deck-for-website01022018.pdf
• https://katacontainers.io/media/uploads/katacontainers/uploads/katacontainers/kata_containers_overview.pdf
• https://itnext.io/kubernetes-authorization-via-open-policy-agent-a9455d9d5ceb
• https://kubernetes.io/docs/concepts/policy/pod-security-policy/

25. Yes! We are hiring!
jhao@yahoo-corp.jp
https://about.yahoo.co.jp/hr/en/
https://about.yahoo.co.jp/hr/

26. Thank you for your listening！
ご静聴ありがとうございました！