This document discusses how to think like a threat actor in Kubernetes environments. It covers three pillars: 1) gaining visibility across the supply chain, 2) starting with RBAC and diving deeper into access controls, and 3) correlating data plane and control plane telemetry for incident response. Examples discussed include malicious and vulnerable packages in container images, masqueraded cluster role bindings, and lateral movements via default service accounts. The document advocates developing a snapshot of the security posture, leveraging concepts like zero trust and IAM, and collecting telemetry across processes, Kubernetes, and cloud platforms to detect attacks.

1. Nov. 29, 2023
How (and why) to think like a
threat actor in your
Kubernetes Environments
Abhinav Mishra
Abhinav Mishra Rewanth Tammana
Director of Product
Management, Uptycs
Consultant, Uptycs

2. 2
Abhinav Mishra
Director of Product Management
› Leading Uptycs product team on Containers & Kubernetes
› 10+ years of Security and Engineering Experience

3. 3
Rewanth Tammana
Consultant
› Speaker, Trainer, Open-source contributor, GSoCer & more
› DevSecOps, Cloud & Container security - Red, Blue & Purple teams

4. By the Numbers - Kubernetes Attacks
Source: 2023 Red Hat State of Kubernetes Security Report
Threat actors are now Kubernetes security experts
4

5. Which room am I in?
How many floors are
in this house?
What doors can I open?
Where are
the security cameras?
Where are the valuable items?

6. Which room am I in?
Specific Pod or Namespace
What other rooms can I get
into?
Kubernetes Network Policies/Lateral Movements
What doors can I open?
Access Controls (Role Bindings)
Where are
the security cameras?
Kubernetes Audit Log Data
Where are the valuable items?
Secrets/Sensitive Data

7. Tools
7
Kubernetes GOAT
RedKube

8. Pillar 1 - Visibility Across Your
Supply Chain

9. Pillar 1 - Visibility Across Your Supply Chain
9
Developer Laptop
Development Container Images
Control
Plane
Data Plane
Code
Development
Git Repository
Code Pull
Node 1
Node 2
Node 3
Registry
Container
Runtime
Container
Orchestration
Registry
Scanning
Confidential. All rights reserved.
CI / CD Tool
CI Scanning

10. Pillar 1 - Visibility Across Your Supply Chain

11. Example - Malicious & Vulnerable Packages
11
Container Image
- Image is Signed
- Registry and CI where image is built
and stored is scanned for vulnerabilities
- Image Layer vulnerabilities are
scanned
- What were the ingredients used to
make the cookie? Are they safe?
- What are the contents of the cookie on
the inside?
- What was the state of the factory when
the cookie was built?

12. Example - Malicious & Vulnerable Packages
Need to inspect:
- the contents of the image
when it is built
- the traceability of the image
- where did it come from,
who built it?
- the provenance of the
image - what was the
security of the supply chain
components?

13. Example - Malicious Admission Controllers
13
Admission Controls:
- enforce sensible & secure defaults
(such as namespace quotas)
- only allow trusted repositories
- don’t allow insecure resources (ex.
wildcard ingress controllers or over
privileged service accounts) to be
deployed
The Challenge: How do I know my admission controller is secure
at any given point in time?

14. Example - Malicious Admission Controllers
14
Source: https://blog.rewanthtammana.com/creating-malicious-admission-controllers

15. Let’s take a look at a crypto mining example to see what
information we need
1. Attacker created malicious mutating webhook to gain persistence to the
system
2. Injects crypto mining init container/side car to each deployment.
3. As an attacker, you want to make everything still seem normal - the application
will still work normal but in the backend, it’s eating your compute resources
4. This cannot be identified with static checks like CIS benchmarking,
misconfiguration checks using kubescape, etc.
15

16. Cryptomining DEMO

17. Pillar 1 Takeaways - Visibility Across Your Supply Chain
- always have point in time snapshot of your security posture
- rely on a combination of the following:
- image scanning: across layers of malicious/vulnerable packages
- image provenance: what was the security posture of my supply chain components at the time
of an image build? need snapshot information
- image traceability: where did the image come from? who committed it? did it go through the
right set of security pipelines?
- Image signing and verification: is the image signed by a trusted author?
17

18. Pillar 2 - Start with RBAC and
Dive Deeper

19. Pillar 2 - Start with RBAC and Dive Deeper
19
Source: MITRE ATT&CK Framework - Containers Matrix

20. Example - Masqueraded Cluster Role Bindings
20
Threat Actors will try to hide behind
benign names or components that seem
important but are actually harmful
Source: https://blog.aquasec.com/leveraging-kubernetes-rbac-to-backdoor-clusters
NOTE: Misconfigurations can also be introduced via
human error or using defaults

21. Example - Lateral Movements via Default Service Accounts
Team Alpha
Namespace
alpha-1
Namespace
alpha-2
SHARED EKS CLUSTER
Team Beta
Namespace alpha-3
The lock/key is now used to access
namespaces including ones belonging to the
other tenant
if malware is present in one namespace or
vulnerabilities, it can laterally move across the
entire cluster!
You need a platform that can tell you where
these misconfigurations are present!
Use Security Tools That Map Real-
Time Threats To Misconfigurations
In Your Cluster

22. Leverage Principles of Zero Trust and IAM in the Cloud
Source: https://blog.rewanthtammana.com/securing-aws-eks-implementing-least-privilege-access-with-irsa

23. Pillar 2 Takeaways - Start with RBAC and Dive Deeper
- always monitor your identities in and across your clusters
- leverage concepts such as IRSAs and Pod Identity to map Kubernetes service
accounts to core IAM roles that are properly managed and audited
- use security platforms that enable you to answer key questions about your
RBAC posture
23

24. DEMO

25. Pillar 3 - Correlating Data Plane
and Control Plane Telemetry for
Incident Response

26. Pillar 3 - Build and Collect Telemetry Across Control and
Data Plane
26
Runtime Security relies on observability - you don’t know what you don’t know
Example: User Space vs Kernel Space

27. Collect and Correlate Across A Security Data Lake
27
SHARED EKS CLUSTER
Audit Logs eBPF Telemetry
- API Calls
- Policy Creations
- User/Service
Account Activity
- Process Events
- Network Events
- File Changes

28. Pillar 3 Takeaways - Correlate Telemetry Across Data Plane
and Control Plane
- Security starts with observability - you need to collect telemetry from the
processes running in a container all the way to your Kubernetes and Cloud
control plane
- Attackers can hide behind seemingly benign processes - leverage eBPF
Telemetry and forensic techniques such as YARA rule scanning to catch these
nasty attacks
28

29. Malware
Developer laptop
Threat Actor
Identity
Provider
Code
Repository
Cloud service
provider
1 2
3 4
5
Uptycs Sensor
How Uptycs Can Help - Pillar 1: Visibility Across the Supply Chain
Cloud Security
Early Warning System
EKS Cluster

30. How Uptycs Can Help - Pillar 2: Start with RBAC and Go Deeper

31. How Uptycs Can Help - Pillar 3: Correlating Data Plane and Control
Plane Telemetry for Incident Response

32. Q&A
Shift up your cybersecurity with Uptycs!
› Learn more at uptycs.com

33. Thank You
Shift up your cybersecurity with Uptycs!
› Learn more at uptycs.com