SlideShare a Scribd company logo
1 of 33
@OtherDevOpsGene #AllThingsOpen
Keeping your
Kubernetes Cluster
Secure
Gene Gotimer
@OtherDevOpsGene
WEDNESDAY, NOVEMBER 2, 2022
Define and Design the Optimal Survey Experience
KUBERNETES SECURITY
Layers
β€’ Infrastructure
β€’ Hosts
β€’ Cluster
β€’ Build
β€’ Application
β€’ Container images
β€’ Deployment code
β€’ Runtime
β€’ Policies
β€’ Resources
β€’ Network
β€’ System calls and interaction
@OtherDevOpsGene #AllThingsOpen
2
Infrastructure
Build
Runtime
Wrap-up
@OtherDevOpsGene #AllThingsOpen
3
Define and Design the Optimal Survey Experience
INFRASTRUCTURE
Hardening
Kubernetes Hardening Guidance,
National Security Agency (NSA) and
Cybersecurity and Infrastructure Security Agency (CISA).
β€’ Start with the kubernetes.io article
Kubernetes Security Technical Implementation Guide,
Cybersecurity and Infrastructure Security Agency (CISA).
β€’ Start with the stigviewer.com client.
CIS Kubernetes Benchmark,
Center for Internet Security (CIS),
non-government, non-profit.
https://www.cisecurity.org/benchmark/kubernetes/
@OtherDevOpsGene #AllThingsOpen
4
Cluster
configuration
INFRASTRUCTURE
@OtherDevOpsGene #AllThingsOpen
5
Is my Kubernetes cluster installed and configured properly?
β€’ Use Aqua kube-bench
β€’ May not apply to master nodes
INFRASTRUCTURE
@OtherDevOpsGene #AllThingsOpen
6
Cluster
configuration
$ kubectl logs kube-bench-kc82n
[INFO] 3 Worker Node Security Configuration
[INFO] 3.1 Worker Node Configuration Files
[PASS] 3.1.1 Ensure that the kubeconfig file permissions are set to 644 or more restrictive (Manual)
[PASS] 3.1.2 Ensure that the kubelet kubeconfig file ownership is set to root:root (Manual)
[PASS] 3.1.3 Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Manual)
[PASS] 3.1.4 Ensure that the kubelet configuration file ownership is set to root:root (Manual)
[INFO] 3.2 Kubelet
[PASS] 3.2.1 Ensure that the --anonymous-auth argument is set to false (Automated)
[PASS] 3.2.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated)
[PASS] 3.2.3 Ensure that the --client-ca-file argument is set as appropriate (Manual)
[PASS] 3.2.4 Ensure that the --read-only-port argument is set to 0 (Manual)
[PASS] 3.2.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Manual)
[PASS] 3.2.6 Ensure that the --protect-kernel-defaults argument is set to true (Automated)
[PASS] 3.2.7 Ensure that the --make-iptables-util-chains argument is set to true (Automated)
[PASS] 3.2.8 Ensure that the --hostname-override argument is not set (Manual)
[WARN] 3.2.9 Ensure that the --eventRecordQPS argument is set to 0 or a level which ensures appropriate event capture
(Automated)
[PASS] 3.2.10 Ensure that the --rotate-certificates argument is not set to false (Manual)
[PASS] 3.2.11 Ensure that the RotateKubeletServerCertificate argument is set to true (Manual)
[INFO] 3.3 Container Optimized OS
[WARN] 3.3.1 Prefer using Container-Optimized OS when possible (Manual)
Infrastructure
Build
Runtime
Wrap-up
@OtherDevOpsGene #AllThingsOpen
7
Static code analysis
BUILD
@OtherDevOpsGene #AllThingsOpen
8
Are resources configured properly?
β€’ Use Checkov by Bridgecrew
β€’ Scans source code for
β€’ Dockerfiles
β€’ Kubernetes manifests
β€’ Terraform
Static code analysis
BUILD
@OtherDevOpsGene #AllThingsOpen
9
$ checkov -d manifests --quiet --compact
kubernetes scan results:
Passed checks: 1066, Failed checks: 166, Skipped checks: 0
Check: CKV_K8S_20: "Containers should not run with allowPrivilegeEscalation"
FAILED for resource: Deployment.sock-shop.carts
File: /manifests/01-carts-dep.yaml:2-51
Check: CKV_K8S_43: "Image should use digest"
FAILED for resource: Deployment.sock-shop.carts
File: /manifests/01-carts-dep.yaml:2-51
Check: CKV_K8S_38: "Ensure that Service Account Tokens are only mounted where necessary"
FAILED for resource: Deployment.sock-shop.carts
File: /manifests/01-carts-dep.yaml:2-51
Check: CKV_K8S_29: "Apply security context to your pods and containers"
FAILED for resource: Deployment.sock-shop.carts
File: /manifests/01-carts-dep.yaml:2-51
Check: CKV_K8S_8: "Liveness Probe Should be Configured"
FAILED for resource: Deployment.sock-shop.carts
File: /manifests/01-carts-dep.yaml:2-51
Check: CKV_K8S_9: "Readiness Probe Should be Configured"
FAILED for resource: Deployment.sock-shop.carts
File: /manifests/01-carts-dep.yaml:2-51
Check: CKV_K8S_31: "Ensure that the seccomp profile is set to docker/default or runtime/default"
FAILED for resource: Deployment.sock-shop.carts
File: /manifests/01-carts-dep.yaml:2-51
Container
image
scanning
BUILD
@OtherDevOpsGene #AllThingsOpen
10
Are there vulnerabilities or misconfigurations on the
container image?
β€’ Use Aqua Security Trivy
β€’ Scan images for vulnerable packages
β€’ Scan infrastructure-as-code for misconfigurations
Container
image
scanning
BUILD
@OtherDevOpsGene #AllThingsOpen
11
$ trivy config manifests/01-carts-dep.yaml
2022-10-22T15:55:21.615Z INFO Misconfiguration scanning is enabled
2022-10-22T15:55:21.806Z INFO Detected config files: 1
01-carts-dep.yaml (kubernetes)
==============================
Tests: 79 (SUCCESSES: 74, FAILURES: 5, EXCEPTIONS: 0)
Failures: 5 (UNKNOWN: 0, LOW: 3, MEDIUM: 2, HIGH: 0, CRITICAL: 0)
MEDIUM: Container 'carts' of Deployment 'carts' should set 'securityContext.allowPrivilegeEscalation' to false
════════════════════════════════════════
A program inside the container can elevate its own privileges and run as root, which might give the program control over the container and node.
See https://avd.aquasec.com/misconfig/ksv001
────────────────────────────────────────
01-carts-dep.yaml:20-45
────────────────────────────────────────
20 β”Œ - name: carts
21 β”‚ image: weaveworksdemos/carts:0.4.8
22 β”‚ env:
23 β”‚ - name: JAVA_OPTS
24 β”‚ value: -Xms64m -Xmx128m -XX:+UseG1GC -Djava.security.egd=file:/dev/urandom -Dspring.zipkin.enabled=false
25 β”‚ resources:
26 β”‚ limits:
27 β”‚ cpu: 300m
28 β”” memory: 500Mi
..
────────────────────────────────────────
Container image
scanning
BUILD
@OtherDevOpsGene #AllThingsOpen
12
$ trivy image weaveworksdemos/carts:0.4.8 --no-progress
2022-10-22T15:48:40.726Z WARN This OS version is no longer supported by the distribution: alpine 3.4.6
2022-10-22T15:48:40.726Z WARN The vulnerability detection may be insufficient because security updates are not provided
weaveworksdemos/carts:0.4.8 (alpine 3.4.6)
==========================================
Total: 40 (UNKNOWN: 0, LOW: 0, MEDIUM: 23, HIGH: 13, CRITICAL: 4)
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Library β”‚ Vulnerability β”‚ Severity β”‚ Installed Version β”‚ Fixed Version β”‚ Title β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ busybox β”‚ CVE-2017-16544 β”‚ HIGH β”‚ 1.24.2-r12 β”‚ 1.24.2-r13 β”‚ busybox: Insufficient sanitization of filenames when β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ autocompleting β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ https://avd.aquasec.com/nvd/cve-2017-16544 β”‚
β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ β”‚ CVE-2017-15873 β”‚ MEDIUM β”‚ β”‚ β”‚ busybox: Integer overflow in the get_next_block function β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ https://avd.aquasec.com/nvd/cve-2017-15873 β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ freetype β”‚ CVE-2017-8105 β”‚ CRITICAL β”‚ 2.6.3-r0 β”‚ 2.6.3-r1 β”‚ freetype: heap-based buffer overflow related to the β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ t1_decoder_parse_charstrings β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ https://avd.aquasec.com/nvd/cve-2017-8105 β”‚
β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ β”‚ CVE-2017-8287 β”‚ β”‚ β”‚ β”‚ freetype: heap-based buffer overflow related to the β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ t1_builder_close_contour function β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ https://avd.aquasec.com/nvd/cve-2017-8287 β”‚
β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ β”‚ CVE-2016-10244 β”‚ HIGH β”‚ β”‚ β”‚ freetype: parse_charstrings function in type1/t1load.c does β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ not ensure that a font contains... β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ https://avd.aquasec.com/nvd/cve-2016-10244 β”‚
Container
image
scanning
BUILD
@OtherDevOpsGene #AllThingsOpen
13
Are there vulnerabilities on the container image?
β€’ Use Anchore Grype
β€’ Scan images for vulnerable packages
β€’ Finds some different vulnerabilities than
Aqua Trivy
Container
image
scanning
BUILD
@OtherDevOpsGene #AllThingsOpen
14
$ grype weaveworksdemos/carts:0.4.8
NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY
busybox 1.24.2-r12 apk CVE-2021-42386 High
busybox 1.24.2-r12 apk CVE-2018-1000500 High
busybox 1.24.2-r12 apk CVE-2021-42379 High
busybox 1.24.2-r12 apk CVE-2021-42381 High
busybox 1.24.2-r12 apk CVE-2021-42384 High
busybox 1.24.2-r12 1.24.2-r13 apk CVE-2017-15873 Medium
busybox 1.24.2-r12 apk CVE-2018-1000517 Critical
busybox 1.24.2-r12 apk CVE-2022-28391 High
busybox 1.24.2-r12 apk CVE-2021-42385 High
busybox 1.24.2-r12 apk CVE-2018-20679 High
busybox 1.24.2-r12 apk CVE-2021-42378 High
busybox 1.24.2-r12 apk CVE-2021-42376 Medium
busybox 1.24.2-r12 1.24.2-r13 apk CVE-2017-16544 High
busybox 1.24.2-r12 apk CVE-2019-5747 High
busybox 1.24.2-r12 apk CVE-2015-9261 Medium
freetype 2.6.3-r0 2.6.3-r1 apk CVE-2016-10244 High
freetype 2.6.3-r0 apk CVE-2022-27404 Critical
freetype 2.6.3-r0 apk CVE-2016-10328 Critical
freetype 2.6.3-r0 apk CVE-2022-27405 High
freetype 2.6.3-r0 apk CVE-2017-7857 Critical
freetype 2.6.3-r0 2.6.3-r1 apk CVE-2017-8287 Critical
freetype 2.6.3-r0 apk CVE-2017-7858 Critical
freetype 2.6.3-r0 2.6.3-r1 apk CVE-2017-8105 Critical
freetype 2.6.3-r0 apk CVE-2020-15999 Medium
Software
bill of materials
(SBOM)
BUILD
@OtherDevOpsGene #AllThingsOpen
15
What components are in my application and
container image?
β€’ Use Anchore Syft
β€’ OS packages
β€’ Libraries
β€’ Frameworks
Software
bill of materials
(SBOM)
BUILD
@OtherDevOpsGene #AllThingsOpen
16
$ syft weaveworksdemos/carts:0.4.8 --output json --file carts-0.4.8.json
$ grype sbom:carts-0.4.8.json
NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY
busybox 1.24.2-r12 apk CVE-2018-20679 High
busybox 1.24.2-r12 apk CVE-2018-1000517 Critical
busybox 1.24.2-r12 apk CVE-2021-42381 High
busybox 1.24.2-r12 1.24.2-r13 apk CVE-2017-16544 High
busybox 1.24.2-r12 1.24.2-r13 apk CVE-2017-15873 Medium
busybox 1.24.2-r12 apk CVE-2021-42386 High
busybox 1.24.2-r12 apk CVE-2021-42385 High
busybox 1.24.2-r12 apk CVE-2019-5747 High
busybox 1.24.2-r12 apk CVE-2021-42376 Medium
busybox 1.24.2-r12 apk CVE-2018-1000500 High
busybox 1.24.2-r12 apk CVE-2022-28391 High
busybox 1.24.2-r12 apk CVE-2021-42378 High
busybox 1.24.2-r12 apk CVE-2015-9261 Medium
busybox 1.24.2-r12 apk CVE-2021-42384 High
busybox 1.24.2-r12 apk CVE-2021-42379 High
freetype 2.6.3-r0 2.6.3-r1 apk CVE-2016-10244 High
freetype 2.6.3-r0 apk CVE-2022-27404 Critical
freetype 2.6.3-r0 apk CVE-2016-10328 Critical
freetype 2.6.3-r0 apk CVE-2022-27405 High
freetype 2.6.3-r0 apk CVE-2017-7857 Critical
freetype 2.6.3-r0 2.6.3-r1 apk CVE-2017-8287 Critical
...
$ syft weaveworksdemos/carts:0.4.8 --output cyclonedx-json --file carts-0.4.8-dx.json
SBOM
monitoring
BUILD
@OtherDevOpsGene #AllThingsOpen
17
Do any components have newly identified
vulnerabilities?
β€’ Use OWASP Dependency-Track
β€’ Track SBOMs
β€’ Identify vulnerabilities
β€’ Notifications
SBOM
monitoring
BUILD
@OtherDevOpsGene #AllThingsOpen
18
Infrastructure
Build
Runtime
Wrap-up
@OtherDevOpsGene #AllThingsOpen
19
Policy
enforcement
RUNTIME
@OtherDevOpsGene #AllThingsOpen
20
Are my Kubernetes workloads and resources
following my rules?
β€’ Use Open Policy Agent
β€’ Admissions controller
β€’ Compliance rules
Policy
enforcement
RUNTIME
@OtherDevOpsGene #AllThingsOpen
21
$ cat allowed.yaml
apiVersion: v1
kind: Pod
metadata:
name: opa-allowed
spec:
containers:
- name: opa
image: openpolicyagent/opa:0.9.2@sha256:04ff8fce2afd1a3bc26260348e5b290e8d945b1fad4b4c16d22834c2f3a1814a
args:
- "run"
- "--server"
- "--addr=localhost:8080β€œ
$ kubectl apply -f allowed.yaml
pod/opa-allowed created
Policy
enforcement
RUNTIME
@OtherDevOpsGene #AllThingsOpen
22
$ cat disallowed.yaml
apiVersion: v1
kind: Pod
metadata:
name: opa-disallowed
spec:
initContainers:
- name: opainit
image: openpolicyagent/opa:0.9.2
args:
- "run"
- "--server"
- "--addr=localhost:8080"
containers:
- name: opa
image: openpolicyagent/opa:0.9.2
args:
- "run"
- "--server"
- "--addr=localhost:8080β€œ
$ kubectl apply -f disallowed.yaml
Error from server (Forbidden): error when creating "disallowed.yaml": admission webhook "validation.gatekeeper.sh"
denied the request: [container-image-must-have-digest] container <opa> uses an image without a digest
<openpolicyagent/opa:0.9.2>
[container-image-must-have-digest] initContainer <opainit> uses an image without a digest <openpolicyagent/opa:0.9.2>
Resource
limits
RUNTIME
@OtherDevOpsGene #AllThingsOpen
23
Can a few containers hog too much
memory or CPU?
β€’ Set the resource requests and limits
for memory and CPU
β€’ Use Fairwinds Goldilocks
β€’ Watches loads
β€’ Makes request and limit
recommendations
Resource
limits
RUNTIME
@OtherDevOpsGene #AllThingsOpen
24
Define and Design the Optimal Survey Experience
RUNTIME
Network isolation
Can Kubernetes resources reach others they don’t
need to?
β€’ Use a service mesh or CNI
β€’ Build a network policy
β€’ Network Policy editor
β€’ https://networkpolicy.io
@OtherDevOpsGene #AllThingsOpen
25
Define and Design the Optimal Survey Experience
RUNTIME
Network isolation
@OtherDevOpsGene #AllThingsOpen
26
Monitor
behavior
RUNTIME
@OtherDevOpsGene #AllThingsOpen
27
Are any workloads doing something unexpected
on the system?
β€’ Use Falco
β€’ Watches system calls
β€’ Privilege escalation
β€’ Ownership and mode changes
β€’ Unexpected network connections
Monitor
behavior
RUNTIME
@OtherDevOpsGene #AllThingsOpen
28
$ kubectl logs -n falco falco-zplnz
Sat Oct 22 19:53:03 2022: Falco version: 0.33.0 (x86_64)
Sat Oct 22 19:53:03 2022: Falco initialized with configuration file: /etc/falco/falco.yaml
Sat Oct 22 19:53:03 2022: Loading rules from file /etc/falco/falco_rules.yaml
Sat Oct 22 19:53:03 2022: Loading rules from file /etc/falco/falco_rules.local.yaml
Sat Oct 22 19:53:03 2022: The chosen syscall buffer dimension is: 8388608 bytes (8 MBs)
Sat Oct 22 19:53:03 2022: Starting health webserver with threadiness 2, listening on port 8765
Sat Oct 22 19:53:03 2022: Enabled event sources: syscall
Sat Oct 22 19:53:03 2022: Opening capture with Kernel module
19:53:09.668585724: Warning Docker or kubernetes client executed in container (user=<NA> user_loginuid=-1
k8s.ns=gatekeeper-system k8s.pod=gatekeeper-update-crds-hook-l5zr2 container=006eacc6d95b parent=<NA> cmdline=kubectl
apply -f crds/ pid=13671 image=openpolicyagent/gatekeeper-crds:v3.10.0)
19:53:09.739647377: Notice Unexpected connection to K8s API Server from container (command=kubectl apply -f crds/
pid=13671 k8s.ns=gatekeeper-system k8s.pod=gatekeeper-update-crds-hook-l5zr2 container=006eacc6d95b
image=openpolicyagent/gatekeeper-crds:v3.10.0 connection=192.168.34.30:45214->10.100.0.1:443)
19:53:21.068878529: Notice Unexpected connection to K8s API Server from container (command=manager --port=8443 --
health-addr=:9090 --prometheus-port=8888 --logtostderr --log-denies=false --emit-admission-events=false --log-
level=INFO --exempt-namespace=gatekeeper-system --operation=webhook --enable-external-data=false --enable-generator-
resource-expansion=false --log-mutations=false --mutation-annotations=false --disable-cert-rotation=false --max-
serving-threads=-1 --tls-min-version=1.3 --metrics-backend=prometheus --operation=mutation-webhook --disable-opa-
builtin={http.send} pid=14628 k8s.ns=gatekeeper-system k8s.pod=gatekeeper-controller-manager-78b8774b7c-wknwm
container=bb34633b4f13 image=openpolicyagent/gatekeeper:v3.10.0 connection=192.168.60.49:34346->10.100.0.1:443)
19:53:21.133021316: Notice Unexpected connection to K8s API Server from container (command=manager --audit-
interval=60 --log-level=INFO --constraint-violations-limit=20 --audit-from-cache=false --audit-chunk-size=500 --
audit-match-kind-only=false --emit-audit-events=false --operation=audit --operation=status --operation=mutation-
status --logtostderr --health-addr=:9090 --prometheus-port=8888 --enable-external-data=false --enable-generator-
resource-expansion=false --metrics-backend=prometheus --disable-cert-rotation=true pid=14691 k8s.ns=gatekeeper-system
k8s.pod=gatekeeper-audit-9b7795dcf-tff44 container=c603d5981dea image=openpolicyagent/gatekeeper:v3.10.0
connection=192.168.62.41:58520->10.100.0.1:443)
Infrastructure
Build
Runtime
Wrap-up
@OtherDevOpsGene #AllThingsOpen
29
Define and Design the Optimal Survey Experience
WRAP-UP
Key takeaways
β€’ Enforce the principle of least privilege.
β€’ Keep everything up to date.
β€’ Scan your container images frequently,
before and after deployment.
β€’ Monitor your systems for expected and
unexpected behavior.
β€’ And disk space.
@OtherDevOpsGene #AllThingsOpen
30
Define and Design the Optimal Survey Experience
WRAP-UP
Reading list
Kubernetes Hardening Guidance,
National Security Agency (NSA) and
Cybersecurity and Infrastructure Security Agency (CISA).
https://media.defense.gov/2021/Aug/03/2002820425/-1/-
1/1/CTR_KUBERNETES%20HARDENING%20GUIDANCE.PDF
A Closer Look at NSA/CISA Kubernetes Hardening Guidance,
Jim Angel, Pushkar Joglekar, and Savitha Raghunathan.
https://kubernetes.io/blog/2021/10/05/nsa-cisa-
kubernetes-hardening-guidance/
Kubernetes Security Technical Implementation Guide,
Cybersecurity and Infrastructure Security Agency (CISA).
https://public.cyber.mil/stigs/downloads/
CIS Kubernetes Benchmark,
Center for Internet Security (CIS),
https://www.cisecurity.org/benchmark/kubernetes/
@OtherDevOpsGene #AllThingsOpen
31
Define and Design the Optimal Survey Experience
WRAP-UP
Tools
Aqua Security kube-bench:
https://github.com/aquasecurity/kube-bench
Checkov by Bridgecrew: https://github.com/bridgecrewio/checkov
Aqua Security Trivy: https://github.com/aquasecurity/trivy
Anchore Grype: https://github.com/anchore/grype
Anchore Syft: https://github.com/anchore/syft
OWASP Dependency-Track: https://dependencytrack.org
Open Policy Agent: https://www.openpolicyagent.org
Fairwinds Goldilocks: https://github.com/fairwindsops/goldilocks
Network Policy Editor: https://networkpolicy.io
Falco: https://falco.org
@OtherDevOpsGene #AllThingsOpen
32
Questions?
@OtherDevOpsGene #AllThingsOpen
33

More Related Content

What's hot

Lecture 9 key distribution and user authentication
Lecture 9 key distribution and user authentication Lecture 9 key distribution and user authentication
Lecture 9 key distribution and user authentication rajakhurram
Β 
Fundamentals of cryptography
Fundamentals of cryptographyFundamentals of cryptography
Fundamentals of cryptographyHossain Md Shakhawat
Β 
Server Consolidation
Server ConsolidationServer Consolidation
Server ConsolidationManuel Padilha
Β 
Secure Socket Layer
Secure Socket LayerSecure Socket Layer
Secure Socket LayerNaveen Kumar
Β 
Security in Cloud Computing
Security in Cloud ComputingSecurity in Cloud Computing
Security in Cloud ComputingRohit Buddabathina
Β 
Network Security Presentation
Network Security PresentationNetwork Security Presentation
Network Security PresentationAllan Pratt MBA
Β 
Cloud federation.pptx
Cloud federation.pptxCloud federation.pptx
Cloud federation.pptxYbhh
Β 
Cloud Security - Security Aspects of Cloud Computing
Cloud Security - Security Aspects of Cloud ComputingCloud Security - Security Aspects of Cloud Computing
Cloud Security - Security Aspects of Cloud ComputingJim Geovedi
Β 
Email Security : PGP & SMIME
Email Security : PGP & SMIMEEmail Security : PGP & SMIME
Email Security : PGP & SMIMERohit Soni
Β 
Cloud Computing and Security - ISACA Hyderabad Chapter Presentation
Cloud Computing and Security - ISACA Hyderabad Chapter PresentationCloud Computing and Security - ISACA Hyderabad Chapter Presentation
Cloud Computing and Security - ISACA Hyderabad Chapter PresentationVenkateswar Reddy Melachervu
Β 
Automation Patterns for Scalable Secret Management
Automation Patterns for Scalable Secret ManagementAutomation Patterns for Scalable Secret Management
Automation Patterns for Scalable Secret ManagementMary Racter
Β 
Data storage security in cloud computing
Data storage security in cloud computingData storage security in cloud computing
Data storage security in cloud computingSonali Jain
Β 
Key management
Key managementKey management
Key managementSujata Regoti
Β 
VMware Cloud on AWS -- A Technical Deep Dive PPT
VMware Cloud on AWS -- A Technical Deep Dive PPTVMware Cloud on AWS -- A Technical Deep Dive PPT
VMware Cloud on AWS -- A Technical Deep Dive PPTAmazon Web Services
Β 
Benefits Of Building Private Cloud
Benefits Of Building Private CloudBenefits Of Building Private Cloud
Benefits Of Building Private Clouddinobusalachi
Β 
20221122-IBM Power10ε…¨ζ–°IBM Power10 ε…₯ι–€ε’ŒδΈ­ιšŽ .pdf
20221122-IBM Power10ε…¨ζ–°IBM Power10 ε…₯ι–€ε’ŒδΈ­ιšŽ .pdf20221122-IBM Power10ε…¨ζ–°IBM Power10 ε…₯ι–€ε’ŒδΈ­ιšŽ .pdf
20221122-IBM Power10ε…¨ζ–°IBM Power10 ε…₯ι–€ε’ŒδΈ­ιšŽ .pdfken761ken1
Β 
How to do Cryptography right in Android Part One
How to do Cryptography right in Android Part OneHow to do Cryptography right in Android Part One
How to do Cryptography right in Android Part OneArash Ramez
Β 
Digital Signature
Digital SignatureDigital Signature
Digital SignatureMohamed Talaat
Β 

What's hot (20)

Lecture 9 key distribution and user authentication
Lecture 9 key distribution and user authentication Lecture 9 key distribution and user authentication
Lecture 9 key distribution and user authentication
Β 
Fundamentals of cryptography
Fundamentals of cryptographyFundamentals of cryptography
Fundamentals of cryptography
Β 
Server Consolidation
Server ConsolidationServer Consolidation
Server Consolidation
Β 
Secure Socket Layer
Secure Socket LayerSecure Socket Layer
Secure Socket Layer
Β 
Security in Cloud Computing
Security in Cloud ComputingSecurity in Cloud Computing
Security in Cloud Computing
Β 
Network Security Presentation
Network Security PresentationNetwork Security Presentation
Network Security Presentation
Β 
Email security
Email securityEmail security
Email security
Β 
Cloud federation.pptx
Cloud federation.pptxCloud federation.pptx
Cloud federation.pptx
Β 
Cloud Security - Security Aspects of Cloud Computing
Cloud Security - Security Aspects of Cloud ComputingCloud Security - Security Aspects of Cloud Computing
Cloud Security - Security Aspects of Cloud Computing
Β 
Email Security : PGP & SMIME
Email Security : PGP & SMIMEEmail Security : PGP & SMIME
Email Security : PGP & SMIME
Β 
Virtualization security and threat
Virtualization security and threatVirtualization security and threat
Virtualization security and threat
Β 
Cloud Computing and Security - ISACA Hyderabad Chapter Presentation
Cloud Computing and Security - ISACA Hyderabad Chapter PresentationCloud Computing and Security - ISACA Hyderabad Chapter Presentation
Cloud Computing and Security - ISACA Hyderabad Chapter Presentation
Β 
Automation Patterns for Scalable Secret Management
Automation Patterns for Scalable Secret ManagementAutomation Patterns for Scalable Secret Management
Automation Patterns for Scalable Secret Management
Β 
Data storage security in cloud computing
Data storage security in cloud computingData storage security in cloud computing
Data storage security in cloud computing
Β 
Key management
Key managementKey management
Key management
Β 
VMware Cloud on AWS -- A Technical Deep Dive PPT
VMware Cloud on AWS -- A Technical Deep Dive PPTVMware Cloud on AWS -- A Technical Deep Dive PPT
VMware Cloud on AWS -- A Technical Deep Dive PPT
Β 
Benefits Of Building Private Cloud
Benefits Of Building Private CloudBenefits Of Building Private Cloud
Benefits Of Building Private Cloud
Β 
20221122-IBM Power10ε…¨ζ–°IBM Power10 ε…₯ι–€ε’ŒδΈ­ιšŽ .pdf
20221122-IBM Power10ε…¨ζ–°IBM Power10 ε…₯ι–€ε’ŒδΈ­ιšŽ .pdf20221122-IBM Power10ε…¨ζ–°IBM Power10 ε…₯ι–€ε’ŒδΈ­ιšŽ .pdf
20221122-IBM Power10ε…¨ζ–°IBM Power10 ε…₯ι–€ε’ŒδΈ­ιšŽ .pdf
Β 
How to do Cryptography right in Android Part One
How to do Cryptography right in Android Part OneHow to do Cryptography right in Android Part One
How to do Cryptography right in Android Part One
Β 
Digital Signature
Digital SignatureDigital Signature
Digital Signature
Β 

Similar to Keeping Your Kubernetes Cluster Secure

Azure from scratch part 4
Azure from scratch part 4Azure from scratch part 4
Azure from scratch part 4Girish Kalamati
Β 
Digital Forensics and Incident Response in The Cloud Part 3
Digital Forensics and Incident Response in The Cloud Part 3Digital Forensics and Incident Response in The Cloud Part 3
Digital Forensics and Incident Response in The Cloud Part 3Velocidex Enterprises
Β 
Weaveworks at AWS re:Invent 2016: Operations Management with Amazon ECS
Weaveworks at AWS re:Invent 2016: Operations Management with Amazon ECSWeaveworks at AWS re:Invent 2016: Operations Management with Amazon ECS
Weaveworks at AWS re:Invent 2016: Operations Management with Amazon ECSWeaveworks
Β 
DevOpsDaysRiga 2018: Andrew Martin - Continuous Kubernetes Security
DevOpsDaysRiga 2018: Andrew Martin - Continuous Kubernetes Security DevOpsDaysRiga 2018: Andrew Martin - Continuous Kubernetes Security
DevOpsDaysRiga 2018: Andrew Martin - Continuous Kubernetes Security DevOpsDays Riga
Β 
Cloud-native .NET-Microservices mit Kubernetes @BASTAcon
Cloud-native .NET-Microservices mit Kubernetes @BASTAconCloud-native .NET-Microservices mit Kubernetes @BASTAcon
Cloud-native .NET-Microservices mit Kubernetes @BASTAconMario-Leander Reimer
Β 
Testing Terraform
Testing TerraformTesting Terraform
Testing TerraformNathen Harvey
Β 
A Developer’s Guide to Kubernetes Security
A Developer’s Guide to Kubernetes SecurityA Developer’s Guide to Kubernetes Security
A Developer’s Guide to Kubernetes SecurityGene Gotimer
Β 
Continuous Security: From tins to containers - now what!
Continuous Security: From tins to containers - now what!Continuous Security: From tins to containers - now what!
Continuous Security: From tins to containers - now what!Michael Man
Β 
The Container Security Checklist
The Container Security Checklist The Container Security Checklist
The Container Security Checklist LibbySchulze
Β 
Behind the Code 'September 2022 // by Exness
Behind the Code 'September 2022 // by ExnessBehind the Code 'September 2022 // by Exness
Behind the Code 'September 2022 // by ExnessMaxim Gaponov
Β 
12 Ways Not to get 'Hacked' your Kubernetes Cluster
12 Ways Not to get 'Hacked' your Kubernetes Cluster12 Ways Not to get 'Hacked' your Kubernetes Cluster
12 Ways Not to get 'Hacked' your Kubernetes ClusterSuman Chakraborty
Β 
Embacing service-level-objectives of your microservices in your Cl/CD
Embacing service-level-objectives of your microservices in your Cl/CDEmbacing service-level-objectives of your microservices in your Cl/CD
Embacing service-level-objectives of your microservices in your Cl/CDNebulaworks
Β 
Introduction to Container Management on AWS
Introduction to Container Management on AWSIntroduction to Container Management on AWS
Introduction to Container Management on AWSAmazon Web Services
Β 
Continuous Delivery: The Next Frontier
Continuous Delivery: The Next FrontierContinuous Delivery: The Next Frontier
Continuous Delivery: The Next FrontierCarlos Sanchez
Β 
Cloud-native .NET Microservices mit Kubernetes
Cloud-native .NET Microservices mit KubernetesCloud-native .NET Microservices mit Kubernetes
Cloud-native .NET Microservices mit KubernetesQAware GmbH
Β 
WWCode Dallas - Kubernetes: Learning from Zero to Production
WWCode Dallas - Kubernetes: Learning from Zero to ProductionWWCode Dallas - Kubernetes: Learning from Zero to Production
WWCode Dallas - Kubernetes: Learning from Zero to ProductionRosemary Wang
Β 
Deploying Cloud Native Red Team Infrastructure with Kubernetes, Istio and Envoy
Deploying Cloud Native Red Team Infrastructure with Kubernetes, Istio and Envoy Deploying Cloud Native Red Team Infrastructure with Kubernetes, Istio and Envoy
Deploying Cloud Native Red Team Infrastructure with Kubernetes, Istio and Envoy Jeffrey Holden
Β 
Whatever it takes - Fixing SQLIA and XSS in the process
Whatever it takes - Fixing SQLIA and XSS in the processWhatever it takes - Fixing SQLIA and XSS in the process
Whatever it takes - Fixing SQLIA and XSS in the processguest3379bd
Β 
Dive into DevOps | March, Building with Terraform, Volodymyr Tsap
Dive into DevOps | March, Building with Terraform, Volodymyr TsapDive into DevOps | March, Building with Terraform, Volodymyr Tsap
Dive into DevOps | March, Building with Terraform, Volodymyr TsapProvectus
Β 

Similar to Keeping Your Kubernetes Cluster Secure (20)

Azure from scratch part 4
Azure from scratch part 4Azure from scratch part 4
Azure from scratch part 4
Β 
Digital Forensics and Incident Response in The Cloud Part 3
Digital Forensics and Incident Response in The Cloud Part 3Digital Forensics and Incident Response in The Cloud Part 3
Digital Forensics and Incident Response in The Cloud Part 3
Β 
Weaveworks at AWS re:Invent 2016: Operations Management with Amazon ECS
Weaveworks at AWS re:Invent 2016: Operations Management with Amazon ECSWeaveworks at AWS re:Invent 2016: Operations Management with Amazon ECS
Weaveworks at AWS re:Invent 2016: Operations Management with Amazon ECS
Β 
DevOpsDaysRiga 2018: Andrew Martin - Continuous Kubernetes Security
DevOpsDaysRiga 2018: Andrew Martin - Continuous Kubernetes Security DevOpsDaysRiga 2018: Andrew Martin - Continuous Kubernetes Security
DevOpsDaysRiga 2018: Andrew Martin - Continuous Kubernetes Security
Β 
Cloud-native .NET-Microservices mit Kubernetes @BASTAcon
Cloud-native .NET-Microservices mit Kubernetes @BASTAconCloud-native .NET-Microservices mit Kubernetes @BASTAcon
Cloud-native .NET-Microservices mit Kubernetes @BASTAcon
Β 
Testing Terraform
Testing TerraformTesting Terraform
Testing Terraform
Β 
A Developer’s Guide to Kubernetes Security
A Developer’s Guide to Kubernetes SecurityA Developer’s Guide to Kubernetes Security
A Developer’s Guide to Kubernetes Security
Β 
Continuous Security: From tins to containers - now what!
Continuous Security: From tins to containers - now what!Continuous Security: From tins to containers - now what!
Continuous Security: From tins to containers - now what!
Β 
The Container Security Checklist
The Container Security Checklist The Container Security Checklist
The Container Security Checklist
Β 
Behind the Code 'September 2022 // by Exness
Behind the Code 'September 2022 // by ExnessBehind the Code 'September 2022 // by Exness
Behind the Code 'September 2022 // by Exness
Β 
12 Ways Not to get 'Hacked' your Kubernetes Cluster
12 Ways Not to get 'Hacked' your Kubernetes Cluster12 Ways Not to get 'Hacked' your Kubernetes Cluster
12 Ways Not to get 'Hacked' your Kubernetes Cluster
Β 
Embacing service-level-objectives of your microservices in your Cl/CD
Embacing service-level-objectives of your microservices in your Cl/CDEmbacing service-level-objectives of your microservices in your Cl/CD
Embacing service-level-objectives of your microservices in your Cl/CD
Β 
Introduction to Container Management on AWS
Introduction to Container Management on AWSIntroduction to Container Management on AWS
Introduction to Container Management on AWS
Β 
Continuous Delivery: The Next Frontier
Continuous Delivery: The Next FrontierContinuous Delivery: The Next Frontier
Continuous Delivery: The Next Frontier
Β 
Cloud-native .NET Microservices mit Kubernetes
Cloud-native .NET Microservices mit KubernetesCloud-native .NET Microservices mit Kubernetes
Cloud-native .NET Microservices mit Kubernetes
Β 
WWCode Dallas - Kubernetes: Learning from Zero to Production
WWCode Dallas - Kubernetes: Learning from Zero to ProductionWWCode Dallas - Kubernetes: Learning from Zero to Production
WWCode Dallas - Kubernetes: Learning from Zero to Production
Β 
Deploying Cloud Native Red Team Infrastructure with Kubernetes, Istio and Envoy
Deploying Cloud Native Red Team Infrastructure with Kubernetes, Istio and Envoy Deploying Cloud Native Red Team Infrastructure with Kubernetes, Istio and Envoy
Deploying Cloud Native Red Team Infrastructure with Kubernetes, Istio and Envoy
Β 
Whatever it takes - Fixing SQLIA and XSS in the process
Whatever it takes - Fixing SQLIA and XSS in the processWhatever it takes - Fixing SQLIA and XSS in the process
Whatever it takes - Fixing SQLIA and XSS in the process
Β 
Kubernetes 101 for_penetration_testers_-_null_mumbai
Kubernetes 101 for_penetration_testers_-_null_mumbaiKubernetes 101 for_penetration_testers_-_null_mumbai
Kubernetes 101 for_penetration_testers_-_null_mumbai
Β 
Dive into DevOps | March, Building with Terraform, Volodymyr Tsap
Dive into DevOps | March, Building with Terraform, Volodymyr TsapDive into DevOps | March, Building with Terraform, Volodymyr Tsap
Dive into DevOps | March, Building with Terraform, Volodymyr Tsap
Β 

More from Gene Gotimer

How I Learned to Stop Worrying and Love Legacy Code
How I Learned to Stop Worrying and Love Legacy CodeHow I Learned to Stop Worrying and Love Legacy Code
How I Learned to Stop Worrying and Love Legacy CodeGene Gotimer
Β 
Ten Ways To Doom Your DevOps
Ten Ways To Doom Your DevOpsTen Ways To Doom Your DevOps
Ten Ways To Doom Your DevOpsGene Gotimer
Β 
Keeping your Kubernetes Cluster Secure
Keeping your Kubernetes Cluster SecureKeeping your Kubernetes Cluster Secure
Keeping your Kubernetes Cluster SecureGene Gotimer
Β 
Explain DevOps To Me Like I’m Five: DevOps for Managers
Explain DevOps To Me Like I’m Five: DevOps for ManagersExplain DevOps To Me Like I’m Five: DevOps for Managers
Explain DevOps To Me Like I’m Five: DevOps for ManagersGene Gotimer
Β 
Keeping your Kubernetes Cluster Secure
Keeping your Kubernetes Cluster SecureKeeping your Kubernetes Cluster Secure
Keeping your Kubernetes Cluster SecureGene Gotimer
Β 
Creative Solutions to Already Solved Problems II
Creative Solutions to Already Solved Problems IICreative Solutions to Already Solved Problems II
Creative Solutions to Already Solved Problems IIGene Gotimer
Β 
Creative Solutions to Already Solved Problems
Creative Solutions to Already Solved ProblemsCreative Solutions to Already Solved Problems
Creative Solutions to Already Solved ProblemsGene Gotimer
Β 
Get to Green: How to Safely Refactor Legacy Code
Get to Green: How to Safely Refactor Legacy CodeGet to Green: How to Safely Refactor Legacy Code
Get to Green: How to Safely Refactor Legacy CodeGene Gotimer
Β 
DevOps for Leadership
DevOps for LeadershipDevOps for Leadership
DevOps for LeadershipGene Gotimer
Β 
Pyramid Discussion: DevOps Adoption in Large, Slow Organizations
Pyramid Discussion: DevOps Adoption in Large, Slow OrganizationsPyramid Discussion: DevOps Adoption in Large, Slow Organizations
Pyramid Discussion: DevOps Adoption in Large, Slow OrganizationsGene Gotimer
Β 
A better faster pipeline for software delivery, even in the government
A better faster pipeline for software delivery, even in the governmentA better faster pipeline for software delivery, even in the government
A better faster pipeline for software delivery, even in the governmentGene Gotimer
Β 
Building the Pipeline of My Dreams
Building the Pipeline of My DreamsBuilding the Pipeline of My Dreams
Building the Pipeline of My DreamsGene Gotimer
Β 
Tests Your Pipeline Might Be Missing
Tests Your Pipeline Might Be MissingTests Your Pipeline Might Be Missing
Tests Your Pipeline Might Be MissingGene Gotimer
Β 
A Definition of Done for DevSecOps
A Definition of Done for DevSecOpsA Definition of Done for DevSecOps
A Definition of Done for DevSecOpsGene Gotimer
Β 
A Better, Faster Pipeline for Software Delivery
A Better, Faster Pipeline for Software DeliveryA Better, Faster Pipeline for Software Delivery
A Better, Faster Pipeline for Software DeliveryGene Gotimer
Β 
Open Source Security Tools for the Pipeline
Open Source Security Tools for the PipelineOpen Source Security Tools for the Pipeline
Open Source Security Tools for the PipelineGene Gotimer
Β 
Which Development Metrics Should I Watch?
Which Development Metrics Should I Watch?Which Development Metrics Should I Watch?
Which Development Metrics Should I Watch?Gene Gotimer
Β 
Add Security Testing Tools to Your Delivery Pipeline
Add Security Testing Tools to Your Delivery PipelineAdd Security Testing Tools to Your Delivery Pipeline
Add Security Testing Tools to Your Delivery PipelineGene Gotimer
Β 
Testing in a Continuous Delivery Pipeline - Better, Faster, Cheaper
Testing in a Continuous Delivery Pipeline - Better, Faster, CheaperTesting in a Continuous Delivery Pipeline - Better, Faster, Cheaper
Testing in a Continuous Delivery Pipeline - Better, Faster, CheaperGene Gotimer
Β 
Experiences Bringing CD to a DoD Project
Experiences Bringing CD to a DoD ProjectExperiences Bringing CD to a DoD Project
Experiences Bringing CD to a DoD ProjectGene Gotimer
Β 

More from Gene Gotimer (20)

How I Learned to Stop Worrying and Love Legacy Code
How I Learned to Stop Worrying and Love Legacy CodeHow I Learned to Stop Worrying and Love Legacy Code
How I Learned to Stop Worrying and Love Legacy Code
Β 
Ten Ways To Doom Your DevOps
Ten Ways To Doom Your DevOpsTen Ways To Doom Your DevOps
Ten Ways To Doom Your DevOps
Β 
Keeping your Kubernetes Cluster Secure
Keeping your Kubernetes Cluster SecureKeeping your Kubernetes Cluster Secure
Keeping your Kubernetes Cluster Secure
Β 
Explain DevOps To Me Like I’m Five: DevOps for Managers
Explain DevOps To Me Like I’m Five: DevOps for ManagersExplain DevOps To Me Like I’m Five: DevOps for Managers
Explain DevOps To Me Like I’m Five: DevOps for Managers
Β 
Keeping your Kubernetes Cluster Secure
Keeping your Kubernetes Cluster SecureKeeping your Kubernetes Cluster Secure
Keeping your Kubernetes Cluster Secure
Β 
Creative Solutions to Already Solved Problems II
Creative Solutions to Already Solved Problems IICreative Solutions to Already Solved Problems II
Creative Solutions to Already Solved Problems II
Β 
Creative Solutions to Already Solved Problems
Creative Solutions to Already Solved ProblemsCreative Solutions to Already Solved Problems
Creative Solutions to Already Solved Problems
Β 
Get to Green: How to Safely Refactor Legacy Code
Get to Green: How to Safely Refactor Legacy CodeGet to Green: How to Safely Refactor Legacy Code
Get to Green: How to Safely Refactor Legacy Code
Β 
DevOps for Leadership
DevOps for LeadershipDevOps for Leadership
DevOps for Leadership
Β 
Pyramid Discussion: DevOps Adoption in Large, Slow Organizations
Pyramid Discussion: DevOps Adoption in Large, Slow OrganizationsPyramid Discussion: DevOps Adoption in Large, Slow Organizations
Pyramid Discussion: DevOps Adoption in Large, Slow Organizations
Β 
A better faster pipeline for software delivery, even in the government
A better faster pipeline for software delivery, even in the governmentA better faster pipeline for software delivery, even in the government
A better faster pipeline for software delivery, even in the government
Β 
Building the Pipeline of My Dreams
Building the Pipeline of My DreamsBuilding the Pipeline of My Dreams
Building the Pipeline of My Dreams
Β 
Tests Your Pipeline Might Be Missing
Tests Your Pipeline Might Be MissingTests Your Pipeline Might Be Missing
Tests Your Pipeline Might Be Missing
Β 
A Definition of Done for DevSecOps
A Definition of Done for DevSecOpsA Definition of Done for DevSecOps
A Definition of Done for DevSecOps
Β 
A Better, Faster Pipeline for Software Delivery
A Better, Faster Pipeline for Software DeliveryA Better, Faster Pipeline for Software Delivery
A Better, Faster Pipeline for Software Delivery
Β 
Open Source Security Tools for the Pipeline
Open Source Security Tools for the PipelineOpen Source Security Tools for the Pipeline
Open Source Security Tools for the Pipeline
Β 
Which Development Metrics Should I Watch?
Which Development Metrics Should I Watch?Which Development Metrics Should I Watch?
Which Development Metrics Should I Watch?
Β 
Add Security Testing Tools to Your Delivery Pipeline
Add Security Testing Tools to Your Delivery PipelineAdd Security Testing Tools to Your Delivery Pipeline
Add Security Testing Tools to Your Delivery Pipeline
Β 
Testing in a Continuous Delivery Pipeline - Better, Faster, Cheaper
Testing in a Continuous Delivery Pipeline - Better, Faster, CheaperTesting in a Continuous Delivery Pipeline - Better, Faster, Cheaper
Testing in a Continuous Delivery Pipeline - Better, Faster, Cheaper
Β 
Experiences Bringing CD to a DoD Project
Experiences Bringing CD to a DoD ProjectExperiences Bringing CD to a DoD Project
Experiences Bringing CD to a DoD Project
Β 

Recently uploaded

CALL ON βž₯8923113531 πŸ”Call Girls Kakori Lucknow best sexual service Online β˜‚οΈ
CALL ON βž₯8923113531 πŸ”Call Girls Kakori Lucknow best sexual service Online  β˜‚οΈCALL ON βž₯8923113531 πŸ”Call Girls Kakori Lucknow best sexual service Online  β˜‚οΈ
CALL ON βž₯8923113531 πŸ”Call Girls Kakori Lucknow best sexual service Online β˜‚οΈanilsa9823
Β 
Hand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxHand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxbodapatigopi8531
Β 
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsUnveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsAlberto GonzΓ‘lez Trastoy
Β 
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...Health
Β 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...kellynguyen01
Β 
TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providerTECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providermohitmore19
Β 
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...ICS
Β 
Diamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with PrecisionDiamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with PrecisionSolGuruz
Β 
The Ultimate Test Automation Guide_ Best Practices and Tips.pdf
The Ultimate Test Automation Guide_ Best Practices and Tips.pdfThe Ultimate Test Automation Guide_ Best Practices and Tips.pdf
The Ultimate Test Automation Guide_ Best Practices and Tips.pdfkalichargn70th171
Β 
HR Software Buyers Guide in 2024 - HRSoftware.com
HR Software Buyers Guide in 2024 - HRSoftware.comHR Software Buyers Guide in 2024 - HRSoftware.com
HR Software Buyers Guide in 2024 - HRSoftware.comFatema Valibhai
Β 
Unlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language ModelsUnlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language Modelsaagamshah0812
Β 
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...panagenda
Β 
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...MyIntelliSource, Inc.
Β 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...MyIntelliSource, Inc.
Β 
Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsRight Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsJhone kinadey
Β 
SyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AI
SyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AISyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AI
SyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AIABDERRAOUF MEHENNI
Β 
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfLearn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfkalichargn70th171
Β 
Microsoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdfMicrosoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdfWilly Marroquin (WillyDevNET)
Β 
Optimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTVOptimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTVshikhaohhpro
Β 

Recently uploaded (20)

CALL ON βž₯8923113531 πŸ”Call Girls Kakori Lucknow best sexual service Online β˜‚οΈ
CALL ON βž₯8923113531 πŸ”Call Girls Kakori Lucknow best sexual service Online  β˜‚οΈCALL ON βž₯8923113531 πŸ”Call Girls Kakori Lucknow best sexual service Online  β˜‚οΈ
CALL ON βž₯8923113531 πŸ”Call Girls Kakori Lucknow best sexual service Online β˜‚οΈ
Β 
Hand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxHand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptx
Β 
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsUnveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Β 
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
Β 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Β 
TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providerTECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service provider
Β 
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
Β 
Diamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with PrecisionDiamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with Precision
Β 
The Ultimate Test Automation Guide_ Best Practices and Tips.pdf
The Ultimate Test Automation Guide_ Best Practices and Tips.pdfThe Ultimate Test Automation Guide_ Best Practices and Tips.pdf
The Ultimate Test Automation Guide_ Best Practices and Tips.pdf
Β 
Vip Call Girls Noida ➑️ Delhi ➑️ 9999965857 No Advance 24HRS Live
Vip Call Girls Noida ➑️ Delhi ➑️ 9999965857 No Advance 24HRS LiveVip Call Girls Noida ➑️ Delhi ➑️ 9999965857 No Advance 24HRS Live
Vip Call Girls Noida ➑️ Delhi ➑️ 9999965857 No Advance 24HRS Live
Β 
HR Software Buyers Guide in 2024 - HRSoftware.com
HR Software Buyers Guide in 2024 - HRSoftware.comHR Software Buyers Guide in 2024 - HRSoftware.com
HR Software Buyers Guide in 2024 - HRSoftware.com
Β 
Unlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language ModelsUnlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language Models
Β 
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
Β 
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Β 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Β 
Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsRight Money Management App For Your Financial Goals
Right Money Management App For Your Financial Goals
Β 
SyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AI
SyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AISyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AI
SyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AI
Β 
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfLearn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Β 
Microsoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdfMicrosoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdf
Β 
Optimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTVOptimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTV
Β 

Keeping Your Kubernetes Cluster Secure

  • 1. @OtherDevOpsGene #AllThingsOpen Keeping your Kubernetes Cluster Secure Gene Gotimer @OtherDevOpsGene WEDNESDAY, NOVEMBER 2, 2022
  • 2. Define and Design the Optimal Survey Experience KUBERNETES SECURITY Layers β€’ Infrastructure β€’ Hosts β€’ Cluster β€’ Build β€’ Application β€’ Container images β€’ Deployment code β€’ Runtime β€’ Policies β€’ Resources β€’ Network β€’ System calls and interaction @OtherDevOpsGene #AllThingsOpen 2
  • 4. Define and Design the Optimal Survey Experience INFRASTRUCTURE Hardening Kubernetes Hardening Guidance, National Security Agency (NSA) and Cybersecurity and Infrastructure Security Agency (CISA). β€’ Start with the kubernetes.io article Kubernetes Security Technical Implementation Guide, Cybersecurity and Infrastructure Security Agency (CISA). β€’ Start with the stigviewer.com client. CIS Kubernetes Benchmark, Center for Internet Security (CIS), non-government, non-profit. https://www.cisecurity.org/benchmark/kubernetes/ @OtherDevOpsGene #AllThingsOpen 4
  • 5. Cluster configuration INFRASTRUCTURE @OtherDevOpsGene #AllThingsOpen 5 Is my Kubernetes cluster installed and configured properly? β€’ Use Aqua kube-bench β€’ May not apply to master nodes
  • 6. INFRASTRUCTURE @OtherDevOpsGene #AllThingsOpen 6 Cluster configuration $ kubectl logs kube-bench-kc82n [INFO] 3 Worker Node Security Configuration [INFO] 3.1 Worker Node Configuration Files [PASS] 3.1.1 Ensure that the kubeconfig file permissions are set to 644 or more restrictive (Manual) [PASS] 3.1.2 Ensure that the kubelet kubeconfig file ownership is set to root:root (Manual) [PASS] 3.1.3 Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Manual) [PASS] 3.1.4 Ensure that the kubelet configuration file ownership is set to root:root (Manual) [INFO] 3.2 Kubelet [PASS] 3.2.1 Ensure that the --anonymous-auth argument is set to false (Automated) [PASS] 3.2.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) [PASS] 3.2.3 Ensure that the --client-ca-file argument is set as appropriate (Manual) [PASS] 3.2.4 Ensure that the --read-only-port argument is set to 0 (Manual) [PASS] 3.2.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Manual) [PASS] 3.2.6 Ensure that the --protect-kernel-defaults argument is set to true (Automated) [PASS] 3.2.7 Ensure that the --make-iptables-util-chains argument is set to true (Automated) [PASS] 3.2.8 Ensure that the --hostname-override argument is not set (Manual) [WARN] 3.2.9 Ensure that the --eventRecordQPS argument is set to 0 or a level which ensures appropriate event capture (Automated) [PASS] 3.2.10 Ensure that the --rotate-certificates argument is not set to false (Manual) [PASS] 3.2.11 Ensure that the RotateKubeletServerCertificate argument is set to true (Manual) [INFO] 3.3 Container Optimized OS [WARN] 3.3.1 Prefer using Container-Optimized OS when possible (Manual)
  • 8. Static code analysis BUILD @OtherDevOpsGene #AllThingsOpen 8 Are resources configured properly? β€’ Use Checkov by Bridgecrew β€’ Scans source code for β€’ Dockerfiles β€’ Kubernetes manifests β€’ Terraform
  • 9. Static code analysis BUILD @OtherDevOpsGene #AllThingsOpen 9 $ checkov -d manifests --quiet --compact kubernetes scan results: Passed checks: 1066, Failed checks: 166, Skipped checks: 0 Check: CKV_K8S_20: "Containers should not run with allowPrivilegeEscalation" FAILED for resource: Deployment.sock-shop.carts File: /manifests/01-carts-dep.yaml:2-51 Check: CKV_K8S_43: "Image should use digest" FAILED for resource: Deployment.sock-shop.carts File: /manifests/01-carts-dep.yaml:2-51 Check: CKV_K8S_38: "Ensure that Service Account Tokens are only mounted where necessary" FAILED for resource: Deployment.sock-shop.carts File: /manifests/01-carts-dep.yaml:2-51 Check: CKV_K8S_29: "Apply security context to your pods and containers" FAILED for resource: Deployment.sock-shop.carts File: /manifests/01-carts-dep.yaml:2-51 Check: CKV_K8S_8: "Liveness Probe Should be Configured" FAILED for resource: Deployment.sock-shop.carts File: /manifests/01-carts-dep.yaml:2-51 Check: CKV_K8S_9: "Readiness Probe Should be Configured" FAILED for resource: Deployment.sock-shop.carts File: /manifests/01-carts-dep.yaml:2-51 Check: CKV_K8S_31: "Ensure that the seccomp profile is set to docker/default or runtime/default" FAILED for resource: Deployment.sock-shop.carts File: /manifests/01-carts-dep.yaml:2-51
  • 10. Container image scanning BUILD @OtherDevOpsGene #AllThingsOpen 10 Are there vulnerabilities or misconfigurations on the container image? β€’ Use Aqua Security Trivy β€’ Scan images for vulnerable packages β€’ Scan infrastructure-as-code for misconfigurations
  • 11. Container image scanning BUILD @OtherDevOpsGene #AllThingsOpen 11 $ trivy config manifests/01-carts-dep.yaml 2022-10-22T15:55:21.615Z INFO Misconfiguration scanning is enabled 2022-10-22T15:55:21.806Z INFO Detected config files: 1 01-carts-dep.yaml (kubernetes) ============================== Tests: 79 (SUCCESSES: 74, FAILURES: 5, EXCEPTIONS: 0) Failures: 5 (UNKNOWN: 0, LOW: 3, MEDIUM: 2, HIGH: 0, CRITICAL: 0) MEDIUM: Container 'carts' of Deployment 'carts' should set 'securityContext.allowPrivilegeEscalation' to false ════════════════════════════════════════ A program inside the container can elevate its own privileges and run as root, which might give the program control over the container and node. See https://avd.aquasec.com/misconfig/ksv001 ──────────────────────────────────────── 01-carts-dep.yaml:20-45 ──────────────────────────────────────── 20 β”Œ - name: carts 21 β”‚ image: weaveworksdemos/carts:0.4.8 22 β”‚ env: 23 β”‚ - name: JAVA_OPTS 24 β”‚ value: -Xms64m -Xmx128m -XX:+UseG1GC -Djava.security.egd=file:/dev/urandom -Dspring.zipkin.enabled=false 25 β”‚ resources: 26 β”‚ limits: 27 β”‚ cpu: 300m 28 β”” memory: 500Mi .. ────────────────────────────────────────
  • 12. Container image scanning BUILD @OtherDevOpsGene #AllThingsOpen 12 $ trivy image weaveworksdemos/carts:0.4.8 --no-progress 2022-10-22T15:48:40.726Z WARN This OS version is no longer supported by the distribution: alpine 3.4.6 2022-10-22T15:48:40.726Z WARN The vulnerability detection may be insufficient because security updates are not provided weaveworksdemos/carts:0.4.8 (alpine 3.4.6) ========================================== Total: 40 (UNKNOWN: 0, LOW: 0, MEDIUM: 23, HIGH: 13, CRITICAL: 4) β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Library β”‚ Vulnerability β”‚ Severity β”‚ Installed Version β”‚ Fixed Version β”‚ Title β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ busybox β”‚ CVE-2017-16544 β”‚ HIGH β”‚ 1.24.2-r12 β”‚ 1.24.2-r13 β”‚ busybox: Insufficient sanitization of filenames when β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ autocompleting β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ https://avd.aquasec.com/nvd/cve-2017-16544 β”‚ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”‚ CVE-2017-15873 β”‚ MEDIUM β”‚ β”‚ β”‚ busybox: Integer overflow in the get_next_block function β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ https://avd.aquasec.com/nvd/cve-2017-15873 β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ freetype β”‚ CVE-2017-8105 β”‚ CRITICAL β”‚ 2.6.3-r0 β”‚ 2.6.3-r1 β”‚ freetype: heap-based buffer overflow related to the β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ t1_decoder_parse_charstrings β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ https://avd.aquasec.com/nvd/cve-2017-8105 β”‚ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”‚ CVE-2017-8287 β”‚ β”‚ β”‚ β”‚ freetype: heap-based buffer overflow related to the β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ t1_builder_close_contour function β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ https://avd.aquasec.com/nvd/cve-2017-8287 β”‚ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”‚ CVE-2016-10244 β”‚ HIGH β”‚ β”‚ β”‚ freetype: parse_charstrings function in type1/t1load.c does β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ not ensure that a font contains... β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ https://avd.aquasec.com/nvd/cve-2016-10244 β”‚
  • 13. Container image scanning BUILD @OtherDevOpsGene #AllThingsOpen 13 Are there vulnerabilities on the container image? β€’ Use Anchore Grype β€’ Scan images for vulnerable packages β€’ Finds some different vulnerabilities than Aqua Trivy
  • 14. Container image scanning BUILD @OtherDevOpsGene #AllThingsOpen 14 $ grype weaveworksdemos/carts:0.4.8 NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY busybox 1.24.2-r12 apk CVE-2021-42386 High busybox 1.24.2-r12 apk CVE-2018-1000500 High busybox 1.24.2-r12 apk CVE-2021-42379 High busybox 1.24.2-r12 apk CVE-2021-42381 High busybox 1.24.2-r12 apk CVE-2021-42384 High busybox 1.24.2-r12 1.24.2-r13 apk CVE-2017-15873 Medium busybox 1.24.2-r12 apk CVE-2018-1000517 Critical busybox 1.24.2-r12 apk CVE-2022-28391 High busybox 1.24.2-r12 apk CVE-2021-42385 High busybox 1.24.2-r12 apk CVE-2018-20679 High busybox 1.24.2-r12 apk CVE-2021-42378 High busybox 1.24.2-r12 apk CVE-2021-42376 Medium busybox 1.24.2-r12 1.24.2-r13 apk CVE-2017-16544 High busybox 1.24.2-r12 apk CVE-2019-5747 High busybox 1.24.2-r12 apk CVE-2015-9261 Medium freetype 2.6.3-r0 2.6.3-r1 apk CVE-2016-10244 High freetype 2.6.3-r0 apk CVE-2022-27404 Critical freetype 2.6.3-r0 apk CVE-2016-10328 Critical freetype 2.6.3-r0 apk CVE-2022-27405 High freetype 2.6.3-r0 apk CVE-2017-7857 Critical freetype 2.6.3-r0 2.6.3-r1 apk CVE-2017-8287 Critical freetype 2.6.3-r0 apk CVE-2017-7858 Critical freetype 2.6.3-r0 2.6.3-r1 apk CVE-2017-8105 Critical freetype 2.6.3-r0 apk CVE-2020-15999 Medium
  • 15. Software bill of materials (SBOM) BUILD @OtherDevOpsGene #AllThingsOpen 15 What components are in my application and container image? β€’ Use Anchore Syft β€’ OS packages β€’ Libraries β€’ Frameworks
  • 16. Software bill of materials (SBOM) BUILD @OtherDevOpsGene #AllThingsOpen 16 $ syft weaveworksdemos/carts:0.4.8 --output json --file carts-0.4.8.json $ grype sbom:carts-0.4.8.json NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY busybox 1.24.2-r12 apk CVE-2018-20679 High busybox 1.24.2-r12 apk CVE-2018-1000517 Critical busybox 1.24.2-r12 apk CVE-2021-42381 High busybox 1.24.2-r12 1.24.2-r13 apk CVE-2017-16544 High busybox 1.24.2-r12 1.24.2-r13 apk CVE-2017-15873 Medium busybox 1.24.2-r12 apk CVE-2021-42386 High busybox 1.24.2-r12 apk CVE-2021-42385 High busybox 1.24.2-r12 apk CVE-2019-5747 High busybox 1.24.2-r12 apk CVE-2021-42376 Medium busybox 1.24.2-r12 apk CVE-2018-1000500 High busybox 1.24.2-r12 apk CVE-2022-28391 High busybox 1.24.2-r12 apk CVE-2021-42378 High busybox 1.24.2-r12 apk CVE-2015-9261 Medium busybox 1.24.2-r12 apk CVE-2021-42384 High busybox 1.24.2-r12 apk CVE-2021-42379 High freetype 2.6.3-r0 2.6.3-r1 apk CVE-2016-10244 High freetype 2.6.3-r0 apk CVE-2022-27404 Critical freetype 2.6.3-r0 apk CVE-2016-10328 Critical freetype 2.6.3-r0 apk CVE-2022-27405 High freetype 2.6.3-r0 apk CVE-2017-7857 Critical freetype 2.6.3-r0 2.6.3-r1 apk CVE-2017-8287 Critical ... $ syft weaveworksdemos/carts:0.4.8 --output cyclonedx-json --file carts-0.4.8-dx.json
  • 17. SBOM monitoring BUILD @OtherDevOpsGene #AllThingsOpen 17 Do any components have newly identified vulnerabilities? β€’ Use OWASP Dependency-Track β€’ Track SBOMs β€’ Identify vulnerabilities β€’ Notifications
  • 20. Policy enforcement RUNTIME @OtherDevOpsGene #AllThingsOpen 20 Are my Kubernetes workloads and resources following my rules? β€’ Use Open Policy Agent β€’ Admissions controller β€’ Compliance rules
  • 21. Policy enforcement RUNTIME @OtherDevOpsGene #AllThingsOpen 21 $ cat allowed.yaml apiVersion: v1 kind: Pod metadata: name: opa-allowed spec: containers: - name: opa image: openpolicyagent/opa:0.9.2@sha256:04ff8fce2afd1a3bc26260348e5b290e8d945b1fad4b4c16d22834c2f3a1814a args: - "run" - "--server" - "--addr=localhost:8080β€œ $ kubectl apply -f allowed.yaml pod/opa-allowed created
  • 22. Policy enforcement RUNTIME @OtherDevOpsGene #AllThingsOpen 22 $ cat disallowed.yaml apiVersion: v1 kind: Pod metadata: name: opa-disallowed spec: initContainers: - name: opainit image: openpolicyagent/opa:0.9.2 args: - "run" - "--server" - "--addr=localhost:8080" containers: - name: opa image: openpolicyagent/opa:0.9.2 args: - "run" - "--server" - "--addr=localhost:8080β€œ $ kubectl apply -f disallowed.yaml Error from server (Forbidden): error when creating "disallowed.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [container-image-must-have-digest] container <opa> uses an image without a digest <openpolicyagent/opa:0.9.2> [container-image-must-have-digest] initContainer <opainit> uses an image without a digest <openpolicyagent/opa:0.9.2>
  • 23. Resource limits RUNTIME @OtherDevOpsGene #AllThingsOpen 23 Can a few containers hog too much memory or CPU? β€’ Set the resource requests and limits for memory and CPU β€’ Use Fairwinds Goldilocks β€’ Watches loads β€’ Makes request and limit recommendations
  • 25. Define and Design the Optimal Survey Experience RUNTIME Network isolation Can Kubernetes resources reach others they don’t need to? β€’ Use a service mesh or CNI β€’ Build a network policy β€’ Network Policy editor β€’ https://networkpolicy.io @OtherDevOpsGene #AllThingsOpen 25
  • 26. Define and Design the Optimal Survey Experience RUNTIME Network isolation @OtherDevOpsGene #AllThingsOpen 26
  • 27. Monitor behavior RUNTIME @OtherDevOpsGene #AllThingsOpen 27 Are any workloads doing something unexpected on the system? β€’ Use Falco β€’ Watches system calls β€’ Privilege escalation β€’ Ownership and mode changes β€’ Unexpected network connections
  • 28. Monitor behavior RUNTIME @OtherDevOpsGene #AllThingsOpen 28 $ kubectl logs -n falco falco-zplnz Sat Oct 22 19:53:03 2022: Falco version: 0.33.0 (x86_64) Sat Oct 22 19:53:03 2022: Falco initialized with configuration file: /etc/falco/falco.yaml Sat Oct 22 19:53:03 2022: Loading rules from file /etc/falco/falco_rules.yaml Sat Oct 22 19:53:03 2022: Loading rules from file /etc/falco/falco_rules.local.yaml Sat Oct 22 19:53:03 2022: The chosen syscall buffer dimension is: 8388608 bytes (8 MBs) Sat Oct 22 19:53:03 2022: Starting health webserver with threadiness 2, listening on port 8765 Sat Oct 22 19:53:03 2022: Enabled event sources: syscall Sat Oct 22 19:53:03 2022: Opening capture with Kernel module 19:53:09.668585724: Warning Docker or kubernetes client executed in container (user=<NA> user_loginuid=-1 k8s.ns=gatekeeper-system k8s.pod=gatekeeper-update-crds-hook-l5zr2 container=006eacc6d95b parent=<NA> cmdline=kubectl apply -f crds/ pid=13671 image=openpolicyagent/gatekeeper-crds:v3.10.0) 19:53:09.739647377: Notice Unexpected connection to K8s API Server from container (command=kubectl apply -f crds/ pid=13671 k8s.ns=gatekeeper-system k8s.pod=gatekeeper-update-crds-hook-l5zr2 container=006eacc6d95b image=openpolicyagent/gatekeeper-crds:v3.10.0 connection=192.168.34.30:45214->10.100.0.1:443) 19:53:21.068878529: Notice Unexpected connection to K8s API Server from container (command=manager --port=8443 -- health-addr=:9090 --prometheus-port=8888 --logtostderr --log-denies=false --emit-admission-events=false --log- level=INFO --exempt-namespace=gatekeeper-system --operation=webhook --enable-external-data=false --enable-generator- resource-expansion=false --log-mutations=false --mutation-annotations=false --disable-cert-rotation=false --max- serving-threads=-1 --tls-min-version=1.3 --metrics-backend=prometheus --operation=mutation-webhook --disable-opa- builtin={http.send} pid=14628 k8s.ns=gatekeeper-system k8s.pod=gatekeeper-controller-manager-78b8774b7c-wknwm container=bb34633b4f13 image=openpolicyagent/gatekeeper:v3.10.0 connection=192.168.60.49:34346->10.100.0.1:443) 19:53:21.133021316: Notice Unexpected connection to K8s API Server from container (command=manager --audit- interval=60 --log-level=INFO --constraint-violations-limit=20 --audit-from-cache=false --audit-chunk-size=500 -- audit-match-kind-only=false --emit-audit-events=false --operation=audit --operation=status --operation=mutation- status --logtostderr --health-addr=:9090 --prometheus-port=8888 --enable-external-data=false --enable-generator- resource-expansion=false --metrics-backend=prometheus --disable-cert-rotation=true pid=14691 k8s.ns=gatekeeper-system k8s.pod=gatekeeper-audit-9b7795dcf-tff44 container=c603d5981dea image=openpolicyagent/gatekeeper:v3.10.0 connection=192.168.62.41:58520->10.100.0.1:443)
  • 30. Define and Design the Optimal Survey Experience WRAP-UP Key takeaways β€’ Enforce the principle of least privilege. β€’ Keep everything up to date. β€’ Scan your container images frequently, before and after deployment. β€’ Monitor your systems for expected and unexpected behavior. β€’ And disk space. @OtherDevOpsGene #AllThingsOpen 30
  • 31. Define and Design the Optimal Survey Experience WRAP-UP Reading list Kubernetes Hardening Guidance, National Security Agency (NSA) and Cybersecurity and Infrastructure Security Agency (CISA). https://media.defense.gov/2021/Aug/03/2002820425/-1/- 1/1/CTR_KUBERNETES%20HARDENING%20GUIDANCE.PDF A Closer Look at NSA/CISA Kubernetes Hardening Guidance, Jim Angel, Pushkar Joglekar, and Savitha Raghunathan. https://kubernetes.io/blog/2021/10/05/nsa-cisa- kubernetes-hardening-guidance/ Kubernetes Security Technical Implementation Guide, Cybersecurity and Infrastructure Security Agency (CISA). https://public.cyber.mil/stigs/downloads/ CIS Kubernetes Benchmark, Center for Internet Security (CIS), https://www.cisecurity.org/benchmark/kubernetes/ @OtherDevOpsGene #AllThingsOpen 31
  • 32. Define and Design the Optimal Survey Experience WRAP-UP Tools Aqua Security kube-bench: https://github.com/aquasecurity/kube-bench Checkov by Bridgecrew: https://github.com/bridgecrewio/checkov Aqua Security Trivy: https://github.com/aquasecurity/trivy Anchore Grype: https://github.com/anchore/grype Anchore Syft: https://github.com/anchore/syft OWASP Dependency-Track: https://dependencytrack.org Open Policy Agent: https://www.openpolicyagent.org Fairwinds Goldilocks: https://github.com/fairwindsops/goldilocks Network Policy Editor: https://networkpolicy.io Falco: https://falco.org @OtherDevOpsGene #AllThingsOpen 32

Editor's Notes

  1. We will look at 10 tools across 3 rough layers of the Kubernetes ecosystem. All are open-source and/or freely available Also, some publicly available guidance Security is a type of quality You cannot be insecure and have high quality You cannot have low quality but high security
  2. Kubernetes clusters consist of servers acting as master nodes and worker nodes. The operating system and processes on these servers have to be secured just like any others. These are the tasks traditionally done by Ops and Security YAGNI K8s hosts need the same security as other hosts Keep the systems up-to-date Easiest to do. Just regularly run apt-get update or dnf update or yum update CISA recommends the following remediation timelines: Critical vulnerabilities should be remediated within 15 calendar days of initial detection. High vulnerabilities should be remediated within 30 calendar days of initial detection.
  3. Least privilege Ronald Reagan 1986 – The nine most terrifying words in the English language are "I'm from the Government and I'm here to help." Google both terms, See the Kubernetes.io article from Oct 2021. See the stigviewer.com link Also, not specific to k8s, but Federal, state, local, tribal and territorial governments, as well as public and private sector critical infrastructure organizations. Can request Cyber Hygiene Services at no cost from CISA. CIS is not a government agency, but it is non-profit. CIS Benchmarks are free checklists, very similar to STIGs, easier to read.
  4. "checks whether Kubernetes is deployed securely by running the checks documented in the CIS Kubernetes Benchmark." If you are running in a managed Kubernetes cluster, such as Amazon EKS or Azure AKS, kube-bench does not have access to the master nodes but can still evaluate the worker nodes. Β  Runs as a kubectl job
  5. Runs as a kubectl job cd ~/git/ggkube/Book/code kubectl apply -f kube-bench-job-eks.yaml kubectl get pods kubectl logs kube-bench-kc82n
  6. These tasks are typically going to fall to the development teams while they are producing their software for deployment All of this is a moot point if the application is security swiss cheese.
  7. Least privilege Checkov by Bridgecrew Frequent updates, sometimes daily I use this extensively on Terraform code
  8. Python pip install or use Docker container pushd ./microservices-demo/deploy/Kubernetes checkov -d manifests --quiet --compact
  9. Can’t just scan once, vulnerabilities can be found even in existing, previously β€œsafe”, containers/code
  10. Installs as package, from script, container, etc. trivy config manifests/01-carts-dep.yaml
  11. trivy image weaveworksdemos/carts:0.4.8 --no-progress
  12. Other options are Clair by Red Hat Can’t just scan once, vulnerabilities can be found even in existing, previously β€œsafe”, containers/code
  13. grype weaveworksdemos/carts:0.4.8
  14. Can feed that back to Grype
  15. Can feed that back to Grype syft weaveworksdemos/carts:0.4.8 --output json --file carts-0.4.8.json grype sbom:carts-0.4.8.json syft weaveworksdemos/carts:0.4.8 --output cyclonedx-json --file carts-0.4.8-dx.json
  16. This is the Ops piece of DevSecOps. After you deploy, the job isn’t finished. Keep monitoring. It is embarrassing how often companies that spend millions to staff a 24x7 SOC have apps go down when they run out of disk space
  17. Oh-pah
  18. https://open-policy-agent.github.io/gatekeeper-library/website/imagedigests kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper-library/master/library/general/imagedigests/template.yaml kubectl describe ConstraintTemplate -A kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper-library/master/library/general/imagedigests/samples/container-image-must-have-digest/constraint.yaml cat allowed.yaml kubectl apply -f allowed.yaml
  19. cat disallowed.yaml kubectl apply -f disallowed.yaml
  20. 014d4142-ca50-4f71-a9f5-a7f727c8df1f kubectl apply -f manifests kubectl label ns sock-shop goldilocks.fairwinds.com/enabled=true kubectl -n goldilocks port-forward svc/goldilocks-dashboard 8444:80& kubectl -n sock-shop port-forward svc/front-end 8555:80&
  21. Least privilege Container network interface Linkerd, Istio, Calico, Cilium mTLS
  22. Least privilege Container network interface Linkerd, Istio, Calico, Cilium mTLS
  23. Install to host so it is isolated from Kubernetes Can also install into cluster using Helm if you don’t control the host, e.g., EKS
  24. kubectl get pods -n falco -o wide kubectl logs -n falco falco-zplnz -f
  25. There are commercial versions that incorporate many of these tools and capabilities, often together in a single package.