1. An air-gapped Kubernetes environment restricts internet access to increase security by preventing downloads of malicious data and attacks from outside entities. 2. Implementing an air-gapped Kubernetes cluster is more difficult than a standard one and requires additional effort for maintenance, but provides protections such as preventing data exfiltration by third parties. 3. Deploying components like the ELK stack in an air-gapped environment requires manually downloading, transferring, and installing charts and images due to the lack of access to external registries and repositories. Processes and permissions must be tightly controlled to maintain security.

1. 1
Air Gap - short intro and why it isn't
complicated at all!

2. 2
Content
01
02
03
04
05
About us
Overview Standard Kubernetes Environment
Air Gap – what is it?
Short demonstration
Conclusion

3. 3
About us

4. 4
Portfolio
www.kubeops.net
• Project support for installation of
Kubernetes
• Implementation of workshops
• Producer of a Kubernetes distribution for
productive environments
• Training on Kubernetes and
Microservices

5. 5
Presenter
Tobias Altmann
DevOps Engineer
Ralf Menti
Kubernetes Trainer
Vincent Mattes
Developer

6. 6
Overview common
Kubernetes Enviroment

7. 7

8. 8
Kubernetes Standard Implementation
(Image Source)
Internet
Internet

9. 9
Some facts about Kubernetes
Source: Canonical /Forbes
• 21.4% of respondents are managing more than 500 machines on a day-to-day basis
• As voted by most professionals, lack of in-house skills/limited manpower is the biggest
challenge to adopting Kubernetes and containers.
• 26% are running a version of Kubernetes that is newer than 1.10 but older than 1.17
• 56% chose security as the key criterion for choosing a base image of a container image
• Namespaces are the preferred way to isolate applications, as mentioned by 63%
• Only 54.2% said that they are running a high availability Kubernetes cluster
Survey from Canonical on Forbes, June 2021

10. 10
• Vulnerable base image and dependencies
• Security features like TLS or RBAC deactivated by
default
Vulnerable
Image-Source: Prevasio.io

11. 11
Where do base images or the chart itself come from?
• Questionable Supply chains
• Risk: containers used for attacks like malicious cryptocurrency mining
Source: https://knowledge-base.prevasio.io/pdf.html?file=Red_Kangaroo.pdf

12. 12
• Enable Kubernetes Role-Based Access Control (RBAC)
Kubernetes Security
• Use Third-Party Authentication for API Server
• Use Process Whitelisting
• Turn on Audit Logging
• Keep Kubernetes Version Up to Date
• Lock Down Kubelet

13. 13
• Secure following directories ! – Only access for Admin!
Kubernetes Security
• /etc/kubernetes/
• /var/lib/kubelet/
• /etc/sysconfig/kubelet
• $HOME/./kube/config.yaml
• Systemctl only for admins of the Cluster!
• https://www.cisecurity.org/benchmark/kubernetes/
Misconfiguration is the most common security incident

14. 14
Example: Security concept whitelisting
Environment
Cluster
Node
Pod
Container

15. 15
What is an Airgap Environment ?

16. 16
Airgap Environment
• Restricted InternetConnection (e.g. by Firewall)
• Cluster can only connect to Software Hub/Registry via
specific ProxyRules
• Restricted CommandSet available
• specific technical user instead of root user
• Only few sudo commands available
Airgap / Security Features

17. 17
Kubernetes Air Gap Implementation
Internet
Image: Faun.pub
Proxy /
Firewall
Controlled Cluster – no connection outside possible Corp. Network
Priv. registry

18. 18
Common Problems
• Missing sudo commands when installing kubernetes with yum
• Restriction of permissions for certain directories:
• /etc/kubernetes/
• /var/lib/kubelet/
• /etc/sysconfig/kubelet
• $HOME/./kube/config.yaml
• Update or downgrade to specific versions
• Accessing the registry via proxy requests might be more complex
• Migration from nfs to longhorn (permission problems)

19. 19
Pro‘s and Con‘s
Pros Cons
Airgap prevents downloading malicious Data Airgap diminishes productivity
Airgap reduces chances for hacks from outside Airgap increases effort/costs (implementation and
maintenance)
Reduce chances of accessing and exfiltrating data by
third-parties
Airgap doesn‘t ensure 100% security (e.g. insider
threats)
Technical User ensures that not everybody can work
with root-permissions
Technical User complicates some tasks (e.g. simple
Linux Commands like cp, mv, ls )

20. 20
Example: Deploying the ELK
Stack on Kubernetes

21. 21
Example: Deploying ELK Stack on Kubernetes
Official Documentation:
https://logz.io/blog/deploying-the-elk-stack-on-kubernetes-with-helm/
Risks:
• Many helm charts contain more content then necessary (bloated images)
• Users don‘t really know what they are installing
• Charts can contain insecure images
• Installscripts can contain any commands
• Therefore an attacker could take control of a cluster

22. 22
Example: Deploying ELK Stack on Kubernetes
Non-Airgap Environment:
Airgap Environment:

23. 23
Example: Deploying ELK Stack on Kubernetes
Non-Airgap Environment:
Airgap Environment:

24. 24
Example: Deploying ELK Stack on Kubernetes
How to install helm in an Airgap Environment
1. Download helm binary to local maschine
2. Transfer binary to admin node (via scp)
3. Transfer to directoryof technical user on admin node
4. Unzip file
5. Make executable by moving to /usr/local/bin
Commands must be in sudoers
file!!!

25. 25
Example: Deploying ELK Stack on Kubernetes
How to deploy ELK Stack in an Airgap Environment
1. Download chartto local machine
2. Grep charts for unnecessarythings
• check values.yaml file
3. Registry:
• Connection to registry must be
given (e.g. by Proxy)
• Images must be available in an
accessible registry
4. Transfer chartto cluster (via sftp, scp)
5. Move chartto directory of technical user
6. Install chart via helm

26. 26
Demonstration
Package through tool SINA

27. 27
More Security in an Air Gap Area

28. 28
Conclusion

29. 29
Conclusion
• Kubernetes standard implementation can be fast installed
• But Kubernetes has open doors
• A safe security concept prior installation is mandatory
• Kubernetes in an airgapped enviroment isn‘t easy at all – but worthwile
• a controlled cluster is worth more than the convenience of the users
• Does Airgap really diminish productivity ? – it depends on tool-sets
• Airgap increases effort/costs (implementation and maintenance) – yes, but a
hacked environment costs more (log4j)
• Protection against malicious data downloads or certain third-party attacks

30. 30
Questions? Let‘s connect!
www.kubeops.net
info@kubeops.ne
t