Kubernetes enables possibilities to develop cloud native microservices or decompose traditional applications making them more technologically advanced with the help of containers. Currently, most of the Kubernetes solutions are offered on top of VMs and there is a room for further improvements. Implementing nested architecture of application containers running inside system containers opens additional flexibility of resource allocation and management, accelerates provisioning of the clusters and pods, as well as cuts the overall costs. In this presentation we cover the possibilities of running a Java-based application inside Kubernetes cluster with nested container architecture, what configurations should be taken into account, and how to overcome the barriers on the way to more efficient Kubernetes hosting.

1. Java Applications
inside Kubernetes
with Nested Container Architecture

2. ● Ruslan Synytsky
● CEO and co-founder of Jelastic PaaS
● Java Champion
● Two-times Duke’s Choice Award Winner
● Former lead of engineering team at
National Data Center (NDC) at National
Space Agency of Ukraine
● @siruslan
About Speaker

3. ● Virtualization and Container Types
● Effective Usage of Infrastructure
● Scaling VMs vs Containers
● Pay-as-you-Go vs Pay-per-Actual-Use
● Overcoming Java Memory Waste
● Kubernetes Challenges & Solutions
Agenda

4. VIRTUALIZATION

5. Virtualization Types

6. Virtualization Types

7. Virtualization Types

8. Container Types
● Used as an OS to run multiple services
● No layered filesystems by default
● Stronger isolation
● Examples: Virtuozzo, LXC
● Used to run for a single service
● Layered filesystems
● Examples: Docker, CRI-O

9. EFFECTIVE USAGE OF
INFRASTRUCTURE

10. Running Kubernetes on VMs vs System Containers

11. Kubernetes on VMs vs System Containers

12. Kubernetes on VMs vs System Containers

13. Kubernetes on VMs vs System Containers

14. Deployment Unification - Kubernetes on Containers in VMs

15. Kubernetes on VMs vs Containers

16. Live Migration of Containers
● Zero downtime hardware maintenance
● Load rebalancing across host nodes
● High-availability across clouds

17. Live Migration across Clouds without Downtime

18. SCALING JAVA IN
VMs vs CONTAINERS

19. Resource Limit vs Real Usage in VM and Container

20. Horizontal Scaling: VMs vs Containers
VMs
Containers

21. Vertical Scaling: VMs vs Containers
Resizing of the same container on the fly is easier, cheaper and faster
than moving to a larger VM.

22. Nested Limits
Kubernetes Node Limit

23. UNDERALLOCATION
OK OVERALLOCATION
JVM Vertical Scaling

24. Important JVM Options When Running in Containers
-XX:+UseContainerSupport
Force JVM to use containers limit instead of the host limits
It’s a must for application containers and can be disregarded for system containers
-XX:InitialRAMPercentage=N
Set initial heap size as a percentage of total memory
If you set a value for -Xms, the -XX:InitialRAMPercentage option is ignored
-XX:MaxRAMPercentage=N
Set maximum heap size as a percentage of total memory
If you set a value for -Xmx, the -XX:MaxRAMPercentage option is ignored

25. Jelastic PaaS adjusts configs of certified Java software stacks based on the
vertical scaling limits
Environment variable
XMX_DEF_PERCENT=80
Automatic Stacks Adjusting based on Scaling Limits

26. OVERCOMING JAVA
MEMORY WASTE

27. Jakarta EE Developer Survey 2018
“The most widely acknowledged issue
when employing with Java EE is
large memory requirements (40%)”
Most Challenging Aspect of Working with Jakarta EE

28. Unreleased Heap Memory
Over-Allocation and Underutilization
The Problem Symptoms

29. Make the G1 garbage collector automatically give back Java heap memory
to the operating system when idle
● -XX:G1PeriodicGCInterval=[milliseconds]
● -XX:G1PeriodicGCSystemLoadThreshold=[float]
● -XX:+G1PeriodicGCInvokesConcurrent
JEP 346: Promptly Return Unused Committed Memory from G1
java -Xmx2g -XX:+UseG1GC -XX:G1PeriodicGCInterval=900k
-XX:G1PeriodicGCSystemLoadThreshold=0.6 -jar app.jar
Available from Java 12
Timely Reduce Unused Committed Memory (JEP 346)

30. Automatically Released Heap
Immediately Improved Heap Elasticity

31. Community Recognition

32. https://github.com/jelastic-
jps/java-memory-agent
As compacting GC cycles are not triggered automatically, we execute them explicitly by
injecting an agent which monitors the memory usage and calls System.gc() periodically:
-javaagent:jelastic-gc-agent.jar=period=300,debug=true
Workaround for Java < 12: Calling Full GC Periodically

33. java -XX:+UseG1GC -Xmx2g -jar app.jar
https://github.com/jelastic/java-vertical-scaling-test
G1 and Full GC

34. Return unused HotSpot class-metadata memory to the operating system
more promptly, reduce metaspace footprint, and simplify the metaspace
code in order to reduce maintenance costs.
● Replace the existing metaspace memory allocator with a buddy-based
allocation scheme to allocate metaspace memory in smaller chunks.
● Commit memory from the operating system to arenas lazily, on
demand to reduce footprint for loaders
● Arrange metaspace memory into uniformly-sized granules which can
be committed and uncommitted independently of each other
-XX:MetaspaceReclaimPolicy=(balanced|aggressive|none)
Available from Java 16
Elastic Metaspace (JEP 387) https://openjdk.java.net/jeps/387

35. PAY-PER-USE
vs
PAY-AS-YOU-GO

36. Pay-As-You-Go vs Pay-per-Actual-Use
Using automatic vertical scaling, cloud provides can offer economically
advantageous pricing based on the actual resource consumption
Forbes - Deceptive Cloud Efficiency: Do You Really Pay As You Use?
Pay-As-You-Go Pay-per-Actual-Use

37. ~$ free -m
● total - memory limit, amount of memory that can be used
● used - currently used memory, calculated as total - free - buffers - cache
● shared - extra used memory and shared with other containers on the same host
● buff/cache - temporary used memory which can be reclaimed any time on demand
Linux Terminology for RAM Consumption

38. ● Physical Memory - the amount of RAM installed
● Memory Used - the amount of RAM being used
○ App Memory - the amount of memory being used by apps
○ Wired Memory - memory required by the system to operate. This memory can’t
be cached and must stay in RAM, so it’s not available to other apps
○ Compressed - the amount of memory that has been compressed to make
more RAM available
To check RAM usage on Mac, go to Activity Monitor (Applications > Utilities)
MacOS Terminology for RAM Consumption

39. ● Total - memory limit, amount of memory
that can be used
● In Use - currently used memory,
calculated as total - free - standby -
modified
● Modified - memory whose contents must
be written to disk before it can be used
for another purpose
● Standby - memory that contains cached data and code that is not actually in use
● Free - memory does not contain any valuable data and that will be used first and
processors drivers or the operating system needs more memory
Windows Terminology for RAM Consumption

40. Pay-as-You-Go
Pay-as-You-Allocate
Pay-as-You-Reserve
= Pay-for-Limits
Pay-per-Use !=
Speculation with Pay-per-Use Term

41. https://aws.amazon.com/ecs/pricing/
AWS’s Pay-per-Use = Pay-per-Allocated-Limits

42. https://aws.amazon.com/fargate/pricing/
AWS’s Pay-per-Use = Pay-per-Allocated-Limits

43. https://cloud.google.com/run#section-2
https://cloud.google.com/run/pricing#cloudrun-pricing
Google’s Pay-per-Use = Pay-per-Allocated-Limits

44. Google’s Pay-per-Use = Pay-per-Allocated-Limits

45. Real Statistics of Resource Consumption with Containers

46. JELASTIC RESPONSE TO
KUBERNETES STRUGGLES

47. Technical Struggles with Kubernetes Services
● Too many components to manage (pod, node, service, ingress
and ingress controller, namespace, deployment, statefulset,
RBAC, nodeport, load balancer, physical volume, physical volume
claim, networks, resource limits, and so on)
● High entry barrier for beginners, most of features are
API-managed only, default Kubernetes Dashboard UI provides
limited functionality
● Migration complexity of traditional and legacy applications
● K8s was designed for large scale cloud-native apps and
microservices, so it’s not suitable for all workloads
● Upgrade to next Kubernetes version requires proper automation
and may be a challenge

48. Kubernetes Issues Solved by Jelastic
● Challenging setup is converted to “one click”
● Manual nodes configuration is fully automated
● Out of the box LB and SSL support
● K8s metrics and monitoring solutions
pre installed
● Replacing VMs with system containers
○ “Pay-Per-Actual-Use” pricing
○ Fast scaling of K8s nodes
● Turnkey solution for Public Hosting
Business
● Multi-cluster and multi-cloud
management
● Built-in billing and monitoring tools
● Product and security updates automation

49. https://www.youtube.com/watch?v=l9H28icAlUg
Automated Kubernetes Cluster Installation

50. ● Nginx, HAProxy or Traefik, ingress
controllers for transferring HTTP/HTTPS
requests to services
● CNI plugin (powered by Weave) for overlay
network support
● CoreDNS for internal names resolution
● Dynamic provisioner of persistent volumes
● Metrics Server for gathering stats
● Jelastic SSL for protecting ingress network
● Kubernetes Dashboard
● HELM package manager to auto-install
pre-packed solutions from repositories
● Jaeger, Prometheus and Grafana
Pre-Installed Kubernetes Components

51. Automatic Vertical Scaling with Pay-Per-Actual-Use Pricing
An example of Workers vertical scaling: available capacity 48 GiB vs actually used and billed 2 GiB

52. Changing Worker Node Resource Limits on the Fly

53. Access to Worker and Master Nodes Via Web SSH

54. Upgrade Procedure

55. Upgrade Procedure
● Check if the cluster is eligible to
upgrade, and availability of final
version(s)
● Upgrade installed cluster
components (Weave, ingresses,
dashboards, hello-world,
metrics-server, Helm,
Prometheus+Grafana, etc.)
● Check if deprecated components are
present in the cluster
● Upgrade master instances
one-by-one via redeploy
● Evict PODs, upgrade worker
instances one-by-one via redeploy

56. Built-In Add-Ons

57. Embedded Cluster Monitoring
Grafana Prometheus

58. Kubernetes Hosting across Clouds & Data Centers
● Provision the clusters across multiple
clouds and on-premises with no
vendor lock-in
● Full interoperability and unification -
no configuration differences in
Kubernetes clusters running on
different clouds
● Mix and match infrastructure for extra
savings and for compliance with strict
data protection requirements

59. 100+ data centers from 65+ local providers in 38 countries
Distributed Network of Service Providers Worldwide

60. Give a Try Yourself
https://jelastic.com/kubernetes-hosting/
Contact for Partnership
and Assistance
info@jelastic.com