This document discusses IBM Semeru Runtimes for improving Java workloads in the cloud. It covers: - Semeru Runtimes are optimized OpenJDK runtimes from IBM that provide smaller container images, faster startup times, and more efficient resource utilization for Java applications in cloud environments. - Techniques used include using smaller base images, ahead-of-time compilation, shared class caching, and container awareness to size memory usage appropriately. - Semeru Runtimes also provide tools for live debugging of containerized Java applications without restarting containers. - Upcoming features will include support for the new cgroups version 2 for more advanced Linux container resource management.

1. Analyzing container
workloads for
improved cloud experience
with
Semeru Runtimes
Sharanabasava, Java Support Engineer for IBM Java

2. Agenda
• Focused on improving Java workloads in the cloud
• Container awareness
• Debugging in the cloud.
2

3. What is Semeru Runtimes?
• IBM Built OpenJDK runtimes powered by the Eclipse OpenJ9 JVM(foundation of IBM’s
java-based-software for 15+ years)
• Two Editions(Functionally 100% compatible) :
• IBM Semeru Runtime Open Edition
Open-Source license(GPLv2+CE)
Available now for Java versions 8,11,17
• IBM Semeru Runtime Certified Edition
IBM license ,TCK certified
Available now for Java 11+ LTS versions download
Download
https://developer.ibm.com/languages/java/semeru-runtimes/downloads/
3

4. What is Eclipse OpenJ9 ?
• Eclipse OpenJ9 JVM previously known as IBM J9.
• Eclipse OpenJ9 was open sourced from IBM J9 JVM.
• stable , secure, high performance ,cloud optimized, multi-platform, ready for development
and production use
Open-sourced Sept 2017
https://www.eclipse.org/openj9/
https://github.com/eclipse/openj9
Dual License:
Eclipse Public License 2.0
Apache 2.0
Users and contributors welcome
https://github.com/eclipse-openj9/openj9/blob/master/CONTRIBUTING.md
4

5. Objectives in the cloud
IBM Runtime Technologies
5
Optimal deployment Size Faster Startup and Ramp up Use computing resources
judiciously

6. IBM Runtime Technologies
6
Footprints

7. Small deployment size == Small container Image
• Reduced push time
- developers
• Reduced pull time
• Add new node for scaling out
• Matter of security
- less attack surface area
7
Docker images: https://hub.docker.com/_/ibm-semeru-runtimes?tab=tags
1. Use jre instead of jdk
2. (Java11+) use jlink
3. Pick a different/smaller OS base image
Advantages Recipes
Docker
Image
Sizes
Latest
(ubuntu)
ubi-
minimal
ubi jre
amd64 240 MB 286MB 380 MB 98 MB

8. More about jlink
• To create custom JRE
• Syntax: jlink [options] - -module-path <modulepath> --add-modules <module> --output
<path>
• Example:
>> <Javapath>/bin/jlink --strip-debug --compress 2 --no-header-files --no-man-pages --module-
path /opt/java/openjdk/jmods --add-modules $(jdeps --print-module-deps
target/AcmeairApp.jar),java.desktop,java.instrument,java.logging,java.management,java.naming
,java.security.jgss,java.sql,java.xml --exclude-
files=**java_**.properties,**J9TraceFormat**.dat,**OMRTraceFormat**.dat,**j9ddr**.dat,**
public_suffix_list**.dat --output jre
• After stripping unwanted modules from the ubuntu based semeru JDK image, image size is
reduced to 152MB from 240MB.
8

9. • In your Dockerfile
• …
FROM ibm-semeru-runtimes:open-11-jre AS semeru
FROM alpine #an example smaller base image
ENV JAVA_HOME=/opt/java/openjdk
COPY --from =semeru $JAVA_HOME $JAVA_HOME
ENV PATH=“${JAVA_HOME}/bin:${PATH}”
…
IBM Runtime Technologies
9
Faster Startup and Ramp up

10. Starting Java workloads quickly
• -Xquickstart
• Faster start-up
• Tradeoff is some throughput loss
• Great for short-lived tasks(i.e. functions)
• -Xtune:virtualized
• Enabled VM idle management
• improve start-up and ramp-up
• Small loss in throughput
10

11. Shared cache cache and Dynamic AOT technology
• Shared class cache(SCC) [-Xshareclasscache:cachedir=/tmp/scc]
• Memory Mapped file for caching
• AOT Compiled code
• Interpreter Profiling information
• Speed-up class loading
• Dynamic Ahead-of-Time(AOT) compilation
• Relocatable format
• AOT loads are 100x faster than JIT compilations
• More generic code -> slightly less optimized
• Generate AOT code only during start-up
11

12. Embedding SCC in containers
• SCC usability problem: Cache goes away when container is destroyed
• Need Persistence!
• Use Volumes; cumbersome with dynamic provisioning; cold start still an issue
• Package SCC inside Openj9 Docker images
• Methodology:
• Start/Stop “training” app(tomcat) when Openj9 docker image is created
• Cons: limited amount of caching
• Pros: any Java app can benefit from cached classes/methods
• Openj9 images(via IBM Semeru Runtimes) now have an embedded SCC named:openj9_system_scc
• https://hub.docker.com/_/ibm-semeru-runtimes
12

13. Multi-layer SCC
• Cache app classes/methods -> significant start-up time benefits
• Methodology: start/stop app during docker image creation time
• Potential issues:
• Oversizing needed to allow addition to cache from a higher Docker layer
• Bloat due to “copy-on-write” mechanism when adding to the cache
• OpenJ9 uses multi-layer SCC
• Each docker layer can add to SCC packaged at that layer independently of lower layers
• Each SCC layer can be trimmed-to-fit because upper layers won’t add to it
13

14. • In your Dockerfile
• …
FROM ibm-semeru-runtimes:open-11-jre AS semeru
FROM alpine #an example smaller base image
ENV JAVA_HOME=/opt/java/openjdk
COPY --from =semeru $JAVA_HOME $JAVA_HOME
ENV PATH=“${JAVA_HOME}/bin:${PATH}”
…
IBM Runtime Technologies
14
Resource utilization

15. Openj9 JVM use computing resources judiciously
• Computing resources ==
• Cloud it’s about sharing; do not be greedy in using all resources
• OpenJ9 is conservative with heap growth
• OpenJ9 frees memory used transiently during JIT compilation
15
0
100
200
300
400
500
600
700
800
1GB 2GB 4GB
Resident
set
size
(MB)
Container limit
Steady state memory footprint in AcmeAir
OpenJ9 Hotspot
For large containers
OpenJ9 uses less than
half the memory !

16. OpenJ9 is Container Aware(Java 8+)
Is your java container aware? Container awareness is critical
Host: 16GB
Host: 16GB
Host: 64 vCPU’s
Host:64 vCPU’s
16
Container
limit: 1G
JVM sets
heap size:
4GB
Container
limit :1P
JVM
thinks can
use 64P
Side effects:
➢ OOM
➢ JVM process gets killed
Side effects:
➢ To many GC/JIT threads
➢ Too many app threads
➢ Likely poor performance
• Better stability
- JVM can properly size data structures(e.g. Java heap) ->
avoid OOM killer
• Better Performance
- JVM can properly size number of GC/JIT threads
- Application can properly size its thread pools

17. Container awareness - usability
17
• -XX:UseContainerSupport option now enabled by default in Java 8 and above
• Automatically increase default -Xmx
(max heap) value when running in a
container
• Better solution than setting –Xmx and –Xms
when same app is deployed with different
memory limits:
“-XX:MaxRAMPercentage=…”
“-XX:IntialRAMpercentage=…”
MemLimit Java8/11 Java8 Java11
<=1G MemLimit/2 RAM/2 RAM/2
1G-2G Memlimit-512M 512M 512M
>=2G MemLimit*3/4 512M RAM/4
Xmx Inside
containers
Xmx outside
containers
-XX:MaxRAMPercentage=80
Heap= 1.6G
Container Mem= 2G
Heap= 2.4G
Container Mem= 3G
Heap= 3.2G
Container Mem= 4G

18. GC on Idle
• -XX:+IdleTuningCompactonIdle
- Compacts the java heap down when idle(think defraging)
• -XX:+IdleTuningGcOnIdle
- Releases heap memory back to system when idle(more effective with above enabled)
18

19. Free resources when applications are idling
-XX:+IdleTuningGcOnIdle
(default setting in container)
19
Benchmark: https://github.com/blueperf/acmeair

20. • In your Dockerfile
• …
FROM ibm-semeru-runtimes:open-11-jre AS semeru
FROM alpine #an example smaller base image
ENV JAVA_HOME=/opt/java/openjdk
COPY --from =semeru $JAVA_HOME $JAVA_HOME
ENV PATH=“${JAVA_HOME}/bin:${PATH}”
…
IBM Runtime Technologies
20
Debugging

21. Get Debugging Info on the Fly !
• Do you have to restart your containerized Java app to dump debugging info ?
NO !
- Use Openj9DiognosticMXBean instead
21

22. Javacore Info
$ docker run –m2g –cpu-quota=“100000”
--cpu-period=“200000” –it semeru11 java
-Xdump:java:events=vmstop App
22
1CICONTINFO Running in container : TRUE
1CICGRPINFO JVM support for cgroups enabled : TRUE
….
1CICGRPINFO Cgroup Information
NULL ------------------------------------------------------------------------
2CICGRPINFO subsystem : cpuset
2CICGRPINFO cgroup name : /
3CICGRPINFO CPU exclusive : 0
3CICGRPINFO Mem exclusive : 0
3CICGRPINFO CPUs : 0-3
3CICGRPINFO Mems : 0
2CICGRPINFO subsystem : memory
2CICGRPINFO cgroup name : /
3CICGRPINFO Memory Limit : 2147483648 bytes
3CICGRPINFO Memory + Swap Limit : Unavailable
3CICGRPINFO Memory Usage : 74129408 bytes
3CICGRPINFO Memory Max Usage : 74752000 bytes
3CICGRPINFO Memory limit exceeded count : 0
3CICGRPINFO OOM Killer Disabled : 0
3CICGRPINFO Under OOM : 0
2CICGRPINFO subsystem : cpu
2CICGRPINFO cgroup name : /
3CICGRPINFO CPU Period : 200000 microseconds
3CICGRPINFO CPU Quota : 100000 microseconds
3CICGRPINFO CPU Shares : 1024
3CICGRPINFO Period intervals elapsed count : 159
3CICGRPINFO Throttled count : 6
3CICGRPINFO Total throttle time : 856140458 nanoseconds

23. Upcoming Feature
Cgroups version 2
• Control group(cgroup) is a linux kernel feature – limits and isolate the resource usage
• Important features in v2:
• Simplified tree architecture for memory and cpu controllers
• Pressure stall information(PSI)
• Docker 20.10 gets cgroups v2 support
• Adding cgroup v2 support to Semeru Runtimes
23

24. Wrap-up
• Reduction of footprints
• Start-up improvements
• Resource utilization
• Debugging in the cloud
24

25. Useful Links
• Semeru Open Edition Docker images : https://hub.docker.com/_/ibm-semeru-runtimes
• Semeru Certified Docker images: ICR.IO
>> docker pull icr.io/appcafe/ibm-semeru-runtimes:certified-11-jdk-focal-amd64
• Dockerfile: http://github.com/ibmruntimes/semeru-containers.git
• Download semeru runtime: https://developer.ibm.com/languages/java/semeru-runtimes/downloads/
25

26. 26
Q&A
You have questions, we have answers