Liberty InstantOn is a technology from IBM that provides fast startup times for Java applications running in containers without compromising features or developer experience. It uses checkpoint and restore to save the state of a pre-initialized Java runtime so new instances can start up almost instantly. This makes Java more suitable for serverless and microservices workloads that require quick startups and the ability to scale to zero. Liberty InstantOn provides the benefits of fast startup times without limiting Java features or requiring changes to how developers build and test applications.

1. Java at light speed

2. 3
Who am I?
• Software Engineer and Developer
Advocate at IBM
• Based in the Austin, TX USA
• Focus on OSS cloud native Java
technologies
Rich Hagarty
@rhagarty8

3. 4
Agenda
• Java runtimes – cloud optimized?
• Open Liberty
• Java on cloud – a good fit?
• JVM and JIT Compiler – good or bad?
• Eclipse OpenJ9 JVM and Semeru Runtimes
• Semeru Cloud Compiler
• Liberty InstantOn

4. 5
5
Java Runtimes
(Application Servers)

5. 6
Many Options

6. 8
Cloud Performance
• Modular runtime
o Low memory, high throughput
o Low operating cost
• CI/CD optimized
o Availability on all Cloud platforms
o Cloud-native tool integration
o Optimized for Kubernetes
• Simple developer experience
o Integration with all popular IDEs and build tools
o API support – Jakarta EE and MicroProfile

7. 10
openliberty.io

8. 11
6 reasons why Liberty
Just enough runtime
Low operating cost
Continuous delivery
Zero migration
Kubernetes optimized
80% disk and 56% memory saving
4x increased density over Tomcat
& Spring Boot
Zero-effort security fixing & zero
technical debt
100% v2v & fixpack migration
saving
Self-tuned optimal perf,
production-ready, kube-native
Developer experience
Container & kube-native
experience, rapid inner loop
Lightweight, highly-
efficient runtime
CI/CD optimized
operational experience
Simple true-to-
production developer
experience

9. 12
<feature>servlet-4.0</feature>
<feature>jaxrs-2.1</feature>
Kernel
• You control which features are loaded into each Liberty instance
Just Enough Runtime
jsonp-1.1
servlet-
4.0
http-2.0
appmgr
Java EE
jaxrs-2.1
jaxrsClien
t-2.1

10. 13
13
Zero Migration
üNo configuration behavior changes
üNo runtime feature behavior
changes
üNo removals Never apply an ifix again
Skipping a release does not introduce
additional migration work
Stay current with a rebuild
(no app or config changes necessary)

11. 15
Developer experience
Repositories Build
IDEs
Dev Mode
APIs Testing

12. 16
Dev Mode
No rebuild
No redeploy
No install
No restart
Just code!

13. Plan
Code
Build
Test
Release
Deploy
Run
Monitor
Cloud-native Liberty
© 2022 IBM Corporation
Testcontainers
Zero Migration
Continuous Delivery
Quality
Security
An end-to-end cloud-native
DevOps delivery experience
with Liberty

14. 19
19
Java on Cloud
Always a Good Fit?

15. Architectural style
Functions
Monolith Microservice
Scale
Smaller
Larger
Macroservice
Lightweight
Runtimes and
Frameworks
Full Function
Runtimes
Function
as a Service
Java runtimes face spectrum of application scale and styles

16. Time
Responses
per second
Startup
Steady state
Time
Memory
Resident
Set
Time
How much of
application has
runtime seen
Varies with load
Varies with load
Varies with load
Traditional
focus of Java
runtime
technologies
Traditional focus on steady state performance
Rampup
CPU
consumption

17. 23
Architecture Styles vs Typical Java workloads
Architecture Characteristics Compatibility
Monoliths Long-lived, efficient
Macroservices Long-lived
Microservices
Fast start-up/ramp-up for scaling up
and down (to zero) to meet demand.
Small footprint.
Serverless/Functions Fast start-up (in MS), short-lived

18. 24
24
The Java Virtual Machine (JVM)

19. 25
What is the JVM?
The Java Virtual Machine (JVM) is the runtime environment to execute Java code.
The JVM is responsible for taking your application bytecode and converting it to a
machines language for execution.
Most other languages compile code for a specific system but in Java, the compiler
converts the code for a Java Virtual Machine that can be used on any system
hence the phrase “Write once, run anywhere”.

20. 26
How the JVM works
Class Loader
Bytecode Verifier
Interpreter JIT
Java Bytecode
JVM
Java Code
Java Compiler
javac
Run Time
Compile Time
OS
Hardware

21. 27
JVM Interpreter
Class Loader
Bytecode Verifier
Interpreter JIT
Java Bytecode
JVM
• Machine independent bytecodes are
interpreted by the JVM at runtime
• This ensures portability of Java programs
across different architectures
• But it affects performance because
interpretation is relatively slow.
OS
Hardware

22. 28
Just-in-Time Compiler
Class Loader
Bytecode Verifier
Interpreter JIT
Java Bytecode
JVM
• The answer to the performance issue is the JIT
compiler, which transforms sequences of
bytecodes into machine code that is cached by
the JVM
• Unit of compilation is a method. To save
overhead, only “hot” methods are compiled
• Compiled native machine code executes 10x
faster than a bytecode-by-bytecode interpreter
• Generated code is saved in a "code cache" for
future use for lifetime of JVM OS
Hardware

23. 29
29
JVM on Cloud – issues?

24. 30
JVM – The Good
30
• Device independent – write once, run anywhere
• > 25 years of improvements
• JIT produces optimized machine code through use of Profilers
• Efficient garbage collection
• Longer it runs, the better it runs (JVM collects more profile data, JIT
compiles more methods)

25. 31
JVM – The Bad
31
• Initial execution run is “interpreted”, which is relatively slow
• “Hot Spot” methods compiled by JIT can create CPU and memory
spikes
• CPU spikes cause lower QoS
• Memory spikes cause OOM issues, including crashes
• Slow start-up time
• Slow ramp-up time

26. 32
JVM Spikes
32
0
50
100
150
200
250
300
350
400
0 30 60 90
CPU
utilization
(%)
Time (sec)
Daytrader7 CPU consumption
CPU spikes caused
by JIT compilation
0
100000
200000
300000
400000
500000
600000
0 30 60 90
Resident
set
size
(KB)
Time (sec)
Daytrader7 memory footprint
Footprint spikes caused
by JIT compilation

27. 33
Problem #1: Container Size
33
Main issues:
•Need to over-provision to
avoid OOM
•Very hard to do – JVMs
have a non-deterministic
behavior
0
100000
200000
300000
400000
500000
600000
0 30 60 90
Resident
set
size
(KB)
Time (sec)
Daytrader7 memory footprint
Footprint spikes caused
by JIT compilation

28. 34
#2: Container Instances (Auto-Scaling)
34
Main issues:
•Slow start-up and ramp-up times
•CPU spikes can cause auto-scaler to incorrectly
launch additional instances

29. 35
Solution Requirements
35
•Minimize/eliminate CPU and memory spikes
•Improve start-up and ramp-up times

30. 36
36
Eclipse OpenJ9 JVM
IBM Semeru Runtimes

31. 37
Overview of Eclipse OpenJ9
Designed from the start to span all the operating
systems needed by IBM products
This JVM can go from small to large
Can handle constrained environments or memory
rich ones
Renowned for its small footprint, fast start-up and
ramp-up time
Is used by the largest enterprises on the planet
37

32. 38
Eclipse OpenJ9
OpenJ9 combines with OpenJDK to form a full JDK
Link to GitHub repo: https://github.com/eclipse-openj9/openj9
38

33. 39
Eclipse OpenJ9 Performance
39

34. 40
IBM Semeru Runtimes
“The part of Java that’s really in the clouds”
IBM-built OpenJDK runtimes powered by the Eclipse OpenJ9 JVM
No cost, stable, secure, high performance, cloud optimized, multi-
platform, ready for development and production use
Open Edition
• Open source license (GPLv2+CE)
• Available for Java 8 (LTS), 11 (LTS), 16, 17 (LTS), 18, 19, 20
Certified Edition
• IBM license
• Java SE TCK certified.
• Available for Java 11, 17
40

35. 41
IBM Semeru Runtimes
“The part of Java that’s really in the clouds”
Other OpenJDK Distributions:
• OpenJDK builds by Oracle
• Adoptium Eclipse Temurin
• Azul Zulu
• BellSoft Liberica
• Amazon Corretto
• Alibaba Dragonwell
• Microsoft Build of OpenJDK
• Red Hat Build of OpenJDK
• Oracle GraalVM
41

36. 42
42
Semeru Cloud Compiler
For Microservices

37. 43
JIT-as-a-Service
Decouple the JIT compiler from the JVM and let it run as an independent
process
Offload JIT
compilation to
remote process
Remote
JIT
Remote
JIT
JVM
JIT
JVM
JIT
Kubernetes
Control Plane
Treat JIT
compilation as a
cloud service
• Auto-managed by orchestrator
• A mono-to-micro solution
• Local JIT still available
43

38. 44
Eclipse OpenJ9 JITServer
• JITServer feature is available in the Eclipse OpenJ9 JVM
• “Semeru Cloud Compiler” when used with Semeru Runtimes
44

39. 45
JITServer advantages for JVM Clients
45
Provisioning
Easier to size; only consider the needs
of the application
Performance
Improved ramp-up time due to JITServer
supplying extra CPU power when the JVM
needs it the most.
Reduced CPU consumption with JITServer AOT
cache
Cost
Reduced memory consumption means
increased application density and reduced
operational cost.
Efficient auto-scaling – only pay for what
you need/use.
Resiliency
If the JITServer crashes, the JVM can
continue to run and compile with its
local JIT

40. 46
JITServer value in Kubernetes
• https://blog.openj9.org/2021/10/20/save-money-with-jitserver-on-the-cloud-an-aws-
experiment/
• Experimental test bed
o ROSA (RedHat OpenShift Service on AWS)
• Demonstrate that JITServer is not tied to IBM HW or SW
o OCP cluster: 3 master nodes, 2 infra nodes, 3 worker nodes
• Worker nodes have 8 vCPUs and 16 GB RAM (only ~12.3 GB available)
o Four different applications
• AcmeAir Microservices
• AcmeAir Monolithic
• Petclinic (Springboot framework)
• Quarkus
o Low amount of load to simulate conditions seen in practice
o OpenShift Scheduler to manage pod and node deployments/placement
46

41. 47
JITServer improves container density and cost
Default config
AM 500
B 550
C 550
F 450 P 450
P 450
B 550
F 450
AM 500
A 350
AM 500
M 200
Q 350
P 450
Q 350
D 600
D 1000
F 450
B 550
Q 350
AM 500
AM 500
AM 500
B 550
B 550
A 350
C 550
F 450
M 200
P 450
P 450
P 450
Q 350
Q 350
D 1000
AM 500 B 550
P 450
AM 500
B 550
B 550
C 550
C 550
F 450
F 450 P 450
Q 350
Q 350
D 1000
D 1000
Q 350
AM 250
AM 250
P 250
P 250
F 250
F 250
B 400 C 350
Q 150
Q 150
M 150
AM 250
AM 250
P 250
P 250
F 250
B 400
Q 150
Q 150
J 1200
A 250
B 400
B 400
C 350
D 1000 D 1000
D 600
AM 250
AM 250
P 250
P 250
F 250
F 250
B 400 C 350
Q 150
Q 150
M 150
AM 250
AM 250
P 250
P 250
F 250
B 400
Q 150
Q 150
J 1200
A 250
B 400
B 400
C 350
D 1000
D 1000
JITServer config
Legend:
AM: AcmeAir monolithic
A: Auth service
B: Booking service
C: Customer service
D: Database (mongo/postgres)
F: Flight service
J: JITServer
M: Main service
P: Petclinic
Q: Quarkus
Total=8250 MB Total=8550 MB Total=8600 MB
Total=9250 MB Total=9850 MB
6.3 GB less
47

42. 48
Throughput comparison
0
200
400
600
800
1000
1200
0 240 480 720 960 1200 1440 1680 1920
Throughput
(pages/sec)
Time (sec)
AcmeAir Microservices
0
200
400
600
800
1000
1200
0 240 480 720 960 1200 1440 1680 1920
Throughput
(pages/sec)
Time (sec)
AcmeAir Monolithic
0
20
40
60
80
100
120
140
160
180
0 240 480 720 960 1200 1440 1680 1920
Throughput
(pages/sec)
Time (sec)
Petclinic
0
500
1000
1500
2000
2500
3000
3500
0 240 480 720 960 1200 1440 1680 1920
Throughput
(pages/sec)
Time (sec)
Quarkus
Machine load:
17.5% from apps
7% from OpenShift
48
è JITServer and default configuration achieve the same level of throughput at steady-state
JITServer Baseline

43. 49
Conclusions from high density experiments
• JITServer can improve container density and reduce operational
costs of Java applications running in the cloud by 20-30%
• Steady-state throughput is the same despite using fewer nodes
49

44. 50
Horizontal Pod Autoscaling in Kubernetes
• Better autoscaling behavior with JITServer due to faster ramp-up
• Less risk to trick the HPA due to transient JIT compilation overhead
50
Setup:
Single node Microk8s cluster (16 vCPUs, 16 GB RAM)
JVMs limited to 1 CPU, 500MB
JITServer limited to 8 CPUs and has AOT cache enabled
Load applied with JMeter, 100 threads, 10 ms think-time,
60s ramp-up time
Autoscaler: scales up when average CPU utilization
exceeds 0.5P. Up to 15 AcmeAir instances
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
0 60 120 180 240 300 360 420 480
Throughput
(pages/sec)
Time (sec)
AcmeAir throughput when using Kubernetes autoscaling
Baseline JITServer+AOTcache

45. 51
51
Demo

46. 52
Final thoughts
• JIT provides advantage, but compilation adds overhead
• Disaggregate JIT from JVM è JIT compilation as a service
• Eclipse OpenJ9 JITServer (a.k.a Semeru Cloud Compiler)
o Available now on Linux for Java 8, Java 11 and Java 17 (IBM Semeru Runtimes)
o Especially good for constrained environments (micro-containers)
o Kubernetes ready (Helm chart available, Prometheus integration, Certified
OpenShift/K8s Operator)
o Can improve ramp-up, autoscaling and performance of short lived applications
o Can reduce peak memory footprint, increase app density and reduce costs
o Competing solutions are proprietary (not open-source) and require licensing
52

47. 53
Link to Semeru Cloud Compiler (JITServer)
articles and blogs:
53

48. 54
54
Liberty InstantOn
For Serverless/Functions/Scale-to-zero

49. 56
Is Native Image the answer?
Static AOT compile to native image (GraalVM Native Image)
56
PROS CONS
• Small footprint
• Fast start-up
• Great for short-lived apps
• Limited JVM (static vs dynamic
compiler optimization, limited GC)
• Limited Java features (dynamic, APIs)
• Moving target (frameworks,
ecosystem, standards)
• Dev/Prod parity (devs use compiler?)

50. What is Liberty InstantOn ?
Liberty InstantOn is a holistic
solution that provides fast
startup without compromise
in containers
• Use checkpoint / restore
- CRIU Project - https://criu.org
- Linux technology used to freeze a
running process state
- Checkpoint state used to restore an
application
• Select point in startup to freeze an application
- Restore application and run from the
point where checkpoint was taken
– Liberty InstantOn feature requires support for
checkpoint / restore to be added to the JDK layer for
Liberty to exploit
– IBM Semeru JDK is used by default in Liberty
containers
– Semeru InstantOn : refers to the checkpoint / restore
support in the IBM Semeru JDK
– Collaborative effort between Java framework
(Liberty) and JDK (Semeru) to create production-
ready InstantOn feature that GAed in Liberty
23006
– Since this feature targets only container
environments, you will need to download the relevant
Liberty 23006 (or newer) container image to try out
57

51. 58
Linux CRIU capable of reducing startup time dramatically
• ~10X reduction in first response time by using Liberty Instant On prototype using CRIU
• Liberty Instant On is an effort to explore productization of CRIU use with Liberty
• Main design goals
• Usability : make it easy for an end user to consume the feature
• Functionality : make it possible for end user to run their application without changes
58
0
1000
2000
3000
4000
5000
6000
Liberty baseline Liberty Instant On prototype
PingPerf - First Response Time(ms)
(lower is better)
0
1000
2000
3000
4000
5000
6000
Liberty baseline Liberty Instant On prototype
Rest CRUD - First Response
Time(ms)
(lower is better)
0
1000
2000
3000
4000
5000
6000
Liberty baseline Liberty Instant On prototype
Daytrader7 - First Response Time(ms)
(lower is better)
1470
128
3082
213
5462
310
System Configuration:
-------------------------------
SUT: LinTel – SLES 15 sp3 - Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz, 2 physical cores(4 cpu), 32GB RAM.

52. Liberty + Semeru InstantOn : fast startup using Linux CRIU
Characteristics
Semeru
InstantOn
Semeru JVM Graal Native
Full Java support
Yes Yes No
‘Instant on’
Yes No Yes
High throughput
Yes Yes No
Low memory (under load)
Yes Yes Yes?
Dev-prod parity
Yes Yes No
Dev Build
Prod
Prod
Prod
checkpoint
restore
restore
restore
Target Liberty application container deployments
Start application containers in milliseconds, ideal for
serverless
Leverages Linux CRIU to perform checkpoint / restore
Make it really easy to consume for a user of Liberty
containers
59

53. Liberty + Semeru InstantOn : easy to try existing apps
Checkpoint and Restore in User Space
Queries the OS for resources a process is using. At
checkpoint it saves the state of the process tree to
files. On restore it resumes the application by re-
creating the process and re-run the system calls
used to create the original state.
Functionality is available in libcriu and exposed via
native APIs.
CRIU Project - https://criu.org
Semeru
JDK
CRIU
Liberty
framework
User
application
Uses the Semeru
JDK’s CRIUSupport
API to control CRIU
Uses CRIU native
APIs from within
the JVM
Does not need any
awareness of CRIU
Uses Linux features
to checkpoint and
restore
60

54. Liberty + Semeru InstantOn : easy to try existing apps
Semeru InstantOn
Semeru InstantOn is based on OpenJ9 CRIUSupport
which is a Java API that offers access to Linux CRIU
functionality, facilitates JVM compensations and
provides tuning and serviceability capabilities. It is
offered as a container image build solution on UBI8.
Features
• Time and CPU count compensations
• Single thread hook support mechanism
• Full Java security function after restore, only a small
subset of security function before checkpoint keeps
checkpoint application container image secure
• Tracing and logging support on restore images
• Add certain JVM options and env vars on restore
Semeru
JDK
CRIU
Liberty
framework
User
application
Uses the Semeru
JDK’s CRIUSupport
API to control CRIU
Uses CRIU native
APIs from within
the JVM
Does not need any
awareness of CRIU
Uses Linux features
to checkpoint and
restore
61

55. Liberty + Semeru InstantOn : easy to try existing apps
Open Liberty InstantOn
A framework that simplifies the use of checkpoint
and restore by creating an abstraction which offers
predetermined phases where a checkpoint can be
taken and standard compensations (hooks) that an
application will need. Makes use of OpenJ9
CRIUSupport to facilitate checkpoint and single
thread run hooks.
Features:
• Easy startup guide
• Container image build solution
• Works on existing applications
Semeru
JDK
CRIU
Liberty
framework
User
application
Uses the Semeru
JDK’s CRIUSupport
API to control CRIU
Uses CRIU native
APIs from within
the JVM
Does not need any
awareness of CRIU
Uses Linux features
to checkpoint and
restore
62

56. Liberty + Semeru InstantOn : easy to try existing apps
User applications on Liberty InstantOn
Liberty InstantOn makes it easy for users to boost
startup time. The user simply selects which phase
they want to take a checkpoint in. No changes need
to be made to the application.
Semeru
JDK
CRIU
Liberty
framework
User
application
Uses the Semeru
JDK’s CRIUSupport
API to control CRIU
Uses CRIU native
APIs from within
the JVM
Does not need any
awareness of CRIU
Uses Linux features
to checkpoint and
restore
63

57. InstantOn:Where to checkpoint?
64
Liberty InstantOn leverages Semeru to provide a
seamless checkpoint/restore solution for
developers. With checkpoint restore, there is a
tradeoff between startup time and the complexity
of the restore.
Checkpoint phases
• beforeAppStart
• afterAppStart
Choice of where to checkpoint depends on what
application start code does, e.g., any of the below
would make afterAppStart unsuitable
• Accessing a remote resource, such as a database
• Reading configuration that is expected to change when
the application is deployed
• Starting a transaction
Kernel
start and
feature
runtime
processing
Process
application
Start
application
Accept
requests
afterAppStart
beforeAppStart
Later
checkpoint
==
Faster
restore
time
Later
checkpoint
==
More
complexity
300 – 2200ms
100 – 3000ms
0 – ???ms

58. 67
67
Demo

59. Summary
Adapting and thriving in the cloud
requires evolving your
development, deployment and
runtime for a cloud-native world
Liberty is a versatile runtime for
these purposes
InstantOn is a key capability that
makes Liberty a great runtime for
serverless use cases
– https://openliberty.io
– https://github.com/eclipse-openj9/openj9
– http://ibm.biz/InstantOn_HowToBlog
– https://openliberty.io/blog/2023/06/29/rapid-
startup-instanton.html
– https://openliberty.io/docs/latest/instanton.html
– https://blog.openj9.org/2022/09/26/getting-started-
with-openj9-criu-support
– https://blog.openj9.org/2022/10/14/openj9-criu-
support-a-look-under-the-hood
68

60. 69
69
Wrap-up

61. 70
Architecture Styles vs Liberty
Architecture Solution Compatibility
Monoliths Liberty + Semeru Runtimes/OpenJ9
Macroservices Liberty + Semeru Runtimes/OpenJ9
Microservices Liberty + Semeru Runtimes/OpenJ9 +
Semeru Cloud Compiler
Serverless/Functions Liberty + Semeru Runtimes/OpenJ9 +
InstantOn

62. 71

63. Open Liberty
© 2021 IBM Corporation 72
Useful Links
Why choose Liberty
for Microservices
https://ibm.biz/6ReasonsW
hyLiberty
Choosing the right
Java runtime
https://ibm.biz/ChooseJava
Runtime
How to approach
application modernization
https://ibm.biz/ModernizeJa
vaApps
Open Liberty Site
https://www.openliberty.io
Open Liberty Guides
https://www.openliberty.io/
guides https://openliberty.io

64. Workloads:
Java EE
⎼ WebSphere
Application Server
⎼ Oracle WebLogic
⎼ Red Hat JBoss
⎼ Apache Tomcat
Integration
⎼ IBM MQ
⎼ App Connect
Enterprise (IIB)
Open Source SDK
for other extenders:
https://github.com/IBM/transfo
rmation-advisor-sdk
IBM Cloud Transformation Advisor
© 2021 IBM Corporation 73
Accelerates the modernization journey by quickly discovering and
analyzing on-premise Java EE and/or messaging workloads in the
enterprise to help in determining and executing the optimum
modernization steps for each.
Optimum modernization depends on workload needs! https://ibm.biz/6ReasonsWhyLiberty
Traditional
WebSphere
network
deployment
Traditional
WebSphere
‘base’
Liberty
OpenShift OpenShift OpenShift
Monolith Modernize
Operations
Modernize
Runtime
Modernize
Architecture
Containers, Java EE, monoliths & microservices…

65. 74
a free virtual consultation
with an expert
Liberty References
Read Watch
Explore the latest on WebSphere and
Liberty
ibm.biz/LibertyTV
View our recent Expert TV episode all
about Liberty https://ibm.biz/BdPn8Y
Learn more about Liberty in containers
and Operator-based deployment
ibm.biz/LibertyContainerOperator
Experience
Try Liberty as a beginner
openliberty.io/guides/getting-
started.html
Learn Liberty, MicroProfile,
Containers, Kubernetes,
Hands-on
openliberty.io/guides
Try dev mode in Containers
https://openliberty.io/guides/doc
ker.html
Why choose Liberty for
Microservices
ibm.biz/6ReasonsWhyLiberty
Choosing the right Java
runtime
ibm.biz/ChooseJavaRuntime
How to approach application
modernization
ibm.biz/ModernizeJavaApps

66. 75
75
Thank You