JIT-as-a-Service can improve the performance and cost-efficiency of running Java applications on cloud platforms. By offloading just-in-time compilation to a separate remote process, JIT-as-a-Service reduces the memory footprint and CPU spikes in Java virtual machines. This allows Java applications to run in smaller and more optimized containers that improve density and reduce costs. It also enhances auto-scaling behavior by speeding up the ramp-up time of JVMs through access to dedicated compilation resources. The Eclipse OpenJ9 JVM includes JITServer support for JIT-as-a-Service to realize these benefits when running Java microservices on Kubernetes.
For Java developers, the Just-In-Time (JIT) compiler is key to improved performance. However, in a container world, the performance gains are often negated due to CPU and memory consumption constraints. To help solve this issue, the Eclipse OpenJ9 JVM provides JITServer technology, which separates the JIT compiler from the application. JITServer allows the user to employ much smaller containers enabling a higher density of applications, resulting in cost savings for end-users and/or cloud providers. Because the CPU and memory surges due to JIT compilation are eliminated, the user has a much easier task of provisioning resources for his/her application. Additional advantages include: faster ramp-up time, better control over resources devoted to compilation, increased reliability (JIT compiler bugs no longer crash the application) and amortization of compilation costs across many application instances. We will dig into JITServer technology, showing the challenges of implementation, detailing its strengths and weaknesses and illustrating its performance characteristics. For the cloud audience we will show how it can be deployed in containers, demonstrate its advantages compared to a traditional JIT compilation technique and offer practical recommendations about when to use this technology.
For Java developers, the Just-In-Time (JIT) compiler is key to improved performance. However, in a container world, the performance gains are often negated due to CPU and memory consumption constraints. To help solve this issue, the Eclipse OpenJ9 JVM provides JITServer technology, which separates the JIT compiler from the application.
JITServer allows the user to employ much smaller containers enabling a higher density of applications, resulting in cost savings for end-users and/or cloud providers. Because the CPU and memory surges due to JIT compilation are eliminated, the user has a much easier task of provisioning resources for his/her application. Additional advantages include: faster ramp-up time, better control over resources devoted to compilation, increased reliability (JIT compiler bugs no longer crash the application) and amortization of compilation costs across many application instances.
We will dig into JITServer technology, showing the challenges of implementation, detailing its strengths and weaknesses and illustrating its performance characteristics. For the cloud audience we will show how it can be deployed in containers, demonstrate its advantages compared to a traditional JIT compilation technique and offer practical recommendations about when to use this technology.
This is an updated version of my JITServer talk that I will present at Open Source Summit North America in May 2023
The Next Frontier in Open Source Java Compilers: Just-In-Time Compilation as a Service
For Java developers, the Just-In-Time (JIT) compiler is key to improved performance. However, in a container world, the performance gains are often negated due to CPU and memory consumption constraints. To help solve this issue, the Eclipse OpenJ9 JVM provides JITServer technology, which separates the JIT compiler from the application.
JITServer allows the user to employ much smaller containers enabling a higher density of applications, resulting in cost savings for end-users and/or cloud providers. Because the CPU and memory surges due to JIT compilation are eliminated, the user has a much easier task of provisioning resources for his/her application. Additional advantages include: faster ramp-up time, better control over resources devoted to compilation, increased reliability (JIT compiler bugs no longer crash the application) and amortization of compilation costs across many application instances.
We will dig into JITServer technology, showing the challenges of implementation, detailing its strengths and weaknesses and illustrating its performance characteristics. For the cloud audience we will show how it can be deployed in containers, demonstrate its advantages compared to a traditional JIT compilation technique and offer practical recommendations about when to use this technology.
Monoliths, macroservices, microservices, cloud-native and serverless... Where do we even start? If you are a Java developer, you will likely have to work with one, some, or even all of these deployment approaches. Does this mean learning multiple frameworks, tools and methods? It certainly looks that way, based on the many deployment-specific solutions being proposed to the Java development community.
In this presentation, we will look into these solutions, weighing their strengths and weaknesses. We will also contrast this with one-size-fits-all solutions being offered by modern open-source cloud-native Java runtimes like Open Liberty. Does it have the right technology to compete in microservice and serverless environments - can one runtime really do it all?
The Next Frontier in Open Source Java Compilers: Just-In-Time Compilation as a Service
For Java developers, the Just-In-Time (JIT) compiler is key to improved performance. However, in a container world, the performance gains are often negated due to CPU and memory consumption constraints. To help solve this issue, the Eclipse OpenJ9 JVM provides JITServer technology, which separates the JIT compiler from the application.
JITServer allows the user to employ much smaller containers enabling a higher density of applications, resulting in cost savings for end-users and/or cloud providers. Because the CPU and memory surges due to JIT compilation are eliminated, the user has a much easier task of provisioning resources for his/her application. Additional advantages include: faster ramp-up time, better control over resources devoted to compilation, increased reliability (JIT compiler bugs no longer crash the application) and amortization of compilation costs across many application instances.
We will dig into JITServer technology, showing the challenges of implementation, detailing its strengths and weaknesses and illustrating its performance characteristics. For the cloud audience we will show how it can be deployed in containers, demonstrate its advantages compared to a traditional JIT compilation technique and offer practical recommendations about when to use this technology.
For Java developers, the Just-In-Time (JIT) compiler is key to improved performance. However, in a container world, the performance gains are often negated due to CPU and memory consumption constraints. To help solve this issue, the Eclipse OpenJ9 JVM provides JITServer technology, which separates the JIT compiler from the application. JITServer allows the user to employ much smaller containers enabling a higher density of applications, resulting in cost savings for end-users and/or cloud providers. Because the CPU and memory surges due to JIT compilation are eliminated, the user has a much easier task of provisioning resources for his/her application. Additional advantages include: faster ramp-up time, better control over resources devoted to compilation, increased reliability (JIT compiler bugs no longer crash the application) and amortization of compilation costs across many application instances. We will dig into JITServer technology, showing the challenges of implementation, detailing its strengths and weaknesses and illustrating its performance characteristics. For the cloud audience we will show how it can be deployed in containers, demonstrate its advantages compared to a traditional JIT compilation technique and offer practical recommendations about when to use this technology.
For Java developers, the Just-In-Time (JIT) compiler is key to improved performance. However, in a container world, the performance gains are often negated due to CPU and memory consumption constraints. To help solve this issue, the Eclipse OpenJ9 JVM provides JITServer technology, which separates the JIT compiler from the application.
JITServer allows the user to employ much smaller containers enabling a higher density of applications, resulting in cost savings for end-users and/or cloud providers. Because the CPU and memory surges due to JIT compilation are eliminated, the user has a much easier task of provisioning resources for his/her application. Additional advantages include: faster ramp-up time, better control over resources devoted to compilation, increased reliability (JIT compiler bugs no longer crash the application) and amortization of compilation costs across many application instances.
We will dig into JITServer technology, showing the challenges of implementation, detailing its strengths and weaknesses and illustrating its performance characteristics. For the cloud audience we will show how it can be deployed in containers, demonstrate its advantages compared to a traditional JIT compilation technique and offer practical recommendations about when to use this technology.
This is an updated version of my JITServer talk that I will present at Open Source Summit North America in May 2023
The Next Frontier in Open Source Java Compilers: Just-In-Time Compilation as a Service
For Java developers, the Just-In-Time (JIT) compiler is key to improved performance. However, in a container world, the performance gains are often negated due to CPU and memory consumption constraints. To help solve this issue, the Eclipse OpenJ9 JVM provides JITServer technology, which separates the JIT compiler from the application.
JITServer allows the user to employ much smaller containers enabling a higher density of applications, resulting in cost savings for end-users and/or cloud providers. Because the CPU and memory surges due to JIT compilation are eliminated, the user has a much easier task of provisioning resources for his/her application. Additional advantages include: faster ramp-up time, better control over resources devoted to compilation, increased reliability (JIT compiler bugs no longer crash the application) and amortization of compilation costs across many application instances.
We will dig into JITServer technology, showing the challenges of implementation, detailing its strengths and weaknesses and illustrating its performance characteristics. For the cloud audience we will show how it can be deployed in containers, demonstrate its advantages compared to a traditional JIT compilation technique and offer practical recommendations about when to use this technology.
Monoliths, macroservices, microservices, cloud-native and serverless... Where do we even start? If you are a Java developer, you will likely have to work with one, some, or even all of these deployment approaches. Does this mean learning multiple frameworks, tools and methods? It certainly looks that way, based on the many deployment-specific solutions being proposed to the Java development community.
In this presentation, we will look into these solutions, weighing their strengths and weaknesses. We will also contrast this with one-size-fits-all solutions being offered by modern open-source cloud-native Java runtimes like Open Liberty. Does it have the right technology to compete in microservice and serverless environments - can one runtime really do it all?
The Next Frontier in Open Source Java Compilers: Just-In-Time Compilation as a Service
For Java developers, the Just-In-Time (JIT) compiler is key to improved performance. However, in a container world, the performance gains are often negated due to CPU and memory consumption constraints. To help solve this issue, the Eclipse OpenJ9 JVM provides JITServer technology, which separates the JIT compiler from the application.
JITServer allows the user to employ much smaller containers enabling a higher density of applications, resulting in cost savings for end-users and/or cloud providers. Because the CPU and memory surges due to JIT compilation are eliminated, the user has a much easier task of provisioning resources for his/her application. Additional advantages include: faster ramp-up time, better control over resources devoted to compilation, increased reliability (JIT compiler bugs no longer crash the application) and amortization of compilation costs across many application instances.
We will dig into JITServer technology, showing the challenges of implementation, detailing its strengths and weaknesses and illustrating its performance characteristics. For the cloud audience we will show how it can be deployed in containers, demonstrate its advantages compared to a traditional JIT compilation technique and offer practical recommendations about when to use this technology.
Microservices is the dominant architecture for developing new applications, as it is ideally suited to cloud deployments. When using JVM-based services, each instance works in isolation and has no awareness of previous runs of a service. The Cloud Native Compiler detaches the JIT compiler from individual JVMs and centralises it in the cloud, effectively a JIT-as-a-Service. The benefits of this are caching of compiled code for instant reuse, use of greater resources for higher optimisation as well as many others. The session will discuss the challenges of cloud-based JVMs and how we can significantly improve performance with reduced costs.
Secrets of Performance Tuning Java on KubernetesBruno Borges
Java on Kubernetes may seem complicated, but after a bit of YAML and Dockerfiles, you will wonder what all that fuss was. But then the performance of your app in 1 CPU/1 GB of RAM makes you wonder. Learn how JVM ergonomics, CPU throttling, and GCs can help increase performance while reducing costs.
Simple tweaks to get the most out of your jvmJamie Coleman
Many developers don’t think about the JVM level when creating applications. It is something that just simply works. Now more applications are becoming cloud-native and we have JVM’s running in every microservice container, each performance gain can have massive benefits when scaled up. Some tweaks are very easy to implement and can have huge impacts on start-up time and performance of your applications. This talk will go through all the different JVM options and give you some easy and simple advice on how to get the most out of your JVM to save not only money but also energy on the cloud.
Amazon EC2 provides a broad selection of instance types to deliver high performance for a diverse mix of applications. In this session, we overview the drivers of system performance and discuss in depth how Amazon EC2 instances deliver system performance while also providing elasticity and complete control over your infrastructure. We also detail best practices and share performance tips for getting the most out of your Amazon EC2 instances.
Simple tweaks to get the most out of your JVMJamie Coleman
Many developers don’t think about the JVM level when creating applications. It is something that just simply works. Now more applications are becoming cloud-native and we have JVM’s running in every microservice container, each performance gain can have massive benefits when scaled up. Some tweaks are very easy to implement and can have huge impacts on start-up time and performance of your applications. This talk will go through all the different JVM options and give you some easy and simple advice on how to get the most out of your JVM to save not only money but also energy on the cloud.
What we've learned from running a PostgreSQL managed service on KubernetesDoKC
In this talk, I will share some of our learnings from running a managed PostgreSQL/TimescaleDB service on Kubernetes on AWS for a little more than a year: I’ll start with the motivation of running managed PostgreSQL on Kubernetes, the benefits and drawbacks. I’ll describe the architecture of the managed PostgreSQL cloud on Kubernetes I’ll zoom in on how we solved some of the Kubernetes-specific issues within our cloud, such as upgrading extensions without downtimes, taming the dreaded OOM killer, and doing regular maintenance and PostgreSQL major upgrades. I’ll share how open-source tools from the PostgreSQL ecosystem helps us to run the service and explain how we use them in a slightly non-trivial way.
This talk was given by Oleksii Kliukin for DoK Day Europe @ KubeCon 2022.
A technical presentation on how Zing changes parts of the JVM to eliminate GC pauses, generate more heavily optimised code from the JIT and reduce the warm up time.
Latest (storage IO) patterns for cloud-native applications OpenEBS
Applying micro service patterns to storage giving each workload its own Container Attached Storage (CAS) system. This puts the DevOps persona within full control of the storage requirements and brings data agility to k8s persistent workloads. We will go over the concept and the implementation of CAS, as well as its orchestration.
The venerable Servlet Container still has some performance tricks up its sleeve - this talk will demonstrate Apache Tomcat's stability under high load, describe some do's (and some don'ts!), explain how to performance test a Servlet-based application, troubleshoot and tune the container and your application and compare the performance characteristics of the different Tomcat connectors. The presenters will share their combined experience supporting real Tomcat applications for over 20 years and show how a few small changes can make a big, big difference.
DevOps for ETL processing at scale with MongoDB, Solr, AWS and ChefGaurav "GP" Pal
Large scale data processing for Extract Transform and Loading (ETL) jobs is a very common practice. The stackArmor DevOps team developed a Chef based automation solution to automate the AWS environment provisioning, code deployment and data ingestion processing to ingest and process over 2 TB of Data.
This presentation covers the technologies used, the planning phase, AWS instance selection and optimizing the ETL processing for not only performance but also cost.
The target was to process 500 million rows within 72 hours with a processing rate of 5 million transactions per hour.
The presentation also provides pitfalls and automation optimizations performed to accomplish the targeted processing rates.
The presentation was delivered at the DevOpsDC Meetup on May 17, 2016
In-memory processing has started to become the norm in large scale data handling. This is aclose to the metal analysis of highly important but often neglected aspects of memory accesstimes and how it impacts big data and NoSQL technologies.We cover aspects such as the TLB, the Transparent Huge Pages, the QPI Link, Hyperthreading and the impact of virtualization on high-memory footprint applications. We present benchmarks of various technologies ranging from Cloudera’s Impala to Couchbase and how they are impacted by the underlying hardware.The key takeaway is a better understanding of how to size a cluster, how to choose a cloud provider and an instance type for big data and NoSQL workloads and why not every core or GB of RAM is created equal.
How to monitor Java application and JVM performance with Flight Recorder and Mission Control. Starts with a discussion of general JVM performance considerations like GC, JIT and threads.
Microservices is the dominant architecture for developing new applications, as it is ideally suited to cloud deployments. When using JVM-based services, each instance works in isolation and has no awareness of previous runs of a service. The Cloud Native Compiler detaches the JIT compiler from individual JVMs and centralises it in the cloud, effectively a JIT-as-a-Service. The benefits of this are caching of compiled code for instant reuse, use of greater resources for higher optimisation as well as many others. The session will discuss the challenges of cloud-based JVMs and how we can significantly improve performance with reduced costs.
Secrets of Performance Tuning Java on KubernetesBruno Borges
Java on Kubernetes may seem complicated, but after a bit of YAML and Dockerfiles, you will wonder what all that fuss was. But then the performance of your app in 1 CPU/1 GB of RAM makes you wonder. Learn how JVM ergonomics, CPU throttling, and GCs can help increase performance while reducing costs.
Simple tweaks to get the most out of your jvmJamie Coleman
Many developers don’t think about the JVM level when creating applications. It is something that just simply works. Now more applications are becoming cloud-native and we have JVM’s running in every microservice container, each performance gain can have massive benefits when scaled up. Some tweaks are very easy to implement and can have huge impacts on start-up time and performance of your applications. This talk will go through all the different JVM options and give you some easy and simple advice on how to get the most out of your JVM to save not only money but also energy on the cloud.
Amazon EC2 provides a broad selection of instance types to deliver high performance for a diverse mix of applications. In this session, we overview the drivers of system performance and discuss in depth how Amazon EC2 instances deliver system performance while also providing elasticity and complete control over your infrastructure. We also detail best practices and share performance tips for getting the most out of your Amazon EC2 instances.
Simple tweaks to get the most out of your JVMJamie Coleman
Many developers don’t think about the JVM level when creating applications. It is something that just simply works. Now more applications are becoming cloud-native and we have JVM’s running in every microservice container, each performance gain can have massive benefits when scaled up. Some tweaks are very easy to implement and can have huge impacts on start-up time and performance of your applications. This talk will go through all the different JVM options and give you some easy and simple advice on how to get the most out of your JVM to save not only money but also energy on the cloud.
What we've learned from running a PostgreSQL managed service on KubernetesDoKC
In this talk, I will share some of our learnings from running a managed PostgreSQL/TimescaleDB service on Kubernetes on AWS for a little more than a year: I’ll start with the motivation of running managed PostgreSQL on Kubernetes, the benefits and drawbacks. I’ll describe the architecture of the managed PostgreSQL cloud on Kubernetes I’ll zoom in on how we solved some of the Kubernetes-specific issues within our cloud, such as upgrading extensions without downtimes, taming the dreaded OOM killer, and doing regular maintenance and PostgreSQL major upgrades. I’ll share how open-source tools from the PostgreSQL ecosystem helps us to run the service and explain how we use them in a slightly non-trivial way.
This talk was given by Oleksii Kliukin for DoK Day Europe @ KubeCon 2022.
A technical presentation on how Zing changes parts of the JVM to eliminate GC pauses, generate more heavily optimised code from the JIT and reduce the warm up time.
Latest (storage IO) patterns for cloud-native applications OpenEBS
Applying micro service patterns to storage giving each workload its own Container Attached Storage (CAS) system. This puts the DevOps persona within full control of the storage requirements and brings data agility to k8s persistent workloads. We will go over the concept and the implementation of CAS, as well as its orchestration.
The venerable Servlet Container still has some performance tricks up its sleeve - this talk will demonstrate Apache Tomcat's stability under high load, describe some do's (and some don'ts!), explain how to performance test a Servlet-based application, troubleshoot and tune the container and your application and compare the performance characteristics of the different Tomcat connectors. The presenters will share their combined experience supporting real Tomcat applications for over 20 years and show how a few small changes can make a big, big difference.
DevOps for ETL processing at scale with MongoDB, Solr, AWS and ChefGaurav "GP" Pal
Large scale data processing for Extract Transform and Loading (ETL) jobs is a very common practice. The stackArmor DevOps team developed a Chef based automation solution to automate the AWS environment provisioning, code deployment and data ingestion processing to ingest and process over 2 TB of Data.
This presentation covers the technologies used, the planning phase, AWS instance selection and optimizing the ETL processing for not only performance but also cost.
The target was to process 500 million rows within 72 hours with a processing rate of 5 million transactions per hour.
The presentation also provides pitfalls and automation optimizations performed to accomplish the targeted processing rates.
The presentation was delivered at the DevOpsDC Meetup on May 17, 2016
In-memory processing has started to become the norm in large scale data handling. This is aclose to the metal analysis of highly important but often neglected aspects of memory accesstimes and how it impacts big data and NoSQL technologies.We cover aspects such as the TLB, the Transparent Huge Pages, the QPI Link, Hyperthreading and the impact of virtualization on high-memory footprint applications. We present benchmarks of various technologies ranging from Cloudera’s Impala to Couchbase and how they are impacted by the underlying hardware.The key takeaway is a better understanding of how to size a cluster, how to choose a cloud provider and an instance type for big data and NoSQL workloads and why not every core or GB of RAM is created equal.
How to monitor Java application and JVM performance with Flight Recorder and Mission Control. Starts with a discussion of general JVM performance considerations like GC, JIT and threads.
This presentation by Morris Kleiner (University of Minnesota), was made during the discussion “Competition and Regulation in Professions and Occupations” held at the Working Party No. 2 on Competition and Regulation on 10 June 2024. More papers and presentations on the topic can be found out at oe.cd/crps.
This presentation was uploaded with the author’s consent.
0x01 - Newton's Third Law: Static vs. Dynamic AbusersOWASP Beja
f you offer a service on the web, odds are that someone will abuse it. Be it an API, a SaaS, a PaaS, or even a static website, someone somewhere will try to figure out a way to use it to their own needs. In this talk we'll compare measures that are effective against static attackers and how to battle a dynamic attacker who adapts to your counter-measures.
About the Speaker
===============
Diogo Sousa, Engineering Manager @ Canonical
An opinionated individual with an interest in cryptography and its intersection with secure software development.
Sharpen existing tools or get a new toolbox? Contemporary cluster initiatives...Orkestra
UIIN Conference, Madrid, 27-29 May 2024
James Wilson, Orkestra and Deusto Business School
Emily Wise, Lund University
Madeline Smith, The Glasgow School of Art
Acorn Recovery: Restore IT infra within minutesIP ServerOne
Introducing Acorn Recovery as a Service, a simple, fast, and secure managed disaster recovery (DRaaS) by IP ServerOne. A DR solution that helps restore your IT infra within minutes.
Have you ever wondered how search works while visiting an e-commerce site, internal website, or searching through other types of online resources? Look no further than this informative session on the ways that taxonomies help end-users navigate the internet! Hear from taxonomists and other information professionals who have first-hand experience creating and working with taxonomies that aid in navigation, search, and discovery across a range of disciplines.
Doctoral Symposium at the 17th IEEE International Conference on Software Test...
JITServerTalk Nebraska 2023.pdf
1. The Next Frontier in
Open Source Java Compilers:
Just-In-Time Compilation as a Service
1
2. Who am I?
• Software Engineer and Developer
Advocate at IBM
• Based in the Austin, TX USA
• Focus on OSS cloud native Java
technologies
Rich Hagarty
@rhagarty8
3. Agenda
REASON:
JVM and JIT compiler
– the good and the
bad
4
JIT-as-a-Service
SOLUTION:
JIT-as-a-Service to
the rescue
PROBLEM:
Java on Cloud - a bad
fit for microservices
5. Legacy Java Apps
6
•Java monolith on
dedicated server
•Plenty of CPU power
and memory
•Never went down
•6 month
upgrade/refresh
schedule
6. Moving to the Cloud
7
-Running in containers
-Managed by Cloud
Provider (and K8s)
-Auto-scaling to meet
demand
Cloud native App talking to Microservices
7. Main Motivators
8
-Flexible & scalable
-Easier to roll-out new releases more frequently
-Take advantage of latest-greatest Cloud technologies
-Less infrastructure to maintain and manage
-Saving money
8. But what about performance?
9
Need to balance cost vs performance
2 variables:
-Container size
-Number of instances (efficient scaling)
14. Java Virtual Machine (JVM)
15
The Good
• Device independent – write once, run anywhere
• > 25 years of improvements
• JIT produces optimized machine code through use of Profilers
• Efficient garbage collection
• Longer it runs, the better it runs (JVM collects more profile data, JIT
compiles more methods)
15. Java Virtual Machine (JVM)
16
The Bad
• Initial execution run is “interpreted”, which is relatively slow
• “Hot Spot” methods compiled by JIT can create CPU and memory
spikes
• CPU spikes cause lower QoS
• Memory spikes cause OOM issues, including crashes
• Slow start-up time
• Slow ramp-up time
16. Java Virtual Machine (JVM)
17
0
50
100
150
200
250
300
350
400
0 30 60 90
CPU
utilization
(%)
Time (sec)
Daytrader7 CPU consumption
CPU spikes caused
by JIT compilation
0
100000
200000
300000
400000
500000
600000
0 30 60 90
Resident
set
size
(KB)
Time (sec)
Daytrader7 memory footprint
Footprint spikes caused
by JIT compilation
18. Container Size
19
Main issues:
•Need to over-provision to
avoid OOM
•Very hard to do – JVMs
have a non-deterministic
behavior
0
100000
200000
300000
400000
500000
600000
0 30 60 90
Resident
set
size
(KB)
Time (sec)
Daytrader7 memory footprint
Footprint spikes caused
by JIT compilation
22. JIT-as-a-Service
Decouple the JIT compiler from the JVM and let it run as an independent process
Offload JIT
compilation to
remote process
Remote
JIT
Remote
JIT
JVM
JIT
JVM
JIT
Kubernetes
Control Plane
Treat JIT
compilation as a
cloud service
• Auto-managed by orchestrator
• A mono-to-micro solution
• Local JIT still available
23
23. Eclipse OpenJ9 JITServer
• JITServer feature is available in the Eclipse OpenJ9 JVM
• “Semeru Cloud Compiler” when used with Semeru Runtimes
• OpenJ9 combines with OpenJDK to form a full JDK
Link to GitHub repo: https://github.com/eclipse-openj9/openj9
24
24. Overview of Eclipse OpenJ9
Designed from the start to span all the operating
systems needed by IBM products
This JVM can go from small to large
Can handle constrained environments or memory
rich ones
Renowned for its small footprint, fast start-up and
ramp-up time
Is used by the largest enterprises on the planet
25
26. IBM Semeru Runtimes
“The part of Java that’s really in the clouds”
IBM-built OpenJDK runtimes powered by the Eclipse OpenJ9 JVM
No cost, stable, secure, high performance, cloud optimized, multi-
platform, ready for development and production use
Open Edition
• Open source license (GPLv2+CE)
• Available for Java 8 (LTS), 11 (LTS), 16, 17 (LTS), 18, 19, 20
Certified Edition
• IBM license
• Java SE TCK certified.
• Available for Java 11, 17
27
27. IBM Semeru Runtimes
“The part of Java that’s really in the clouds”
Other OpenJDK Distributions:
• OpenJDK builds by Oracle
• Adoptium Eclipse Temurin
• Azul Zulu
• BellSoft Liberica
• Amazon Corretto
• Alibaba Dragonwell
• Microsoft Build of OpenJDK
• Red Hat Build of OpenJDK
• Oracle GraalVM
28
28. JITServer advantages for JVM Clients
29
Provisioning
Easier to size; only consider the needs
of the application
Performance
Improved ramp-up time due to JITServer
supplying extra CPU power when the JVM
needs it the most.
Reduced CPU consumption with JITServer AOT
cache
Cost
Reduced memory consumption means
increased application density and reduced
operational cost.
Efficient auto-scaling – only pay for what
you need/use.
Resiliency
If the JITServer crashes, the JVM can
continue to run and compile with its
local JIT
30. JITServer value in Kubernetes
• https://blog.openj9.org/2021/10/20/save-money-with-jitserver-on-the-
cloud-an-aws-experiment/
• Experimental test bed
• ROSA (RedHat OpenShift Service on AWS)
• Demonstrate that JITServer is not tied to IBM HW or SW
• OCP cluster: 3 master nodes, 2 infra nodes, 3 worker nodes
• Worker nodes have 8 vCPUs and 16 GB RAM (only ~12.3 GB available)
• Four different applications
• AcmeAir Microservices
• AcmeAir Monolithic
• Petclinic (Springboot framework)
• Quarkus
• Low amount of load to simulate conditions seen in practice
• OpenShift Scheduler to manage pod and node deployments/placement
31
31. JITServer improves container density and cost
Default config
AM 500
B 550
C 550
F 450 P 450
P 450
B 550
F 450
AM 500
A 350
AM 500
M 200
Q 350
P 450
Q 350
D 600
D 1000
F 450
B 550
Q 350
AM 500
AM 500
AM 500
B 550
B 550
A 350
C 550
F 450
M 200
P 450
P 450
P 450
Q 350
Q 350
D 1000
AM 500 B 550
P 450
AM 500
B 550
B 550
C 550
C 550
F 450
F 450 P 450
Q 350
Q 350
D 1000
D 1000
Q 350
AM 250
AM 250
P 250
P 250
F 250
F 250
B 400 C 350
Q 150
Q 150
M 150
AM 250
AM 250
P 250
P 250
F 250
B 400
Q 150
Q 150
J 1200
A 250
B 400
B 400
C 350
D 1000 D 1000
D 600
AM 250
AM 250
P 250
P 250
F 250
F 250
B 400 C 350
Q 150
Q 150
M 150
AM 250
AM 250
P 250
P 250
F 250
B 400
Q 150
Q 150
J 1200
A 250
B 400
B 400
C 350
D 1000
D 1000
JITServer config
Legend:
AM: AcmeAir monolithic
A: Auth service
B: Booking service
C: Customer service
D: Database (mongo/postgres)
F: Flight service
J: JITServer
M: Main service
P: Petclinic
Q: Quarkus
Total=8250 MB Total=8550 MB Total=8600 MB
Total=9250 MB Total=9850 MB
6.3 GB less
32
32. Throughput comparison
0
200
400
600
800
1000
1200
0 240 480 720 960 1200 1440 1680 1920
Throughput
(pages/sec)
Time (sec)
AcmeAir Microservices
0
200
400
600
800
1000
1200
0 240 480 720 960 1200 1440 1680 1920
Throughput
(pages/sec)
Time (sec)
AcmeAir Monolithic
0
20
40
60
80
100
120
140
160
180
0 240 480 720 960 1200 1440 1680 1920
Throughput
(pages/sec)
Time (sec)
Petclinic
0
500
1000
1500
2000
2500
3000
3500
0 240 480 720 960 1200 1440 1680 1920
Throughput
(pages/sec)
Time (sec)
Quarkus
Machine load:
17.5% from apps
7% from OpenShift
33
è JITServer and default configuration achieve the same level of throughput at steady-state
JITServer Baseline
33. Conclusions from high density experiments
• JITServer can improve container density and reduce operational costs
of Java applications running in the cloud by 20-30%
• Steady-state throughput is the same despite using fewer nodes
34
34. Horizontal Pod Autoscaling in Kubernetes
• Better autoscaling behavior with JITServer due to faster ramp-up
• Less risk to trick the HPA due to transient JIT compilation overhead
35
Setup:
Single node Microk8s cluster (16 vCPUs, 16 GB RAM)
JVMs limited to 1 CPU, 500MB
JITServer limited to 8 CPUs and has AOT cache enabled
Load applied with JMeter, 100 threads, 10 ms think-time,
60s ramp-up time
Autoscaler: scales up when average CPU utilization
exceeds 0.5P. Up to 15 AcmeAir instances
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
0 60 120 180 240 300 360 420 480
Throughput
(pages/sec)
Time (sec)
AcmeAir throughput when using Kubernetes autoscaling
Baseline JITServer+AOTcache
36. Improve ramp-up time with JITServer
• Experiment in docker containers
• Show that JITServer improves ramp-up
• Show that JITServer allows a lower memory limit for JVM containers
37
OpenLiberty+
AcmeAir
1P, 400 MB
limit
OpenLiberty+
AcmeAir
1P, 200 MB
limit
OpenLiberty+
AcmeAir
1P, 200 MB
limit
MongoDB
JITServer
4P, 1GB limit
JMeter
JMeter
JMeter
InfluxDB
Grafana
Collect throughput
data from JMeter
Display throughput
data
Apply load
to AcmeAir
instances
Provide data
persistence
services
Run the AcmeAir
application
Provide JIT
compilation
services
Grafana
Prometheus
Scrape
metrics
Display
JITServer
metrics
40. JITServer usage basics
• One JDK, three different personas
• Normal JVM: $JAVA_HOME/bin/java MyApp
• JITServer: $JAVA_HOME/bin/jitserver
• Client JVM: $JAVA_HOME/bin/java -XX:+UseJITServer MyApp
• Optional further configuration through JVM command line options
• At the server:
-XX:JITServerPort=… default: 38400
• At the client:
-XX:JITServerAddress=… default: ‘localhost’
-XX:JITServerPort=… default: 38400
• Full list of options: https://www.eclipse.org/openj9/docs/jitserver/
• Note: Java version and OpenJ9 release at client and server must match
41
41. JITServer usage in Kubernetes
• Typically we create/configure
• JITServer deployment
• JITServer service (clients interact with service)
• Use
• Yaml files
• Helm charts: repo https://raw.githubusercontent.com/eclipse/openj9-utils/master/helm-chart/
• Certified OpenShift/K8s Operators from Open Liberty
• Tutorial: https://developer.ibm.com/tutorials/using-openj9-jitserver-in-
kubernetes/
42
42. JITServer encryption/authentication through TLS
• Needs additional JVM options
• Server: -XX:JITServerSSLKey=key.pem -XX:JITServerSSLCert=cert.pem
• Client: -XX:JITServerSSLRootCerts=cert.pem
• Certificates and keys can be provided using Kubernetes TLS Secrets
• Create TLS secret:
• kubectl create secret tls my-tls-secret --key <private-key-filename> --cert <certificate-filename>
• Use a volume to map “pem” files
43
apiVersion: v1
kind: Pod
metadata:
name: my-pod
spec:
containers:
- name: my-container-name
image: my-image
volumeMounts:
- name: secret-volume
mountPath: /etc/secret-volume
volumes:
- name: secret-volume
secret:
secretName: my-tls-secret
43. Monitoring
• Support for custom metrics for Prometheus
• Metrics scrapping: GET request to http://<jitserveraddress>:<port>/metrics
• Command line options:
-XX:+JITServerMetrics -XX:JITServerMetricsPort=<port>
• Metrics available
• jitserver_cpu_utilization
• jitserver_available_memory
• jitserver_connected_clients
• jitserver_active_threads
• Verbose logging
• Print client/server connections
-XX:+JITServerLogConnections
• Heart-beat: periodically print to verbose log some JITServer stats
• -Xjit:statisticsFrequency=<period-in-ms>
• Print detailed information about client/server behavior
-Xjit:verbose={JITServer},verbose={compilePerformance},vlog=…
44
44. JITServer usage recommendations
• When to use it
• JVM needs to compile many methods in a relatively short time
• JVM is running in a CPU/memory constrained environment, which can worsen
interference from the JIT compiler
• The network latency between JITServer and client VM is relatively low (<1ms)
• To keep network latency low, use “latency-performance” profile for tuned and configure your VM
with SR-IOV
• Recommendations
• 10-20 client JVMs connected to a single JITServer instance
• JITServer needs 1-2 GB of RAM
• Better performance if the compilation phases from different JVM clients do not overlap
(stagger)
• Encryption adds to the communication overhead; avoid if possible
• In K8s use “sessionAffinity” to ensure a client always connects to the same server
• Enable JITServer AOT cache: -XX:+JITServerUseAOTCache (client needs to have shared
class cache enabled)
45
45. Final thoughts
• JIT provides advantage, but compilation adds overhead
• Disaggregate JIT from JVM è JIT compilation as a service
• Eclipse OpenJ9 JITServer (a.k.a Semeru Cloud Compiler)
• Available now on Linux for Java 8, Java 11 and Java 17 (IBM Semeru Runtimes)
• Especially good for constrained environments (micro-containers)
• Kubernetes ready (Helm chart available, Prometheus integration)
• Can improve ramp-up, autoscaling and performance of short lived applications
• Can reduce peak memory footprint, increase app density and reduce costs
• Java solution to Java problem, with no compromise
47
48. 50
a free virtual consultation
with an expert
Liberty References
Read Watch
Explore the latest on WebSphere and
Liberty
ibm.biz/LibertyTV
View our recent Expert TV episode all
about Liberty https://ibm.biz/BdPn8Y
Learn more about Liberty in containers
and Operator-based deployment
ibm.biz/LibertyContainerOperator
Experience
Try Liberty as a beginner
openliberty.io/guides/getting-
started.html
Learn Liberty, MicroProfile,
Containers, Kubernetes,
Hands-on
openliberty.io/guides
Try dev mode in Containers
https://openliberty.io/guides/doc
ker.html
Why choose Liberty for
Microservices
ibm.biz/6ReasonsWhyLiberty
Choosing the right Java
runtime
ibm.biz/ChooseJavaRuntime
How to approach application
modernization
ibm.biz/ModernizeJavaApps