SlideShare a Scribd company logo
1 of 56
Payments to grow your world
TechSquad PPT Toolbox
Loom
promises
be there!
Get in touch:
@jefrajames
jean-francois.james@worldline.com
linkedin.com/ln/jefrajames
Jean-François JAMES
Software Architect
• Focus on Java, Jakarta EE, MicroProfile
• Open Source contributor
• Head of DevRel
Get in touch:
@foxlegend
david.pequegnot@worldline.com
linkedin.com/in/davidpequegnot/
David PEQUEGNOT
Performance Engineer
• Focus on JVM technologies
• ❤ Observability
We design payments technology
that powers the growth of millions
of businesses around the world.
Who are we?
7000+ engineers
in over 40 countries
Managing 28+ billion
transactions per year
€250M spent in R&D
every year
Handling 150+
payment methods
How your application server handles requests?
So, what’s the problem with our Java apps?
Application Server
Request 1
Request 2
Request n
Thread 1
Thread 2
Thread n
…
… Thread pool
Images: Flaticon.com/Freepik
Blocking, waiting, blocking, waiting, blocking, waiting …
So, what’s the problem with our Java apps?
Wait for WebService response Wait for database
Image: Flaticon.com/Freepik
May Reactive Be With You
Reactive vs Imperative
Request 1
Request 2
Request 3
Request 4
Request 5
…
Event
Loop
Image: Flaticon.com/Freepik
Imperative Reactive
Pro. & Cons.
Reactive vs Imperative
Request 1
Request 2
Request 3
Request 4
Request 5
…
Event
Loop
Image: Flaticon.com/Freepik
Code imperative, work reactive!
Block for free!
Loom promises
One project, 3 JEPs
Virtual
Threads
Lightweight threads
JEP 444
Stable with
Java 21 LTS
Structured
Concurrency
Facilitate tasks dev&
run on top of Virtual
Threads
JEP 453
Preview with
Java 21 LTS
Scoped
Values
Modernization,
optimization of
Thread Locals
JEP 446
Preview with
Java 21 LTS
Evolution of Java Concurrency
1997
Java 1.1
Green Threads
2004
Java 1.5
Executor
2014
Java 8
CompletableFuture
Parallel Streams
1998
Java 1.2
Native Threads
2011
Java 1.7
ForkJoinPool
2023
Java 21
Virtual Threads
From a few Platform Threads …
Platform Threads
OS Threads
OS Scheduling
Virtual Threads
OS
JVM
Kernel mode
User mode
… to millions Virtual Threads
Platform (Carrier) Threads
OS Threads
OS Scheduling
JVM Scheduling (ForkJoinPool)
Virtual Threads
OS
JVM
Inside the JVM: the magic of Continuation
Heap
Continuation.yield(scope) continuation.run()
Virtual Thread 1
Carrier Thread a
Carrier Thread b
Carrier Thread c
…
unmount
mount
ForkJoin
Pool
wait
Many changes in the JVM
No impact on user code (vs Kotlin’s coroutines)
Heap and GC activity to be further monitored and tuned
Not suitable for CPU-hungry processing
Loom promises
Keypoints
Payments to grow your world
TechSquad PPT Toolbox
Demo time!
Round 1
Java SE
• Basic Java SE tests
• Inspired from Eclipse Jetty loom-trial (Webtide)
• Code on GitHub
• Measured on MacOS (ARM M1, 8 CPU, 16 G RAM) with Java 20 (Temurin distro)
Methodology
Some code
Cheap to create?
0
50
100
150
200
250
1 2 4 8 16 32 32 64
vthread started/ms
Really cheap to create?
0
1
2
3
4
5
6
7
8
9
1 2 4 8 16 32 32 64
Carrier Threads
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
80,000
1 2 4 8 16 32 32 64
Heap size (MB)
• Up to 64 millions Virtual Threads in a single JVM!
• Just 8 Carrier Threads used in the background
• Same test with Platform Threads: 4K+ on MacOS and 30k+ on Linux
• About 200 Virtual Threads created/msec, 1 MB Heap/thread
• No hard limit with Virtual Threads: the limit is the Garbage Collector efficiency!
How long to start Virtual Threads?
One million Threads running a recursive genetic
mutation algorithm: is it possible?
TechSquad PPT Toolbox
DeepStack
Test conditions:
• Stack depth: 1000
• Timeout if Thread creation > 3s
Platform Threads
TechSquad PPT Toolbox
DeepStack
Virtual Threads
DeepStack
First results
How to increase the number of Virtual Threads? …
DeepStack
Platform Threads Virtual Threads
4k+ 30k+ 30k+
Images: Flaticon.com/Freepik
Threads over Heap Size
TechSquad PPT Toolbox
DeepStack
0
10000
20000
30000
40000
50000
60000
70000
1GB 2GB 4GB 8GB
Max
Threads
JVM Heap Size
Platform Virtual
Platform and Virtual Threads: Take aways
Platform Threads
Java’s traditional unit of concurrency
Trivial wrappers around OS Threads
Scarce resources
Expensive to create & block
Counter measures: thread pools, reactive
programming, cooperative scheduling
(async/await construct)
Virtual Threads
A new lightweight construct
Not an OS resource
Cheap to create & block
No counter measures needed
Price to pay: heap memory footprint
Risk of thread pinning
Continuation.yield(scope)
Virtual Thread 1
Carrier Thread a
Carrier Thread b
Carrier Thread c
…
unmount
ForkJoin
Pool
wait
{native code/JNI}
May reference addresses on the stack
(not supported by unmount/mount mechanism)
Images: Flaticon.com/Freepik
Avoid long synchronized blocks/methods
• Replace with ReentrantLock
• Check your dependencies
Make your code Loom-friendly
Thread pool not needed with VT
• Use Semaphore to limit the access of resources
Use Thread Locals with caution
• Virtual Threads support Thread Locals but amplify an existing issue
• Plenty of useless and Local Threads with critical data: transactional context, user id, permissions …
• Thread Locals should be removed!
• To be replaced with Scoped Variables in the mid-term: immutable, lifecycle limited to a task
DZone: Pitfalls to avoid when switching to virtual threads
0 200 400 600 800 1000 1200 1400 1600
PostgreSQL Driver 42.5.4 (non-loom friendly)
PostgreSQL Driver 42.6.0 (loom friendly)
Max. Requests Per Seconds (Virtual Threads)
Monitor
• Memory footprint
• GC activity
Observability & monitoring
-xlog:gc
Pinned Carrier Threads
Java options: -Djdk.tracePinnedThreads=short or -Djdk.tracePinnedThreads=full
Images: Flaticon.com/Freepik
Monitor
• Memory footprint
• GC activity
Observability & monitoring
-xlog:gc
Pinned Carrier Threads
Java options: -Djdk.tracePinnedThreads=short or -Djdk.tracePinnedThreads=full
Pinned thread occurrences
Thread[#50,ForkJoinPool-1-worker-1,5,CarrierThreads]
io.jefrajames.loomdemo.basic.LoomResource.pinnedRequest(LoomResource.java:76) <== monitors:1
Others
• Java Flight Recorder
• jstack / jcmd
Images: Flaticon.com/Freepik
Loom promises
Observability & monitoring
Virtual Threads Scheduling/ForkJoinPool
Java jdk.virtualThreadScheduler options:
• parallelism: number of Carrier Threads, default to CPU processors
• maxPoolSize: the parallelism may be temporarily expanded, at most 256
• minRunnable: number of Carrier Threads not blocked, half the pool size
Payments to grow your world
TechSquad PPT Toolbox
Back to demo!
Round Two
Helidon, Quarkus, Spring
• Inspired from Oracle Helidon demo
• Code on GitHub
• Comparing imperative programming with Virtual & Platform Threads, reactive programming
• Measuring throughput, memory footprint, quick vs slow request collision
• Same benchmark mixing REST API, dependency injection and JPA persistence
• REST/HTTP injector sending 10 threads * 6000 requests in //
• 10 worker threads configured on the server side
Methodology
Helidon 3
Reactive Web Server (Netty)
Plugable ExecutorService
Platform or Virtual Threads
Support reactive programming
Helidon 4
Aka NIMA
Opinionated approach
Natively based on Virtual Threads
Built in tight collaboration with the Java team
Web Server mixing Platform & Virtual Threads
Business Logic run on Virtual Threads
Open-Source from Oracle
Java libs for Cloud Native, Microservices
Focus on simplicity and performance
2 flavors: SE and MP
Native Image support
Helidon Performance
27,000
28,000
29,000
30,000
31,000
32,000
33,000
34,000
35,000
36,000
Helidon 3
Imperative code
Platform Threads
Helidon 3
Imperative code
Virtual Threads
Helidon 3
Reactive code
Nima
Imperative code
Virtual Threads
Throughput (req/sec)
Helidon Memory Footprint
0
50
100
150
200
250
Helidon 3
Imperative code
Platform Threads
Helidon 3
Imperative code
Virtual Threads
Helidon 3
Reactive code
Nima
Imperative code
Virtual Threads
Heap size
Max Heap (MB)
• Basic adoption of VirtualThreads with Helidon 3 Executors.newVirtualThreadPerTaskExecutor()
• Not optimal: more memory, less performance!
• Nima refactored to be nativelly based on Virtual Threads
• Nima Performance and memory footprint comparable to Reactive Programming
• With standard blocking Jakarta EE & MicroProfile code!
Helidon & Virtual Threads
Aka. the "pragmatic"
Open-Source from Red Hat
Java libs for Cloud Native, Microservices
« Supersonic Subatomic Java »
Native Image support
IO Threads
Aka Event Loop
Can’t block
Worker Threads
Can block
Platform
Threads
Default
Virtual
Threads
@RunOnDefaultThread
• Reactive programming with Mutiny delivers the best performance
• Quarkus not (yet) nativelly based on Virtual Threads, based on IO & Worker Threads
• Optionnaly, Worker threads can be Virtual
• Performance & memory footprint comparable with Platform & Virtual Threads
• So … Is it worth using Virtual Threads with Quarkus?
• Let’s see what happens, when calling concurrently quick and slow endpoints
Quarkus and Virtual Threads
Some benchmarks
PostgreSQL
300ms
25ms
System
Under
Test
JVM heap size: 4g
4 vCPU
16GB RAM
Constant load: 500VU
Images: Flaticon.com/Freepik
Code!
@Path("/uuid")
public class UUIDResource {
@Inject
UUIDRepository uuid;
@Inject@RestClient
ThirdPartySleepServiceClient client;
@GET
@Path("/platform")
@Produces(MediaType.APPLICATION_JSON)
public UUID uuidPlatform() {
client.timed(300);
return uuid.generateRandom(0.025);
}
}
@Path("/uuid")
public class UUIDResource {
@Inject
UUIDRepository uuid;
@Inject@RestClient
ThirdPartySleepServiceClient client;
@GET
@Path("/virtual")
@Produces(MediaType.APPLICATION_JSON)
@RunOnVirtualThread
public UUID uuidVirtual() {
client.timed(300);
return uuid.generateRandom(0.025);
}
}
Platform Threads Virtual Threads
Code!
Reactive Model
@Path("/uuid")
public class UUIDResource {
@Inject @RestClient
ThirdPartySleepServiceClient client;
@Inject
UUIDRepository repository;
@GET
@Produces(MediaType.APPLICATION_JSON)
public Uni<UUID> uuid() {
return client.timed(300).onItem().transformToUni(x -> repository.generateRandomUUID(0.025));
}
}
Requests Per Second and Average Response Time per Thread Model
141
283
564
843
1121
1390
1493
1388
1334
3520
1760
885
592
446 360 334 360 374
0
500
1000
1500
2000
2500
3000
3500
4000
0
200
400
600
800
1000
1200
1400
1600
ms
Requests
per
Second
Max RPS Average Response Time
Thread per Request
Process Max Memory Usage (Resident Set Size) per Thread Model
645
325
497
423
0
100
200
300
400
500
600
700
Thread per Request (500 PT) Reactive Virtual Virtual (native)
Memory
Usage
(MB)
Replace Tomcat's Executor
@SpringBootApplication
@EnableFeignClients
public class SpringBoot3WebApplication {
@Bean
AsyncTaskExecutor taskExecutor() {
ExecutorService executorService = Executors.newVirtualThreadPerTaskExecutor();
return new TaskExecutorAdapter(executorService::execute);
}
@Bean
TomcatProtocolHandlerCustomizer<?> protocolHandlerVirtualThreadsCustomizer() {
return protocolHandler ->
protocolHandler.setExecutor(Executors.newVirtualThreadPerTaskExecutor());
}
public static void main(String[] args) {
SpringApplication.run(SpringBoot3WebApplication.class, args);
}
}
Requests Per Second and Average Response Time per Thread Model
141
280
562
837
1109
1369 1390
1338
3500
1760
885
596
450 365 359 373
0
500
1000
1500
2000
2500
3000
3500
4000
0
200
400
600
800
1000
1200
1400
1600
50 PT 100 PT 200 PT 300 PT 400 PT 500 PT Reactive Virtual
Threads
ms
Requests
per
Second
Max RPS Average Response Time
Thread per Request
Process Max Memory Usage (Resident Set Size) per Thread Model
727
609
804
0
100
200
300
400
500
600
700
800
900
Thread per Request
(500)
Reactive Virtual
Memory
Usage
(MB)
1. Understand the principles of Virtual Threads
2. Check how your favorite frameworks and libs have adopted them
3. Do some realistic benchmarks before going to production, monitor in production
4. Bench and monitor in JVM and in GraalVM/Mandrel Native Image mode
5. Things are improving fast: re-bench regularly!
6. Reminder: no benefit for CPU-hungry processing, …
Getting ready for Virtual Threads
Loom promises
The more you block,
The better!
Contact:
JF James & David Pequegnot
Worldline
Thanks for your
attention!
Open Feedback
Contact:
JF James & David Pequegnot
Worldline
Appendix
• Asbtract away the use of Virtual Threads
• Enable to coordinate tasks running on Virtual Threads in the context of a “scope”
• Built-in coordination strategy: any, all
• Extensible coordination strategy
• No back pressure
• Still incubator
TechSquad PPT Toolbox
Structured Concurrency
• Thread locals not designed to be shared by “millions” of threads
• Unclear lifecycle: not always cleaned up
• Uncontrolled mutability: can be changed at any time
• Inheritance: risk of high memory footprint
• Bound to a callable (not a Thread)
TechSquad PPT Toolbox
Scoped Values

More Related Content

What's hot

Jakarta EE 10 - Simplicity for Modern and Lighweight Cloud
Jakarta EE 10 - Simplicity for Modern and Lighweight CloudJakarta EE 10 - Simplicity for Modern and Lighweight Cloud
Jakarta EE 10 - Simplicity for Modern and Lighweight CloudIvar Grimstad
 
Intro to InfluxDB 2.0 and Your First Flux Query by Sonia Gupta
Intro to InfluxDB 2.0 and Your First Flux Query by Sonia GuptaIntro to InfluxDB 2.0 and Your First Flux Query by Sonia Gupta
Intro to InfluxDB 2.0 and Your First Flux Query by Sonia GuptaInfluxData
 
6 Nines: How Stripe keeps Kafka highly-available across the globe with Donny ...
6 Nines: How Stripe keeps Kafka highly-available across the globe with Donny ...6 Nines: How Stripe keeps Kafka highly-available across the globe with Donny ...
6 Nines: How Stripe keeps Kafka highly-available across the globe with Donny ...HostedbyConfluent
 
GraalVM: Run Programs Faster Everywhere
GraalVM: Run Programs Faster EverywhereGraalVM: Run Programs Faster Everywhere
GraalVM: Run Programs Faster EverywhereJ On The Beach
 
Vectorized Query Execution in Apache Spark at Facebook
Vectorized Query Execution in Apache Spark at FacebookVectorized Query Execution in Apache Spark at Facebook
Vectorized Query Execution in Apache Spark at FacebookDatabricks
 
Cloud-Native Integration with Apache Camel on Kubernetes (Copenhagen October ...
Cloud-Native Integration with Apache Camel on Kubernetes (Copenhagen October ...Cloud-Native Integration with Apache Camel on Kubernetes (Copenhagen October ...
Cloud-Native Integration with Apache Camel on Kubernetes (Copenhagen October ...Claus Ibsen
 
Profiling your Java Application
Profiling your Java ApplicationProfiling your Java Application
Profiling your Java ApplicationVictor Rentea
 
Performance Tuning RocksDB for Kafka Streams’ State Stores
Performance Tuning RocksDB for Kafka Streams’ State StoresPerformance Tuning RocksDB for Kafka Streams’ State Stores
Performance Tuning RocksDB for Kafka Streams’ State Storesconfluent
 
2장. Runtime Data Areas
2장. Runtime Data Areas2장. Runtime Data Areas
2장. Runtime Data Areas김 한도
 
Deep Dive Java 17 Devoxx UK
Deep Dive Java 17 Devoxx UKDeep Dive Java 17 Devoxx UK
Deep Dive Java 17 Devoxx UKJosé Paumard
 
VictoriaLogs: Open Source Log Management System - Preview
VictoriaLogs: Open Source Log Management System - PreviewVictoriaLogs: Open Source Log Management System - Preview
VictoriaLogs: Open Source Log Management System - PreviewVictoriaMetrics
 
Kafka’s New Control Plane: The Quorum Controller | Colin McCabe, Confluent
Kafka’s New Control Plane: The Quorum Controller | Colin McCabe, ConfluentKafka’s New Control Plane: The Quorum Controller | Colin McCabe, Confluent
Kafka’s New Control Plane: The Quorum Controller | Colin McCabe, ConfluentHostedbyConfluent
 
Graal in GraalVM - A New JIT Compiler
Graal in GraalVM - A New JIT CompilerGraal in GraalVM - A New JIT Compiler
Graal in GraalVM - A New JIT CompilerKoichi Sakata
 
Producer Performance Tuning for Apache Kafka
Producer Performance Tuning for Apache KafkaProducer Performance Tuning for Apache Kafka
Producer Performance Tuning for Apache KafkaJiangjie Qin
 
Devoxx France 2023 - Les nouveautés de Java 19 et 20
Devoxx France 2023 - Les nouveautés de Java 19 et 20Devoxx France 2023 - Les nouveautés de Java 19 et 20
Devoxx France 2023 - Les nouveautés de Java 19 et 20Jean-Michel Doudoux
 
Kafka: All an engineer needs to know
Kafka: All an engineer needs to knowKafka: All an engineer needs to know
Kafka: All an engineer needs to knowThao Huynh Quang
 

What's hot (20)

Jakarta EE 10 - Simplicity for Modern and Lighweight Cloud
Jakarta EE 10 - Simplicity for Modern and Lighweight CloudJakarta EE 10 - Simplicity for Modern and Lighweight Cloud
Jakarta EE 10 - Simplicity for Modern and Lighweight Cloud
 
Intro to InfluxDB 2.0 and Your First Flux Query by Sonia Gupta
Intro to InfluxDB 2.0 and Your First Flux Query by Sonia GuptaIntro to InfluxDB 2.0 and Your First Flux Query by Sonia Gupta
Intro to InfluxDB 2.0 and Your First Flux Query by Sonia Gupta
 
6 Nines: How Stripe keeps Kafka highly-available across the globe with Donny ...
6 Nines: How Stripe keeps Kafka highly-available across the globe with Donny ...6 Nines: How Stripe keeps Kafka highly-available across the globe with Donny ...
6 Nines: How Stripe keeps Kafka highly-available across the globe with Donny ...
 
GraalVM: Run Programs Faster Everywhere
GraalVM: Run Programs Faster EverywhereGraalVM: Run Programs Faster Everywhere
GraalVM: Run Programs Faster Everywhere
 
Vectorized Query Execution in Apache Spark at Facebook
Vectorized Query Execution in Apache Spark at FacebookVectorized Query Execution in Apache Spark at Facebook
Vectorized Query Execution in Apache Spark at Facebook
 
Cloud-Native Integration with Apache Camel on Kubernetes (Copenhagen October ...
Cloud-Native Integration with Apache Camel on Kubernetes (Copenhagen October ...Cloud-Native Integration with Apache Camel on Kubernetes (Copenhagen October ...
Cloud-Native Integration with Apache Camel on Kubernetes (Copenhagen October ...
 
Profiling your Java Application
Profiling your Java ApplicationProfiling your Java Application
Profiling your Java Application
 
Performance Tuning RocksDB for Kafka Streams’ State Stores
Performance Tuning RocksDB for Kafka Streams’ State StoresPerformance Tuning RocksDB for Kafka Streams’ State Stores
Performance Tuning RocksDB for Kafka Streams’ State Stores
 
Project Reactor By Example
Project Reactor By ExampleProject Reactor By Example
Project Reactor By Example
 
Envoy and Kafka
Envoy and KafkaEnvoy and Kafka
Envoy and Kafka
 
2장. Runtime Data Areas
2장. Runtime Data Areas2장. Runtime Data Areas
2장. Runtime Data Areas
 
Deep Dive Java 17 Devoxx UK
Deep Dive Java 17 Devoxx UKDeep Dive Java 17 Devoxx UK
Deep Dive Java 17 Devoxx UK
 
VictoriaLogs: Open Source Log Management System - Preview
VictoriaLogs: Open Source Log Management System - PreviewVictoriaLogs: Open Source Log Management System - Preview
VictoriaLogs: Open Source Log Management System - Preview
 
De Mensajería hacia Logs con Apache Kafka
De Mensajería hacia Logs con Apache KafkaDe Mensajería hacia Logs con Apache Kafka
De Mensajería hacia Logs con Apache Kafka
 
BWE in Janus
BWE in JanusBWE in Janus
BWE in Janus
 
Kafka’s New Control Plane: The Quorum Controller | Colin McCabe, Confluent
Kafka’s New Control Plane: The Quorum Controller | Colin McCabe, ConfluentKafka’s New Control Plane: The Quorum Controller | Colin McCabe, Confluent
Kafka’s New Control Plane: The Quorum Controller | Colin McCabe, Confluent
 
Graal in GraalVM - A New JIT Compiler
Graal in GraalVM - A New JIT CompilerGraal in GraalVM - A New JIT Compiler
Graal in GraalVM - A New JIT Compiler
 
Producer Performance Tuning for Apache Kafka
Producer Performance Tuning for Apache KafkaProducer Performance Tuning for Apache Kafka
Producer Performance Tuning for Apache Kafka
 
Devoxx France 2023 - Les nouveautés de Java 19 et 20
Devoxx France 2023 - Les nouveautés de Java 19 et 20Devoxx France 2023 - Les nouveautés de Java 19 et 20
Devoxx France 2023 - Les nouveautés de Java 19 et 20
 
Kafka: All an engineer needs to know
Kafka: All an engineer needs to knowKafka: All an engineer needs to know
Kafka: All an engineer needs to know
 

Similar to Loom promises: be there!

Realtime traffic analyser
Realtime traffic analyserRealtime traffic analyser
Realtime traffic analyserAlex Moskvin
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...smallerror
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...xlight
 
Fixing twitter
Fixing twitterFixing twitter
Fixing twitterRoger Xia
 
"Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft...
"Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft..."Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft...
"Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft...Dataconomy Media
 
Writing high performance code in NetCore 3.0
Writing high performance code in NetCore 3.0Writing high performance code in NetCore 3.0
Writing high performance code in NetCore 3.0Javier Cantón Ferrero
 
DotNet 2019 | Javier Cantón - Writing high performance code in NetCore 3.0
DotNet 2019 | Javier Cantón - Writing high performance code in NetCore 3.0DotNet 2019 | Javier Cantón - Writing high performance code in NetCore 3.0
DotNet 2019 | Javier Cantón - Writing high performance code in NetCore 3.0Plain Concepts
 
Chirp 2010: Scaling Twitter
Chirp 2010: Scaling TwitterChirp 2010: Scaling Twitter
Chirp 2010: Scaling TwitterJohn Adams
 
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...Chester Chen
 
Secrets of Performance Tuning Java on Kubernetes
Secrets of Performance Tuning Java on KubernetesSecrets of Performance Tuning Java on Kubernetes
Secrets of Performance Tuning Java on KubernetesBruno Borges
 
Introduction to Micronaut - JBCNConf 2019
Introduction to Micronaut - JBCNConf 2019Introduction to Micronaut - JBCNConf 2019
Introduction to Micronaut - JBCNConf 2019graemerocher
 
Are you ready for cloud-native java JavaCro2019
Are you ready for cloud-native java JavaCro2019Are you ready for cloud-native java JavaCro2019
Are you ready for cloud-native java JavaCro2019Jamie Coleman
 
Celery: The Distributed Task Queue
Celery: The Distributed Task QueueCelery: The Distributed Task Queue
Celery: The Distributed Task QueueRichard Leland
 
"You Don't Know NODE.JS" by Hengki Mardongan Sihombing (Urbanhire)
"You Don't Know NODE.JS" by Hengki Mardongan Sihombing (Urbanhire)"You Don't Know NODE.JS" by Hengki Mardongan Sihombing (Urbanhire)
"You Don't Know NODE.JS" by Hengki Mardongan Sihombing (Urbanhire)Tech in Asia ID
 
Node.js meetup at Palo Alto Networks Tel Aviv
Node.js meetup at Palo Alto Networks Tel AvivNode.js meetup at Palo Alto Networks Tel Aviv
Node.js meetup at Palo Alto Networks Tel AvivRon Perlmuter
 
Machine Learning With H2O vs SparkML
Machine Learning With H2O vs SparkMLMachine Learning With H2O vs SparkML
Machine Learning With H2O vs SparkMLArnab Biswas
 
FreeSWITCH as a Microservice
FreeSWITCH as a MicroserviceFreeSWITCH as a Microservice
FreeSWITCH as a MicroserviceEvan McGee
 
NodeJS ecosystem
NodeJS ecosystemNodeJS ecosystem
NodeJS ecosystemYukti Kaura
 

Similar to Loom promises: be there! (20)

Realtime traffic analyser
Realtime traffic analyserRealtime traffic analyser
Realtime traffic analyser
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
 
Fixing twitter
Fixing twitterFixing twitter
Fixing twitter
 
Fixing_Twitter
Fixing_TwitterFixing_Twitter
Fixing_Twitter
 
"Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft...
"Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft..."Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft...
"Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft...
 
Writing high performance code in NetCore 3.0
Writing high performance code in NetCore 3.0Writing high performance code in NetCore 3.0
Writing high performance code in NetCore 3.0
 
DotNet 2019 | Javier Cantón - Writing high performance code in NetCore 3.0
DotNet 2019 | Javier Cantón - Writing high performance code in NetCore 3.0DotNet 2019 | Javier Cantón - Writing high performance code in NetCore 3.0
DotNet 2019 | Javier Cantón - Writing high performance code in NetCore 3.0
 
Chirp 2010: Scaling Twitter
Chirp 2010: Scaling TwitterChirp 2010: Scaling Twitter
Chirp 2010: Scaling Twitter
 
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
 
Secrets of Performance Tuning Java on Kubernetes
Secrets of Performance Tuning Java on KubernetesSecrets of Performance Tuning Java on Kubernetes
Secrets of Performance Tuning Java on Kubernetes
 
Introduction to Micronaut - JBCNConf 2019
Introduction to Micronaut - JBCNConf 2019Introduction to Micronaut - JBCNConf 2019
Introduction to Micronaut - JBCNConf 2019
 
Are you ready for cloud-native java JavaCro2019
Are you ready for cloud-native java JavaCro2019Are you ready for cloud-native java JavaCro2019
Are you ready for cloud-native java JavaCro2019
 
Celery: The Distributed Task Queue
Celery: The Distributed Task QueueCelery: The Distributed Task Queue
Celery: The Distributed Task Queue
 
"You Don't Know NODE.JS" by Hengki Mardongan Sihombing (Urbanhire)
"You Don't Know NODE.JS" by Hengki Mardongan Sihombing (Urbanhire)"You Don't Know NODE.JS" by Hengki Mardongan Sihombing (Urbanhire)
"You Don't Know NODE.JS" by Hengki Mardongan Sihombing (Urbanhire)
 
20120306 dublin js
20120306 dublin js20120306 dublin js
20120306 dublin js
 
Node.js meetup at Palo Alto Networks Tel Aviv
Node.js meetup at Palo Alto Networks Tel AvivNode.js meetup at Palo Alto Networks Tel Aviv
Node.js meetup at Palo Alto Networks Tel Aviv
 
Machine Learning With H2O vs SparkML
Machine Learning With H2O vs SparkMLMachine Learning With H2O vs SparkML
Machine Learning With H2O vs SparkML
 
FreeSWITCH as a Microservice
FreeSWITCH as a MicroserviceFreeSWITCH as a Microservice
FreeSWITCH as a Microservice
 
NodeJS ecosystem
NodeJS ecosystemNodeJS ecosystem
NodeJS ecosystem
 

More from Jean-Francois James

More from Jean-Francois James (7)

When GenAI meets with Java with Quarkus and langchain4j
When GenAI meets with Java with Quarkus and langchain4jWhen GenAI meets with Java with Quarkus and langchain4j
When GenAI meets with Java with Quarkus and langchain4j
 
LyonJUG-2023-v1.0.pdf
LyonJUG-2023-v1.0.pdfLyonJUG-2023-v1.0.pdf
LyonJUG-2023-v1.0.pdf
 
ParisJUG-2022-v0.4.pdf
ParisJUG-2022-v0.4.pdfParisJUG-2022-v0.4.pdf
ParisJUG-2022-v0.4.pdf
 
Boost your APIs with GraphQL
Boost your APIs with GraphQLBoost your APIs with GraphQL
Boost your APIs with GraphQL
 
Tnt 2020-jf-james
Tnt 2020-jf-jamesTnt 2020-jf-james
Tnt 2020-jf-james
 
Talk Oracle Code One 2019
Talk Oracle Code One 2019Talk Oracle Code One 2019
Talk Oracle Code One 2019
 
Boost your API with GraphQL
Boost your API with GraphQLBoost your API with GraphQL
Boost your API with GraphQL
 

Recently uploaded

『澳洲文凭』买麦考瑞大学毕业证书成绩单办理澳洲Macquarie文凭学位证书
『澳洲文凭』买麦考瑞大学毕业证书成绩单办理澳洲Macquarie文凭学位证书『澳洲文凭』买麦考瑞大学毕业证书成绩单办理澳洲Macquarie文凭学位证书
『澳洲文凭』买麦考瑞大学毕业证书成绩单办理澳洲Macquarie文凭学位证书rnrncn29
 
Novel 3D-Printed Soft Linear and Bending Actuators
Novel 3D-Printed Soft Linear and Bending ActuatorsNovel 3D-Printed Soft Linear and Bending Actuators
Novel 3D-Printed Soft Linear and Bending ActuatorsResearcher Researcher
 
Gravity concentration_MI20612MI_________
Gravity concentration_MI20612MI_________Gravity concentration_MI20612MI_________
Gravity concentration_MI20612MI_________Romil Mishra
 
Mine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxMine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxRomil Mishra
 
Immutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdfImmutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdfDrew Moseley
 
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTIONTHE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTIONjhunlian
 
Engineering Drawing section of solid
Engineering Drawing     section of solidEngineering Drawing     section of solid
Engineering Drawing section of solidnamansinghjarodiya
 
multiple access in wireless communication
multiple access in wireless communicationmultiple access in wireless communication
multiple access in wireless communicationpanditadesh123
 
Virtual memory management in Operating System
Virtual memory management in Operating SystemVirtual memory management in Operating System
Virtual memory management in Operating SystemRashmi Bhat
 
List of Accredited Concrete Batching Plant.pdf
List of Accredited Concrete Batching Plant.pdfList of Accredited Concrete Batching Plant.pdf
List of Accredited Concrete Batching Plant.pdfisabel213075
 
DEVICE DRIVERS AND INTERRUPTS SERVICE MECHANISM.pdf
DEVICE DRIVERS AND INTERRUPTS  SERVICE MECHANISM.pdfDEVICE DRIVERS AND INTERRUPTS  SERVICE MECHANISM.pdf
DEVICE DRIVERS AND INTERRUPTS SERVICE MECHANISM.pdfAkritiPradhan2
 
Comparative study of High-rise Building Using ETABS,SAP200 and SAFE., SAFE an...
Comparative study of High-rise Building Using ETABS,SAP200 and SAFE., SAFE an...Comparative study of High-rise Building Using ETABS,SAP200 and SAFE., SAFE an...
Comparative study of High-rise Building Using ETABS,SAP200 and SAFE., SAFE an...Erbil Polytechnic University
 
Energy Awareness training ppt for manufacturing process.pptx
Energy Awareness training ppt for manufacturing process.pptxEnergy Awareness training ppt for manufacturing process.pptx
Energy Awareness training ppt for manufacturing process.pptxsiddharthjain2303
 
Levelling - Rise and fall - Height of instrument method
Levelling - Rise and fall - Height of instrument methodLevelling - Rise and fall - Height of instrument method
Levelling - Rise and fall - Height of instrument methodManicka Mamallan Andavar
 
CS 3251 Programming in c all unit notes pdf
CS 3251 Programming in c all unit notes pdfCS 3251 Programming in c all unit notes pdf
CS 3251 Programming in c all unit notes pdfBalamuruganV28
 
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTESCME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTESkarthi keyan
 
Earthing details of Electrical Substation
Earthing details of Electrical SubstationEarthing details of Electrical Substation
Earthing details of Electrical Substationstephanwindworld
 
ROBOETHICS-CCS345 ETHICS AND ARTIFICIAL INTELLIGENCE.ppt
ROBOETHICS-CCS345 ETHICS AND ARTIFICIAL INTELLIGENCE.pptROBOETHICS-CCS345 ETHICS AND ARTIFICIAL INTELLIGENCE.ppt
ROBOETHICS-CCS345 ETHICS AND ARTIFICIAL INTELLIGENCE.pptJohnWilliam111370
 
Comprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdfComprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdfalene1
 
Ch10-Global Supply Chain - Cadena de Suministro.pdf
Ch10-Global Supply Chain - Cadena de Suministro.pdfCh10-Global Supply Chain - Cadena de Suministro.pdf
Ch10-Global Supply Chain - Cadena de Suministro.pdfChristianCDAM
 

Recently uploaded (20)

『澳洲文凭』买麦考瑞大学毕业证书成绩单办理澳洲Macquarie文凭学位证书
『澳洲文凭』买麦考瑞大学毕业证书成绩单办理澳洲Macquarie文凭学位证书『澳洲文凭』买麦考瑞大学毕业证书成绩单办理澳洲Macquarie文凭学位证书
『澳洲文凭』买麦考瑞大学毕业证书成绩单办理澳洲Macquarie文凭学位证书
 
Novel 3D-Printed Soft Linear and Bending Actuators
Novel 3D-Printed Soft Linear and Bending ActuatorsNovel 3D-Printed Soft Linear and Bending Actuators
Novel 3D-Printed Soft Linear and Bending Actuators
 
Gravity concentration_MI20612MI_________
Gravity concentration_MI20612MI_________Gravity concentration_MI20612MI_________
Gravity concentration_MI20612MI_________
 
Mine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxMine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptx
 
Immutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdfImmutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdf
 
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTIONTHE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
 
Engineering Drawing section of solid
Engineering Drawing     section of solidEngineering Drawing     section of solid
Engineering Drawing section of solid
 
multiple access in wireless communication
multiple access in wireless communicationmultiple access in wireless communication
multiple access in wireless communication
 
Virtual memory management in Operating System
Virtual memory management in Operating SystemVirtual memory management in Operating System
Virtual memory management in Operating System
 
List of Accredited Concrete Batching Plant.pdf
List of Accredited Concrete Batching Plant.pdfList of Accredited Concrete Batching Plant.pdf
List of Accredited Concrete Batching Plant.pdf
 
DEVICE DRIVERS AND INTERRUPTS SERVICE MECHANISM.pdf
DEVICE DRIVERS AND INTERRUPTS  SERVICE MECHANISM.pdfDEVICE DRIVERS AND INTERRUPTS  SERVICE MECHANISM.pdf
DEVICE DRIVERS AND INTERRUPTS SERVICE MECHANISM.pdf
 
Comparative study of High-rise Building Using ETABS,SAP200 and SAFE., SAFE an...
Comparative study of High-rise Building Using ETABS,SAP200 and SAFE., SAFE an...Comparative study of High-rise Building Using ETABS,SAP200 and SAFE., SAFE an...
Comparative study of High-rise Building Using ETABS,SAP200 and SAFE., SAFE an...
 
Energy Awareness training ppt for manufacturing process.pptx
Energy Awareness training ppt for manufacturing process.pptxEnergy Awareness training ppt for manufacturing process.pptx
Energy Awareness training ppt for manufacturing process.pptx
 
Levelling - Rise and fall - Height of instrument method
Levelling - Rise and fall - Height of instrument methodLevelling - Rise and fall - Height of instrument method
Levelling - Rise and fall - Height of instrument method
 
CS 3251 Programming in c all unit notes pdf
CS 3251 Programming in c all unit notes pdfCS 3251 Programming in c all unit notes pdf
CS 3251 Programming in c all unit notes pdf
 
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTESCME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
 
Earthing details of Electrical Substation
Earthing details of Electrical SubstationEarthing details of Electrical Substation
Earthing details of Electrical Substation
 
ROBOETHICS-CCS345 ETHICS AND ARTIFICIAL INTELLIGENCE.ppt
ROBOETHICS-CCS345 ETHICS AND ARTIFICIAL INTELLIGENCE.pptROBOETHICS-CCS345 ETHICS AND ARTIFICIAL INTELLIGENCE.ppt
ROBOETHICS-CCS345 ETHICS AND ARTIFICIAL INTELLIGENCE.ppt
 
Comprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdfComprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdf
 
Ch10-Global Supply Chain - Cadena de Suministro.pdf
Ch10-Global Supply Chain - Cadena de Suministro.pdfCh10-Global Supply Chain - Cadena de Suministro.pdf
Ch10-Global Supply Chain - Cadena de Suministro.pdf
 

Loom promises: be there!

  • 1. Payments to grow your world TechSquad PPT Toolbox Loom promises be there!
  • 2. Get in touch: @jefrajames jean-francois.james@worldline.com linkedin.com/ln/jefrajames Jean-François JAMES Software Architect • Focus on Java, Jakarta EE, MicroProfile • Open Source contributor • Head of DevRel
  • 3. Get in touch: @foxlegend david.pequegnot@worldline.com linkedin.com/in/davidpequegnot/ David PEQUEGNOT Performance Engineer • Focus on JVM technologies • ❤ Observability
  • 4. We design payments technology that powers the growth of millions of businesses around the world. Who are we? 7000+ engineers in over 40 countries Managing 28+ billion transactions per year €250M spent in R&D every year Handling 150+ payment methods
  • 5. How your application server handles requests? So, what’s the problem with our Java apps? Application Server Request 1 Request 2 Request n Thread 1 Thread 2 Thread n … … Thread pool Images: Flaticon.com/Freepik
  • 6. Blocking, waiting, blocking, waiting, blocking, waiting … So, what’s the problem with our Java apps? Wait for WebService response Wait for database Image: Flaticon.com/Freepik
  • 7. May Reactive Be With You Reactive vs Imperative Request 1 Request 2 Request 3 Request 4 Request 5 … Event Loop Image: Flaticon.com/Freepik Imperative Reactive
  • 8. Pro. & Cons. Reactive vs Imperative Request 1 Request 2 Request 3 Request 4 Request 5 … Event Loop Image: Flaticon.com/Freepik
  • 9. Code imperative, work reactive! Block for free! Loom promises
  • 10. One project, 3 JEPs Virtual Threads Lightweight threads JEP 444 Stable with Java 21 LTS Structured Concurrency Facilitate tasks dev& run on top of Virtual Threads JEP 453 Preview with Java 21 LTS Scoped Values Modernization, optimization of Thread Locals JEP 446 Preview with Java 21 LTS
  • 11. Evolution of Java Concurrency 1997 Java 1.1 Green Threads 2004 Java 1.5 Executor 2014 Java 8 CompletableFuture Parallel Streams 1998 Java 1.2 Native Threads 2011 Java 1.7 ForkJoinPool 2023 Java 21 Virtual Threads
  • 12. From a few Platform Threads … Platform Threads OS Threads OS Scheduling Virtual Threads OS JVM Kernel mode User mode
  • 13. … to millions Virtual Threads Platform (Carrier) Threads OS Threads OS Scheduling JVM Scheduling (ForkJoinPool) Virtual Threads OS JVM
  • 14. Inside the JVM: the magic of Continuation Heap Continuation.yield(scope) continuation.run() Virtual Thread 1 Carrier Thread a Carrier Thread b Carrier Thread c … unmount mount ForkJoin Pool wait
  • 15. Many changes in the JVM No impact on user code (vs Kotlin’s coroutines) Heap and GC activity to be further monitored and tuned Not suitable for CPU-hungry processing Loom promises Keypoints
  • 16. Payments to grow your world TechSquad PPT Toolbox Demo time! Round 1 Java SE
  • 17. • Basic Java SE tests • Inspired from Eclipse Jetty loom-trial (Webtide) • Code on GitHub • Measured on MacOS (ARM M1, 8 CPU, 16 G RAM) with Java 20 (Temurin distro) Methodology
  • 19. Cheap to create? 0 50 100 150 200 250 1 2 4 8 16 32 32 64 vthread started/ms
  • 20. Really cheap to create? 0 1 2 3 4 5 6 7 8 9 1 2 4 8 16 32 32 64 Carrier Threads 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 1 2 4 8 16 32 32 64 Heap size (MB)
  • 21. • Up to 64 millions Virtual Threads in a single JVM! • Just 8 Carrier Threads used in the background • Same test with Platform Threads: 4K+ on MacOS and 30k+ on Linux • About 200 Virtual Threads created/msec, 1 MB Heap/thread • No hard limit with Virtual Threads: the limit is the Garbage Collector efficiency! How long to start Virtual Threads?
  • 22. One million Threads running a recursive genetic mutation algorithm: is it possible? TechSquad PPT Toolbox DeepStack Test conditions: • Stack depth: 1000 • Timeout if Thread creation > 3s
  • 23. Platform Threads TechSquad PPT Toolbox DeepStack
  • 25. First results How to increase the number of Virtual Threads? … DeepStack Platform Threads Virtual Threads 4k+ 30k+ 30k+ Images: Flaticon.com/Freepik
  • 26. Threads over Heap Size TechSquad PPT Toolbox DeepStack 0 10000 20000 30000 40000 50000 60000 70000 1GB 2GB 4GB 8GB Max Threads JVM Heap Size Platform Virtual
  • 27. Platform and Virtual Threads: Take aways Platform Threads Java’s traditional unit of concurrency Trivial wrappers around OS Threads Scarce resources Expensive to create & block Counter measures: thread pools, reactive programming, cooperative scheduling (async/await construct) Virtual Threads A new lightweight construct Not an OS resource Cheap to create & block No counter measures needed Price to pay: heap memory footprint
  • 28. Risk of thread pinning Continuation.yield(scope) Virtual Thread 1 Carrier Thread a Carrier Thread b Carrier Thread c … unmount ForkJoin Pool wait {native code/JNI} May reference addresses on the stack (not supported by unmount/mount mechanism) Images: Flaticon.com/Freepik
  • 29. Avoid long synchronized blocks/methods • Replace with ReentrantLock • Check your dependencies Make your code Loom-friendly Thread pool not needed with VT • Use Semaphore to limit the access of resources Use Thread Locals with caution • Virtual Threads support Thread Locals but amplify an existing issue • Plenty of useless and Local Threads with critical data: transactional context, user id, permissions … • Thread Locals should be removed! • To be replaced with Scoped Variables in the mid-term: immutable, lifecycle limited to a task DZone: Pitfalls to avoid when switching to virtual threads 0 200 400 600 800 1000 1200 1400 1600 PostgreSQL Driver 42.5.4 (non-loom friendly) PostgreSQL Driver 42.6.0 (loom friendly) Max. Requests Per Seconds (Virtual Threads)
  • 30. Monitor • Memory footprint • GC activity Observability & monitoring -xlog:gc Pinned Carrier Threads Java options: -Djdk.tracePinnedThreads=short or -Djdk.tracePinnedThreads=full Images: Flaticon.com/Freepik
  • 31.
  • 32. Monitor • Memory footprint • GC activity Observability & monitoring -xlog:gc Pinned Carrier Threads Java options: -Djdk.tracePinnedThreads=short or -Djdk.tracePinnedThreads=full Pinned thread occurrences Thread[#50,ForkJoinPool-1-worker-1,5,CarrierThreads] io.jefrajames.loomdemo.basic.LoomResource.pinnedRequest(LoomResource.java:76) <== monitors:1 Others • Java Flight Recorder • jstack / jcmd Images: Flaticon.com/Freepik
  • 34. Observability & monitoring Virtual Threads Scheduling/ForkJoinPool Java jdk.virtualThreadScheduler options: • parallelism: number of Carrier Threads, default to CPU processors • maxPoolSize: the parallelism may be temporarily expanded, at most 256 • minRunnable: number of Carrier Threads not blocked, half the pool size
  • 35. Payments to grow your world TechSquad PPT Toolbox Back to demo! Round Two Helidon, Quarkus, Spring
  • 36. • Inspired from Oracle Helidon demo • Code on GitHub • Comparing imperative programming with Virtual & Platform Threads, reactive programming • Measuring throughput, memory footprint, quick vs slow request collision • Same benchmark mixing REST API, dependency injection and JPA persistence • REST/HTTP injector sending 10 threads * 6000 requests in // • 10 worker threads configured on the server side Methodology
  • 37. Helidon 3 Reactive Web Server (Netty) Plugable ExecutorService Platform or Virtual Threads Support reactive programming Helidon 4 Aka NIMA Opinionated approach Natively based on Virtual Threads Built in tight collaboration with the Java team Web Server mixing Platform & Virtual Threads Business Logic run on Virtual Threads Open-Source from Oracle Java libs for Cloud Native, Microservices Focus on simplicity and performance 2 flavors: SE and MP Native Image support
  • 38. Helidon Performance 27,000 28,000 29,000 30,000 31,000 32,000 33,000 34,000 35,000 36,000 Helidon 3 Imperative code Platform Threads Helidon 3 Imperative code Virtual Threads Helidon 3 Reactive code Nima Imperative code Virtual Threads Throughput (req/sec)
  • 39. Helidon Memory Footprint 0 50 100 150 200 250 Helidon 3 Imperative code Platform Threads Helidon 3 Imperative code Virtual Threads Helidon 3 Reactive code Nima Imperative code Virtual Threads Heap size Max Heap (MB)
  • 40. • Basic adoption of VirtualThreads with Helidon 3 Executors.newVirtualThreadPerTaskExecutor() • Not optimal: more memory, less performance! • Nima refactored to be nativelly based on Virtual Threads • Nima Performance and memory footprint comparable to Reactive Programming • With standard blocking Jakarta EE & MicroProfile code! Helidon & Virtual Threads
  • 41. Aka. the "pragmatic" Open-Source from Red Hat Java libs for Cloud Native, Microservices « Supersonic Subatomic Java » Native Image support IO Threads Aka Event Loop Can’t block Worker Threads Can block Platform Threads Default Virtual Threads @RunOnDefaultThread
  • 42. • Reactive programming with Mutiny delivers the best performance • Quarkus not (yet) nativelly based on Virtual Threads, based on IO & Worker Threads • Optionnaly, Worker threads can be Virtual • Performance & memory footprint comparable with Platform & Virtual Threads • So … Is it worth using Virtual Threads with Quarkus? • Let’s see what happens, when calling concurrently quick and slow endpoints Quarkus and Virtual Threads
  • 43. Some benchmarks PostgreSQL 300ms 25ms System Under Test JVM heap size: 4g 4 vCPU 16GB RAM Constant load: 500VU Images: Flaticon.com/Freepik
  • 44. Code! @Path("/uuid") public class UUIDResource { @Inject UUIDRepository uuid; @Inject@RestClient ThirdPartySleepServiceClient client; @GET @Path("/platform") @Produces(MediaType.APPLICATION_JSON) public UUID uuidPlatform() { client.timed(300); return uuid.generateRandom(0.025); } } @Path("/uuid") public class UUIDResource { @Inject UUIDRepository uuid; @Inject@RestClient ThirdPartySleepServiceClient client; @GET @Path("/virtual") @Produces(MediaType.APPLICATION_JSON) @RunOnVirtualThread public UUID uuidVirtual() { client.timed(300); return uuid.generateRandom(0.025); } } Platform Threads Virtual Threads
  • 45. Code! Reactive Model @Path("/uuid") public class UUIDResource { @Inject @RestClient ThirdPartySleepServiceClient client; @Inject UUIDRepository repository; @GET @Produces(MediaType.APPLICATION_JSON) public Uni<UUID> uuid() { return client.timed(300).onItem().transformToUni(x -> repository.generateRandomUUID(0.025)); } }
  • 46. Requests Per Second and Average Response Time per Thread Model 141 283 564 843 1121 1390 1493 1388 1334 3520 1760 885 592 446 360 334 360 374 0 500 1000 1500 2000 2500 3000 3500 4000 0 200 400 600 800 1000 1200 1400 1600 ms Requests per Second Max RPS Average Response Time Thread per Request
  • 47. Process Max Memory Usage (Resident Set Size) per Thread Model 645 325 497 423 0 100 200 300 400 500 600 700 Thread per Request (500 PT) Reactive Virtual Virtual (native) Memory Usage (MB)
  • 48. Replace Tomcat's Executor @SpringBootApplication @EnableFeignClients public class SpringBoot3WebApplication { @Bean AsyncTaskExecutor taskExecutor() { ExecutorService executorService = Executors.newVirtualThreadPerTaskExecutor(); return new TaskExecutorAdapter(executorService::execute); } @Bean TomcatProtocolHandlerCustomizer<?> protocolHandlerVirtualThreadsCustomizer() { return protocolHandler -> protocolHandler.setExecutor(Executors.newVirtualThreadPerTaskExecutor()); } public static void main(String[] args) { SpringApplication.run(SpringBoot3WebApplication.class, args); } }
  • 49. Requests Per Second and Average Response Time per Thread Model 141 280 562 837 1109 1369 1390 1338 3500 1760 885 596 450 365 359 373 0 500 1000 1500 2000 2500 3000 3500 4000 0 200 400 600 800 1000 1200 1400 1600 50 PT 100 PT 200 PT 300 PT 400 PT 500 PT Reactive Virtual Threads ms Requests per Second Max RPS Average Response Time Thread per Request
  • 50. Process Max Memory Usage (Resident Set Size) per Thread Model 727 609 804 0 100 200 300 400 500 600 700 800 900 Thread per Request (500) Reactive Virtual Memory Usage (MB)
  • 51. 1. Understand the principles of Virtual Threads 2. Check how your favorite frameworks and libs have adopted them 3. Do some realistic benchmarks before going to production, monitor in production 4. Bench and monitor in JVM and in GraalVM/Mandrel Native Image mode 5. Things are improving fast: re-bench regularly! 6. Reminder: no benefit for CPU-hungry processing, … Getting ready for Virtual Threads
  • 52. Loom promises The more you block, The better!
  • 53. Contact: JF James & David Pequegnot Worldline Thanks for your attention! Open Feedback
  • 54. Contact: JF James & David Pequegnot Worldline Appendix
  • 55. • Asbtract away the use of Virtual Threads • Enable to coordinate tasks running on Virtual Threads in the context of a “scope” • Built-in coordination strategy: any, all • Extensible coordination strategy • No back pressure • Still incubator TechSquad PPT Toolbox Structured Concurrency
  • 56. • Thread locals not designed to be shared by “millions” of threads • Unclear lifecycle: not always cleaned up • Uncontrolled mutability: can be changed at any time • Inheritance: risk of high memory footprint • Bound to a callable (not a Thread) TechSquad PPT Toolbox Scoped Values

Editor's Notes

  1. JF
  2. Diapo : changer par gros chiffre (30K+ et 60k+) et listing plus visual (icons & co)
  3. Cpu-bound -> virtual threads moins efficaces
  4. TODO: Example with pinned carrier thread ? Jcmd to dump virtual threads ?
  5. JF PodCast Thomas Langer : Episode 29 “helidon níma & virtual threads” – Inside.java
  6. TODO: Animate + 600PT + native benchmark
  7. 3 images représentant