SlideShare a Scribd company logo
1 of 62
Download to read offline
1
JVM JIT-compiler overview
Vladimir Ivanov
HotSpot JVM Compiler
Oracle Corp.
2
Agenda
§  about compilers in general
–  … and JIT-compilers in particular
§  about JIT-compilers in HotSpot JVM
§  monitoring JIT-compilers in HotSpot JVM
3
Static vs Dynamic
AOT vs JIT
4
Dynamic and Static Compilation Differences
§  Static compilation
–  “ahead-of-time”(AOT) compilation
–  Source code → Native executable
–  Most of compilation work happens before executing
5
Dynamic and Static Compilation Differences
§  Static compilation
–  “ahead-of-time”(AOT) compilation
–  Source code → Native executable
–  Most of compilation work happens before executing
§  Modern Java VMs use dynamic compilers (JIT)
–  “just-in-time” (JIT) compilation
–  Source code → Bytecode → Interpreter + JITted executable
–  Most of compilation work happens during application execution
6
Dynamic and Static Compilation Differences
§  Static compilation (AOT)
–  can utilize complex and heavy analyses and optimizations
7
Dynamic and Static Compilation Differences
§  Static compilation (AOT)
–  can utilize complex and heavy analyses and optimizations
§  … but static information sometimes isn’t enough
§  … and it’s hard to rely on profiling info, if any
8
Dynamic and Static Compilation Differences
§  Static compilation (AOT)
–  can utilize complex and heavy analyses and optimizations
§  … but static information sometimes isn’t enough
§  … and it’s hard to rely on profiling info, if any
–  moreover, how to utilize specific platform features?
§  like SSE4.2 / AVX / AVX 2, TSX, AES-NI, RdRand
9
Dynamic and Static Compilation Differences
§  Modern Java VMs use dynamic compilers (JIT)
–  aggressive optimistic optimizations
§  through extensive usage of profiling info
10
Dynamic and Static Compilation Differences
§  Modern Java VMs use dynamic compilers (JIT)
–  aggressive optimistic optimizations
§  through extensive usage of profiling info
§  … but budget is limited and shared with an application
11
Dynamic and Static Compilation Differences
§  Modern Java VMs use dynamic compilers (JIT)
–  aggressive optimistic optimizations
§  through extensive usage of profiling info
§  … but budget is limited and shared with an application
–  thus:
§  startup speed suffers
§  peak performance may suffer as well (but not necessarily)
12
Profiling
§  Gathers data about code during execution
–  invariants
§  types, constants (e.g. null pointers)
–  statistics
§  branches, calls
§  Gathered data is used during optimization
–  Educated guess
–  Guess can be wrong
13
Optimistic Compilers
§  Assume profile is accurate
–  Aggressively optimize based on profile
–  Bail out if they’re wrong
§  ...and hope that they’re usually right
14
Profile-guided optimizations (PGO)
§  Use profile for more efficient optimization
§  PGO in JVMs
–  Always have it, turned on by default
–  Developers (usually) not interested or concerned about it
–  Profile is always consistent to execution scenario
15
Dynamic Compilation
in JVM
16
Dynamic Compilation (JIT)
§  Can do non-conservative optimizations in dynamic
§  Separates optimization from product delivery cycle
–  Update JVM, run the same application, realize improved performance!
–  Can be "tuned" to the target platform
17
Dynamic Compilation (JIT)
§  Knows about
–  loaded classes, methods the program has executed
§  Makes optimization decisions based on code paths executed
–  Code generation depends on what is observed:
§  loaded classes, code paths executed, branches taken
§  May re-optimize if assumption was wrong, or alternative code paths
taken
–  Instruction path length may change between invocations of methods as a
result of de-optimization / re-compilation
18
JVM
§  Runtime
–  class loading, bytecode verification, synchronization
§  JIT
–  profiling, compilation plans, OSR
–  aggressive optimizations
§  GC
–  different algorithms: throughput vs. response time
19
JVM: Makes Bytecodes Fast
§  JVMs eventually JIT bytecodes
–  To make them fast
–  compiled when needed
§  Maybe immediately before execution
§  ...or when we decide it’s important
§  ...or never?
–  Some JITs are high quality optimizing compilers
20
JVM: Makes Bytecodes Fast
§  JVMs eventually JIT bytecodes
§  But cannot use existing static compilers directly
–  different cost model
§  time & resource constraints (CPU, memory)
–  tracking OOPs (ptrs) for GC
–  Java Memory Model (volatile reordering & fences)
–  New code patterns to optimize
21
JVM: Makes Bytecodes Fast
§  JIT'ing requires Profiling
–  Because you don't want to JIT everything
§  Profiling allows focused code-gen
§  Profiling allows better code-gen
–  Inline what’s hot
–  Loop unrolling, range-check elimination, etc
–  Branch prediction, spill-code-gen, scheduling
22
Dynamic Compilation (JIT)
§  Is dynamic compilation overhead essential?
–  The longer your application runs, the less the overhead
§  Trading off compilation time, not application time
–  Steal some cycles very early in execution
–  Done automagically and transparently to application
§  Most of “perceived” overhead is compiler waiting for more data
–  ...thus running semi-optimal code for time being
Overhead
23
JVM
Author: Aleksey Shipilev
24
Mixed-Mode Execution
§  Interpreted
–  Bytecode-walking
–  Artificial stack machine
§  Compiled
–  Direct native operations
–  Native register machine
25
Bytecode Execution
1 2
34
Interpretation Profiling
Dynamic
Compilation
Deoptimization
26
Deoptimization
§  Bail out of running native code
–  stop executing native (JIT-generated) code
–  start interpreting bytecode
§  It’s a complicated operation at runtime…
27
OSR: On-Stack Replacement
§  Running method never exits?
§  But it’s getting really hot?
§  Generally means loops, back-branching
§  Compile and replace while running
§  Not typically useful in large systems
§  Looks great on benchmarks!
28
Optimizations
29
Optimizations in HotSpot JVM
§  compiler tactics
delayed compilation
tiered compilation
on-stack replacement
delayed reoptimization
program dependence graph rep.
static single assignment rep.
§  proof-based techniques
exact type inference
memory value inference
memory value tracking
constant folding
reassociation
operator strength reduction
null check elimination
type test strength reduction
type test elimination
algebraic simplification
common subexpression elimination
integer range typing
§  flow-sensitive rewrites
conditional constant propagation
dominating test detection
flow-carried type narrowing
dead code elimination
§  language-specific techniques
class hierarchy analysis
devirtualization
symbolic constant propagation
autobox elimination
escape analysis
lock elision
lock fusion
de-reflection
§  speculative (profile-based) techniques
optimistic nullness assertions
optimistic type assertions
optimistic type strengthening
optimistic array length strengthening
untaken branch pruning
optimistic N-morphic inlining
branch frequency prediction
call frequency prediction
§  memory and placement transformation
expression hoisting
expression sinking
redundant store elimination
adjacent store fusion
card-mark elimination
merge-point splitting
§  loop transformations
loop unrolling
loop peeling
safepoint elimination
iteration range splitting
range check elimination
loop vectorization
§  global code shaping
inlining (graph integration)
global code motion
heat-based code layout
switch balancing
throw inlining
§  control flow graph transformation
local code scheduling
local code bundling
delay slot filling
graph-coloring register allocation
linear scan register allocation
live range splitting
copy coalescing
constant splitting
copy removal
address mode matching
instruction peepholing
DFA-based code generator
30
JVM: Makes Virtual Calls Fast
§  C++ avoids virtual calls – because they are slow
§  Java embraces them – and makes them fast
–  Well, mostly fast – JIT's do Class Hierarchy Analysis (CHA)
–  CHA turns most virtual calls into static calls
–  JVM detects new classes loaded, adjusts CHA
§  May need to re-JIT
–  When CHA fails to make the call static, inline caches
–  When IC's fail, virtual calls are back to being slow
31
Inlining
§  Combine caller and callee into one unit
–  e.g. based on profile
–  … or prove smth using CHA (Class Hierarchy Analysis)
–  Perhaps with a guard/test
§  Optimize as a whole
–  More code means better visibility
32
Inlining
Before
33
Inlining
After
34
Inlining and devirtualization
§  Inlining is the most profitable compiler optimization
–  Rather straightforward to implement
–  Huge benefits: expands the scope for other optimizations
§  OOP needs polymorphism, that implies virtual calls
–  Prevents naïve inlining
–  Devirtualization is required
–  (This does not mean you should not write OOP code)
35
Call Site
§  The place where you make a call
§  Monomorphic (“one shape”)
–  Single target class
§  Bimorphic (“two shapes”)
§  Polymorphic (“many shapes”)
§  Megamorphic
36
JVM Devirtualization
§  Analyzes hierarchy of currently loaded classes
§  Efficiently devirtualizes all monomorphic calls
§  Able to devirtualize polymorphic calls
§  JVM may inline dynamic methods
–  Reflection calls
–  Runtime-synthesized methods
–  JSR 292
37
Feedback multiplies optimizations
§  Profiling and CHA produces information
–  ...which lets the JIT ignore unused paths
–  ...and helps the JIT sharpen types on hot paths
–  ...which allows calls to be devirtualized
–  ...allowing them to be inlined
–  ...expanding an ever-widening optimization horizon
§  Result:
Large native methods containing tightly optimized machine code for
hundreds of inlined calls!
38
HotSpot JVM
39
Existing JVMs
§  Oracle HotSpot
§  Oracle JRockit
§  IBM J9
§  Excelsior JET
§  Azul Zing
§  SAPJVM
§  …
40
HotSpot JVM
§  client / C1
§  server / C2
§  tiered mode (C1 + C2)
JIT-compilers
41
HotSpot JVM
§  client / C1
–  $ java –client
§  only available in 32-bit VM
–  fast code generation of acceptable quality
–  basic optimizations
–  doesn’t need profile
–  compilation threshold: 1,5k invocations
JIT-compilers
42
HotSpot JVM
§  server / C2
–  $ java –server
–  highly optimized code for speed
–  many aggressive optimizations which rely on profile
–  compilation threshold: 10k invocations
JIT-compilers
43
HotSpot JVM
§  Client / C1
+ fast startup
–  peak performance suffers
§  Server / C2
+ very good code for hot methods
–  slow startup / warmup
JIT-compilers comparison
44
Tiered compilation
§  -XX:+TieredCompilation
§  Multiple tiers of interpretation, C1, and C2
§  Level0=Interpreter
§  Level1-3=C1
–  #1: C1 w/o profiling
–  #2: C1 w/ basic profiling
–  #3: C1 w/ full profiling
§  Level4=C2
C1 + C2
45
Monitoring JIT
46
Monitoring JIT-Compiler
§  how to print info about compiled methods?
–  -XX:+PrintCompilation
§  how to print info about inlining decisions
–  -XX:+PrintInlining
§  how to control compilation policy?
–  -XX:CompileCommand=…
§  how to print assembly code?
–  -XX:+PrintAssembly
–  -XX:+PrintOptoAssembly (C2-only)
47
Print Compilation
§  -XX:+PrintCompilation
§  Print methods as they are JIT-compiled
§  Class + name + size
48
Print Compilation
$ java -XX:+PrintCompilation
988 1 java.lang.String::hashCode (55 bytes)
1271 2 sun.nio.cs.UTF_8$Encoder::encode (361 bytes)
1406 3 java.lang.String::charAt (29 bytes)
Sample output
49
Print Compilation
§  2043 470 % ! jdk.nashorn.internal.ir.FunctionNode::accept @ 136 (265 bytes)
% == OSR compilation
! == has exception handles (may be expensive)
s == synchronized method
§  2028 466 n java.lang.Class::isArray (native)
n == native method
Other useful info
50
Print Compilation
§  621 160 java.lang.Object::equals (11 bytes) made not entrant
–  don‘t allow any new calls into this compiled version
§  1807 160 java.lang.Object::equals (11 bytes) made zombie
–  can safely throw away compiled version
Not just compilation notifications
51
No JIT At All?
§  Code is too large
§  Code isn’t too «hot»
–  executed not too often
52
Print Inlining
§  -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining
§  Shows hierarchy of inlined methods
§  Prints reason, if a method isn’t inlined
53
Print Inlining
$ java -XX:+PrintCompilation -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining
75 1 java.lang.String::hashCode (55 bytes)
88 2 sun.nio.cs.UTF_8$Encoder::encode (361 bytes)
@ 14 java.lang.Math::min (11 bytes) (intrinsic)
@ 139 java.lang.Character::isSurrogate (18 bytes) never executed
103 3 java.lang.String::charAt (29 bytes)
54
Print Inlining
$ java -XX:+PrintCompilation -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining
75 1 java.lang.String::hashCode (55 bytes)
88 2 sun.nio.cs.UTF_8$Encoder::encode (361 bytes)
@ 14 java.lang.Math::min (11 bytes) (intrinsic)
@ 139 java.lang.Character::isSurrogate (18 bytes) never executed
103 3 java.lang.String::charAt (29 bytes)
55
Intrinsic
§  Known to the JIT compiler
–  method bytecode is ignored
–  inserts “best” native code
§  e.g. optimized sqrt in machine code
§  Existing intrinsics
–  String::equals, Math::*, System::arraycopy, Object::hashCode,
Object::getClass, sun.misc.Unsafe::*
56
Inlining Tuning
§  -XX:MaxInlineSize=35
–  Largest inlinable method (bytecode)
§  -XX:InlineSmallCode=#
–  Largest inlinable compiled method
§  -XX:FreqInlineSize=#
–  Largest frequently-called method…
§  -XX:MaxInlineLevel=9
–  How deep does the rabbit hole go?
§  -XX:MaxRecursiveInlineLevel=#
–  recursive inlining
57
Machine Code
§  -XX:+PrintAssembly
§  http://wikis.sun.com/display/HotSpotInternals/PrintAssembly
§  Knowing code compiles is good
§  Knowing code inlines is better
§  Seeing the actual assembly is best!
58
-XX:CompileCommand=
§  Syntax
–  “[command] [method] [signature]”
§  Supported commands
–  exclude – never compile
–  inline – always inline
–  dontinline – never inline
§  Method reference
–  class.name::methodName
§  Method signature is optional
59
What Have We Learned?
§  How JIT compilers work
§  How HotSpot’s JIT works
§  How to monitor the JIT in HotSpot
60
Future work
§  Da Vinci Machine Project (MLVM)
–  project page: http://openjdk.java.net/projects/mlvm/
–  repository: http://hg.openjdk.java.net/mlvm/mlvm
§  JSR292 matured there
§  Some other projects:
–  value objects
–  coroutines / continuations
–  tail calls (hard / guaranteed)
Where does innovation take place?
61
Questions?
vladimir.x.ivanov@oracle.com
@iwanowww
62
Graphic Section Divider

More Related Content

What's hot

What's hot (20)

CMake - Introduction and best practices
CMake - Introduction and best practicesCMake - Introduction and best practices
CMake - Introduction and best practices
 
Rust: Systems Programming for Everyone
Rust: Systems Programming for EveryoneRust: Systems Programming for Everyone
Rust: Systems Programming for Everyone
 
Type Casting Operator
Type Casting OperatorType Casting Operator
Type Casting Operator
 
Java static keyword
Java static keywordJava static keyword
Java static keyword
 
JVM Under The Hood WDI.pdf
JVM Under The Hood WDI.pdfJVM Under The Hood WDI.pdf
JVM Under The Hood WDI.pdf
 
Native Java with GraalVM
Native Java with GraalVMNative Java with GraalVM
Native Java with GraalVM
 
Memory model
Memory modelMemory model
Memory model
 
GDB Rocks!
GDB Rocks!GDB Rocks!
GDB Rocks!
 
LLVM Backend Porting
LLVM Backend PortingLLVM Backend Porting
LLVM Backend Porting
 
55 New Features in Java SE 8
55 New Features in Java SE 855 New Features in Java SE 8
55 New Features in Java SE 8
 
Something About Dynamic Linking
Something About Dynamic LinkingSomething About Dynamic Linking
Something About Dynamic Linking
 
Introduction to GNU Make Programming Language
Introduction to GNU Make Programming LanguageIntroduction to GNU Make Programming Language
Introduction to GNU Make Programming Language
 
Advanced c programming in Linux
Advanced c programming in Linux Advanced c programming in Linux
Advanced c programming in Linux
 
Core java
Core javaCore java
Core java
 
Making Linux do Hard Real-time
Making Linux do Hard Real-timeMaking Linux do Hard Real-time
Making Linux do Hard Real-time
 
Memory safety in rust
Memory safety in rustMemory safety in rust
Memory safety in rust
 
Java Garbage Collection - How it works
Java Garbage Collection - How it worksJava Garbage Collection - How it works
Java Garbage Collection - How it works
 
Automating OpenSCAP with Foreman
Automating OpenSCAP with ForemanAutomating OpenSCAP with Foreman
Automating OpenSCAP with Foreman
 
Graal in GraalVM - A New JIT Compiler
Graal in GraalVM - A New JIT CompilerGraal in GraalVM - A New JIT Compiler
Graal in GraalVM - A New JIT Compiler
 
Qemu JIT Code Generator and System Emulation
Qemu JIT Code Generator and System EmulationQemu JIT Code Generator and System Emulation
Qemu JIT Code Generator and System Emulation
 

Viewers also liked

JVM and Garbage Collection Tuning
JVM and Garbage Collection TuningJVM and Garbage Collection Tuning
JVM and Garbage Collection Tuning
Kai Koenig
 
Revisiting the Applicability of the Pareto Principle to Core Development Team...
Revisiting the Applicability of the Pareto Principle to Core Development Team...Revisiting the Applicability of the Pareto Principle to Core Development Team...
Revisiting the Applicability of the Pareto Principle to Core Development Team...
SAIL_QU
 
Владимир Иванов. JIT для Java разработчиков
Владимир Иванов. JIT для Java разработчиковВладимир Иванов. JIT для Java разработчиков
Владимир Иванов. JIT для Java разработчиков
Volha Banadyseva
 
Compiler optimization
Compiler optimizationCompiler optimization
Compiler optimization
liu_ming50
 
Android Code Optimization Techniques 2
Android Code Optimization Techniques 2Android Code Optimization Techniques 2
Android Code Optimization Techniques 2
Ishrat khan
 
Android Code Optimization Techniques
Android Code Optimization TechniquesAndroid Code Optimization Techniques
Android Code Optimization Techniques
Ishrat khan
 

Viewers also liked (20)

JVM Mechanics: When Does the JVM JIT & Deoptimize?
JVM Mechanics: When Does the JVM JIT & Deoptimize?JVM Mechanics: When Does the JVM JIT & Deoptimize?
JVM Mechanics: When Does the JVM JIT & Deoptimize?
 
Understanding Garbage Collection
Understanding Garbage CollectionUnderstanding Garbage Collection
Understanding Garbage Collection
 
JVM and Garbage Collection Tuning
JVM and Garbage Collection TuningJVM and Garbage Collection Tuning
JVM and Garbage Collection Tuning
 
Revisiting the Applicability of the Pareto Principle to Core Development Team...
Revisiting the Applicability of the Pareto Principle to Core Development Team...Revisiting the Applicability of the Pareto Principle to Core Development Team...
Revisiting the Applicability of the Pareto Principle to Core Development Team...
 
Basics of JVM Tuning
Basics of JVM TuningBasics of JVM Tuning
Basics of JVM Tuning
 
Владимир Иванов. JIT для Java разработчиков
Владимир Иванов. JIT для Java разработчиковВладимир Иванов. JIT для Java разработчиков
Владимир Иванов. JIT для Java разработчиков
 
JIT Soultions: Overview
 JIT Soultions: Overview JIT Soultions: Overview
JIT Soultions: Overview
 
Five cool ways the JVM can run Apache Spark faster
Five cool ways the JVM can run Apache Spark fasterFive cool ways the JVM can run Apache Spark faster
Five cool ways the JVM can run Apache Spark faster
 
Øredev 2011 - JVM JIT for Dummies (What the JVM Does With Your Bytecode When ...
Øredev 2011 - JVM JIT for Dummies (What the JVM Does With Your Bytecode When ...Øredev 2011 - JVM JIT for Dummies (What the JVM Does With Your Bytecode When ...
Øredev 2011 - JVM JIT for Dummies (What the JVM Does With Your Bytecode When ...
 
The Performance Engineer's Guide To HotSpot Just-in-Time Compilation
The Performance Engineer's Guide To HotSpot Just-in-Time CompilationThe Performance Engineer's Guide To HotSpot Just-in-Time Compilation
The Performance Engineer's Guide To HotSpot Just-in-Time Compilation
 
Under the Hood of the Testarossa JIT Compiler
Under the Hood of the Testarossa JIT CompilerUnder the Hood of the Testarossa JIT Compiler
Under the Hood of the Testarossa JIT Compiler
 
Profile Guided Optimization
Profile Guided OptimizationProfile Guided Optimization
Profile Guided Optimization
 
C++ Optimization Tips
C++ Optimization TipsC++ Optimization Tips
C++ Optimization Tips
 
Compiler optimization
Compiler optimizationCompiler optimization
Compiler optimization
 
IoT
IoTIoT
IoT
 
Android Code Optimization Techniques 2
Android Code Optimization Techniques 2Android Code Optimization Techniques 2
Android Code Optimization Techniques 2
 
Android Code Optimization Techniques
Android Code Optimization TechniquesAndroid Code Optimization Techniques
Android Code Optimization Techniques
 
Android Code Optimization Techniques 3
Android Code Optimization Techniques 3Android Code Optimization Techniques 3
Android Code Optimization Techniques 3
 
Nashorn on JDK 8 (ADC2013)
Nashorn on JDK 8 (ADC2013)Nashorn on JDK 8 (ADC2013)
Nashorn on JDK 8 (ADC2013)
 
Abdelrahman Al-Ogail Resume
Abdelrahman Al-Ogail ResumeAbdelrahman Al-Ogail Resume
Abdelrahman Al-Ogail Resume
 

Similar to JVM JIT compilation overview by Vladimir Ivanov

"JIT compiler overview" @ JEEConf 2013, Kiev, Ukraine
"JIT compiler overview" @ JEEConf 2013, Kiev, Ukraine"JIT compiler overview" @ JEEConf 2013, Kiev, Ukraine
"JIT compiler overview" @ JEEConf 2013, Kiev, Ukraine
Vladimir Ivanov
 
Byteman and The Jokre, Sanne Grinovero (JBoss by RedHat)
Byteman and The Jokre, Sanne Grinovero (JBoss by RedHat)Byteman and The Jokre, Sanne Grinovero (JBoss by RedHat)
Byteman and The Jokre, Sanne Grinovero (JBoss by RedHat)
OpenBlend society
 
Architecture for Massively Parallel HDL Simulations
Architecture for Massively Parallel HDL Simulations Architecture for Massively Parallel HDL Simulations
Architecture for Massively Parallel HDL Simulations
DVClub
 

Similar to JVM JIT compilation overview by Vladimir Ivanov (20)

"JIT compiler overview" @ JEEConf 2013, Kiev, Ukraine
"JIT compiler overview" @ JEEConf 2013, Kiev, Ukraine"JIT compiler overview" @ JEEConf 2013, Kiev, Ukraine
"JIT compiler overview" @ JEEConf 2013, Kiev, Ukraine
 
Scala & Spark(1.6) in Performance Aspect for Scala Taiwan
Scala & Spark(1.6) in Performance Aspect for Scala TaiwanScala & Spark(1.6) in Performance Aspect for Scala Taiwan
Scala & Spark(1.6) in Performance Aspect for Scala Taiwan
 
Unsafe Java
Unsafe JavaUnsafe Java
Unsafe Java
 
Apache Big Data Europe 2016
Apache Big Data Europe 2016Apache Big Data Europe 2016
Apache Big Data Europe 2016
 
A Java Implementer's Guide to Boosting Apache Spark Performance by Tim Ellison.
A Java Implementer's Guide to Boosting Apache Spark Performance by Tim Ellison.A Java Implementer's Guide to Boosting Apache Spark Performance by Tim Ellison.
A Java Implementer's Guide to Boosting Apache Spark Performance by Tim Ellison.
 
Java under the hood
Java under the hoodJava under the hood
Java under the hood
 
Cpu Cache and Memory Ordering——并发程序设计入门
Cpu Cache and Memory Ordering——并发程序设计入门Cpu Cache and Memory Ordering——并发程序设计入门
Cpu Cache and Memory Ordering——并发程序设计入门
 
White and Black Magic on the JVM
White and Black Magic on the JVMWhite and Black Magic on the JVM
White and Black Magic on the JVM
 
Java Jit. Compilation and optimization by Andrey Kovalenko
Java Jit. Compilation and optimization by Andrey KovalenkoJava Jit. Compilation and optimization by Andrey Kovalenko
Java Jit. Compilation and optimization by Andrey Kovalenko
 
Cloud Native Compiler
Cloud Native CompilerCloud Native Compiler
Cloud Native Compiler
 
Java virtual machine 101
Java virtual machine 101Java virtual machine 101
Java virtual machine 101
 
Java virtual machine 101
Java virtual machine 101Java virtual machine 101
Java virtual machine 101
 
A Java Implementer's Guide to Better Apache Spark Performance
A Java Implementer's Guide to Better Apache Spark PerformanceA Java Implementer's Guide to Better Apache Spark Performance
A Java Implementer's Guide to Better Apache Spark Performance
 
FOSDEM 2017 - Open J9 The Next Free Java VM
FOSDEM 2017 - Open J9 The Next Free Java VMFOSDEM 2017 - Open J9 The Next Free Java VM
FOSDEM 2017 - Open J9 The Next Free Java VM
 
Byteman and The Jokre, Sanne Grinovero (JBoss by RedHat)
Byteman and The Jokre, Sanne Grinovero (JBoss by RedHat)Byteman and The Jokre, Sanne Grinovero (JBoss by RedHat)
Byteman and The Jokre, Sanne Grinovero (JBoss by RedHat)
 
Jvm problem diagnostics
Jvm problem diagnosticsJvm problem diagnostics
Jvm problem diagnostics
 
Groovy In the Cloud
Groovy In the CloudGroovy In the Cloud
Groovy In the Cloud
 
The Diabolical Developers Guide to Performance Tuning
The Diabolical Developers Guide to Performance TuningThe Diabolical Developers Guide to Performance Tuning
The Diabolical Developers Guide to Performance Tuning
 
Java concurrency
Java concurrencyJava concurrency
Java concurrency
 
Architecture for Massively Parallel HDL Simulations
Architecture for Massively Parallel HDL Simulations Architecture for Massively Parallel HDL Simulations
Architecture for Massively Parallel HDL Simulations
 

More from ZeroTurnaround

Top Java IDE keyboard shortcuts for Eclipse, IntelliJIDEA, NetBeans (report p...
Top Java IDE keyboard shortcuts for Eclipse, IntelliJIDEA, NetBeans (report p...Top Java IDE keyboard shortcuts for Eclipse, IntelliJIDEA, NetBeans (report p...
Top Java IDE keyboard shortcuts for Eclipse, IntelliJIDEA, NetBeans (report p...
ZeroTurnaround
 
Language Design Tradeoffs - Kotlin and Beyond, by Andrey Breslav
Language Design Tradeoffs - Kotlin and Beyond, by Andrey BreslavLanguage Design Tradeoffs - Kotlin and Beyond, by Andrey Breslav
Language Design Tradeoffs - Kotlin and Beyond, by Andrey Breslav
ZeroTurnaround
 
Runtime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan Sciampacone
Runtime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan SciampaconeRuntime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan Sciampacone
Runtime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan Sciampacone
ZeroTurnaround
 
Easy Scaling with Open Source Data Structures, by Talip Ozturk
Easy Scaling with Open Source Data Structures, by Talip OzturkEasy Scaling with Open Source Data Structures, by Talip Ozturk
Easy Scaling with Open Source Data Structures, by Talip Ozturk
ZeroTurnaround
 
Blast your app with Gatling! by Stephane Landelle
Blast your app with Gatling! by Stephane LandelleBlast your app with Gatling! by Stephane Landelle
Blast your app with Gatling! by Stephane Landelle
ZeroTurnaround
 
How To Do Kick-Ass Software Development, by Sven Peters
How To Do Kick-Ass Software Development, by Sven PetersHow To Do Kick-Ass Software Development, by Sven Peters
How To Do Kick-Ass Software Development, by Sven Peters
ZeroTurnaround
 
Level Up Your Git and GitHub Experience by Jordan McCullough and Brent Beer
Level Up Your Git and GitHub Experience by Jordan McCullough and Brent BeerLevel Up Your Git and GitHub Experience by Jordan McCullough and Brent Beer
Level Up Your Git and GitHub Experience by Jordan McCullough and Brent Beer
ZeroTurnaround
 
AST Transformations: Groovy’s best kept secret by Andres Almiray
AST Transformations: Groovy’s best kept secret by Andres AlmirayAST Transformations: Groovy’s best kept secret by Andres Almiray
AST Transformations: Groovy’s best kept secret by Andres Almiray
ZeroTurnaround
 
Tap into the power of slaves with Jenkins by Kohsuke Kawaguchi
Tap into the power of slaves with Jenkins by Kohsuke KawaguchiTap into the power of slaves with Jenkins by Kohsuke Kawaguchi
Tap into the power of slaves with Jenkins by Kohsuke Kawaguchi
ZeroTurnaround
 
Language Design Tradeoffs (Kotlin and Beyond) by Andrey Breslav
Language Design Tradeoffs (Kotlin and Beyond) by Andrey BreslavLanguage Design Tradeoffs (Kotlin and Beyond) by Andrey Breslav
Language Design Tradeoffs (Kotlin and Beyond) by Andrey Breslav
ZeroTurnaround
 
Spring 4 on Java 8 by Juergen Hoeller
Spring 4 on Java 8 by Juergen HoellerSpring 4 on Java 8 by Juergen Hoeller
Spring 4 on Java 8 by Juergen Hoeller
ZeroTurnaround
 

More from ZeroTurnaround (20)

XRebel - Real Time Insight, Faster Apps
XRebel - Real Time Insight, Faster AppsXRebel - Real Time Insight, Faster Apps
XRebel - Real Time Insight, Faster Apps
 
Redeploy chart
Redeploy chartRedeploy chart
Redeploy chart
 
Top Reasons Why Java Rocks (report preview) - http:0t.ee/java-rocks
Top Reasons Why Java Rocks (report preview) - http:0t.ee/java-rocksTop Reasons Why Java Rocks (report preview) - http:0t.ee/java-rocks
Top Reasons Why Java Rocks (report preview) - http:0t.ee/java-rocks
 
Top Java IDE keyboard shortcuts for Eclipse, IntelliJIDEA, NetBeans (report p...
Top Java IDE keyboard shortcuts for Eclipse, IntelliJIDEA, NetBeans (report p...Top Java IDE keyboard shortcuts for Eclipse, IntelliJIDEA, NetBeans (report p...
Top Java IDE keyboard shortcuts for Eclipse, IntelliJIDEA, NetBeans (report p...
 
Java Tools and Technologies Landscape for 2014 (image gallery)
Java Tools and Technologies Landscape for 2014 (image gallery)Java Tools and Technologies Landscape for 2014 (image gallery)
Java Tools and Technologies Landscape for 2014 (image gallery)
 
Getting Started with IntelliJ IDEA as an Eclipse User
Getting Started with IntelliJ IDEA as an Eclipse UserGetting Started with IntelliJ IDEA as an Eclipse User
Getting Started with IntelliJ IDEA as an Eclipse User
 
[Image Results] Java Build Tools: Part 2 - A Decision Maker's Guide Compariso...
[Image Results] Java Build Tools: Part 2 - A Decision Maker's Guide Compariso...[Image Results] Java Build Tools: Part 2 - A Decision Maker's Guide Compariso...
[Image Results] Java Build Tools: Part 2 - A Decision Maker's Guide Compariso...
 
DevOps vs Traditional IT Ops (DevOps Days ignite talk by Oliver White)
DevOps vs Traditional IT Ops (DevOps Days ignite talk by Oliver White)DevOps vs Traditional IT Ops (DevOps Days ignite talk by Oliver White)
DevOps vs Traditional IT Ops (DevOps Days ignite talk by Oliver White)
 
Lazy Coder's Visual Guide to RebelLabs' Developer Productivity Report 2013
Lazy Coder's Visual Guide to RebelLabs' Developer Productivity Report 2013Lazy Coder's Visual Guide to RebelLabs' Developer Productivity Report 2013
Lazy Coder's Visual Guide to RebelLabs' Developer Productivity Report 2013
 
The State of Managed Runtimes 2013, by Attila Szegedi
The State of Managed Runtimes 2013, by Attila SzegediThe State of Managed Runtimes 2013, by Attila Szegedi
The State of Managed Runtimes 2013, by Attila Szegedi
 
Language Design Tradeoffs - Kotlin and Beyond, by Andrey Breslav
Language Design Tradeoffs - Kotlin and Beyond, by Andrey BreslavLanguage Design Tradeoffs - Kotlin and Beyond, by Andrey Breslav
Language Design Tradeoffs - Kotlin and Beyond, by Andrey Breslav
 
Runtime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan Sciampacone
Runtime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan SciampaconeRuntime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan Sciampacone
Runtime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan Sciampacone
 
Easy Scaling with Open Source Data Structures, by Talip Ozturk
Easy Scaling with Open Source Data Structures, by Talip OzturkEasy Scaling with Open Source Data Structures, by Talip Ozturk
Easy Scaling with Open Source Data Structures, by Talip Ozturk
 
Blast your app with Gatling! by Stephane Landelle
Blast your app with Gatling! by Stephane LandelleBlast your app with Gatling! by Stephane Landelle
Blast your app with Gatling! by Stephane Landelle
 
How To Do Kick-Ass Software Development, by Sven Peters
How To Do Kick-Ass Software Development, by Sven PetersHow To Do Kick-Ass Software Development, by Sven Peters
How To Do Kick-Ass Software Development, by Sven Peters
 
Level Up Your Git and GitHub Experience by Jordan McCullough and Brent Beer
Level Up Your Git and GitHub Experience by Jordan McCullough and Brent BeerLevel Up Your Git and GitHub Experience by Jordan McCullough and Brent Beer
Level Up Your Git and GitHub Experience by Jordan McCullough and Brent Beer
 
AST Transformations: Groovy’s best kept secret by Andres Almiray
AST Transformations: Groovy’s best kept secret by Andres AlmirayAST Transformations: Groovy’s best kept secret by Andres Almiray
AST Transformations: Groovy’s best kept secret by Andres Almiray
 
Tap into the power of slaves with Jenkins by Kohsuke Kawaguchi
Tap into the power of slaves with Jenkins by Kohsuke KawaguchiTap into the power of slaves with Jenkins by Kohsuke Kawaguchi
Tap into the power of slaves with Jenkins by Kohsuke Kawaguchi
 
Language Design Tradeoffs (Kotlin and Beyond) by Andrey Breslav
Language Design Tradeoffs (Kotlin and Beyond) by Andrey BreslavLanguage Design Tradeoffs (Kotlin and Beyond) by Andrey Breslav
Language Design Tradeoffs (Kotlin and Beyond) by Andrey Breslav
 
Spring 4 on Java 8 by Juergen Hoeller
Spring 4 on Java 8 by Juergen HoellerSpring 4 on Java 8 by Juergen Hoeller
Spring 4 on Java 8 by Juergen Hoeller
 

Recently uploaded

Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
panagenda
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
Wonjun Hwang
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
FIDO Alliance
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
FIDO Alliance
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc
 

Recently uploaded (20)

AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDM
 
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfFrisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptx
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهله
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 

JVM JIT compilation overview by Vladimir Ivanov

  • 1. 1 JVM JIT-compiler overview Vladimir Ivanov HotSpot JVM Compiler Oracle Corp.
  • 2. 2 Agenda §  about compilers in general –  … and JIT-compilers in particular §  about JIT-compilers in HotSpot JVM §  monitoring JIT-compilers in HotSpot JVM
  • 4. 4 Dynamic and Static Compilation Differences §  Static compilation –  “ahead-of-time”(AOT) compilation –  Source code → Native executable –  Most of compilation work happens before executing
  • 5. 5 Dynamic and Static Compilation Differences §  Static compilation –  “ahead-of-time”(AOT) compilation –  Source code → Native executable –  Most of compilation work happens before executing §  Modern Java VMs use dynamic compilers (JIT) –  “just-in-time” (JIT) compilation –  Source code → Bytecode → Interpreter + JITted executable –  Most of compilation work happens during application execution
  • 6. 6 Dynamic and Static Compilation Differences §  Static compilation (AOT) –  can utilize complex and heavy analyses and optimizations
  • 7. 7 Dynamic and Static Compilation Differences §  Static compilation (AOT) –  can utilize complex and heavy analyses and optimizations §  … but static information sometimes isn’t enough §  … and it’s hard to rely on profiling info, if any
  • 8. 8 Dynamic and Static Compilation Differences §  Static compilation (AOT) –  can utilize complex and heavy analyses and optimizations §  … but static information sometimes isn’t enough §  … and it’s hard to rely on profiling info, if any –  moreover, how to utilize specific platform features? §  like SSE4.2 / AVX / AVX 2, TSX, AES-NI, RdRand
  • 9. 9 Dynamic and Static Compilation Differences §  Modern Java VMs use dynamic compilers (JIT) –  aggressive optimistic optimizations §  through extensive usage of profiling info
  • 10. 10 Dynamic and Static Compilation Differences §  Modern Java VMs use dynamic compilers (JIT) –  aggressive optimistic optimizations §  through extensive usage of profiling info §  … but budget is limited and shared with an application
  • 11. 11 Dynamic and Static Compilation Differences §  Modern Java VMs use dynamic compilers (JIT) –  aggressive optimistic optimizations §  through extensive usage of profiling info §  … but budget is limited and shared with an application –  thus: §  startup speed suffers §  peak performance may suffer as well (but not necessarily)
  • 12. 12 Profiling §  Gathers data about code during execution –  invariants §  types, constants (e.g. null pointers) –  statistics §  branches, calls §  Gathered data is used during optimization –  Educated guess –  Guess can be wrong
  • 13. 13 Optimistic Compilers §  Assume profile is accurate –  Aggressively optimize based on profile –  Bail out if they’re wrong §  ...and hope that they’re usually right
  • 14. 14 Profile-guided optimizations (PGO) §  Use profile for more efficient optimization §  PGO in JVMs –  Always have it, turned on by default –  Developers (usually) not interested or concerned about it –  Profile is always consistent to execution scenario
  • 16. 16 Dynamic Compilation (JIT) §  Can do non-conservative optimizations in dynamic §  Separates optimization from product delivery cycle –  Update JVM, run the same application, realize improved performance! –  Can be "tuned" to the target platform
  • 17. 17 Dynamic Compilation (JIT) §  Knows about –  loaded classes, methods the program has executed §  Makes optimization decisions based on code paths executed –  Code generation depends on what is observed: §  loaded classes, code paths executed, branches taken §  May re-optimize if assumption was wrong, or alternative code paths taken –  Instruction path length may change between invocations of methods as a result of de-optimization / re-compilation
  • 18. 18 JVM §  Runtime –  class loading, bytecode verification, synchronization §  JIT –  profiling, compilation plans, OSR –  aggressive optimizations §  GC –  different algorithms: throughput vs. response time
  • 19. 19 JVM: Makes Bytecodes Fast §  JVMs eventually JIT bytecodes –  To make them fast –  compiled when needed §  Maybe immediately before execution §  ...or when we decide it’s important §  ...or never? –  Some JITs are high quality optimizing compilers
  • 20. 20 JVM: Makes Bytecodes Fast §  JVMs eventually JIT bytecodes §  But cannot use existing static compilers directly –  different cost model §  time & resource constraints (CPU, memory) –  tracking OOPs (ptrs) for GC –  Java Memory Model (volatile reordering & fences) –  New code patterns to optimize
  • 21. 21 JVM: Makes Bytecodes Fast §  JIT'ing requires Profiling –  Because you don't want to JIT everything §  Profiling allows focused code-gen §  Profiling allows better code-gen –  Inline what’s hot –  Loop unrolling, range-check elimination, etc –  Branch prediction, spill-code-gen, scheduling
  • 22. 22 Dynamic Compilation (JIT) §  Is dynamic compilation overhead essential? –  The longer your application runs, the less the overhead §  Trading off compilation time, not application time –  Steal some cycles very early in execution –  Done automagically and transparently to application §  Most of “perceived” overhead is compiler waiting for more data –  ...thus running semi-optimal code for time being Overhead
  • 24. 24 Mixed-Mode Execution §  Interpreted –  Bytecode-walking –  Artificial stack machine §  Compiled –  Direct native operations –  Native register machine
  • 25. 25 Bytecode Execution 1 2 34 Interpretation Profiling Dynamic Compilation Deoptimization
  • 26. 26 Deoptimization §  Bail out of running native code –  stop executing native (JIT-generated) code –  start interpreting bytecode §  It’s a complicated operation at runtime…
  • 27. 27 OSR: On-Stack Replacement §  Running method never exits? §  But it’s getting really hot? §  Generally means loops, back-branching §  Compile and replace while running §  Not typically useful in large systems §  Looks great on benchmarks!
  • 29. 29 Optimizations in HotSpot JVM §  compiler tactics delayed compilation tiered compilation on-stack replacement delayed reoptimization program dependence graph rep. static single assignment rep. §  proof-based techniques exact type inference memory value inference memory value tracking constant folding reassociation operator strength reduction null check elimination type test strength reduction type test elimination algebraic simplification common subexpression elimination integer range typing §  flow-sensitive rewrites conditional constant propagation dominating test detection flow-carried type narrowing dead code elimination §  language-specific techniques class hierarchy analysis devirtualization symbolic constant propagation autobox elimination escape analysis lock elision lock fusion de-reflection §  speculative (profile-based) techniques optimistic nullness assertions optimistic type assertions optimistic type strengthening optimistic array length strengthening untaken branch pruning optimistic N-morphic inlining branch frequency prediction call frequency prediction §  memory and placement transformation expression hoisting expression sinking redundant store elimination adjacent store fusion card-mark elimination merge-point splitting §  loop transformations loop unrolling loop peeling safepoint elimination iteration range splitting range check elimination loop vectorization §  global code shaping inlining (graph integration) global code motion heat-based code layout switch balancing throw inlining §  control flow graph transformation local code scheduling local code bundling delay slot filling graph-coloring register allocation linear scan register allocation live range splitting copy coalescing constant splitting copy removal address mode matching instruction peepholing DFA-based code generator
  • 30. 30 JVM: Makes Virtual Calls Fast §  C++ avoids virtual calls – because they are slow §  Java embraces them – and makes them fast –  Well, mostly fast – JIT's do Class Hierarchy Analysis (CHA) –  CHA turns most virtual calls into static calls –  JVM detects new classes loaded, adjusts CHA §  May need to re-JIT –  When CHA fails to make the call static, inline caches –  When IC's fail, virtual calls are back to being slow
  • 31. 31 Inlining §  Combine caller and callee into one unit –  e.g. based on profile –  … or prove smth using CHA (Class Hierarchy Analysis) –  Perhaps with a guard/test §  Optimize as a whole –  More code means better visibility
  • 34. 34 Inlining and devirtualization §  Inlining is the most profitable compiler optimization –  Rather straightforward to implement –  Huge benefits: expands the scope for other optimizations §  OOP needs polymorphism, that implies virtual calls –  Prevents naïve inlining –  Devirtualization is required –  (This does not mean you should not write OOP code)
  • 35. 35 Call Site §  The place where you make a call §  Monomorphic (“one shape”) –  Single target class §  Bimorphic (“two shapes”) §  Polymorphic (“many shapes”) §  Megamorphic
  • 36. 36 JVM Devirtualization §  Analyzes hierarchy of currently loaded classes §  Efficiently devirtualizes all monomorphic calls §  Able to devirtualize polymorphic calls §  JVM may inline dynamic methods –  Reflection calls –  Runtime-synthesized methods –  JSR 292
  • 37. 37 Feedback multiplies optimizations §  Profiling and CHA produces information –  ...which lets the JIT ignore unused paths –  ...and helps the JIT sharpen types on hot paths –  ...which allows calls to be devirtualized –  ...allowing them to be inlined –  ...expanding an ever-widening optimization horizon §  Result: Large native methods containing tightly optimized machine code for hundreds of inlined calls!
  • 39. 39 Existing JVMs §  Oracle HotSpot §  Oracle JRockit §  IBM J9 §  Excelsior JET §  Azul Zing §  SAPJVM §  …
  • 40. 40 HotSpot JVM §  client / C1 §  server / C2 §  tiered mode (C1 + C2) JIT-compilers
  • 41. 41 HotSpot JVM §  client / C1 –  $ java –client §  only available in 32-bit VM –  fast code generation of acceptable quality –  basic optimizations –  doesn’t need profile –  compilation threshold: 1,5k invocations JIT-compilers
  • 42. 42 HotSpot JVM §  server / C2 –  $ java –server –  highly optimized code for speed –  many aggressive optimizations which rely on profile –  compilation threshold: 10k invocations JIT-compilers
  • 43. 43 HotSpot JVM §  Client / C1 + fast startup –  peak performance suffers §  Server / C2 + very good code for hot methods –  slow startup / warmup JIT-compilers comparison
  • 44. 44 Tiered compilation §  -XX:+TieredCompilation §  Multiple tiers of interpretation, C1, and C2 §  Level0=Interpreter §  Level1-3=C1 –  #1: C1 w/o profiling –  #2: C1 w/ basic profiling –  #3: C1 w/ full profiling §  Level4=C2 C1 + C2
  • 46. 46 Monitoring JIT-Compiler §  how to print info about compiled methods? –  -XX:+PrintCompilation §  how to print info about inlining decisions –  -XX:+PrintInlining §  how to control compilation policy? –  -XX:CompileCommand=… §  how to print assembly code? –  -XX:+PrintAssembly –  -XX:+PrintOptoAssembly (C2-only)
  • 47. 47 Print Compilation §  -XX:+PrintCompilation §  Print methods as they are JIT-compiled §  Class + name + size
  • 48. 48 Print Compilation $ java -XX:+PrintCompilation 988 1 java.lang.String::hashCode (55 bytes) 1271 2 sun.nio.cs.UTF_8$Encoder::encode (361 bytes) 1406 3 java.lang.String::charAt (29 bytes) Sample output
  • 49. 49 Print Compilation §  2043 470 % ! jdk.nashorn.internal.ir.FunctionNode::accept @ 136 (265 bytes) % == OSR compilation ! == has exception handles (may be expensive) s == synchronized method §  2028 466 n java.lang.Class::isArray (native) n == native method Other useful info
  • 50. 50 Print Compilation §  621 160 java.lang.Object::equals (11 bytes) made not entrant –  don‘t allow any new calls into this compiled version §  1807 160 java.lang.Object::equals (11 bytes) made zombie –  can safely throw away compiled version Not just compilation notifications
  • 51. 51 No JIT At All? §  Code is too large §  Code isn’t too «hot» –  executed not too often
  • 52. 52 Print Inlining §  -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining §  Shows hierarchy of inlined methods §  Prints reason, if a method isn’t inlined
  • 53. 53 Print Inlining $ java -XX:+PrintCompilation -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining 75 1 java.lang.String::hashCode (55 bytes) 88 2 sun.nio.cs.UTF_8$Encoder::encode (361 bytes) @ 14 java.lang.Math::min (11 bytes) (intrinsic) @ 139 java.lang.Character::isSurrogate (18 bytes) never executed 103 3 java.lang.String::charAt (29 bytes)
  • 54. 54 Print Inlining $ java -XX:+PrintCompilation -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining 75 1 java.lang.String::hashCode (55 bytes) 88 2 sun.nio.cs.UTF_8$Encoder::encode (361 bytes) @ 14 java.lang.Math::min (11 bytes) (intrinsic) @ 139 java.lang.Character::isSurrogate (18 bytes) never executed 103 3 java.lang.String::charAt (29 bytes)
  • 55. 55 Intrinsic §  Known to the JIT compiler –  method bytecode is ignored –  inserts “best” native code §  e.g. optimized sqrt in machine code §  Existing intrinsics –  String::equals, Math::*, System::arraycopy, Object::hashCode, Object::getClass, sun.misc.Unsafe::*
  • 56. 56 Inlining Tuning §  -XX:MaxInlineSize=35 –  Largest inlinable method (bytecode) §  -XX:InlineSmallCode=# –  Largest inlinable compiled method §  -XX:FreqInlineSize=# –  Largest frequently-called method… §  -XX:MaxInlineLevel=9 –  How deep does the rabbit hole go? §  -XX:MaxRecursiveInlineLevel=# –  recursive inlining
  • 57. 57 Machine Code §  -XX:+PrintAssembly §  http://wikis.sun.com/display/HotSpotInternals/PrintAssembly §  Knowing code compiles is good §  Knowing code inlines is better §  Seeing the actual assembly is best!
  • 58. 58 -XX:CompileCommand= §  Syntax –  “[command] [method] [signature]” §  Supported commands –  exclude – never compile –  inline – always inline –  dontinline – never inline §  Method reference –  class.name::methodName §  Method signature is optional
  • 59. 59 What Have We Learned? §  How JIT compilers work §  How HotSpot’s JIT works §  How to monitor the JIT in HotSpot
  • 60. 60 Future work §  Da Vinci Machine Project (MLVM) –  project page: http://openjdk.java.net/projects/mlvm/ –  repository: http://hg.openjdk.java.net/mlvm/mlvm §  JSR292 matured there §  Some other projects: –  value objects –  coroutines / continuations –  tail calls (hard / guaranteed) Where does innovation take place?