This document discusses techniques used by just-in-time (JIT) compilers to optimize method calls in dynamic programming languages. It describes how call sites are often monomorphic, calling the same method, and how tracing JITs can exploit this by compiling call sequences into optimized machine code traces. Whole method JITs used in virtual machines like the JVM additionally profile call sites and can devirtualize polymorphic calls. The document outlines challenges for JITs in dynamic languages and techniques used to address these challenges.

1. Devirtualization of method calls
Dirty tricks of JIT compilers
Alexey Ragozin
alexey.ragozin@gmail.com
Apr 2013

2. Old good C++
010110010010
101010100110
101010100101
010101001010
101010101010
101010101010
101010101010
101010100010
00: methodA
02: methodC
03: methodD
CODEOBJECT
VTABLE
01: methodB
Simple case

3. Old good C++
010110010010
101010100110
101010100101
010101001010
101010101010
101010101010
101010101010
101010100010
111010100100
011110000010
101001010100
00: methodA
02: methodC
03: methodD
CODEOBJECT
VTABLE
01: methodB
00: methodX
02: methodZ
01: methodY
VTABLE
Multiple inheritance

4. Old good C++
Multiple inheritance – more fun
A
B C
D D
A
B C
D
Vs

5. Problems
Two memory reads before jump
 Memory access is serialized
 instruction pipeline is blocked

6. Dynamic languages
 Every call is virtual
 No predefined classes
 Multiple inheritance
Nightmare for super scalar CPU

7. Call site polymorphism
A particular call site
in most case
would be calling
exactly one method instance
Call sites are mostly monomorthic!

8. Exploiting monomorthism
Tracing JIT
 JavaScript – Mozila TraceMonkey
 Python – PyPy
Profiling for whole method JIT
 JVM

9. Tracing JIT
Interpretation phase
 Record actions, record branch conditions
Compiling trace
 Actions compiled into branchless machine code
 Guards added for conditions
 If guard is broken – fall back to interpretation

10. Tracing JIT
Mozila’s TraceMonkey
1. IncrementalDynamicCodeGenerationwithTraceTrees
http://www.ics.uci.edu/~franz/Site/pubs-pdf/ICS-TR-06-16.pdf
PyPy
 RPython – restrict python dialect compiling to C
 Rpython based interpreter trace JITed by runtime
1. http://tratt.net/laurie/research/pubs/papers/bolz_tratt__the_impact_of_metatra
cing_on_vm_design_and_implementation.pdf

11. Tracing JIT
Limitation
 Trace is EXPENSIVE
 Falling out of path is EXPENSIVE
 A lot of code is executed just once
– wasting resources on tracing

12. Whole method JIT
JVM
 Classic whole method JIT
 Profile “morphism” of call sites
 Multi tier compiler
 Dynamics recompilation / decompilation

13. Whole method JIT
Call sites
 Monomorphic
 Bimorphic
 Megamorphic

14. Invoke dynamic
Problem
 Interpreters tends to produce
megamorphic sites
Invoke dynamic
 Detach call site into manageable object

15. More JVM back magic
On stack replacement
 Code for method could be replaced without
reentering to a method
Scalar replacement
 Object not leaving method scope
can be reduced to a number of local variables

16. THANK YOU
alexey.ragozin@gmail.com