Introduction to the Compiler Runtime and the problematics of ahead-of-time compilation.

1. Lifecycle of a JIT compiled code
Ivan Krylov
Azul Systems

2. 2 Ivan Krylov

3. Is this the right talk?
• JVM Internals and Performance Talk:
• General Interest
• Got a performance critical app
• You have a reproducible case
• Have a way to collect JVM logs
3

4. Scope and Agenda
• Code flow inside JVM
• Profiles that JVM collects
• Deoptimization
• Four ways to control compilation
4
Ivan Krylov

5. Code Pipeline in JVM
5
Sources
in .java
Bytecodes
Machine
code from
the
Interpreter
Machine code
from a
compiler
Javac
1) Static verification
2) Compilation with near-0 opts
Runtime
1) Dynamic verification
2) Linking
One of N compilers
Governed by runtime
next tierdeoptimization
live
profile data
JAOT in JDK9

6. • Bytecodes coming not from javac
• Any JVM language; jasm-like tools; Byte Buddy
• May be no interpreter
• JRockit VM
• May be only interpreter
• Zero-port or -Xint
• May be a pure AOT (no bytecodes, no JIT)
• Pure AOT can’t implement the entire Java SE standard
• Hence a hybrid approach : Excelsior JET, Graal-based AOT (in works)
Everything can be different
6

7. Template Interpreter - a tier 0 compiler
• Machine code creation
• From asm(masm) templates
• Populated with addresses in VM
• Generated at start-up
• Maintains invocation and Back branch C-rs
• Mixed native/java stack frame
• All bytescodes in a single blob with GoTo-s
7

8. Template Interpreter - ladd example
8
0x000000010a59a920: mov (%rsp),%rax
0x000000010a59a924: add $0x10,%rsp
0x000000010a59a928: mov (%rsp),%rdx
0x000000010a59a92c: add $0x10,%rsp
0x000000010a59a930: add %rdx,%rax
0 0x000000010a590933: movzbl 0x1(%r13),%ebx
0x000000010a59a938: inc %r13
0x000000010a59a93b: movabs $0x10a0f6600,%r10
0x000000010a59a945: jmpq *(%r10,%rbx,8)
Entry if %RAX
doesn’t contain
first operand
Entry otherwise
Read next
bytecode
& dispatch
№1
№2
return value
bytecode cntr next bytecode
method start
0x000000010a0f6600:

9. What is a profile ?
• It is all about counters
• Inaccurate but indicative
• If overflows - stays at
saturated level
• Some counters with properties
• Actual types observed, etc..
9
https://www.flickr.com/photos/icemanphoto/1739731523

10. Profile is.. methodCounters.hpp
• interpreter_invocation_count &
invocation_counter
• interpreter_throwout_count
• backedge_counter
• number_of_breakpoints (4 dbg)
• nmethod_age
• interpreter_invocation_limit &
interpreter_backward_branch_limit
• interpreter_profile_limit
• invoke_mask
• backedge_mask
• rate & prev_time
• highest_comp_level &
highest_osr_comp_level
10

11. Anything else needed?
11
class A {
static List<String> list = new LinkedList<>();
static {
DateFormat dateFormat = new
SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
list.add(dateFormat.format(new Date()));
}
static List<String> getList() {
return list;
}
}
class B
{
String process(String s, int i)
{
if (i < A.list.size())
return s.concat( A.list.get(i) );
else
return s;
}
}
B.process() may
be compiled
only after A is initialized
A may be initialized only at first
invocation of B.process()
(If B.process() is the only user of A)

12. Is SimpleEnum simple?
12
public enum SimpleEnum {
One
};
>ll SimpleEnum.java
-rw-r--r-- 1 ivan staff 35 Feb 23 21:30 SimpleEnum.java
>ll SimpleEnum.class
-rw-r--r-- 1 ivan staff 732 Feb 23 21:30 SimpleEnum.class

13. Is SimpleEnum simple?
12
public enum SimpleEnum {
One
};
Compiled from "SimpleEnum.java"
public final class SimpleEnum extends java.lang.Enum<SimpleEnum> {
public static final SimpleEnum One;
public static SimpleEnum[] values();
Code:
0: getstatic #1 // Field $VALUES:[LSimpleEnum;
3: invokevirtual #2 // Method "[LSimpleEnum;".clone:()Ljava/lang/Object;
6: checkcast #3 // class "[LSimpleEnum;"
9: areturn
public static SimpleEnum valueOf(java.lang.String);
Code:
0: ldc #4 // class SimpleEnum
2: aload_0
3: invokestatic #5 // Method java/lang/Enum.valueOf:(Ljava/lang/Class;Ljava/lang/String;)Ljava/lang/Enum;
6: checkcast #4 // class SimpleEnum
9: areturn
static {};
Code:
0: new #4 // class SimpleEnum
3: dup
4: ldc #7 // String One
6: iconst_0
7: invokespecial #8 // Method "<init>":(Ljava/lang/String;I)V
10: putstatic #9 // Field One:LSimpleEnum;
13: iconst_1
14: anewarray #4 // class SimpleEnum
17: dup
18: iconst_0
19: getstatic #9 // Field One:LSimpleEnum;
22: aastore
23: putstatic #1 // Field $VALUES:[LSimpleEnum;
26: return
}

14. 2: aload_0
3: invokestatic #5 // Method java/lang/Enum.valueOf:(Ljava/lang/C
6: checkcast #4 // class SimpleEnum
9: areturn
static {};
Code:
0: new #4 // class SimpleEnum
3: dup
4: ldc #7 // String One
6: iconst_0
7: invokespecial #8 // Method "<init>":(Ljava/lang/String;I)V
10: putstatic #9 // Field One:LSimpleEnum;
13: iconst_1
14: anewarray #4 // class SimpleEnum
17: dup
18: iconst_0
19: getstatic #9 // Field One:LSimpleEnum;
22: aastore
23: putstatic #1 // Field $VALUES:[LSimpleEnum;
26: return
}
Is SimpleEnum simple?
13
public enum SimpleEnum {
One
};

15. The Rumor:Inlining is important
14
• Hide the costs of preparing arguments
• Hide the costs of marshaling the result
• Code is co-located (I-cache friendly)
• Makes executable smaller in some cases
• Unlocks a bunch of optimizations
• Because the scope of a compilation block is bigger
• Easy to test: disable Inlining with -XX:-Inline - and observe XX% perf loss
• -XX:+PrintInlining to see what methods’re inlined and where

16. Inlining that didn’t happen
15
• Inlining Callee makes a caller too big
• Inlining too deep / recursive inlining
• Callee has exception handlers? (InlineMethodsWithExceptionHandlers)
• Callee is synchronized (InlineSynchronizedMethods )
• Class of Callee is not initialized
• Callee has unbalanced monitors
• Caller has jsr bytecode (!)
}
http://cliffhacks.blogspot.ru/2008/02/java-6-tryfinally-compilation-without.html
Overrulable

17. Did you see tiers?
16
495 7 n 0 jdk.internal.reflect.Reflection::getCallerClass (native) (static)
495 8 b 2 java.util.Properties::getProperty (49 bytes)
497 9 b 2 java.util.concurrent.ConcurrentHashMap::get (162 bytes)
500 10 b 3 java.lang.String::hashCode (48 bytes)
503 11 b 4 java.lang.String::hashCode (48 bytes)
507 10 3 java.lang.String::hashCode (48 bytes) made not entrant
507 12 b 2 java.lang.StringLatin1::hashCode (42 bytes)
509 13 b 3 java.lang.Boolean::<clinit> (31 bytes)
510 14 b 4 java.lang.Boolean::<clinit> (31 bytes)
510 13 3 java.lang.Boolean::<clinit> (31 bytes) made not entrant
511 15 b 3 java.lang.Boolean::<init> (10 bytes)
511 16 b 4 java.lang.Boolean::<init> (10 bytes)
513 15 3 java.lang.Boolean::<init> (10 bytes) made not entrant
513 17 b 2 java.lang.Object::<init> (1 bytes)
513 18 n 0 java.lang.Class::getPrimitiveClass (native) (static)
514 19 b 3 java.lang.Boolean::parseBoolean (19 bytes)
514 20 b 4 java.lang.Boolean::parseBoolean (19 bytes)
516 19 3 java.lang.Boolean::parseBoolean (19 bytes) made not entrant
516 21 b 2 java.lang.String::isLatin1 (19 bytes)
517 22 b 2 java.lang.Integer::parseInt (259 bytes)
533 23 b 3 java.lang.Character::<clinit> (25 bytes)

18. 4 - Full Optimisation - C2
Tiered compilation
Fastest
Slow
Mediocre
Fast
Slowest
3 - Full Profile - C1
(incl. MethodData)
2 - Limited Profile - C1
(+ invocation & backed counters)
1 - Simple Profile - C1
0 - Interpreter
17

19. • … Compiler does
“optimistic” (or “heroic”)
optimizations
• … Something AOT can’t
do
• & “Whole World” is not
developed at time of
compilation
Deoptimizations happen because…
18

20. 19
Not all opts or speculative
• Constant
propagation
• Loop invariant
• …
• Bias locking
• NUMA-aware
allocation
• TSX transactions
• Uncommon trap
• …
Deterministic opts Speculative
w/fallback
Speculative
w/hard stop
• CHA invalidation
• Effectively final
fields modified
• …

21. Implicit code
• Mandatory operations from JVMLS
• Zeroing of Objects
• Out of boundary checks
• Divide by 0 in math
• Null Pointer protection
• Checkcasts
• ….
20
class C {
int val;
C[] kInverseVector(int k, C[] v) {
C[] result = new C[v.length];
for (int i=0; i < v.length; i++){
result[i] = new C();
result[i].val = k / v[i].val ;
}
return result;
}
}

22. Per bci deoptimization reasons
• Null Check (null object or div 0)
• Null Assert
• Range Check (OOB)
• Class Check (Unexpected class)
• Array Check (Unexpected array class)
• Intrinsic operand
• Bimorphic inlining failed
21

23. Per method deoptimization reasons
• Unloaded class
• Uninitialized class
• Unreached code
• Unhandled exception
• Unexpected Constraint
• Div0 check
• Age (tier threshold reached)
• Predicate failed
• Loop limit check
• Speculate class check
• Speculate Null Check
• Rm state change
• Unstable if
• Reason unstable fused if
22

24. Uncommon trap happened. Now what?
• 1-off event?
• A change in a pattern?
• Keep old around?
• Recompile? Never compile?
• What the next invocation will be like?
23
class C {
int val;
C[] kInverseVector(int k, C[] v) {
C[] result = new C[v.length];
for (int i=0; i < v.length; i++){
result[i]=new C();
result[i].val = k / v[i].val ;
}
return result;
}
}

25. How to control a compiler?
24

26. I. Legacy CompileCommand
25
• Provided statically via
• Command line: -XX:HotspotCompile
• .hotspot_compiler or
-XX:CompileCommandFile=/path/to/thefile
• Dynamically
• jcmd Compiler.directives_add
• Syntax:
command package/Class methodName
• Actions:
• dontinline or exclude or excludec2 or excludec1
exclude java/lang/Thread setPriority
dontinline java/lang/String charAt

27. II. Compiler Control in Java 9
26
{
// matching several patterns require an array
match: ["steve.*","alex.*"]
c2: {
Enable: false, // Ignore this directive for c2.
BreakAtExecute: true // This will not be applied since Enable is false above
}
// applies to all compilers
// + force inline, - dont inline
inline : [ "+java/util.*", "-com/sun.*"],
PrintInlining: true
}

28. Common Compiler Control Directives
27
• Enable
• bool Exclude
• bool Inline
• bool BreakAtExecute
• bool BreakAtCompile
• bool Log
• bool PrintAssembly

29. Common Compiler Control Directives
27
• Enable
• bool Exclude
• bool Inline
• bool BreakAtExecute
• bool BreakAtCompile
• bool Log
• bool PrintAssembly
• bool PrintInlining
• bool PrintNMethods
• bool ReplayInline
• bool DumpReplay
• bool DumpInline
• bool
CompilerDirectivesIgnoreCompileCommands

30. • bool PrintOptoAssembly
• bool PrintIntrinsics
• bool TraceOptoOutput
• bool TraceSpilling
• bool CloneMapDebug
• bool IGVPrintLevel
C2 Compiler Control Directives
28
• BlockLayoutByFrequency
• bool raceOptoPipelining
• bool Vectorize
• bool VectorizeDebug
• intx MaxNodeLimit
• intx DisableIntrinsics

31. III. java.lang.Compiler
29
• OpenJDK / Oracle don’t support it.
• IBM J9 supports it
http://www.ibm.com/support/knowledgecenter/en/
SSSTCZ_2.0.0/com.ibm.rt.doc.20/realtime/rt_jit.html
• Zing has a slight different flavour
http://docs.azul.com/zing/Zing_UserGuide/
#Zing_UserGuide/
Zing_AT_ReadyNow_EnsureCriticalMethodsareCom
piled_CompilerAPI.htm
• Oracle wants to deprecate
See https://bugs.openjdk.java.net/browse/
JDK-4285505
{
Compiler.enable(); // ensure compiler is active
Compiler.command("{com.mycompany.*}(compile)");
System.out.println("Now let’s wait till compilation is done");
Compiler.command("waitOnCompilationQueue");
System.out.println("Compilation is complete");
Compiler.disable(); // turn the compiler off
}

32. IV. ReadyNow
30
• Between AOT and JIT
• Designed to compile sooner than normal JIT
• “Your JIT’s yesterday’s mistakes wont hurt you today”
• Designed to deoptimize less often than normal JIT
• Works well in good “scenarios”
• Designed to never produce an invalid code
• The fallback is the normal JIT

33. Three requirements of ReadyNow
31
I. Profiles must be adequate
II. Profile needs to be applicable
III. All necessary types are initialized or …
1. all the ones we need for fastpath
2. can force eager initialization
in case of no/trivial static initializers

34. 32
Recording a profile
CT1 CT2 CT3
profile

35. Using a profile
33
CT1 CT2 CT3

36. ReadyNow
34

37. Bonus: JDK9 Ahead of Time Compilation
35
• Targeting Fast startup
• Can interoperate with JIT
• Only Linux x64
• Only java.base is guaranteed
• Must use the same JVM flags
• Not for dynamically generated classes
• No reflection support
• No instrumentation support
jaotc --output libjava.base.so --module java.base
jaotc --output libHelloWorld.so HelloWorld.class
java -XX:AOTLibrary=./libHelloWorld.so HelloWorld

38. 36
• Zing: A better JVM for the enterprise
• Azul’s enterprise JVM focused on better metrics
• Consistent performance - not just fast, always fast
• Eliminate GC as a concern for enterprise apps
• Very wide operating range
• From human-sensitive app responsiveness to low-latency
trading
• From microservices to huge in-memory apps
• Eliminates an entire class of engineering workarounds common in
Java
•Zulu Embedded: When you need embedded Support
• 100% open source, based on OpenJDK
• Certified Java SE compliant and compatible
• Verified “non-contaminating” open source license
• Identical metrics to OpenJDK and Oracle Java SE
• World-class support offerings
• Support for Linux & Windows; x86, ARM32, ARM64, PPC32,
MIPS
36

39. Ivan Krylov www.azul.com
Azul Systems ivan @ azul.com