Geekout 2016 presentation. Ahead-of-time (AOT) native compilation has been available for .NET for quite some time, whereas Java SE, while being older than .NET, still does not have a standard AOT compiler. Some people ask why, some people think that it could only work for certain limited types of Java applications, some people think that it is not needed. The truth, however, is that a Java AOT compiler has been available for over a decade, as part of a compliant Java SE implementation called Excelsior JET, and it proved to be useful for many organizations around the world. In this session, I would like to dispel the common myths about Java AOT compilation and show where it has advantages over JIT compilation from the technical point of view.

1. Ahead-Of-Time Compilation
of Java Applications
1
Nikita Lipsky
Excelsior LLC

2. Once upon a time,
when C++ was new and cool,
+++++++++
2

3. Once upon a time,
when C++ was new and cool,
+++++++++
3

4. Once upon a time,
when C++ was new and cool,
all compilers were static
4

5. Until into this world came…
Java
5

6. Until into this world came…
Java
6

7. Two Ways To Execute Java Bytecode
• Interpret
Slow but portable
• Compile to native code
Runs on the actual hardware
7

8. When to compile to native code?
• At application run time
Dynamic (Just-In-Time, JIT) Compilation
• Before the application is launched/deployed
Static (Ahead-Of-Time, AOT) Compilation
8

9. 9

10. Static compilation of Java
• Is it at all possible?
– If yes, what are the conditions and would it still be Java?
• What are the benefits?
– Obfuscation vs. static compilation
– Application startup
– Smaller installers with no external dependencies
– Performance: comparison with JIT compilation
– Why is AOT better for Desktop, Embedded, Mobile
– Are there any benefits for server-side apps?
10

11. Nikita Lipsky
• 20+ years in software development
• Excelsior JET project initiator
– 16+ years of contributions
– compiler engineer
– team lead
– product lead
– etc.
• Open source projects WebFX and Java ReStart
– in a spare time
• Twitter: @pjBooms
11

12. 12

13. Who needs Java AOT
13
Кто знает про Excelsior JET?Кто знает про Excelsior JET?

14. Java AOT Compilation Myths
14

15. Myth 1. Java is “too dynamic”
Reflection Dynamic class loading
15

16. Myth 2. AOT kills WORA
WORA
Write Once
Run Anywhere
BORA
Build Once
Run Anywhere
16
!=

17. Myth 3. AOT = Small EXE
”I would get a small executable that works
without a JVM (as if I wrote my app in C)”
17

18. Myth 3. AOT = Small EXE
• What about thousands of standard classes?
• Who will collect the garbage?
• What about reflection?
18

19. AOT = Smaller Package
Yes, AOT compilation can help you create
a JRE-independent installer smaller than the JRE
installer alone
Javа Runtime Slim-Down:
www.excelsiorjet.com/solutions/java-download-size
19

20. Java IS “too dynamic”
20

21. Non-standard Class Loaders
• Override the default reference resolution logic
• Unique namespaces
• Dependency management frameworks
– OSGi, Java Module System, etc.
Solve JAR hell problem
• Java EE servers, Eclipse RCP, plugin architectures
21

22. Non-standard Class Loaders
How to compile such classes statically?
• Compile each class in isolation
– Bad for performance
• Reproduce reference resolution logic of popular
class loaders
– Does not work for arbitrary class loaders
22

23. Non-standard Class Loaders
23
Classes
Classe
sClasses

24. Classes
CL1
CL2
CL3
CL4
Non-standard Class Loaders
24CLi: class loader

25. • CL4• CL3
• CL2• CL1
classes classes
classesclasses
Non-standard Class Loaders
25

26. • CL4• CL3
• CL2• CL1
classes
classes
classesclasses
Non-standard Class Loaders
26

27. • CL4• CL3
• CL2• CL1
classes classes
classesclasses
Non-standard Class Loaders
27

28. Non-standard Class Loaders
28
• CL4• CL3
• CL2• CL1
classes classes
classes

29. • CL4• CL3
• CL2• CL1
classes classes
Non-standard Class Loaders
29

30. Non-standard Class Loaders
AOT class loader support scheme:
• Reference resolver in the AOT compiler:
– Determines which class loaders will be created at
run time and assigns a static ID to each
– Breaks down the set of classes by class loader IDs
– Resolves references between classes
30

31. Non-standard Class Loaders
AOT class loader support scheme contd.:
• At application run time:
– For each class loader instance, compute its ID
– Knowing the ID, “load” the precompiled classes:
• Create a java.lang.Class instance
• Fill it with reflection information
• Let the O/S load code from the executable
31

32. Non-standard Class Loaders
AOT class loader support scheme contd.:
• At application run time:
– For each class loader instance, compute its ID
– Knowing the ID, “load” the precompiled classes
Known to work for Eclipse RCP
and Tomcat Web apps
32

33. Why AOT for Java?
33

34. Protect Code from Decompilers
34

35. Application Code Protection
• Bytecode emitted by javac is extremely easy to
decompile almost to original source
Proof: http://bytecodeviewer.com/
• Reflection makes name obfuscation labor-
consuming and error prone, hinders code
maintenance
• Possible to guess what obfuscated code does
References to JDK classes remain intact
35

36. Application Code Protection
• Native machine code can only be effectively
disassembled, but not decompiled
• Application structure not deductible
from disassembler output
• Aggressively optimized native code only
remotely resembles original
36

37. Application Startup Time
37

38. Cold Start vs Warm Start
Why a re-started application starts much faster?
– No need to load app code and data from disk
– No need to load system/third-party dynamic
libraries (.DLL, .so) either
38

39. AOT Is Faster?
• Native code is “thicker” than Java bytecode
• Reading code from disk often takes more
time than its execution, esp. on cold starts
39

40. AOT IS Faster!
1. Identify code that gets executed on startup
2. Place it at the beginning of the executable
3. Preload the entire “startup segment” in one
sequential read (Makes most difference on rotating media,
slow USB sticks, etc., not so much on SSDs)
40

41. 41

42. Performance
42

43. Myth 4. AOT Is Faster
“By compiling Java statically, we are making it
equivalent to C, C is faster than Java,
therefore statically compiled Java is faster”
43

44. Myth 5. JIT Is Faster
“Effective optimization of a Java application
is only possible in the presence
of its dynamic execution profile”
44

45. Compiler Optimizations
– Constant propagation
– Dead code elimination
– Common subexpression
elimination
– Inline substitution
– Method specialization
– Loop unrolling
– Loop versioning
– Loop-invariant code motion
– Tail recursion elimination
– Call devirtualization
– On-stack allocation and
explosion of objects
– Runtime checks removal
– Excess synchronization removal
– Optimal code selection and
bundling
– Instruction scheduling
– Optimal register allocation
– etc.
45

46. Java == OOP
• Lots of methods
• Lots of small methods (get/set)
• Lots of virtual calls of small methods
46

47. Call Devirtualization
• Precondition for subsequent inline substitution
• Class hierarchy analysis
Method not overridden => non-virtual call
• Type analysis
new T().foo(); // Non-virtual
// call of T.foo()
• Inline caches
47

48. A a;
…
a.foo(); 
if (RT_Type(a) in CHA) {
inlined body of foo()
} else {
a.foo();
}
Idea: method not overriden => non-virtual call
Class Hierarchy Analysis (CHA)
48

49. Type Analysis
Idea:
new A().foo(); // non-virtual call. Always!
49

50. Type Analysis
A a = b ? new B() : new C();
a.foo();// is it virtual call?
// B, C extend A and do not override A.foo
50

51. Type Analysis
A a = b ? new B() : new C();
a.foo();
If neither B nor С override A.foo(),
then a.foo()is a non-virtual call again!
(can be inlined)
51

52. Type Analysis
A a = b ? bar() : baz();
…
a.foo();
If bar() only returns results of new B,
and baz() - only returns results of new С,
then a.foo() is again non-virtual
(can be inlined)
52

53. Type Analysis
How do we know
what bar() and baz() return?
53

54. Global Analysis
• Analyses all methods of the program, computing
useful information about each
– whether a method always returns new T();
– whether an argument of a method does not escape to
a shared memory
• Results are then used for optimization of each
particular method
54

55. Stack Allocation of Objects
• All Java objects are supposed to reside in dynamic
memory – on the Java heap
• But, most objects are small and temporary
• It is desirable to allocate them on the stack
Escape analysis determines whether a locally
created object escapes from the method
stack frame into shared memory
55

56. Example
for (Object o: getCollection()) {
doSomething(o);
}
56

57. Example
Iterator iter = getCollection().iterator();
while (iter.hasNext()) {
Object o = iter.next();
doSomething(o);
}
57

58. Example
Suppose analysis has shown that getCollection()
always returns an ArrayList
ArrayList list = getCollection();
Iterator iter = list.iterator();
while (iter.hasNext()) {
Object o = iter.next();
doSomething(o);
}
58

59. Example
ArrayList list = getCollection();
ArrayList.ListItr iter = new ListItr(list);
while (iter.hasNext()) {
Object o = iter.next();
doSomething(o);
}
59

60. Example
ArrayList list = getCollection();
ArrayList.ListItr iter = onStack ListItr();
iter.this$0 = list;
iter.cursor = 0;
iter.size = list.elemData.length;
while (iter.hasNext()) {
Object o = iter.next();
doSomething(o);
}
60

61. Example
ArrayList list = getCollection();
ArrayList.ListItr iter = onStack ListItr(list);
iter.this$0 = list;
iter.cursor = 0;
iter.size = list.elemData.length;
while (iter.cursor < iter.size) {
int index = iter.cursor++;
Object o = iter.this$0.elemData[index];
doSomething(o);
}
61

62. Example
ArrayList list = getCollection();
int cursor = 0;
int size = list.elemData.length;
while (cursor < size) {
Object o = list.elemData[cursor++];
doSomething(o);
}
62

63. Example
ArrayList list = getCollection();
int size = list.elemData.length;
for (int i = 0; i < size; i++) {
doSomething(list.elemData[i]);
}
63

64. Analysis & Optimizations…
• …are often quite complicated
• …require iterative re-computation
• …, if global, depend on the entire program
64

65. Analysis & Optimizations…
• …are often quite complicated
• …require iterative recomputation
• …, if global, depend on the entire program
Can a JIT compiler afford
all or any of that?
65

66. Analysis & Optimizations…
• …are often quite complicated
• …require iterative recomputations
• …, if global, depend on the entire program
Can a JIT compiler afford
all or any of that?
66

67. Analysis & Optimizations…
• …are often quite complicated
• …require iterative recomputations
• …, if global, depend on the entire program
Can a JIT compiler afford
all or any of that?
67

68. Dynamic Optimizations
• Profiling and selective compilation
• Inline substitution based on execution profile
• Hot execution traces optimization
• Optimal instruction selection
68

69. Hot Code vs Warm Code
Q: What happens when an app with no
distinctly hot code runs on a typical JVM?
A: Long warmup with results not stored
for future reuse
Typical case: UI-centric applications
69

70. JFCMark (Short Run)
0%
50%
100%
150%
200%
250%
2 core Celeron, 2.60Ghz 4 core i5, 3.8GZ
Excelsior JET
HotSpot client
HotSpot server
Bigger – better
70

71. JFCMark (Long Run)
0%
20%
40%
60%
80%
100%
120%
140%
160%
2 core Celeron, 2.60Ghz 4 core i5, 3.8GZ
Excelsior JET
HotSpot client
HotSpot server
71
Bigger – better

72. Profile Guided Optimizations
Can a static compiler use
dynamic execution profiles as input?
72

73. Server side
• CPU time in the cloud costs
money
• After some time, execution profile of a server app
stabilizes
• Why not pass it over to the AOT compiler?
73

74. AOT on the Server side
• Stable performance and predictable latency
– no code de-optimizations occur at run-time
• Work at full speed right from the start
– good for load balancing
• Better startup time
– good when many servers start simultaneously
• Still protects the code from decompilation
74

75. Embedded/IoT
• The less powerful the hardware, the more
expensive dynamic compilation becomes
• Embedded systems typically have less
computing power than desktops and servers
75

76. Mobile
• JVM on Mobile:
– Platform classes
– Memory Manager and GC
– Reflection
– JIT Compiler
What is missing?
76

77. Mobile
• JVM on Mobile:
– Platform classes
– Memory Manager and GC
– Reflection
– JIT Compiler
What is missing?
77

78. Mobile
• JVM on Mobile:
– Platform classes
– Memory Manager and GC
– Reflection
– JIT Compiler
What is missing?
78

79. Mobile
• JVM on Mobile:
– Platform classes
– Memory Manager and GC
– Reflection
– JIT Compiler
What is missing?
79

80. Mobile
• JVM on Mobile:
– Platform classes
– Memory Manager and GC
– Reflection
– JIT Compiler
What is missing?
80

81. Mobile
• JVM on Mobile:
– Platform classes
– Memory Manager and GC
– Reflection
– JIT Compiler
What is missing?
81

82. Mobile
• JVM on Mobile:
– Platform classes
– Memory Manager and GC
– Reflection
– JIT Compiler
Charger!
82

83. Mobile
• Wireless devices have batteries
• Does it make sense to spend power on
dynamic compilation?
83

84. iOS
• iOS policy prohibits creation of any native
code during application run time
• Getting Java apps to work on iOS requires
either an interpreting JVM or an AOT compiler
84

85. AOT Java Compilers in 2000
Desktop/Server:
BulletTrain
Excelsior JET
GNU Compiler for Java (GCJ)
IBM VisualAge for Java
Supercede/JOVE
TowerJ
Embedded/Mobile:
Diab FastJ
Esmertec Jbed ME
GCJ
IBM J9
Sun Java ME
(custom offerings)
85

86. AOT Java Compilers in 2016
Desktop/Server:
Excelsior JET
GCJ
IBM Java SDK for AIX
IBM Java SDK for z/OS
Embedded:
Excelsior JET Embedded
IBM WebSphere Real Time
Oracle Java ME Embedded Client
Codename One
Avian
Android ART
RoboVM (RIP)
Migeran (acquired by Intel)
86

87. 87

88. Static compilation of Java apps:
88

89. Static compilation of Java apps:
89
Possible

90. Static compilation of Java apps:
90
Possible
Preserving all Java features

91. Static compilation of Java apps:
91
Possible
Preserving all Java features
Useful in many aspects

92. Q & A
Nikita Lipsky,
Excelsior
nlipsky@excelsior-usa.com
twitter: @pjBooms 92