This paper proposes using machine learning techniques to automatically construct inlining heuristics that improve performance over default heuristics. The researchers used NeuroEvolution of Augmenting Topologies (NEAT) to evolve neural networks that determine whether to inline functions based on call site features. Their best NEAT-generated heuristic provided an 11% average speedup over the Java HotSpot VM heuristic. They also generated a decision tree heuristic that explained the NEAT heuristic and provided 10% speedup. The machine-learned heuristics outperformed default heuristics by considering a wider range of inlining factors.

1. AUTOMATIC CONSTRUCTION OF
INLINING HEURISTICS USING
MACHINE LEARNING
(S. KULKARNI, J. CAVAZOS, C. WIMMER,
D. SIMON, 2013)
NATALLIE BAIKEVICH
@LU-A-JALLA
PAPERS WE LOVE: ZÜRICH

2. WHY THIS PAPER?
Research papers in the .NET source (post
by Matt Warren, @matthewwarren) :
http://mattwarren.org/2016/12/12/Research-
papers-in-the-.NET-source/
Inspired by Java papers:
https://lowlevelbits.org/java-papers/

3. MOTIVATION
 Inlining can improve performance
A LOT
 Things can go REALLY WRONG
 The only thing worse than no
optimization is a harmful
optimization

4. INLINING:
FUNCTION CALL -> BODY
+ eliminates the overhead of a call
+ broadens the scope for other
optimizations
- increases register pressure
- increases memory footprint
- slower compilation

5. FACTORS
 calling context of the method
 target platform*
 other optimizations
 effect of previous inlining decisions
*Java HotSpot & Maxine VM C1X compiler: static,
hard-coded heuristics, same for all platforms

6. APPROACH AND
IMPLEMENTATION
DEFAULT HEURISTICS AND NEAT

7. FEATURES
Caller and Callee
- Simple instructions count
- Method calls
- Conditional branches
- Unconditional branches
- Memory load/store
- New objects
- Other instructions
- Size (all instructions)
Calling Context
- In loop or not
- Depth of recursive call
- Depth of non-recursice
call
- Current graph size
- Synchronized or not
HotSpot Only
- Loop depth
- Block weight

8. SENSITIVITY(SPECJVM98)

9. SEARCHSPACE(SPECJVM98)

10. METHODS
Traditional supervised
learning: requires
labeled training set
Each decision affects
next: difficult to study
the effect
Unsupervised
learning: generate
solution, measure
effectiveness
METRIC: average
speedup
Large space of possible solutions -> ML

11. INLINING HEURISTIC
(HOTSPOT)
if (calleeSize > ALWAYS_INLINE_SIZE)
if (calleeSize > CALLER_MAX_SIZE)
return NO;
if (inlineDepth > MAX_INLINE_DEPTH)
return NO;
if (callWarmth > CALL_WARMTH_THRESHOLD*)
return NO;
// Passed all tests so we inline
return YES;
* # of call invocation + est. profit from
inlining + est. amount of work

12. NEAT: NEURO-EVOLUTION OF
AUGMENTING TOPOLOGIES
 Randomly generate an initial generation
of neural networks
 Evaluate the performance of each NN
 Use the best NNs to produce next
generation
 Repeat
Interpretation?

13. CREATING
HEURISTICS
Input: Features
Multi-layer NN
Output: Number
between 0 and 1
(>0.5 for inline)

14. RESULTS
BENCHMARKING INLINERS

15. DECISION TREE FOR
READABLE HEURISTICS
 The best NEAT NN -> training dataset
 Construct decision tree
 Prune
Average performance gain
NEAT 11% over default heuristic
Decision Tree 10%

16. Probability of
execution
WHATISUSEFUL?
Close to size
threshold
BUT
many memory
operations
=> INLINE
Factors
depend on
other factors

17. BENCHMARKS: ALGORITHMS

18. BENCHMARKS: FEATURES

19. INLINING:
GOOD OR EVIL?
MANUAL INLINING EXAMPLES IN F#

20. OUTSMARTING COMPILERS:
TO BE OR NOT TO BE?
Let’s take .NET world as an example:
 JIT-inlining: automatic + manual hints
(e.g. MethodImplAttribute)
 Manual inlining (e.g. inline keyword in
F#)
What could possibly go
wrong?

21. CONCLUSIONS
 Inlining is important
 NEAT and Decision Trees allow to
construct good heuristics
 Average performance improvement
over Java HotSpot VM and Maxine
VM - 11%
 Decision Trees for readable
heuristics

22. REFERENCES: PAPERS
 Automatic construction of inlining heuristics using
Machine Learning:
https://www.eecis.udel.edu/~skulkarn/papers/cgo-
2013.pdf
 Evolving Neural Networks through Augmenting
Topologies:
http://nn.cs.utexas.edu/downloads/papers/stanley.ec
02.pdf
 Meta optimization: improving compiler heuristics
with Machine
Learning:http://groups.csail.mit.edu/commit/papers/0
3/metaopt-pldi.pdf
 Adaptive online context-sensitive
inlining:http://www.research.ibm.com/people/d/dgrov
e/papers/cgo03.pdf

23. REFERENCES: .NET & JAVA
 Research papers in the .NET
source:http://mattwarren.org/2016/12/12/Research-
papers-in-the-.NET-source/
 CoreCLR
inlining:https://github.com/dotnet/coreclr/blob/maste
r/src/jit/inline.cpp
 CoreCLR
intrinsics:https://github.com/dotnet/coreclr/blob/mas
ter/src/inc/corinfo.h#L908-L965
 Java papers: https://lowlevelbits.org/java-papers/
 Scala Benchmarking project: http://scalabench.org/