Jit builder status and directions 2018 03-28

Eclipse OMR
JitBuilder
Current status and directions 2018
Mark Stoodley
An Eclipse OMR project lead and creator of JitBuilder
March 28, 2018

Agenda
• Quick introduction to the JitBuilder library
• 2018 focus areas
1. Enable JitBuilder as a multi-language native acceleration library
2. Integrate with coming-soon JIT as a Service work at Eclipse OpenJ9
3. Address some functional gaps
• Easy ways to get involved!
2

The JitBuilder Library
• JitBuilder is a native acceleration library
• Simplified API wrapping around the OMR compiler component
• Emphasis on ease of use
• Designed for use by programs to generate native code
• Easy for humans to understand, but not meant to be a programming language
• Created for writing Just In Time (JIT) compilers
• Can be used to accelerate just about anything
• JitBuilder is already very useful, but there is still lots more to do
• Currently best supports C++ clients
• Supports a wide variety of data types, arithmetic and control flow operations
3

JitBuilder features at-a-glance
• Data Types
• Primitives: Int8, Int16, Int32, Int64, Float, Double (with Vector equivalents, e.g. VectorDouble)
• Arrays and pointers, arbitrary structs of primitive values, including arrays and pointers
• Explicit type conversion via ConvertTo / UnsignedConvertTo
• Arithmetic
• Add, Sub, Mul, Div, And, Xor, (Unsigned)Shift(L,R), some ”WithOverflow” / ”WithUnsignedOverflow” variants
• Conditional
• EqualTo, NotEqualTo, LessThan, LessOrEqualTo, GreaterThan, GreaterOrEqualTo (with Unsigned variants)
• Memory
• Load, Store, StoreOver, IndexAt, LoadAt, StoreAt, LoadIndirect, StoreIndirect, VectorLoad/Store, VectorLoad/StoreAt
• CreateLocalStruct, CreateLocalArray to allocate stack memory at compile time
• Call to call any C function directly, ComputedCall for indirect calls (via function pointer or table)
• ThunkBuilder to assemble an array of arguments to call a C function by address
• Control flow
• IfThen, IfThenElse, Switch, ForLoopUp, ForLoopDown, DoWhile, WhileDo, variants with break, continue, etc.
• Return, Goto
• IfAnd and IfOr to easily build up short-circuit conditional expressions
• BytecodeBuilders: package up key operations (as “bytecodes”) and describe how to sequence them
• VirtualMachineState modelling: optimize accesses to e.g. operand stacks
• Can use local variables / hardware registers inside compiled function but synchronize VM state variable value when needed (e.g. On callout) 4

Projects using JitBuilder
1. Lua Vermelha (JIT for Lua 5.3 implementing “all but one” opcodes) ~3,000 lines of C++ code
• https://github.com/Leonardo2718/lua-vermelha
2. JIT for LPEG runtime used by Rosie Pattern Language (all opcodes used by Rosie) ~1,200 lines of C++ code
• https://github.com/mstoodle/rosie-lpeg
3. JIT for Swift (about 1/3 of SIL opcodes) ~2,000 lines of C++ code
• To be open sourced, some details here: http://www.ustream.tv/recorded/105013815
4. Smalltalk: SOM++ runtime ~2,500 lines of C++ code
• https://github.com/charliegracie/SOMpp/tree/jitbuilder_vmstate
5. Base9 JIT: Javascript-like tutorial language ~1,000 lines of C++ code
• https://github.com/b9org/b9
6. WebAssembly JIT: U of Calgary undergraduate project (144 of 177 opcodes!) ~2,000 lines of C++ code
• https://github.com/wasmjit-omr/wasmjit-omr
7. T4 JIT: experimental Javascript implementation ~2,700 lines of C++ code
• Written by John Duimovich, not yet open sourced
8. Kaleidoscope JIT: JitBuilder JIT implementation for LLVM tutorial language ~300 lines of C++ code
• Written by Mark Stoodley, some details here: http://www.ustream.tv/recorded/105013815
9. Java matrix multiply accelerator prototype with Vector math services 58 lines of Java code
• https://ibm.box.com/v/JavaJitBuilder-x86-64
5

What’s next for JitBuilder?
6

3 Focus Areas for JitBuilder in 2018
1. Enable JitBuilder as multi-language native acceleration library
2. Integrate with JIT as a Service work happening at Eclipse OpenJ9
3. Address functional gaps via OMR and new JitBuilder API
7

1. Multi-language native acceleration library
• JitBuilder is currently most easily used by C++ projects
• Mainly because Eclipse OMR compiler component is a C++ component
• Possible to hide behind a C interface, but bulk of code still needs to be C++
8

News Flash!
Not everyone likes
writing C++ code (!)
10

1. Multi-language native acceleration library
• JitBuilder is currently most easily used by C++ projects
• Mainly because Eclipse OMR compiler component is a C++ component
• Possible to hide behind a C interface, but bulk of code still needs to be C++
• 2018 goal: sustainably enable language bindings for JitBuilder API by:
1. Separate client JitBuilder API from “public” implementation API
2. Introduce a maintainable data format to describe JitBuilder API
3. Write binding generator to create multiple language “shim"s from API
description
11

Our ultimate goal is a world where:
C++ programmers can easily write a JIT for any language in C++
C programmers can easily write a JIT for any language in C
Java programmers can easily write a JIT for any language in Java
Ruby programmers can easily write a JIT for any language in Ruby
Python programmers can easily write a JIT for any language in Python
…and so on…
Leveraging a similar JitBuilder API and all backed by Eclipse OMR
So that improvements in JitBuilder and OMR can reach Everyone
12

13
Eclipse OMR
compiler
JitBuilder Implementation classes + JitBuilder API Description
C++
JitBuilder
API
C++
library
C++
program
Auto
generated
C++ JitBuilder Library
akin to what exists today
JitBuilder
like API

14
Eclipse OMR
compiler
C++
JitBuilder
API
C
JitBuilder
API
C++
library
C
library
C++
program
C
program
Auto
generated
per language
JitBuilder library with
100% C interface
JitBuilder
like API

15
Eclipse OMR
compiler
C++
JitBuilder
API
C
JitBuilder
API
Java
JitBuilder
API
C++
library
C
library
Java
library
C++
program
C
program
Java
program
(a JVM)
Auto
generated
per language
100% Java interface
- Independent native code cache?
- Integrated code cache?
JitBuilder
like API

16
Eclipse OMR
compiler
C++
JitBuilder
API
C
JitBuilder
API
Java
JitBuilder
API
Ruby
JitBuilder
API
C++
library
C
library
Java
library
Ruby
gem
C++
program
C
program
Java
program
(a JVM)
Ruby
program
(CRuby?)
Auto
generated
per language
100% Ruby interface
Write any (even Ruby) JIT in
pure Ruby!
JitBuilder
like API

17
Eclipse OMR
compiler
C++
JitBuilder
API
C
JitBuilder
API
Java
JitBuilder
API
Ruby
JitBuilder
API
Python
JitBuilder
API
C++
library
C
library
Java
library
Ruby
gem
Python
package
C++
program
C
program
Java
program
(a JVM)
Ruby
program
(CRuby?)
Python
program
(CPython?)
Auto
generated
per language
100% Python interface
Write any (even Python) JIT in
pure Python
JitBuilder
like API

18
Eclipse OMR
compiler
C++
JitBuilder
API
C
JitBuilder
API
Java
JitBuilder
API
Ruby
JitBuilder
API
Python
JitBuilder
API
??
JitBuilder
API
…
C++
library
C
library
Java
library
Ruby
gem
Python
package
??
“module”
C++
program
C
program
Java
program
(a JVM)
Ruby
program
(CRuby?)
Python
program
(CPython?)
??
program
(?? VM)
Auto
generated
per language
The Grand JitBuilder Vision!
JitBuilder
like API

Sadly, there will be some breakage L
• JitBuilder classes right now are, in fact, OMR compiler classes
• So: names are TR::IlBuilder, TR::MethodBuilder, TR::TypeDictionary, etc.
• Probably to become either JB::IlBuilder or JitBuilder::IlBuilder
• Preferences?
• JitBuilder services are currently capitalized (e.g. Add, Sub, IfThen, etc.)
• Capital identifies it as a “client” service versus implementation service
• No longer needed once client API classes are independent of implementation
• Should these names change? (add, sub, ifthen, etc.) ?
• Please join discussion in https://github.com/eclipse/omr/issues/2397
• Semantic versioning to be introduced: https://github.com/eclipse/omr/issues/1819
• What language do you want? https://github.com/eclipse/omr/issues/2401
19

API description example: IlBuilder class
Class( /* .doc = */ ”<Doxygen text describing IlBuilder class>",
/* .name = */ IlBuilder,
/* .shortName = */ "IB",
/* .flags = */ 0,
/* .super = */ NO_SUPER,
Fields(IlBuilder,
Field("NoType", T_IlType, IS_PUBLIC | ASSIGN_AT_CREATE)
Field("Int8", T_IlType, IS_PUBLIC | ASSIGN_AT_CREATE)
Field("Float", T_IlType, IS_PUBLIC | ASSIGN_AT_CREATE)
Field("Double", T_IlType, IS_PUBLIC | ASSIGN_AT_CREATE)
Field("Address", T_IlType, IS_PUBLIC | ASSIGN_AT_CREATE)
Field("VectorInt8", T_IlType, IS_PUBLIC | ASSIGN_AT_CREATE)
Field("VectorFloat", T_IlType, IS_PUBLIC | ASSIGN_AT_CREATE)
Field("VectorDouble", T_IlType, IS_PUBLIC | ASSIGN_AT_CREATE)
Field("Word", T_IlType, IS_PUBLIC | ASSIGN_AT_CREATE)
Field("_impl", T_pointer, IS_PUBLIC )
)
…
20

Example API description for functions
Functions(IlBuilder,
…
Function(/* .doc = */ ”<Doxygen text describing Load>",
/* .name = */ Load,
/* .flags = */ IS_PUBLIC,
/* .overloadSuffix = */ "",
/* .returnType = */ T_IlValue,
/* .numParameters = */ 1,
/* .parms = */ Parms( Parm("name", T_string) ) )
Function(/* .doc = */ ”<Doxygen text describing LoadAt>",
/* .name = */ LoadAt,
/* .flags = */ IS_PUBLIC,
/* .overloadSuffix = */ "",
/* .returnType = */ T_IlValue,
/* .numParameters = */ 2,
/* .parms = */ Parms( Parm("type", T_IlType),
Parm("address", T_IlValue) )
)
…
)
) /// Class(…IlBuilder…) 21

Example (partial) language binding generator
void
CBinding::generateHeader(JBClass *clazz)
{
HeaderCFile header;
string ext(".h");
header.open(_dir, clazz->name, ext);
header.prolog(clazz->name);
header.includeFile("api/c/Common.h");
// typedefs so everyone knows about them
for ( auto it = _api.classesBegin(); it != _api.classesEnd(); ++it )
{
auto otherClass = *it;
header.typedefClass(otherClass);
}
if (clazz->hasSuper())
header.includeFile("api/c/" + clazz->super->name + ".h");
…
22

A few words about “sustainable”
• Why do this “generator” thing…sounds complicated?
• Goal: it should stay easy to add new services
• Enabling N language bindings should not always mean N times the work it is now
• With an API description, the typical work for new service should be:
• Implement the new service
• Add it to the API description, bump the version appropriately
• Re-run the generators for all existing language bindings
• Some services will require work in the binding generators too
• But we’ll try to avoid doing substantially weirder things without strong
justification
• Still that pesky “testing” challenge!
• But hopefully enablement will usually be an O(1) activity 23

2. Integrate with Jit as a Service work
• JIT as a Service == move JIT compiler out of process to a “service”
• Most of JIT compiler implementation not needed in runtime process
• Most of JIT compiler work can move to a different process, even another machine
• Much of JIT compiler footprint moves to a different process, even another machine
• Opportunity to share JIT compilation effort across multiple runtime instances
• Opportunity to identify optimization opportunities across multiple runtimes
• Some interesting service, test, and maintenance opportunities as well
• Not without challenges, of course J
• Higher latencies with added communication paths could mean slower ramp-up
• OMR compiler IL has no persistent form so IL generation can be very “chatty”
• But could be good match for low footprint runtimes that lack a JIT
24

Eclipse OpenJ9 is working on JITaaS now
• Community members at IBM Canada have a prototype implementation
• Working through some kinks
• Preparing to open source at Eclipse OpenJ9 and Eclipse OMR
• Goal: steer fundamental technology parts toward Eclipse OMR
• Establishing compiler service, communication paths, messaging mechanisms
• Rely on other open source projects where possible (e.g. gRPC, maybe flatbuffers)
• Enable more development in the open
• Open question: how can more runtimes take advantage of JIT as a Service?
25

JitBuilder to the rescue!
• OMR compiler IL (TRIL) has no fully implemented persistent form
• Very detailed and complex data structures (many of them extensible classes)
• Footprint and compile-time sensitive
• But JitBuilder API is less complicated and potentially easier to persist
• See: Provide record and replay capability for JitBuilder API at
https://github.com/eclipse/omr/issues/1818
• Define “JBIL” binary format:
• An enabler for JIT as a Service for all JitBuilder based clients
• Secondary benefit: enables OMR project testing to capture how JitBuilder
clients generate JBIL
26

Our ultimate goal is a world
where a JitBuilder client can:
Compile in-process with a statically linked compiler
Compile in-process with a dynamically linked compiler
Compile out-of-process with a remote service on same machine
Compile out-of-process with a remote service on another machine
Store JBIL and reload it to be compiled in- or out-of-process
…without modifying the client’s use of JitBuilder
27

3. Address Functional Gaps
• More work here than people I know of to resolve everything
• These items will probably get the earliest attention this year:
• Make JitBuilder work on all platforms
• https://github.com/eclipse/omr/issues/297
• Implement an InlineCall() mechanism
• Improve integration with OMR garbage collector component
• Define MethodMetaData: https://github.com/eclipse/omr/issues/2369
• Stack and register maps (also in https://github.com/eclipse/omr/issues/2369)
• Introduce a “managed heap pointer” IlType: https://github.com/eclipse/omr/issues/2399
• Improve code cache management: delete bodies and support recompilation
28

Less likely to get attention soon (help wanted!)
• Profile data interface
• Expressing type relationships
• Nop-able guards and runtime assumptions
• Full On Stack Replacement interface
• Link to all open JitBuilder issues:
• https://github.com/eclipse/omr/issues?q=is%3Aissue+is%3Aopen+label%3Aji
tbuilder
29

Want to help?
• We want to grow our user base, so please let us know what you think!
• If you’re excited by what we’re doing, please let us know
• If we’re missing the mark for you, please let us know
• If you have suggestions or new ideas, please let us know
• Interested to pick up an item that’s not currently active?
• Please register your interest at https://github.com/eclipse/omr/issues/2400
• Write down your experience trying out JitBuilder and let us know how it went!
• Tell us what language you want to use JitBuilder from at
https://github.com/eclipse/omr/issues/2401
• Even just refer this project to someone you think would be interested!
• We’ll always welcome a new face/voice and look forward to growing the OMR community
30

Feedback?
• mstoodle@ca.ibm.com
• @mstoodle
• https://github.com/eclipse/omr
• 2018 JitBuilder top level issue:
• Reach out at omr-dev mailing list:
• https://accounts.eclipse.org/mailing-list/omr-dev
31

Jit builder status and directions 2018 03-28

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Jit builder status and directions 2018 03-28

Similar to Jit builder status and directions 2018 03-28 (20)

More from Mark Stoodley

More from Mark Stoodley (6)

Recently uploaded

Recently uploaded (20)

Jit builder status and directions 2018 03-28