Beachhead implements new opcode on CLR JIT

Beachhead implements
new opcode on CLR JIT
.NET FRINGE JAPAN 2016 KOUJI MATSUI (@KEKYO2)

2
Kouji Matsui - kekyo
• NAGOYA city, AICHI pref., JP
• Twitter – @kekyo2 / Facebook
• ux-spiral corporation
• Microsoft Most Valuable Professional VS
and DevTech 2015-
• Certified Scrum master / Scrum product
owner
• Center CLR organizer.
• .NET/C#/F#/IL/metaprogramming or
like…
• Bike rider
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI

3
You can beginning hack:
“CoreCLR”
CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI
MATSUI

4
Agenda
• Introduction / Background
• How to build coreclr/corefx
• Add custom IL opcode
• Deep-dive CLR JIT
• Verify custom IL opcode to work
• Conclusion

5
Introduction / Background
• .NET Core is open-sourced!!
• Become clearing the .NET internal implementations.
• .NET Framework noeq .NET Core, but very interesting internal
implements anythings…

6
Introduction / Background
• I am joining .NET Fringe Japan organizer teams. And thinking
what about speaks first conference…
• Roslyn and corefx already exploring and explaining any people
(in Japan) … Hmm.
• If can add custom IL opcode and build custom CLR ?
Fun, interesting and understanding internal CoreCLR ! :)

7
How to build coreclr/corefx
• Development and test bench requirements:
• Windows 10 x64
• Visual Studio 2015 Update 3 (Using C++ compiler)
• CMake 3.6.2 (Multiplatform building tool) https://cmake.org/
• Python 3.5.2 https://www.python.org/
• Official docs: “Building and running tests on Windows”
https://github.com/dotnet/coreclr/blob/master/Documentat
ion/building/windows-test-instructions.md

8
• Test summary:
1. Get source codes from GitHub dotnet/coreclr, corefx.
• git clone https://github.com/dotnet/coreclr
• git clone https://github.com/dotnet/corefx
2. Build coreclr and corefx.
• Run build.cmd both coreclr and corefx.

9
3. Test running minimum sample code using coreclr/corefx.
• Copy System.Runtime.dll and some assemblies from corefx into
coreclr.
• Compile the C# Hello world code using VS2015 C# compiler (csc.exe),
with /nostdlib /r:System.Runtime.dll and another strict options.
• Run the code, “CoreRun.exe Program.exe”
Need more informations?
see documents previous links.

10
•TIPS!!!
• Must use stable version commits for coreclr
and corefx!
• They are developing continuously and
worldwide, 10 or more commits/day.
• Hint: Look for CI status on GitHub
coreclr/corefx page.
https://github.com/dotnet/coreclr
https://github.com/dotnet/corefx

11

12
• This is just beginning :)
• Suggest first step: Very simple spec opcode:
• Opcode name: “customcode”
• No operand, no IL stack consume/produce.
• Use opcode: affect output demonstration string to Windows Debug log.
(Use Win32 API “OutputDebugStringW”)
Add custom IL opcode

13
• Debug log can capture use Sysinternals DebugView utility.
https://technet.microsoft.com/en-
us/sysinternals/debugview.aspx

14
• Thinking what currently declared opcode for completely
nothing input/output and no side-effect in CLR ?

15
• ex: Opcode “break” – Break execution the attached debugger current
position.
https://msdn.microsoft.com/en-
us/library/system.reflection.emit.opcodes.break(v=vs.110).aspx
• “Debugger break” means raise interruption native CPU (x64), such as
“DebugBreak” API or “__debugbreak” intrinsic.
• So, maybe contains invoke these API in coreclr source codes. I can find
related code easier, try using base for this opcode…

16
• Grep special-like or unique naming opcode in coreclr:
ex: “initobj”, “ldftn” etc…
--> Opcodes declared in src/inc/opcode.def by OPDEF() macro.
• Opcode break: “CEE_BREAK”
OPDEF(CEE_BREAK, "break", Pop0, Push0, InlineNone, IPrimitive, 1,
0xFF, 0x01, BREAK)
• Add “CEE_CUSTOMCODE” for last opcode “CEE_UNUSED70”’s next:
OPDEF(CEE_CUSTOMCODE, “customcode", Pop0, Push0, InlineNone,
IPrimitive, 2, 0xFE, 0x23, NEXT)
New 2 words opcode: fe,23
Instruction move hint:
“NEXT” is execute next opcode.
(Standard behavior)
No stack consume/produce

17
Deep-dive CLR JIT
•Oh, I’m just declared new opcode “customcode” now!! :)
•But this opcode used no coreclr runtime…
Require giving new opcode behavior
MANUALLY

18
Compiler-Importer JIT_CustomCode()JIT
Deep-dive CLR JIT
•How interpret opcodes in coreclr:
Assembly file:
MSIL section Parse Call OutputDebugStringW()
Parse IL opcodes
GTNODE
Call
Peek from JIT helper
function pointer table.
Internal IL stream
tree structures

19
Deep-dive CLR JIT
• Compiler-Importer (src/jit/importer.cpp) is IL opcode stream
parser use declaring CEE_* macros.
• CEE_BREAK case example:
case CEE_BREAK:
op1 = gtNewHelperCallNode(CORINFO_HELP_USER_BREAKPOINT, TYP_VOID);
goto SPILL_APPEND;
• “gtNewHelperCallNode” is construction GTNODE internal tree
structure node for invoke JIT helper function.
• “CORINFO_HELP_USER_BREAKPOINT” is JIT helper function index
symbol.

20
Deep-dive CLR JIT
• “CORINFO_HELP_USER_BREAKPOINT” declared in
src/inc/corinfo.h:
• Symbol declared in “enum CorInfoHelpFunc”. Because JIT helper
functions management by function pointer table. This table size
calculated from enum symbols count.
• And src/inc/jithelper.h:
JITHELPER(CORINFO_HELP_USER_BREAKPOINT, JIT_UserBreakpoint,
CORINFO_HELP_SIG_REG_ONLY)
REAL helper function name

21
Deep-dive CLR JIT
• Add “CORINFO_HELP_CUSTOMCODE” into CorInfoHelpFunc
and provide JIT helper function information by JITHELPER()
macro.
JITHELPER(CORINFO_HELP_CUSTOMCODE, JIT_CustomCode,
CORINFO_HELP_SIG_REG_ONLY)
• Back to Importer and add “CEE_CUSTOMCODE” handler:
case CEE_CUSTOMCODE:
op1 = gtNewHelperCallNode(CORINFO_HELP_CUSTOMCODE, TYP_VOID);
goto SPILL_APPEND;
• “TYP_VOID” is hold opcode value type (ex: operand type).
“customcode” opcode hold no value, so this ID is TYP_VOID.
• Importer done!
New JIT helper
function name

22
Deep-dive CLR JIT
• Implement JIT helper function “JIT_CustomCode”:
HCIMPL0(void, JIT_CustomCode)
{
FCALL_CONTRACT;
HELPER_METHOD_FRAME_BEGIN_NOPOLL();
::OutputDebugStringW(L"Triggered custom code!!!!!!! (for JIT)");
HELPER_METHOD_FRAME_END_POLL();
}
HCIMPLEND
• HCIMPL0(), FCALL_CONTRACT,
HELPER_METHOD_FRAME_BEGIN_NOPOLL(),
HELPER_METHOD_FRAME_END_POLL() macros are required for construct
hard-coded low level prologue/epilogue codes JIT helper functions.
• JIT helper function done!!
THIS IS CORE CODE!!

23
Deep-dive CLR JIT
• Anything done ?
• More two non-important points:
1. Implement interpreter-based code. Interpreter is
src/vm/interpreter.cpp.
• But Windows-x64 environments nothing to use interpreter, all
situation works only use JIT.
2. ILFormatter (src/utilcode/ilformatter.cpp).
• Format printer-friendly string from IL opcode. But default
implementation is printing uses CEE_* macro information, this
session’s custom code is not required.

24

25
Verify custom IL opcode to work
• OK, ready to run… How?
• The “customcode” IL opcode can work with CLR now, but how to generate
“customcode” contained assembly??
Manually paching with BINARY EDITOR…??
(; ﾟДﾟ)
This is TOP SECRET:
I fogot IMAGE_DOS_HEADER, IMAGE_FILE_HEADER, IMAGE_NT_HEADER,
IMAGE_OPTIONAL_HEADER, IMAGE_DATA_DIRECTORY, IMAGE…

26
• Thinking about more easy construction:
1. Compile standard C# sample code by .NET Core 1.0.
2. Use “ildasm” to disassembled.
3. Insert “customcode” opcode into disassembled IL source code.
4. Use custom-opcode enabled “idasm” to build new assembly.
• The “ilasm” and “ildasm” are built with coreclr. New opcodes
can handling from “opcode.def” automatically.

27
• Bootstrap test code in C#:
Generate template code from “dotnet new” command and simplied:
namespace ConsoleApplication
{
public static class Program
{
public static void Main(string[] args)
{
}
}
}
• Compile:
• dotnet restore
• dotnet build
• Storing compiled assembly: bin¥Debug¥netcoreapp1.0¥addil.dll

28
• Disassembling:
• ..¥ildasm.exe bin¥Debug¥netcoreapp1.0¥addil.dll > addil.il
• Fixed attributes for referenced System.Runtime:
.assembly extern System.Runtime
{
.publickeytoken = (B0 3F 5F 7F 11 D5 0A 3A )
.ver 4:2:0:0
}
Fix pubkey token and version similer to your
local built corefx binaries if different.

29
• Insert “customcode” opcode into Main method body:
.method public hidebysig static void Main(string[] args) cil managed
{
.entrypoint
.maxstack 8
IL_0000: nop
customcode
IL_0001: ret
}
Insert “customcode” opcode!!

30
• Assemble by customcode-enabled “ilasm”:
• ..¥ilasm.exe Program.il
Success with nothing error.
If use official ilasm.exe, will cause error:
“syntax error at token ‘ret’”

31
• Run the assembly and verify with DebugView:
• Before execute DbgView.exe
• ..¥CoreRun.exe Program.exe

32

33
Conclusion
• Custom IL opcode declare and implement:
• Declare opcode into opcode.def with OPDEF() macro.
• Declare JIT helper function into corinfo.h and jithelper.h with
JITHELPER() macro.
• Implement JIT helper function with HCIMPL() macros.
• Implement custom opcode handler into Compile-Importer.

34
Conclusion
• Verification:
• Generate IL codes from disassembled C# codes with “ildasm.”
(Or, write from scratch IL codes…)
• Using custom-opcode enabled “ilasm” to generate final binary.

35
Conclusion
• coreclr is truly OSS: Custom IL opcode can implements with
average difficulity.
• This session explain with constraints “No operand, No stack
consume/produce opcode.”
Maybe more hard work for intermediate usage opcode design…
• But YOU CAN DO THAT!!
• This session’s demonstration code:
• https://github.com/kekyo/coreclr git branch:addil

36
Thank you joining my session!!
• Become slides on slideshare and my blog entry.
http://www.kekyo.net/ (Sorry blog only Japanese language)
• My twitter account @kekyo2, follow <3
• GitHub https://github.com/kekyo/ contains:
• FusionTasks, RelaxVersioner, fscx and more…
• Open conference with “Center CLR” in Aichi
pref., Japan! Join us!!
https://www.meetup.com/en-US/CenterCLR/

Beachhead implements new opcode on CLR JIT

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (7)

Similar to Beachhead implements new opcode on CLR JIT

Similar to Beachhead implements new opcode on CLR JIT (20)

More from Kouji Matsui

More from Kouji Matsui (20)

Recently uploaded

Recently uploaded (20)

Beachhead implements new opcode on CLR JIT