Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Beachhead implements new opcode on CLR JIT

13,057 views

Published on

How to implement new custom IL Opcode on .NET Core CLR.

Published in: Software
  • Be the first to comment

Beachhead implements new opcode on CLR JIT

  1. 1. Beachhead implements new opcode on CLR JIT .NET FRINGE JAPAN 2016 KOUJI MATSUI (@KEKYO2)
  2. 2. 2 Kouji Matsui - kekyo • NAGOYA city, AICHI pref., JP • Twitter – @kekyo2 / Facebook • ux-spiral corporation • Microsoft Most Valuable Professional VS and DevTech 2015- • Certified Scrum master / Scrum product owner • Center CLR organizer. • .NET/C#/F#/IL/metaprogramming or like… • Bike rider CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  3. 3. 3 You can beginning hack: “CoreCLR” CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  4. 4. 4 Agenda • Introduction / Background • How to build coreclr/corefx • Add custom IL opcode • Deep-dive CLR JIT • Verify custom IL opcode to work • Conclusion CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  5. 5. 5 Introduction / Background • .NET Core is open-sourced!! • Become clearing the .NET internal implementations. • .NET Framework noeq .NET Core, but very interesting internal implements anythings… CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  6. 6. 6 Introduction / Background • I am joining .NET Fringe Japan organizer teams. And thinking what about speaks first conference… • Roslyn and corefx already exploring and explaining any people (in Japan) … Hmm. • If can add custom IL opcode and build custom CLR ? Fun, interesting and understanding internal CoreCLR ! :) CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  7. 7. 7 How to build coreclr/corefx • Development and test bench requirements: • Windows 10 x64 • Visual Studio 2015 Update 3 (Using C++ compiler) • CMake 3.6.2 (Multiplatform building tool) https://cmake.org/ • Python 3.5.2 https://www.python.org/ • Official docs: “Building and running tests on Windows” https://github.com/dotnet/coreclr/blob/master/Documentat ion/building/windows-test-instructions.md CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  8. 8. 8 How to build coreclr/corefx • Test summary: 1. Get source codes from GitHub dotnet/coreclr, corefx. • git clone https://github.com/dotnet/coreclr • git clone https://github.com/dotnet/corefx 2. Build coreclr and corefx. • Run build.cmd both coreclr and corefx. CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  9. 9. 9 How to build coreclr/corefx 3. Test running minimum sample code using coreclr/corefx. • Copy System.Runtime.dll and some assemblies from corefx into coreclr. • Compile the C# Hello world code using VS2015 C# compiler (csc.exe), with /nostdlib /r:System.Runtime.dll and another strict options. • Run the code, “CoreRun.exe Program.exe” Need more informations? see documents previous links. CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  10. 10. 10 How to build coreclr/corefx •TIPS!!! • Must use stable version commits for coreclr and corefx! • They are developing continuously and worldwide, 10 or more commits/day. • Hint: Look for CI status on GitHub coreclr/corefx page. https://github.com/dotnet/coreclr https://github.com/dotnet/corefx CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  11. 11. 11 CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  12. 12. 12 • This is just beginning :) • Suggest first step: Very simple spec opcode: • Opcode name: “customcode” • No operand, no IL stack consume/produce. • Use opcode: affect output demonstration string to Windows Debug log. (Use Win32 API “OutputDebugStringW”) Add custom IL opcode CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  13. 13. 13 • Debug log can capture use Sysinternals DebugView utility. https://technet.microsoft.com/en- us/sysinternals/debugview.aspx Add custom IL opcode CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  14. 14. 14 Add custom IL opcode • Thinking what currently declared opcode for completely nothing input/output and no side-effect in CLR ? CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  15. 15. 15 Add custom IL opcode • ex: Opcode “break” – Break execution the attached debugger current position. https://msdn.microsoft.com/en- us/library/system.reflection.emit.opcodes.break(v=vs.110).aspx • “Debugger break” means raise interruption native CPU (x64), such as “DebugBreak” API or “__debugbreak” intrinsic. • So, maybe contains invoke these API in coreclr source codes. I can find related code easier, try using base for this opcode… CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  16. 16. 16 Add custom IL opcode • Grep special-like or unique naming opcode in coreclr: ex: “initobj”, “ldftn” etc… --> Opcodes declared in src/inc/opcode.def by OPDEF() macro. • Opcode break: “CEE_BREAK” OPDEF(CEE_BREAK, "break", Pop0, Push0, InlineNone, IPrimitive, 1, 0xFF, 0x01, BREAK) • Add “CEE_CUSTOMCODE” for last opcode “CEE_UNUSED70”’s next: OPDEF(CEE_CUSTOMCODE, “customcode", Pop0, Push0, InlineNone, IPrimitive, 2, 0xFE, 0x23, NEXT) New 2 words opcode: fe,23 Instruction move hint: “NEXT” is execute next opcode. (Standard behavior) No stack consume/produce CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  17. 17. 17 Deep-dive CLR JIT •Oh, I’m just declared new opcode “customcode” now!! :) •But this opcode used no coreclr runtime… Require giving new opcode behavior MANUALLY CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  18. 18. 18 Compiler-Importer JIT_CustomCode()JIT Deep-dive CLR JIT •How interpret opcodes in coreclr: Assembly file: MSIL section Parse Call OutputDebugStringW() Parse IL opcodes GTNODE Call Peek from JIT helper function pointer table. Internal IL stream tree structures CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  19. 19. 19 Deep-dive CLR JIT • Compiler-Importer (src/jit/importer.cpp) is IL opcode stream parser use declaring CEE_* macros. • CEE_BREAK case example: case CEE_BREAK: op1 = gtNewHelperCallNode(CORINFO_HELP_USER_BREAKPOINT, TYP_VOID); goto SPILL_APPEND; • “gtNewHelperCallNode” is construction GTNODE internal tree structure node for invoke JIT helper function. • “CORINFO_HELP_USER_BREAKPOINT” is JIT helper function index symbol. CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  20. 20. 20 Deep-dive CLR JIT • “CORINFO_HELP_USER_BREAKPOINT” declared in src/inc/corinfo.h: • Symbol declared in “enum CorInfoHelpFunc”. Because JIT helper functions management by function pointer table. This table size calculated from enum symbols count. • And src/inc/jithelper.h: JITHELPER(CORINFO_HELP_USER_BREAKPOINT, JIT_UserBreakpoint, CORINFO_HELP_SIG_REG_ONLY) REAL helper function name CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  21. 21. 21 Deep-dive CLR JIT • Add “CORINFO_HELP_CUSTOMCODE” into CorInfoHelpFunc and provide JIT helper function information by JITHELPER() macro. JITHELPER(CORINFO_HELP_CUSTOMCODE, JIT_CustomCode, CORINFO_HELP_SIG_REG_ONLY) • Back to Importer and add “CEE_CUSTOMCODE” handler: case CEE_CUSTOMCODE: op1 = gtNewHelperCallNode(CORINFO_HELP_CUSTOMCODE, TYP_VOID); goto SPILL_APPEND; • “TYP_VOID” is hold opcode value type (ex: operand type). “customcode” opcode hold no value, so this ID is TYP_VOID. • Importer done! New JIT helper function name CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  22. 22. 22 Deep-dive CLR JIT • Implement JIT helper function “JIT_CustomCode”: HCIMPL0(void, JIT_CustomCode) { FCALL_CONTRACT; HELPER_METHOD_FRAME_BEGIN_NOPOLL(); ::OutputDebugStringW(L"Triggered custom code!!!!!!! (for JIT)"); HELPER_METHOD_FRAME_END_POLL(); } HCIMPLEND • HCIMPL0(), FCALL_CONTRACT, HELPER_METHOD_FRAME_BEGIN_NOPOLL(), HELPER_METHOD_FRAME_END_POLL() macros are required for construct hard-coded low level prologue/epilogue codes JIT helper functions. • JIT helper function done!! THIS IS CORE CODE!! CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  23. 23. 23 Deep-dive CLR JIT • Anything done ? • More two non-important points: 1. Implement interpreter-based code. Interpreter is src/vm/interpreter.cpp. • But Windows-x64 environments nothing to use interpreter, all situation works only use JIT. 2. ILFormatter (src/utilcode/ilformatter.cpp). • Format printer-friendly string from IL opcode. But default implementation is printing uses CEE_* macro information, this session’s custom code is not required. CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  24. 24. 24 CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  25. 25. 25 Verify custom IL opcode to work • OK, ready to run… How? • The “customcode” IL opcode can work with CLR now, but how to generate “customcode” contained assembly?? Manually paching with BINARY EDITOR…?? (; ゚Д゚) This is TOP SECRET: I fogot IMAGE_DOS_HEADER, IMAGE_FILE_HEADER, IMAGE_NT_HEADER, IMAGE_OPTIONAL_HEADER, IMAGE_DATA_DIRECTORY, IMAGE… CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  26. 26. 26 Verify custom IL opcode to work • Thinking about more easy construction: 1. Compile standard C# sample code by .NET Core 1.0. 2. Use “ildasm” to disassembled. 3. Insert “customcode” opcode into disassembled IL source code. 4. Use custom-opcode enabled “idasm” to build new assembly. • The “ilasm” and “ildasm” are built with coreclr. New opcodes can handling from “opcode.def” automatically. CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  27. 27. 27 Verify custom IL opcode to work • Bootstrap test code in C#: Generate template code from “dotnet new” command and simplied: namespace ConsoleApplication { public static class Program { public static void Main(string[] args) { } } } • Compile: • dotnet restore • dotnet build • Storing compiled assembly: bin¥Debug¥netcoreapp1.0¥addil.dll CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  28. 28. 28 Verify custom IL opcode to work • Disassembling: • ..¥ildasm.exe bin¥Debug¥netcoreapp1.0¥addil.dll > addil.il • Fixed attributes for referenced System.Runtime: .assembly extern System.Runtime { .publickeytoken = (B0 3F 5F 7F 11 D5 0A 3A ) .ver 4:2:0:0 } Fix pubkey token and version similer to your local built corefx binaries if different. CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  29. 29. 29 Verify custom IL opcode to work • Insert “customcode” opcode into Main method body: .method public hidebysig static void Main(string[] args) cil managed { .entrypoint .maxstack 8 IL_0000: nop customcode IL_0001: ret } Insert “customcode” opcode!! CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  30. 30. 30 Verify custom IL opcode to work • Assemble by customcode-enabled “ilasm”: • ..¥ilasm.exe Program.il Success with nothing error. If use official ilasm.exe, will cause error: “syntax error at token ‘ret’” CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  31. 31. 31 Verify custom IL opcode to work • Run the assembly and verify with DebugView: • Before execute DbgView.exe • ..¥CoreRun.exe Program.exe CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  32. 32. 32 CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  33. 33. 33 Conclusion • Custom IL opcode declare and implement: • Declare opcode into opcode.def with OPDEF() macro. • Declare JIT helper function into corinfo.h and jithelper.h with JITHELPER() macro. • Implement JIT helper function with HCIMPL() macros. • Implement custom opcode handler into Compile-Importer. CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  34. 34. 34 Conclusion • Verification: • Generate IL codes from disassembled C# codes with “ildasm.” (Or, write from scratch IL codes…) • Using custom-opcode enabled “ilasm” to generate final binary. CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  35. 35. 35 Conclusion • coreclr is truly OSS: Custom IL opcode can implements with average difficulity. • This session explain with constraints “No operand, No stack consume/produce opcode.” Maybe more hard work for intermediate usage opcode design… • But YOU CAN DO THAT!! • This session’s demonstration code: • https://github.com/kekyo/coreclr git branch:addil CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI
  36. 36. 36 Thank you joining my session!! • Become slides on slideshare and my blog entry. http://www.kekyo.net/ (Sorry blog only Japanese language) • My twitter account @kekyo2, follow <3 • GitHub https://github.com/kekyo/ contains: • FusionTasks, RelaxVersioner, fscx and more… • Open conference with “Center CLR” in Aichi pref., Japan! Join us!! https://www.meetup.com/en-US/CenterCLR/ CC-BY-SA 4.0 COPYRIGHT (C) 2016 KOUJI MATSUI

×