Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hijacking Ruby Syntax in Ruby


Published on

RubyKaigi 2018 Day 1
Joint talk by #asakusarb by @joker1007 and me.

Published in: Software
  • Be the first to comment

Hijacking Ruby Syntax in Ruby

  1. 1. HIJACKING RUBY SYNTAX IN RUBY RubyKaigi 2018 Sendai #rubykaigi 2018/05/31 (Fri) @joker1007 & @tagomoris
  2. 2. Self-Intro Joker ▸ id: joker1007 ▸ Repro inc. CTO ▸ I’m familiar with Ruby/Rails/Fluentd/ECS/Presto. ▸ I ♥ “Jojo’s Bizarre Adventure” so much. ▸ I’m very happy to talk in Sendai (Moriocho)
  3. 3. Satoshi Tagomori (@tagomoris) Fluentd, MessagePack-Ruby, Norikra, Woothee, ... Treasure Data, Inc.
  4. 4. Agenda ▸ Ruby Features ▸ Hacks ▸ finalist, overrider, abstriker ▸ binding_ninja ▸ with_resources, deferral
  5. 5. Ruby Features: Binding Binding ▸ Context object, includes: ▸ Kernel#binding receiver ▸ local variables ▸ For template engines (?) ▸ Methods: ▸ #receiver, #eval,
  6. 6. テキスト
  7. 7. テキスト
  8. 8. テキスト
  9. 9. Ruby Features: Binding Binding#local_variable_set ▸ Method to ▸ add a variable only in a binding instance ▸ overwrite values of existing variables in original context
  10. 10. Ruby Features: TracePoint TracePoint ▸ Tracing events in VM ▸ For various events: ▸ :line , :raise ▸ :class , :end ▸ :call , :return , :c_call , :c_return , :b_call , :b_return ▸ :thread_begin , :thread_end , :fiber_switch ▸ "We can use TracePoint to gather information specifically for exceptions:" (from Ruby doc) ▸ This is COMPLETELY WRONG statement...
  11. 11. Ruby Features: TracePoint TracePoint Methods ▸ Methods: ▸ Control: disable, enable, enabled? ▸ Event what/where: event, defined_class, path, lineno ▸ Method names: method_id, callee_id ▸ Event special: raised_exception, return_value ▸ And: binding ▸ So what? ▸ We can use TracePoint  ▸ to gather information ▸ to overwrite everything
  12. 12. Ruby Features: TracePoint
  13. 13. テキスト 10min Break (10min)
  14. 14. Ruby Features: Refinements ▸ Refinements provide a way to extend a class locally ▸ Useful use case. (Safety monkey patching) Ruby Features: Refinements
  15. 15. Another use case ▸ Super private method Ruby Features: Refinements
  16. 16. Hacks 1 Method modifiers ▸ final ( ▸ forbid method override ▸ override ( ▸ enforce method has super method ▸ abstract ( ▸ enforce method override These method modifiers work when Ruby defines class. It is actually runtime, but in most case, before main logic. Hacks: method modifiers
  17. 17. Sample code: finalist Hacks: method modifiers
  18. 18. Sample code: overrider Hacks: method modifiers
  19. 19. Sample code: abstriker Hacks: method modifiers
  20. 20. How to implement method modifiers I use so many hook methods. `included`, `extended`, `method_added`, and `TracePoint`. And I use `Ripper`. In other words, I use the power of many black magics !! Hacks: method modifiers
  21. 21. Ruby Features: Method Hooks ▸ Ruby has six method hooks ▸ Module#method_added ▸ Module#method_removed ▸ Module#method_undefined ▸ BasicObject#singleton_method_added ▸ BasicObject#singleton_method_removed ▸ BasicObject#singleton_method_undefined Ruby Features: method hooks
  22. 22. Sample: #method_added Ruby Features: method hooks
  23. 23. Sample code: #method_removed Ruby Features: method hooks
  24. 24. Sample code: #method_undefined Ruby Features: method hooks
  25. 25. Use case in finalist (Simplified sample) Ruby Features: method hooks
  26. 26. call method_added -> call verify_final_method
  27. 27. Usage: Method Hook ▸ Method Hook provides a way to implement declarations (like protected, private) ▸ With class inheritance, Method Hook seems magic !! ▸ But it is not enough to implement “finalist” actually ▸ Ruby has so many cases of method definition ▸ `def` or `define_method` ▸ `include` module ▸ `extend`, `prepend ▸ Each case calls different hooks Ruby Features: method hooks
  28. 28. “include” does not add method ▸ Because `include` insert module to method lookup hierarchy, it does not touch method entry table. Ruby Features: method hooks
  29. 29. “include” changes only chain of method discovering Foo Object Bar Insert module to hierarchy It is different from to add method Class#ancestors displays class-module hierarchy Ruby Features: method hooks
  30. 30. Hooks in finalist gem ▸ `method_added` to detect override by subclass ▸ `singleton_method_added` to detect override class method ▸ `included` to detect override by module include ▸ `extended` to detect override by module extend
  31. 31. And I talk about TracePoint too
  32. 32. overrider and abstriker uses TracePoint ▸ `inherited` and `included` to start `TracePoint`` call main logic call main logic Tracepoint in overrider, abstriker
  33. 33. Tracepoint in overrider, abstriker Check abstract method existence in class hierarchy
  34. 34. Why use TracePoint? ▸ In order to verify method existence at the end of class definition. ▸ Ruby interpreter needs to wait until the end of class definition to know a method absence. ▸ override and abstract cannot detect violation just when they are called. ▸ In ruby, The only way to detect the violation is `TracePoint`.
  35. 35. Advanced TracePoint: Detect particular class end Advanced Tracepoint :end event cannot trace definition by ``. Use :c_return and return_value to detect particular class end
  36. 36. Advanced TracePoint: Ripper combination Advanced Tracepoint Detect target Sexp node by TracePoint#linen Sexp node type expresses the style of method cal
  37. 37. Ripper empowers TracePoint ▸ `Ripper.sexp` returns token position ▸ And TracePoint returns lineno an event occurs. ▸ Ripper provides more information about event context. ▸ ex. power_assert gem is implemented by this combination. Advanced Tracepoint
  38. 38. Hacks 2 (binding_ninja) ▸ ▸ It is method wrapper to pass binding of method caller implicitly. ▸ I talked about it at Rubykaigi 2017 LT. Hacks: binding_ninja
  39. 39. Sample code: binding_ninja Hacks: binding_ninja Pass binding of caller implicitly
  40. 40. How To Implement ▸ Binding is based on Ruby level control frame. ▸ When Ruby calls c-method, Ruby does not change Ruby level control frame. ▸ It is core logic of binding_ninja create binding in C level, binding keeps caller stack in Ruby level Hacks: binding_ninja
  41. 41. Use case: binding_ninja ▸ I implement scala like implicit parameter in Ruby. ▸ joker1007/ implicit_parameter Hacks: binding_ninja
  42. 42. Hacks: binding_ninja Wrap method by `define_method` And wrap method by binding_ninja
  43. 43. Black Magic is dangerous actually, but it is very fun, and it extends Ruby potential These gems is for proof of concept. But these work and decent practical.
  44. 44. Ruby Quiz Break (25-30min) What the difference between: - #undef_method - #remove_method
  45. 45. RUBY QUIZ class Foo def foo class Foo def foo class Bar < Foo def foo class Bar < Foo def foo
  46. 46. RUBY QUIZ class Foo def foo class Foo def foo class Bar < Foo def foo class Bar < Foo undef_method(:foo) NoMethodError remove_method(:foo)
  47. 47. Hack: with_resources Add "with" Statement in Ruby ▸ Safe resource allocate/release ▸ Ensure to release resources ▸ after a lexical scope ▸ in reverse order of allocation ▸ Idioms used very frequently ▸ Other languages: ▸ Java: try-with-resources ▸ Python: with ▸ C#: using
  48. 48. Hack: with_resources Safe Resource Allocation/Release Statement in Ruby ▸ Open method with blocks ▸{|f| ... } ▸ Ruby way (?) ▸ More indentation ▸ Not implemented sometimes
 (e.g., TCPSocket) ▸ Simple begin-ensure ▸ Anomaly cases can't be
  49. 49. Hack: with_resources with_resources.gem ▸ Safe resource allocation ▸ Top level "with" ▸ via Kernel refinement ▸ Resource allocation as lambda ▸ Multi statements to allocate resources ▸ to release first resource
 if second resource allocation raises exception ▸ Block to define scope for resources
  50. 50. Hack: with_resources Implementing "with" in Ruby ▸ TracePoint ▸ "b_return": pass allocated resources to block arguments ▸ "line": identify allocated resources in lambda ▸ Binding ▸ detect newly defined
 local variables
 in allocation lambda ▸ Refinements ▸ introduce "with"
 in top-level without side effects
  51. 51. Hack: deferral Alternative: defer? ▸ Multi-step resource
 allocation in a method ▸ using "with" ▸ Nesting! ▸ not so bad ▸ many nesting looks
 a bit messy :( ▸ Alternative? ▸ "defer" in golang
  52. 52. Hack: deferral Making "defer" in Ruby ▸ Block based "defer" ▸ Should work ▸ Requires 1-level
 nesting *always* ▸ Defer.start, end (+ nesting)
 look too much (than golang) ▸ Abnormal case:
 reassigning variables
  53. 53. Hack: deferral deferral.gem ▸ Safe resource release ▸ Top level "defer" ▸ via Kernel refinements ▸ Deferred processing to release resources ▸ at the end of scope (method, block) ▸ or exception raised
  54. 54. Hack: deferral Implementing "defer" in Ruby ▸ #defer ▸ Enable TracePoint if not yet ▸ Initialize internal stack frame ▸ TracePoint ▸ Monitor method call stack ▸ Get the snapshot of local variables in defer block ▸ Call release blocks at the end of scope ▸ Binding ▸ save/restore local variables of release block ▸ Refinements ▸ introduce "defer" in top-level without side effects stack level 0 stack level 1
  55. 55. The Hard Thing in Magical World: Debugging!
  56. 56. "Are You Ready? I'm OK." Blono Bucciarati Thank You! @joker1007 & @tagomoris