Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hijacking Ruby Syntax in Ruby (RubyConf 2018)

#rubyconf 2018 presentation in RubyKaigi track, discussing about how to use (or abuse) Ruby's meta programming features

  • Be the first to comment

  • Be the first to like this

Hijacking Ruby Syntax in Ruby (RubyConf 2018)

  1. 1. HIJACKING RUBY SYNTAX IN RUBY RubyConf 2018 LA 2018/11/15 (Thu) @joker1007 & @tagomoris
  2. 2. Self-Intro Joker ▸ id: joker1007 ▸ Repro inc. CTO ▸ I’m familiar with Ruby/Rails/Fluentd/ECS/Presto. ▸ This talk is my first speech abroad!!
  3. 3. Satoshi Tagomori (@tagomoris) Fluentd, MessagePack-Ruby, Norikra, Woothee, ... Arm Ltd.
  4. 4. Agenda ▸ Ruby Features ▸ binding ▸ Tracepoint ▸ method hook ▸ Hacks ▸ finalist, overrider, abstriker ▸ with_resources, deferral
  5. 5. Ruby Features: Binding Binding ▸ Context object, includes: ▸ Kernel#binding receiver ▸ local variables ▸ For template engines (?) ▸ Methods: ▸ #receiver, #eval,
  6. 6. Binding creates new object per call
  7. 7. Setting variables are ignored on binding
  8. 8. Overwriting existing variable is effective!
  9. 9. Ruby Features: Binding Binding#local_variable_set ▸ Method to ▸ add a variable only in a binding instance ▸ overwrite values of existing variables in original context
  10. 10. Ruby Features: TracePoint TracePoint ▸ Tracing events in VM ▸ and call hook block ▸ For various events: ▸ :line , :raise ▸ :class , :end ▸ :call , :return , :c_call , :c_return , :b_call , :b_return ▸ :thread_begin , :thread_end , :fiber_switch ▸ "We can use TracePoint to gather information specifically for exceptions:" (from Ruby doc) ▸ This is COMPLETELY WRONG statement...
  11. 11. Ruby Features: TracePoint
  12. 12. TracePoint interrupted the argument (stringified) and return value (upcased)
  13. 13. Ruby Features: TracePoint TracePoint Methods ▸ Methods: ▸ Control: disable, enable, enabled? ▸ Event what/where: event, defined_class, path, lineno ▸ Method names: method_id, callee_id ▸ Event special: raised_exception, return_value ▸ And: binding ▸ So what? ▸ We can use TracePoint  ▸ to gather information ▸ to overwrite everything
  14. 14. テキスト Break (10min)
  15. 15. Ruby Features: Refinements ▸ Refinements provide a way to extend a class locally ▸ Useful use case. (Safety monkey patching) Ruby Features: Refinements
  16. 16. Another use case ▸ Super private method Ruby Features: Refinements
  17. 17. Ruby Features: Method Hooks ▸ Ruby has six method hooks ▸ Module#method_added ▸ Module#method_removed ▸ Module#method_undefined ▸ BasicObject#singleton_method_added ▸ BasicObject#singleton_method_removed ▸ BasicObject#singleton_method_undefined Ruby Features: method hooks
  18. 18. Sample: #method_added Ruby Features: method hooks
  19. 19. Method hook provides a way to implement method modifier.
  20. 20. Hacks: Method modifiers ▸ final ( ▸ forbid method override ▸ override ( ▸ enforce method has super method ▸ abstract ( ▸ enforce method override These method modifiers work when Ruby defines class. It is runtime, but in most case, before main logic. Hacks: method modifiers
  21. 21. Sample code: finalist Hacks: method modifiers
  22. 22. Hacks: method modifiers
  23. 23. Sample code: abstriker Hacks: method modifiers
  24. 24. How to implement method modifiers I use so many hook methods. #included, #extended, #method_added, and TracePoint. And I use Ripper. In other words, I use the power of many black magics !! Hacks: method modifiers
  25. 25. Use case of method_added in finalist Ruby Features: method hooks MyObject Finalist def method_added def verify_final_method ▸ #method_added checks violations call
  26. 26. Limitation of Method Hooks ▸ But it is not enough to implement “finalist” actually ▸ Ruby has so many cases of method definition ▸ def or #define_method ▸ #include module ▸ #extend, #prepend ▸ Each case has dedicated hooks Ruby Features: method hooks
  27. 27. #include changes only chain of method discovering Foo Object Bar Insert module to hierarchy It is different from method adding Class#ancestors displays class-module hierarchy Ruby Features: method hooks
  28. 28. Hooks in finalist gem hook method override event by method_added subclass singleton_method_added subclass class methods included included modules extended extended moduled
  29. 29. And I talk about TracePoint too
  30. 30. overrider and abstriker use TracePoint ▸ #inherited and #included to start TracePoint Tracepoint in overrider, abstriker MyObject Overrider/Abstriker def included(or inherited) TracePoint.trace(:end, :c_return, :raise) call
  31. 31. Tracepoint in overrider, abstriker TracePoint Hook Overrider self.override_methods = [ :method_a, :method_b, ] ▸ Use Method#super_method method to check method existence (ex. method(:method_a).super_method)
  32. 32. Why use TracePoint? ▸ In order to verify method existence at the end of class definition. ▸ Ruby interpreter needs to wait until the end of class definition to know a method absence. ▸ override and abstract cannot detect violation just when they are called. ▸ In ruby, The only way to detect the violation is TracePoint.
  33. 33. Advanced TracePoint: Detect particular class end Advanced Tracepoint :end event cannot trace definition by ``. Use :c_return and return_value
 to detect particular class end
  34. 34. Advanced TracePoint: Ripper combination Advanced Tracepoint ▸ Ripper: ▸ a standard library, a parser for Ruby code ▸ outputs token list and S-expression ▸ S-expression is similar to AST ▸ has token string and token position ▸ Future option: RubyVM::AbstractSyntaxTree (2.6.0 or later)
  35. 35. Ripper sample: Advanced Tracepoint
  36. 36. Advanced TracePoint: Ripper combination Advanced Tracepoint Detect target Sexp node by TracePoint#lineno Sexp node type expresses the style of method cal
  37. 37. Ripper empowers TracePoint ▸ Conclusion: ▸ TracePoint detects events and where it occurs ▸ Ripper.sexp provides how methods were called ▸ Other use case ▸ power_assert gem also uses this combination Advanced Tracepoint
  38. 38. Black Magic is dangerous actually, but it is very fun, and it extends Ruby potential These gems are proof of concepts, But these are decent practical.
  39. 39. Ruby Quiz Break (25-30min) What the difference between: - #undef_method - #remove_method
  40. 40. RUBY QUIZ class Foo def foo class Foo def foo class Bar < Foo def foo class Bar < Foo def foo
  41. 41. RUBY QUIZ class Foo def foo class Foo def foo class Bar < Foo class Bar < Foo remove_method(:foo) def foo NoMethodError undef_method(:foo)
  42. 42. Hack: with_resources Add "with" Statement to Ruby ▸ Safe resource allocate/release ▸ Ensure to release resources ▸ at the end of a lexical scope ▸ in reverse order of allocation ▸ Idioms used very frequently ▸ Other languages: ▸ Java:
 try-with-resources ▸ Python: with ▸ C#: using
  43. 43. Hack: with_resources Safe Resource Allocation/Release Statement in Ruby ▸ Open method with blocks ▸{|f| ... } ▸ Ruby way (?) ▸ More indentation ▸ Not implemented sometimes
 (e.g., TCPSocket)
  44. 44. Hack: with_resources with_resources.gem ▸ Safe resource allocation ▸ Top level #with ▸ by Kernel refinement ▸ Resource allocation as lambda ▸ Multi statements to allocate resources ▸ to release first resource
 if second resource allocation raises exception ▸ Block to define scope for resources
  45. 45. Hack: with_resources Implementing #with in Ruby ▸ TracePoint ▸ "b_return": pass allocated resources to block arguments ▸ "line": identify allocated resources in lambda ▸ Binding ▸ detect newly defined
 local variables
 in allocation lambda ▸ Refinements ▸ introduce #with
 in top-level without side effects
  46. 46. Hack: deferral Alternative: defer? ▸ Multi-step resource
 allocation in a method ▸ Nesting! w/ #with ▸ not so bad ▸ many nesting looks
 a bit messy :( ▸ Alternative? ▸ defer in golang
  47. 47. Hack: deferral Adding #defer to Ruby ▸ Block based #defer ▸ Should work ▸ Requires 1-level
 nesting *always* ▸ Defer.start, end (+ nesting)
 look too much (than golang) ▸ Abnormal case:
 reassigning variables
  48. 48. Hack: deferral deferral.gem ▸ Safe resource release ▸ Top level #defer ▸ by Kernel refinements ▸ Deferred processing to release resources ▸ at the end of scope (method, block) ▸ or exception raised
  49. 49. Hack: deferral Implementing "defer" in Ruby ▸ #defer ▸ Enable TracePoint if not yet ▸ Initialize internal stack frame ▸ TracePoint ▸ Monitor method call stack ▸ Get the snapshot of local variables in defer block ▸ Call release blocks at the end of scope ▸ Binding ▸ save/restore local variables of release block ▸ Refinements ▸ introduce #defer in top-level without side effects stack level 0 stack level 1
  50. 50. The Hard Thing in Magical World: Debugging!
  51. 51. "Give yourself to the dark side. It is the only way you can save your (Ruby)friends." - Darth vader Thank You! @joker1007 & @tagomoris