Hijacking Ruby Syntax in Ruby

1. HIJACKING RUBY SYNTAX IN RUBY RubyKaigi 2018 Sendai #rubykaigi 2018/05/31 (Fri) @joker1007 & @tagomoris

2. Self-Intro Joker ▸ id: joker1007 ▸ Repro inc. CTO ▸ I’m familiar with Ruby/Rails/Fluentd/ECS/Presto. ▸ I ♥ “Jojo’s Bizarre Adventure” so much. ▸ I’m very happy to talk in Sendai (Moriocho)

3. Satoshi Tagomori (@tagomoris) Fluentd, MessagePack-Ruby, Norikra, Woothee, ... Treasure Data, Inc.

5. Agenda ▸ Ruby Features ▸ Hacks ▸ ﬁnalist, overrider, abstriker ▸ binding_ninja ▸ with_resources, deferral

6. Ruby Features: Binding Binding ▸ Context object, includes: ▸ Kernel#binding receiver ▸ local variables ▸ For template engines (?) ▸ Methods: ▸ #receiver, #eval,  #local_variables,  #local_variable_get,  #local_variable_deﬁned?,  #local_variable_set

7. テキスト

8. テキスト

9. テキスト

10. Ruby Features: Binding Binding#local_variable_set ▸ Method to ▸ add a variable only in a binding instance ▸ overwrite values of existing variables in original context

11. Ruby Features: TracePoint TracePoint ▸ Tracing events in VM ▸ For various events: ▸ :line , :raise ▸ :class , :end ▸ :call , :return , :c_call , :c_return , :b_call , :b_return ▸ :thread_begin , :thread_end , :ﬁber_switch ▸ "We can use TracePoint to gather information speciﬁcally for exceptions:" (from Ruby doc) ▸ This is COMPLETELY WRONG statement...

12. Ruby Features: TracePoint TracePoint Methods ▸ Methods: ▸ Control: disable, enable, enabled? ▸ Event what/where: event, deﬁned_class, path, lineno ▸ Method names: method_id, callee_id ▸ Event special: raised_exception, return_value ▸ And: binding ▸ So what? ▸ We can use TracePoint ▸ to gather information ▸ to overwrite everything

13. Ruby Features: TracePoint

16. テキスト 10min Break (10min)

17. Ruby Features: Refinements ▸ Reﬁnements provide a way to extend a class locally ▸ Useful use case. (Safety monkey patching) Ruby Features: Refinements

18. Another use case ▸ Super private method Ruby Features: Refinements

19. Hacks 1 Method modifiers ▸ final (https://github.com/joker1007/finalist) ▸ forbid method override ▸ override (https://github.com/joker1007/overrider) ▸ enforce method has super method ▸ abstract (https://github.com/joker1007/abstriker) ▸ enforce method override These method modifiers work when Ruby defines class. It is actually runtime, but in most case, before main logic. Hacks: method modifiers

20. Sample code: finalist Hacks: method modifiers

21. Sample code: overrider Hacks: method modifiers

22. Sample code: abstriker Hacks: method modifiers

23. How to implement method modifiers I use so many hook methods. `included`, `extended`, `method_added`, and `TracePoint`. And I use `Ripper`. In other words, I use the power of many black magics !! Hacks: method modifiers

24. Ruby Features: Method Hooks ▸ Ruby has six method hooks ▸ Module#method_added ▸ Module#method_removed ▸ Module#method_undeﬁned ▸ BasicObject#singleton_method_added ▸ BasicObject#singleton_method_removed ▸ BasicObject#singleton_method_undeﬁned Ruby Features: method hooks

25. Sample: #method_added Ruby Features: method hooks

26. Sample code: #method_removed Ruby Features: method hooks

27. Sample code: #method_undefined Ruby Features: method hooks

28. Use case in finalist (Simplified sample) Ruby Features: method hooks

29. call method_added -> call verify_ﬁnal_method

30. Usage: Method Hook ▸ Method Hook provides a way to implement declarations (like protected, private) ▸ With class inheritance, Method Hook seems magic !! ▸ But it is not enough to implement “finalist” actually ▸ Ruby has so many cases of method definition ▸ `def` or `define_method` ▸ ìnclude` module ▸ èxtend`, `prepend ▸ Each case calls different hooks Ruby Features: method hooks

31. “include” does not add method ▸ Because `include` insert module to method lookup hierarchy, it does not touch method entry table. Ruby Features: method hooks

32. “include” changes only chain of method discovering Foo Object Bar Insert module to hierarchy It is different from to add method Class#ancestors displays class-module hierarchy Ruby Features: method hooks

33. Hooks in finalist gem ▸ `method_added` to detect override by subclass ▸ `singleton_method_added` to detect override class method ▸ `included` to detect override by module include ▸ `extended` to detect override by module extend

34. And I talk about TracePoint too

35. overrider and abstriker uses TracePoint ▸ `inherited` and `included` to start `TracePoint`` call main logic call main logic Tracepoint in overrider, abstriker

36. Tracepoint in overrider, abstriker Check abstract method existence in class hierarchy

37. Why use TracePoint? ▸ In order to verify method existence at the end of class deﬁnition. ▸ Ruby interpreter needs to wait until the end of class deﬁnition to know a method absence. ▸ override and abstract cannot detect violation just when they are called. ▸ In ruby, The only way to detect the violation is `TracePoint`.

38. Advanced TracePoint: Detect particular class end Advanced Tracepoint :end event cannot trace deﬁnition by `Class.new`. Use :c_return and return_value to detect particular class end

39. Advanced TracePoint: Ripper combination Advanced Tracepoint Detect target Sexp node by TracePoint#linen Sexp node type expresses the style of method cal

40. Ripper empowers TracePoint ▸ `Ripper.sexp` returns token position ▸ And TracePoint returns lineno an event occurs. ▸ Ripper provides more information about event context. ▸ ex. power_assert gem is implemented by this combination. Advanced Tracepoint

41. Hacks 2 (binding_ninja) ▸ https://github.com/joker1007/binding_ninja ▸ It is method wrapper to pass binding of method caller implicitly. ▸ I talked about it at Rubykaigi 2017 LT. Hacks: binding_ninja

42. Sample code: binding_ninja Hacks: binding_ninja Pass binding of caller implicitly

43. How To Implement ▸ Binding is based on Ruby level control frame. ▸ When Ruby calls c-method, Ruby does not change Ruby level control frame. ▸ It is core logic of binding_ninja create binding in C level, binding keeps caller stack in Ruby level Hacks: binding_ninja

44. Use case: binding_ninja ▸ I implement scala like implicit parameter in Ruby. ▸ https://github.com/ joker1007/ implicit_parameter Hacks: binding_ninja

45. Hacks: binding_ninja Wrap method by `deﬁne_method` And wrap method by binding_ninja

46. Black Magic is dangerous actually, but it is very fun, and it extends Ruby potential These gems is for proof of concept. But these work and decent practical.

47. Ruby Quiz Break (25-30min) What the difference between: - #undef_method - #remove_method

48. RUBY QUIZ class Foo def foo class Foo def foo class Bar < Foo def foo class Bar < Foo def foo Bar.new.foo()

49. RUBY QUIZ class Foo def foo class Foo def foo class Bar < Foo def foo class Bar < Foo Bar.new.foo() undef_method(:foo) NoMethodError remove_method(:foo)

50. Hack: with_resources Add "with" Statement in Ruby ▸ Safe resource allocate/release ▸ Ensure to release resources ▸ after a lexical scope ▸ in reverse order of allocation ▸ Idioms used very frequently ▸ Other languages: ▸ Java: try-with-resources ▸ Python: with ▸ C#: using

51. Hack: with_resources Safe Resource Allocation/Release Statement in Ruby ▸ Open method with blocks ▸ File.open(path){|f| ... } ▸ Ruby way (?) ▸ More indentation ▸ Not implemented sometimes  (e.g., TCPSocket) ▸ Simple begin-ensure ▸ Anomaly cases can't be  handled

52. Hack: with_resources with_resources.gem ▸ Safe resource allocation ▸ Top level "with" ▸ via Kernel refinement ▸ Resource allocation as lambda ▸ Multi statements to allocate resources ▸ to release first resource  if second resource allocation raises exception ▸ Block to define scope for resources https://github.com/tagomoris/with_resources

53. Hack: with_resources Implementing "with" in Ruby ▸ TracePoint ▸ "b_return": pass allocated resources to block arguments ▸ "line": identify allocated resources in lambda ▸ Binding ▸ detect newly deﬁned  local variables  in allocation lambda ▸ Reﬁnements ▸ introduce "with"  in top-level without side effects https://github.com/tagomoris/with_resources

54. Hack: deferral Alternative: defer? ▸ Multi-step resource  allocation in a method ▸ using "with" ▸ Nesting! ▸ not so bad ▸ many nesting looks  a bit messy :( ▸ Alternative? ▸ "defer" in golang

55. Hack: deferral Making "defer" in Ruby ▸ Block based "defer" ▸ Should work ▸ Requires 1-level  nesting *always* ▸ Defer.start, end (+ nesting)  look too much (than golang) ▸ Abnormal case:  reassigning variables

56. Hack: deferral deferral.gem ▸ Safe resource release ▸ Top level "defer" ▸ via Kernel reﬁnements ▸ Deferred processing to release resources ▸ at the end of scope (method, block) ▸ or exception raised

57. Hack: deferral Implementing "defer" in Ruby ▸ #defer ▸ Enable TracePoint if not yet ▸ Initialize internal stack frame ▸ TracePoint ▸ Monitor method call stack ▸ Get the snapshot of local variables in defer block ▸ Call release blocks at the end of scope ▸ Binding ▸ save/restore local variables of release block ▸ Reﬁnements ▸ introduce "defer" in top-level without side effects stack level 0 stack level 1

58. The Hard Thing in Magical World: Debugging!

59. "Are You Ready? I'm OK." Blono Bucciarati Thank You! @joker1007 & @tagomoris

Hijacking Ruby Syntax in Ruby

Hijacking Ruby Syntax in Ruby

Recommended

More Related Content

What's hot

What's hot(20)

Similar to Hijacking Ruby Syntax in Ruby

Similar to Hijacking Ruby Syntax in Ruby(20)

More from SATOSHI TAGOMORI

More from SATOSHI TAGOMORI(20)

Recently uploaded

Recently uploaded(20)

Hijacking Ruby Syntax in Ruby