Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Plan towards Ruby 3 Types

3,883 views

Published on

EuRuKo 2019, 22rd Jun. 2019
http://euruko2019.org

Published in: Technology
  • Be the first to comment

  • Be the first to like this

A Plan towards Ruby 3 Types

  1. 1. A Plan towards Ruby 3 Types Yusuke Endoh EuRuKo '19 - 22rd Jun. 2019 1
  2. 2. Yusuke Endoh (@mametter) •A full-time Ruby committer at Cookpad Inc. •with @ko1 (Ruby's speed freak) 2
  3. 3. PR: Cookpad Inc. •Mission: "Make Everyday Cooking Fun" •cookpad.com: A recipe sharing service •Monthly Average Users: 96 million 3
  4. 4. PR: Cookpad Inc. We're hiring! HQ is in Bristol, UK Aim to be No.1 in 100 countries (Now in 29 Languages and 72 Countries) 4
  5. 5. Yusuke Endoh (@mametter) •A full-time Ruby committer at Cookpad Inc. • with @ko1 (Ruby's speed freak) •My contributions: • Keyword arguments since Ruby 2.0 • Test coverage measurement • Optcarrot: a benckmark for Ruby 3x3 • Recent: Beginless/endless range: (1..), (..1) 5
  6. 6. Ruby 3 goals •Performance (Ruby 3x3) •Concurrency •Static analysis  My mission 6
  7. 7. This Talk •Matz's plan for Ruby 3 Types •An introduction of Type Profiler 7
  8. 8. Matz's plan for Ruby 3 Types 8
  9. 9. Ruby 3 Types' •Objective: Find a possible type bug •To help development •Requirement: Keep Ruby's experience •Type annotation is optional 9 # s is String def foo(s) ... end type annotation
  10. 10. Items for Ruby 3 Static Analysis 1. Standard type signature format 2. Level-1 type checker without signatures + Type inference to suggest type signature for non-annotated Ruby code 3. Level-2 type checker with signatures 10
  11. 11. 1. Type Signature Format (.rbi) •What types a method accepts/returns class Array[A] include Enumerable def []: (Integer) -> A? def each: { (A)->void } -> self ... end mix-in generics option type interface any type Proposal: ruby-signature 11
  12. 12. 2. Type Checker without Signature •Finds possible NoMethodError/TypeErrors •Work without application's type signature •May report false positive def foo(s) s.gsuub!(//, "") s + 42 end foo("foo") NoMethodError? TypeError? Proposals: • Type Profiler • mruby's JIT compiler 12
  13. 13. 2'. A kind of Type Inference •Suggest a prototype of type signature for non-annotated Ruby code • Possible approach • virtually simulating the code in type-level def foo(s) s.to_s end foo("foo") foo(42) (String | Integer) -> String ?Proposals: • Type Profiler • mruby's JIT compiler 13
  14. 14. 3. Type Checker with Signatures •Conservatively find NoMethod/TypeErrors •Verifies that the code complies with signature • Possible implementation • Gradual type checking • Tool-defined annotation def foo(s) s.gsuub!(//, "") s + 42 end TypeError! def foo: (String) -> Integer Proposals: • Steep • Sorbet • RDL NoMethodError! 14
  15. 15. Items for Ruby 3 Static Analysis Library code type signature Sorbet Steep RDL Type error warnings Application code Type Profiler (mruby JIT) Type error warnings type signature 15
  16. 16. Use Cases • I want to find a possible bug • Write a code →Use 2 (level-1 type checker without signature) • I want to verify my code • Write a code → Use 2' (type inference) → Use 3 (level-2 type checker with signature) • I want to write a code in "type-driven" style • Hand-write a signature and then a code → Use 3 (level-2 type checker with signature) 16
  17. 17. Current Development Status •ruby-signature: discussed by tool developers • Any comment is welcome: https://github.com/ruby/ruby-signature •Type-Profiler: Very experimental... •Steep: Trial use in Sider, Inc. • Good for duck typing / requires many annotations •Sorbet: Trial use in some companies • Gradually applicable / less support for duck typing 17
  18. 18. What Ruby 3 will Ship •Matz wants to bundle "type signatures" •A parser library for the format •Type signature files for stdlibs •RubyGems with type signature support •No plan to bundle the checkers •They will be released as external gems 18
  19. 19. An introduction of Type Profiler 19
  20. 20. Type Profiler for Checking •Finds NoMethodError, TypeError, etc. def foo(n) if n < 10 n.timees {|x| } end end foo(42) Type Profiler t.rb:3: [error] undefined method: Integer#timees Typo 20
  21. 21. Type Profiler for Inference •Generates a prototype of type definition def foo(n) n.to_s end foo(42) Type Profiler Object#foo :: (Integer) -> String 21
  22. 22. Demo: Overloading def my_to_s(x) x.to_s end my_to_s(42) my_to_s("STR") my_to_s(:sym) Type Profiler Object#my_to_s :: (Integer) -> String Object#my_to_s :: (String) -> String Object#my_to_s :: (Symbol) -> String 22
  23. 23. Demo: Recursive Method def fib(n) if n > 1 fib(n-1) + fib(n-2) else n end end fib(10000) Type Profiler Object#fib :: (Integer) -> Integer 23
  24. 24. How Type Profiler Does •Runs a Ruby code in "type-level" Normal interpreter def foo(n) n.to_s end foo(42) Calls w/ 42 Returns "42" Type profiler def foo(n) n.to_s end foo(42) Calls w/ Integer Returns String Object#foo :: (Integer) -> String 24
  25. 25. Type Profiler and Branch •"Forks" the execution def foo(n) if n < 10 n else "error" end end foo(42) Fork! Now here We cannot tell if n<10 or not Returns Integer Returns String Object#foo :: (Integer) -> (Integer | String) 25
  26. 26. The Problem: State Explosion •Analysis time (at April, at RubyKaigi) •Analyzing the code of Type Profiler: 10 min. •Analyzing optcarrot: 3 min. a=b=c=d=e=nil a = 42 if n < 10 b = 42 if n < 10 c = 42 if n < 10 d = 42 if n < 10 e = 42 if n < 10 Fork! Fork! Fork! Fork! Fork! 2 4 8 16 32 State numbers 26
  27. 27. The Progress since RubyKaigi •Revamped the analysis algorithm •"state merging" technique called in symbolic execution 27
  28. 28. Basic Analysis Algorithm •"Environment": A map from variable to type •Example: •TP runs each instruction iteratively •by propagating the environments •until no environments are updated 28 x = 1 x = 1 x = 1 x y { int, str } { int } x y { nil } { int } ∅ ∅ x y { nil } { int } { int } { int }
  29. 29. Example of Analysis 1: def foo(a) 2: if a < 10 3: b = 42 4: else 5: b = "str" 6: end 7: c = b 8: c 9: end 10: 11: ret = foo(42) Line# a b c 1 ∅ ∅ ∅ 2 ∅ ∅ ∅ 3 ∅ ∅ ∅ 4 - - - 5 ∅ ∅ ∅ 6 - 7 ∅ ∅ ∅ 8 ∅ ∅ ∅ Line# a b c 1 { Int } ∅ ∅ 2 ∅ ∅ ∅ 3 ∅ ∅ ∅ 4 - - - 5 ∅ ∅ ∅ 6 - 7 ∅ ∅ ∅ 8 ∅ ∅ ∅ Line# a b c 1 { Int } ∅ ∅ 2 { Int } ∅ ∅ 3 ∅ ∅ ∅ 4 - - - 5 ∅ ∅ ∅ 6 - 7 ∅ ∅ ∅ 8 ∅ ∅ ∅ Line# a b c 1 { Int } ∅ ∅ 2 { Int } ∅ ∅ 3 { Int } ∅ ∅ 4 - - - 5 { Int } ∅ ∅ 6 - 7 ∅ ∅ ∅ 8 ∅ ∅ ∅ Line# a b c 1 { Int } ∅ ∅ 2 { Int } ∅ ∅ 3 { Int } ∅ ∅ 4 - - - 5 { Int } ∅ ∅ 6 - 7 { Int } { Int } ∅ 8 ∅ ∅ ∅ Line# a b c 1 { Int } ∅ ∅ 2 { Int } ∅ ∅ 3 { Int } ∅ ∅ 4 - - - 5 { Int } ∅ ∅ 6 - 7 { Int } { Int } ∅ 8 ∅ ∅ ∅ Line# a b c 1 { Int } ∅ ∅ 2 { Int } ∅ ∅ 3 { Int } ∅ ∅ 4 - - - 5 { Int } ∅ ∅ 6 - 7 { Int } { Int, Str } ∅ 8 ∅ ∅ ∅ Line# a b c 1 { Int } ∅ ∅ 2 { Int } ∅ ∅ 3 { Int } ∅ ∅ 4 - - - 5 { Int } ∅ ∅ 6 - 7 { Int } { Int, Str } ∅ 8 ∅ ∅ ∅ Line# a b c 1 { Int } ∅ ∅ 2 { Int } ∅ ∅ 3 { Int } ∅ ∅ 4 - - - 5 { Int } ∅ ∅ 6 - 7 { Int } { Int, Str } ∅ 8 { Int } { Int, Str } { Int, Str } Object#foo :: (Int) -> (Int|Str) 29
  30. 30. Experiment result •Type Profiler: 10 min. ➔ 2.5 sec. •optcarrot: 3 min. ➔ 6 sec. 30
  31. 31. Analysis Time of Type Profiler 0 200 400 600 800 self-profiling optcarrot seconds old new 31
  32. 32. Other Problems of Type Profiler •It requires a test •False positive and false suggestion •Some Ruby features cannot be handled • e.g., Object#send, singleton classes # b: Integer or String c = b # c: Integer or String # We lose the correspond between b and c b + c # "May call Integer#+(String)!" 32
  33. 33. Development Progress: Done •Design the basic analysis algorithm •Based on abstract interpretation •Support basic language features •Variables, methods, user-defined classes, etc. •Blocks and arrays (maybe) •A limited set of built-in classes 33
  34. 34. Development Progress: To Do •Support more features •More built-in classes (Hash!) •Complex arguments (optional/rest/keyword) •Exception •Modules •Input/output type signature format •A lot of improvements for practical use... 34
  35. 35. Acknowledgement •Hideki Miura •Matz, Akr, Ko1 •PPL paper co-authors • Soutaro Matsumoto • Katsuhiro Ueno • Eijiro Sumii •Stripe team & Jeff Foster •And many people 35
  36. 36. Conclusion •Explained Matz's plan for Ruby 3 static analysis •Introduced Type Profiler • A type analyzer for Ruby 3 applicable to a non-annotated Ruby code • Based on abstract interpretation technique • Little change for Ruby programming experience •Any comments and/or contribution are welcome! • https://github.com/mame/ruby-type-profiler 36

×