TypeProf for IDE:
Enrich Development Experience
without Annotations
Yusuke Endoh (@mametter)
RubyKaigi Takeout 2021
Yusuke Endoh / @mametter
•A Ruby committer working at Cookpad w/ @ko1
• TypeProf: A static type analyzer for Ruby 3
•An esoteric Ruby programmer
• See my RubyKaigi bio
2
A recent contribution: error_highlight
3
json = { foo: { bar: { baz: 42 } } }
json[:foo][:barr][:baz]
# Ruby 3.0
$ ruby t.rb
t.rb:2:in `<main>': undefined method `[]' for nil:NilClass (NoMethodError)
# Ruby 3.1
$ ruby t.rb
t.rb:2:in `<main>': undefined method `[]' for nil:NilClass (NoMethodError)
json[:foo][:barr][:baz]
^^^^^^
Credit: The original author of its prototype is @yui-knk
https://github.com/ruby/ruby/pull/4470
"I love static typing" … or do you?
•Static typing is a means, not an end
•The end is the modern development experience
•Demo: TypeScript
4
The modern development experience
with TypeScript
5
On-the-fly error reporting
The modern development experience
with TypeScript
6
On-the-fly type inference
The modern development experience
with TypeScript
7
Go to definition
The modern development experience
with TypeScript
8
•Completion
Completion
The modern development experience
with TypeScript
9
Hint for arguments
"I love static typing" … or do you?
•Static typing is a means, not an end
•The end is the modern development experience
• On-the-fly error reporting, go-to-definition,
completion, argument hint, etc.
•Q: Isn't it possible to achieve the experience
without changing the Ruby language?
• My answer is "Possible!"
10
(under some conditions)
Today's topic: TypeProf for IDE
•A VSCode extension for Ruby
• Powered by TypeProf (a static type analyzer for Ruby)
•Demo
11
The modern development experience
with TypeProf for IDE
12
On-the-fly method signature
The modern development experience
with TypeProf for IDE
13
On-the-fly error reporting
The modern development experience
with TypeProf for IDE
14
The modern development experience
with TypeProf for IDE
15
On-the-fly type inference
Completion
The modern development experience
with TypeProf for IDE
16
Hint for arguments
Demo: Summary
•The modern development experience is possible
without full type annotations by TypeProf for IDE
17
Agenda
➔ Recap: Types for Ruby 3 
•TypeProf
•TypeProf for IDE
•Conclusion
18
Recap: Types for Ruby 3
19
Name What
RBS
TypeProf
Steep
Sorbet
The Ruby official type definition language
A static type analyzer for Ruby
A static type analyzer for Ruby
A static type analyzer for Ruby
RBS: Ruby official type definition language
20
class User
def initialize: (String) -> void
def name: () -> String
end
What a static type analyzer does
•Infers the types of a given program
•Reports a potential runtime type error
•Supports IDE
21
msg = "Hello, " + user.name
msg:
String
msg = "Hello, " + user.naame Error!
Differences of each static analyzer
22
Type
inference
Error
reporting
IDE
support
Analysis
speed
TypeProf 🙂Strong ☹️Weak No → Yes ☹️Slow
Steep ☹️Weak 🙂Strong Yes 🙂Fast
Agenda
•Recap: Types for Ruby 3
➔ TypeProf 
•TypeProf for IDE
•Conclusion
23
The key idea of TypeProf
Runs a Ruby code in "type-level"
Traditional interpreter
def foo(n)
n.to_s
end
foo(42)
Calls w/
42
Returns
"42"
TypeProf
def foo(n)
n.to_s
end
foo(42)
Calls w/
Integer
Returns
String
Object#foo ::
(Integer) -> String
24
The user-facing feature of TypeProf
•Ruby 3.0
• A type inference tool to create a prototype of RBS
• No IDE support
•Ruby 3.1~
• + IDE support!
25
Demo (memo)
•Test instead of type annotations
•Passing an unknown type
•Manual RBS specification
•Flow-sensitive analysis
•Manual overloading
•Block, RBSWiki example
26
Demo: Tests instead of type annotations
27
untyped
String
Demo: Passing an unknown type
28
The method signature is changed
to accept a new type
Error in the callee side
Demo: Manual RBS specification
29
Click here
A prototype RBS is
created
Demo: Manual RBS specification (cont'd)
30
Error in the caller side
# means RBS-defined
Demo: Flow-sensitive analysis
31
No error
type case
Demo: Manual overloading
32
No error
Write RBS
for overloading Argument hint is selectable
Demo: Block
33
Demo: Works with a simple Rack app
34
Agenda
•Recap: Types for Ruby 3
•TypeProf
➔TypeProf for IDE
•Conclusion
35
Language Server Protocol (LSP)
36
LSP client
(vscode)
LSP server
(TypeProf for IDE)
code changed
error found
complete "5.ti"
maybe "5.times"
5.ti|
Do you mean:
5.times
1 + "str"
1 + "str"
Is this a bug?
Currently supported LSP features
•Show method signatures
•Show potential errors
•Go to definition of methods/variables
•Find references of methods/variables
•Completion
•Show hint about method arguments
37
Not-implemented-yet LSP Features
38
Type hover
Documentation
• More challenging features
• Auto refactoring (renaming), quick fix suggestion, etc.
• Contribution is welcome!!!
Future work: the parser needs more ♥
•An AST node knows only the beginning and end
of the source code
•The parser returns nothing when syntax is broken
• Cannot complete an incomplete call
39
current implementation desired behavior
foo(k:)
About analysis performance
•TypeProf is very slow
• It traces "require 'foo'" (even if it is a gem)
•TypeProf for IDE stops the analysis in one second
• Nothing will be reported where it couldn't analyze
within the time limit
40
How to make TypeProf fast
•Write RBS for gems
• so that TypeProf don't have to analyze gem sources
• Watch pocke's talk (Day 2)
•Divide your application to components
• and write RBS against their interfaces
• We need experiment with a practical use case
41
Agenda
•Recap: Types for Ruby 3
•TypeProf
•TypeProf for IDE
•Implementation
➔Conclusion
42
Release plan
•TypeProf for IDE will be released in Ruby 3.1
• Happy if you could play with it and give us feedback
• How to run the development version
https://gist.github.com/mame/86234de6a58352b9f994e0f8a6d6fbc2 (in Japanese)
•Ready for production?
• Experimental, but hopefully works for small programs
• For large code base, please write RBS for gems first!
43
Special thanks
•Hideki Miura
•Ruby committers: matz, akr, ko1, soutaro
•Katsuhiro Ueno & Eijiro Sumii
•Stripe team & Shopify team & Jeff Foster
•Yuta Saito (@kateinoigakukun)
• Many improvements of TypeProf for IDE
44
Conclusion
•The modern development experience is possible
without full type annotations by TypeProf for IDE
•Ruby 3.1 will bundle TypeProf for IDE
45

TypeProf for IDE: Enrich Development Experience without Annotations

  • 1.
    TypeProf for IDE: EnrichDevelopment Experience without Annotations Yusuke Endoh (@mametter) RubyKaigi Takeout 2021
  • 2.
    Yusuke Endoh /@mametter •A Ruby committer working at Cookpad w/ @ko1 • TypeProf: A static type analyzer for Ruby 3 •An esoteric Ruby programmer • See my RubyKaigi bio 2
  • 3.
    A recent contribution:error_highlight 3 json = { foo: { bar: { baz: 42 } } } json[:foo][:barr][:baz] # Ruby 3.0 $ ruby t.rb t.rb:2:in `<main>': undefined method `[]' for nil:NilClass (NoMethodError) # Ruby 3.1 $ ruby t.rb t.rb:2:in `<main>': undefined method `[]' for nil:NilClass (NoMethodError) json[:foo][:barr][:baz] ^^^^^^ Credit: The original author of its prototype is @yui-knk https://github.com/ruby/ruby/pull/4470
  • 4.
    "I love statictyping" … or do you? •Static typing is a means, not an end •The end is the modern development experience •Demo: TypeScript 4
  • 5.
    The modern developmentexperience with TypeScript 5 On-the-fly error reporting
  • 6.
    The modern developmentexperience with TypeScript 6 On-the-fly type inference
  • 7.
    The modern developmentexperience with TypeScript 7 Go to definition
  • 8.
    The modern developmentexperience with TypeScript 8 •Completion Completion
  • 9.
    The modern developmentexperience with TypeScript 9 Hint for arguments
  • 10.
    "I love statictyping" … or do you? •Static typing is a means, not an end •The end is the modern development experience • On-the-fly error reporting, go-to-definition, completion, argument hint, etc. •Q: Isn't it possible to achieve the experience without changing the Ruby language? • My answer is "Possible!" 10 (under some conditions)
  • 11.
    Today's topic: TypeProffor IDE •A VSCode extension for Ruby • Powered by TypeProf (a static type analyzer for Ruby) •Demo 11
  • 12.
    The modern developmentexperience with TypeProf for IDE 12 On-the-fly method signature
  • 13.
    The modern developmentexperience with TypeProf for IDE 13 On-the-fly error reporting
  • 14.
    The modern developmentexperience with TypeProf for IDE 14
  • 15.
    The modern developmentexperience with TypeProf for IDE 15 On-the-fly type inference Completion
  • 16.
    The modern developmentexperience with TypeProf for IDE 16 Hint for arguments
  • 17.
    Demo: Summary •The moderndevelopment experience is possible without full type annotations by TypeProf for IDE 17
  • 18.
    Agenda ➔ Recap: Typesfor Ruby 3  •TypeProf •TypeProf for IDE •Conclusion 18
  • 19.
    Recap: Types forRuby 3 19 Name What RBS TypeProf Steep Sorbet The Ruby official type definition language A static type analyzer for Ruby A static type analyzer for Ruby A static type analyzer for Ruby
  • 20.
    RBS: Ruby officialtype definition language 20 class User def initialize: (String) -> void def name: () -> String end
  • 21.
    What a statictype analyzer does •Infers the types of a given program •Reports a potential runtime type error •Supports IDE 21 msg = "Hello, " + user.name msg: String msg = "Hello, " + user.naame Error!
  • 22.
    Differences of eachstatic analyzer 22 Type inference Error reporting IDE support Analysis speed TypeProf 🙂Strong ☹️Weak No → Yes ☹️Slow Steep ☹️Weak 🙂Strong Yes 🙂Fast
  • 23.
    Agenda •Recap: Types forRuby 3 ➔ TypeProf  •TypeProf for IDE •Conclusion 23
  • 24.
    The key ideaof TypeProf Runs a Ruby code in "type-level" Traditional interpreter def foo(n) n.to_s end foo(42) Calls w/ 42 Returns "42" TypeProf def foo(n) n.to_s end foo(42) Calls w/ Integer Returns String Object#foo :: (Integer) -> String 24
  • 25.
    The user-facing featureof TypeProf •Ruby 3.0 • A type inference tool to create a prototype of RBS • No IDE support •Ruby 3.1~ • + IDE support! 25
  • 26.
    Demo (memo) •Test insteadof type annotations •Passing an unknown type •Manual RBS specification •Flow-sensitive analysis •Manual overloading •Block, RBSWiki example 26
  • 27.
    Demo: Tests insteadof type annotations 27 untyped String
  • 28.
    Demo: Passing anunknown type 28 The method signature is changed to accept a new type Error in the callee side
  • 29.
    Demo: Manual RBSspecification 29 Click here A prototype RBS is created
  • 30.
    Demo: Manual RBSspecification (cont'd) 30 Error in the caller side # means RBS-defined
  • 31.
  • 32.
    Demo: Manual overloading 32 Noerror Write RBS for overloading Argument hint is selectable
  • 33.
  • 34.
    Demo: Works witha simple Rack app 34
  • 35.
    Agenda •Recap: Types forRuby 3 •TypeProf ➔TypeProf for IDE •Conclusion 35
  • 36.
    Language Server Protocol(LSP) 36 LSP client (vscode) LSP server (TypeProf for IDE) code changed error found complete "5.ti" maybe "5.times" 5.ti| Do you mean: 5.times 1 + "str" 1 + "str" Is this a bug?
  • 37.
    Currently supported LSPfeatures •Show method signatures •Show potential errors •Go to definition of methods/variables •Find references of methods/variables •Completion •Show hint about method arguments 37
  • 38.
    Not-implemented-yet LSP Features 38 Typehover Documentation • More challenging features • Auto refactoring (renaming), quick fix suggestion, etc. • Contribution is welcome!!!
  • 39.
    Future work: theparser needs more ♥ •An AST node knows only the beginning and end of the source code •The parser returns nothing when syntax is broken • Cannot complete an incomplete call 39 current implementation desired behavior foo(k:)
  • 40.
    About analysis performance •TypeProfis very slow • It traces "require 'foo'" (even if it is a gem) •TypeProf for IDE stops the analysis in one second • Nothing will be reported where it couldn't analyze within the time limit 40
  • 41.
    How to makeTypeProf fast •Write RBS for gems • so that TypeProf don't have to analyze gem sources • Watch pocke's talk (Day 2) •Divide your application to components • and write RBS against their interfaces • We need experiment with a practical use case 41
  • 42.
    Agenda •Recap: Types forRuby 3 •TypeProf •TypeProf for IDE •Implementation ➔Conclusion 42
  • 43.
    Release plan •TypeProf forIDE will be released in Ruby 3.1 • Happy if you could play with it and give us feedback • How to run the development version https://gist.github.com/mame/86234de6a58352b9f994e0f8a6d6fbc2 (in Japanese) •Ready for production? • Experimental, but hopefully works for small programs • For large code base, please write RBS for gems first! 43
  • 44.
    Special thanks •Hideki Miura •Rubycommitters: matz, akr, ko1, soutaro •Katsuhiro Ueno & Eijiro Sumii •Stripe team & Shopify team & Jeff Foster •Yuta Saito (@kateinoigakukun) • Many improvements of TypeProf for IDE 44
  • 45.
    Conclusion •The modern developmentexperience is possible without full type annotations by TypeProf for IDE •Ruby 3.1 will bundle TypeProf for IDE 45