Sorbet at Grailed
Typing a Large Rails Codebase to Ship with Confidence
About me
> $ Hi, I’m Jose Rosello!
> $ Live in Brooklyn, NY
> $ Staff Engineer at Grailed
> $ Worked on infrastructure, security, and payments over the past 4 years
About this talk
> $ About Grailed
> $ Motivation for Static Types
> $ Intro to Sorbet
> $ Rolling out Sorbet at Grailed
> $ Current Challenges
> $ Parting Thoughts
About Grailed
> $ Peer-to-peer marketplace focusing on style and fashion
> $ Grown to support 7+ million users
> $ We use Rails to power our API
> $ 30k lines of code in 2017 - 150k lines of code in 2021
> $ Typescript for our client and Next.js code
> $ Introduced Sorbet in early 2020 by an engineer who shipped a null
pointer bug and said “nevermore”
Why static typing in Ruby?
> $ Refactoring widely used interfaces was scary
Good grepping / code finding skills required
> $ Type enforcement subject to human error
Unit tests could miss it
Null pointers find a way (especially ActiveRecord ones)
Changes
> $ Stale documentation around types
# Converts the object into textual markup given a specific format.
#
# @param format [Symbol] the format type, `:text` or `:html`
# @return [String] the object converted into the expected format.
def to_format(format = :html)
# format the object
end
> $ Better enforce documentation to begin with (especially for a big
codebase!)
Legibility
Intro to Sorbet
What can Sorbet do?
> $ Sorbet will help address these (with some tradeoffs)
> $ Research is split on the overall benefit of static typing, but researchers
aren’t looking at your codebase
(not everyone agrees!)
> $ Gradual typing
> $ Sigils:
typed: ignore -- Sorbet won’t look at this file
typed: false -- (Default) Checks for missing constants and syntax errors
typed: true -- All of the above + methods and their arity and any existing signatures
typed: strict -- All methods and constants must have types declared
typed: strong -- No dynamic typing at all, all call issued within must also return typed values
> $ We aim to have true for existing files, and strict for all new code
Signature Types
# typed: strict
class PaymentTransaction < ApplicationRecord
extend T::Sig # Exposes sig and other Sorbet helpers
sig { returns(T::Boolean) } # Declare signature, return T::Boolean
def committed?
status == "success"
end
end
Sigils
# typed: true
class Photo
COMPANY = "Acme"
PHOTOGRAPHERS = {
principal: "Jenna",
backup: "Aaron",
}
def watermark
"#{COMPANY - #{PHOTOGRAPHERS[:principal]}"
end
end
# typed: strict
class Photo
extend T::Sig
COMPANY = T.let("Acme", String)
PHOTOGRAPHERS = T.let({
principal: "Jenna",
backup: "Aaron",
}, T::Hash[Symbol, String])
sig { returns(String) }
def watermark
"#{COMPANY} -
#{PHOTOGRAPHERS[:principal]}"
end
end
> $ Runtime checks
Important, since not all code is typed
We don’t treat them as exceptions (for now)
> $ RBI’s (Ruby Interface files)
Used to declare types for dependencies or dynamically generated code
Separate files
Not “real” code
Valid Ruby
RBI’s (Ruby Interface Files)
class Pry
extend ::Forwardable
extend ::Pry::Forwardable
def initialize(options = T.unsafe(nil)); end
def add_sticky_local(name, &block); end
def backtrace; end
def backtrace=(_arg0); end
def binding_stack; end
end
RBI’s (Ruby Interface Files)
module Designer::GeneratedAttributeMethods
sig { returns(T.nilable(T::Boolean)) }
def basic?; end
sig { params(value: T.nilable(T::Boolean)).void }
def basic=(value); end
sig { params(value: T.nilable(T::Boolean)).void }
def classify_enabled=(value); end
sig { returns(T::Boolean) }
def classify_enabled?; end
end
Our Favorite Sorbet Features
T.nilable to enforce nil checks
sig {params(label:
T.nilable(String)).returns(T.nilable(String))}
def label_id(label)
"#{label.upcase}-#{item_id}"
end
sig {params(label:
T.nilable(String)).returns(T.nilable(String))}
def label_id(label)
if label
"#{label.upcase}-#{item_id}"
end
end
type error, label can be nil
Exhaustiveness checking with Enum
class FeeCalculator
extend T::Sig
sig {params(fee: Fee).returns(Integer)}
def self.calculate_percentage(fee)
case fee
when Fee::None
0
when Fee::StandardFee
2
when Fee::NewMemberFee
3
else
T.absurd(fee)
end
end
end
class Fee < T::Enum
extend T::Sig
enums do
None = new
StandardFee = new
NewMemberFee = new
end
end
Sorbet will complain
if this code is reachable!
Enforce data shape with T::Struct
class Reply < T::Struct
const :valid, T::Boolean
const :title, T.nilable(String)
const :body, T.nilable(String)
const :actions, T::Array[Action], default: []
const :takeover, T.nilable(String)
end
Void return types
sig { void }
def send_messages!
shipping_address = Grailed::ShippingAddress.find_through_order(listing: @listing)
Conversations::GrailedBot::PurchaseConfirmation.call(
listing: @listing,
shipping_address: shipping_address,
shipping_label: @shipping_label,
)
end
Aid in discovery and refactoring code (like improving callsite to only take symbols and not string, or getting rid of symbol and
string altogether!)
class AutomaticAction < T::Enum
extend T::Sig
enums do
Disburse = new("disburse")
Refund = new("refund")
end
sig { returns(ActiveSupport::Duration) }
def shipment_required_period
case self
when Disburse then 5.days
when Refund then 7.days
else T.absurd(self)
end
end
end
Aid in discovery
Rolling out Sorbet
First steps
> $ Big initial PR to perform `srb init`
> $ Some things Sorbet did not like:
No dynamic imports (we used DryMonad)
Undefined constants and dead code
> $ Issues with Rails in general
Tooling
> $ sorbet-rails to create RBI’s for all the dynamic methods Rails generates
(on models, etc)
> $ Tapioca to create RBI’s for gems
> $ rubocop-sorbet to enforce sigils and other linting errors
> $ spoom for reporting and auto-bumping the sigil strictness across files if
possible
Evangelize and Enforce
> $ Finally ready to start using typed: true and typed: strict around our codebase
as code is refactored
> $ Evangelize Sorbet and encourage folks to type code as things went along
> $ Enforce it on CI
Type your Dynamic Code: Service Objects at Grailed
class SaveService
# This lets us use SaveService.call(...)
instead of SaveService.new(...).call
include ServiceObject
sig { params(sales_tax: SalesTax, order:
Order).void }
def initialize(sales_tax, order)
@sales_tax = sales_tax
@order = order
end
sig { returns(SalesTax) }
def call
# ...
end
end
module ServiceObject
def self.included(klass)
klass.extend(ClassMethods)
end
module ClassMethods
def call(*args, **kwargs)
if kwargs.present?
new(*args, **kwargs).call
else
new(*args).call
end
end
end
end
Type your Dynamic Code: Breaking Things
class SaveService
include ServiceObject
sig { params(sales_tax: SalesTax, order:
Order).void }
def initialize(sales_tax, order)
@sales_tax = sales_tax
@order = order
end
sig { returns(SalesTax) }
def call
# ...
end
sig { params(sales_tax: SalesTax, order:
Order).returns(SalesTax) }
def self.call
# ...
end
end
We broke
ServiceObject.call(...)
because we only statically declared
ServiceObject.new(...).call
Type your Dynamic Code: Parlour to the Rescue
ParlourHelper.traverse_rbi_tree(source_location) do |current_child| # source_location is our service object
if ParlourHelper.instance_method?(current_child)
if current_child.name == "initialize"
init_method = current_child
elsif current_child.name == "call"
call_method = current_child
end
end
# Creates an RBI for our class with the correct `call` class method signature
generator.root.create_class(service_object.to_s) do |klass|
klass.create_method(
"call",
class_method: true,
parameters: init_method.parameters,
return_type: call_method.return_type,
)
end
end
Removing Dynamic Code
> $ Removed dynamic imports
> $ Moved away from DryMonads as a whole - wasn’t really possible to
enforce types
Replace Result Monad with Type union
ErrorType = T.type_alias {
T.any(InvalidPermissionsError, NoListingError)
}
sig { returns(T.any(Listing, ErrorType)) }
def call
err = validate
return err if err.present?
# ...
@listing
end
def listing
result = Service.call(
listing: @listing,
)
case result
when Listing
render_json(result)
when Service::InvalidPermissionsError
render_error(...)
else
render_error(...)
end
end
Challenges We Still Face
> $ Manual refreshing of RBI’s for gems and Rails models
> $ Not confident in runtime checks yet
> $ Gradual typing still lets some type errors through
> $ Hacks for testing (RSpec ignore)
> $ Impedance mismatch with Rails
> $ Learning curve for new developers
> $ It’s very much in beta, you will almost certainly run into bugs and
limitations
Parting Words
> $ Almost 2 years after the initial rollout, we’re still very invested in Sorbet
> $ Has met our goals in preventing a whole class of type errors, has aided in
documenting our code
Grailed is Hiring!
● From 50 to 105 employees in 2021, and we expect to
continue that trend
● Excellent work/life balance, benefits, fully remote
● Engineers value empathy as much as they do good
code
● Tackling a lot of interesting problems in the peer to
peer marketplace place
● Honestly, my favorite employer in my career
See more at grailed.com/jobs or reach out to me at
jose@grailed.com
Appendix
Some papers on the impact of static typing
> $ An empirical study on the impact of static typing on software
maintainability
> $ Gradual Typing of Erlang Programs: A Wrangler Experience
> $ Unit testing isn't enough. You need static typing too.
Thank You!

Sorbet at Grailed

  • 1.
    Sorbet at Grailed Typinga Large Rails Codebase to Ship with Confidence
  • 2.
    About me > $Hi, I’m Jose Rosello! > $ Live in Brooklyn, NY > $ Staff Engineer at Grailed > $ Worked on infrastructure, security, and payments over the past 4 years
  • 3.
    About this talk >$ About Grailed > $ Motivation for Static Types > $ Intro to Sorbet > $ Rolling out Sorbet at Grailed > $ Current Challenges > $ Parting Thoughts
  • 4.
    About Grailed > $Peer-to-peer marketplace focusing on style and fashion > $ Grown to support 7+ million users > $ We use Rails to power our API > $ 30k lines of code in 2017 - 150k lines of code in 2021 > $ Typescript for our client and Next.js code > $ Introduced Sorbet in early 2020 by an engineer who shipped a null pointer bug and said “nevermore”
  • 5.
  • 6.
    > $ Refactoringwidely used interfaces was scary Good grepping / code finding skills required > $ Type enforcement subject to human error Unit tests could miss it Null pointers find a way (especially ActiveRecord ones) Changes
  • 7.
    > $ Staledocumentation around types # Converts the object into textual markup given a specific format. # # @param format [Symbol] the format type, `:text` or `:html` # @return [String] the object converted into the expected format. def to_format(format = :html) # format the object end > $ Better enforce documentation to begin with (especially for a big codebase!) Legibility
  • 8.
  • 9.
    What can Sorbetdo? > $ Sorbet will help address these (with some tradeoffs) > $ Research is split on the overall benefit of static typing, but researchers aren’t looking at your codebase (not everyone agrees!)
  • 10.
    > $ Gradualtyping > $ Sigils: typed: ignore -- Sorbet won’t look at this file typed: false -- (Default) Checks for missing constants and syntax errors typed: true -- All of the above + methods and their arity and any existing signatures typed: strict -- All methods and constants must have types declared typed: strong -- No dynamic typing at all, all call issued within must also return typed values > $ We aim to have true for existing files, and strict for all new code
  • 11.
    Signature Types # typed:strict class PaymentTransaction < ApplicationRecord extend T::Sig # Exposes sig and other Sorbet helpers sig { returns(T::Boolean) } # Declare signature, return T::Boolean def committed? status == "success" end end
  • 12.
    Sigils # typed: true classPhoto COMPANY = "Acme" PHOTOGRAPHERS = { principal: "Jenna", backup: "Aaron", } def watermark "#{COMPANY - #{PHOTOGRAPHERS[:principal]}" end end # typed: strict class Photo extend T::Sig COMPANY = T.let("Acme", String) PHOTOGRAPHERS = T.let({ principal: "Jenna", backup: "Aaron", }, T::Hash[Symbol, String]) sig { returns(String) } def watermark "#{COMPANY} - #{PHOTOGRAPHERS[:principal]}" end end
  • 13.
    > $ Runtimechecks Important, since not all code is typed We don’t treat them as exceptions (for now) > $ RBI’s (Ruby Interface files) Used to declare types for dependencies or dynamically generated code Separate files Not “real” code Valid Ruby
  • 14.
    RBI’s (Ruby InterfaceFiles) class Pry extend ::Forwardable extend ::Pry::Forwardable def initialize(options = T.unsafe(nil)); end def add_sticky_local(name, &block); end def backtrace; end def backtrace=(_arg0); end def binding_stack; end end
  • 15.
    RBI’s (Ruby InterfaceFiles) module Designer::GeneratedAttributeMethods sig { returns(T.nilable(T::Boolean)) } def basic?; end sig { params(value: T.nilable(T::Boolean)).void } def basic=(value); end sig { params(value: T.nilable(T::Boolean)).void } def classify_enabled=(value); end sig { returns(T::Boolean) } def classify_enabled?; end end
  • 16.
  • 17.
    T.nilable to enforcenil checks sig {params(label: T.nilable(String)).returns(T.nilable(String))} def label_id(label) "#{label.upcase}-#{item_id}" end sig {params(label: T.nilable(String)).returns(T.nilable(String))} def label_id(label) if label "#{label.upcase}-#{item_id}" end end type error, label can be nil
  • 18.
    Exhaustiveness checking withEnum class FeeCalculator extend T::Sig sig {params(fee: Fee).returns(Integer)} def self.calculate_percentage(fee) case fee when Fee::None 0 when Fee::StandardFee 2 when Fee::NewMemberFee 3 else T.absurd(fee) end end end class Fee < T::Enum extend T::Sig enums do None = new StandardFee = new NewMemberFee = new end end Sorbet will complain if this code is reachable!
  • 19.
    Enforce data shapewith T::Struct class Reply < T::Struct const :valid, T::Boolean const :title, T.nilable(String) const :body, T.nilable(String) const :actions, T::Array[Action], default: [] const :takeover, T.nilable(String) end
  • 20.
    Void return types sig{ void } def send_messages! shipping_address = Grailed::ShippingAddress.find_through_order(listing: @listing) Conversations::GrailedBot::PurchaseConfirmation.call( listing: @listing, shipping_address: shipping_address, shipping_label: @shipping_label, ) end
  • 21.
    Aid in discoveryand refactoring code (like improving callsite to only take symbols and not string, or getting rid of symbol and string altogether!) class AutomaticAction < T::Enum extend T::Sig enums do Disburse = new("disburse") Refund = new("refund") end sig { returns(ActiveSupport::Duration) } def shipment_required_period case self when Disburse then 5.days when Refund then 7.days else T.absurd(self) end end end Aid in discovery
  • 22.
  • 23.
    First steps > $Big initial PR to perform `srb init` > $ Some things Sorbet did not like: No dynamic imports (we used DryMonad) Undefined constants and dead code > $ Issues with Rails in general
  • 24.
    Tooling > $ sorbet-railsto create RBI’s for all the dynamic methods Rails generates (on models, etc) > $ Tapioca to create RBI’s for gems > $ rubocop-sorbet to enforce sigils and other linting errors > $ spoom for reporting and auto-bumping the sigil strictness across files if possible
  • 25.
    Evangelize and Enforce >$ Finally ready to start using typed: true and typed: strict around our codebase as code is refactored > $ Evangelize Sorbet and encourage folks to type code as things went along > $ Enforce it on CI
  • 26.
    Type your DynamicCode: Service Objects at Grailed class SaveService # This lets us use SaveService.call(...) instead of SaveService.new(...).call include ServiceObject sig { params(sales_tax: SalesTax, order: Order).void } def initialize(sales_tax, order) @sales_tax = sales_tax @order = order end sig { returns(SalesTax) } def call # ... end end module ServiceObject def self.included(klass) klass.extend(ClassMethods) end module ClassMethods def call(*args, **kwargs) if kwargs.present? new(*args, **kwargs).call else new(*args).call end end end end
  • 27.
    Type your DynamicCode: Breaking Things class SaveService include ServiceObject sig { params(sales_tax: SalesTax, order: Order).void } def initialize(sales_tax, order) @sales_tax = sales_tax @order = order end sig { returns(SalesTax) } def call # ... end sig { params(sales_tax: SalesTax, order: Order).returns(SalesTax) } def self.call # ... end end We broke ServiceObject.call(...) because we only statically declared ServiceObject.new(...).call
  • 28.
    Type your DynamicCode: Parlour to the Rescue ParlourHelper.traverse_rbi_tree(source_location) do |current_child| # source_location is our service object if ParlourHelper.instance_method?(current_child) if current_child.name == "initialize" init_method = current_child elsif current_child.name == "call" call_method = current_child end end # Creates an RBI for our class with the correct `call` class method signature generator.root.create_class(service_object.to_s) do |klass| klass.create_method( "call", class_method: true, parameters: init_method.parameters, return_type: call_method.return_type, ) end end
  • 29.
    Removing Dynamic Code >$ Removed dynamic imports > $ Moved away from DryMonads as a whole - wasn’t really possible to enforce types
  • 30.
    Replace Result Monadwith Type union ErrorType = T.type_alias { T.any(InvalidPermissionsError, NoListingError) } sig { returns(T.any(Listing, ErrorType)) } def call err = validate return err if err.present? # ... @listing end def listing result = Service.call( listing: @listing, ) case result when Listing render_json(result) when Service::InvalidPermissionsError render_error(...) else render_error(...) end end
  • 31.
    Challenges We StillFace > $ Manual refreshing of RBI’s for gems and Rails models > $ Not confident in runtime checks yet > $ Gradual typing still lets some type errors through > $ Hacks for testing (RSpec ignore) > $ Impedance mismatch with Rails > $ Learning curve for new developers > $ It’s very much in beta, you will almost certainly run into bugs and limitations
  • 32.
    Parting Words > $Almost 2 years after the initial rollout, we’re still very invested in Sorbet > $ Has met our goals in preventing a whole class of type errors, has aided in documenting our code
  • 33.
    Grailed is Hiring! ●From 50 to 105 employees in 2021, and we expect to continue that trend ● Excellent work/life balance, benefits, fully remote ● Engineers value empathy as much as they do good code ● Tackling a lot of interesting problems in the peer to peer marketplace place ● Honestly, my favorite employer in my career See more at grailed.com/jobs or reach out to me at jose@grailed.com
  • 34.
    Appendix Some papers onthe impact of static typing > $ An empirical study on the impact of static typing on software maintainability > $ Gradual Typing of Erlang Programs: A Wrangler Experience > $ Unit testing isn't enough. You need static typing too.
  • 35.