MULTI-TENANCY AND RAILS House party, slumlord, or mogul? RedDotRubyConf – Singapore 22-Apr-2011
Why do I care about this? The product: 360º engagement between you and your customers  Unify and socialize support, brand management, product management, feedback & innovation Launched privately April 1 st Objective is a public SaaS launch within 6 months Many customers Customers of customers Single, cohesive (rails) application ..hence we’ve been thinking a lot about multi-tenancy ;-)  Paul Gallagher [email_address] evendis.com personal: tardate.com twitter.com/tardate
A long time ago in a galaxy far, far away…. My first encounter with “multi-tenancy” Take a single-tenant on-premise application and implement for 8 related entities (fiduciary requirement for separation) Sure, no problem! Physical separation of the app tier Shared database cluster hosting separate database instances Shared identity management service Custom perl-based provisioning system OMG the hardware 4-node db cluster (9 db instances) 9x2 app tier servers Another 4 infrastructure service nodes These days we’d call this  virtualization  at best, and  cloud-washing  otherwise! .. and they say rails can’t scale!
Three Thoughts for Today
A GENERAL MODEL
Multi-tenancy Simplest definition possible: a principle in  software architecture where a  single instance  of the software serves  multiple client organizations
Tenancy Models
Tenancy Models Social networks, LBS, Web 2.0 MSP/ASP, PaaS, IaaS Business, Enterprise 2.0, SaaS The same idea can manifest itself across the continuum – depending how you approach it
Tenancy Models Social networks, LBS, Web 2.0 MSP/ASP, PaaS, IaaS Business, Enterprise 2.0, SaaS Hardware/infrastructure challenge Application architecture challenge Explicit Rails support.. ..but our usage increasingly extends here
Tenancy Considerations
Key Concern: Data Partitioning
RE-USE / RE-INVENT
Four Data Partioning Techniques.. Instance Partitioning RBAC Partitioning Model Partitioning Schema Partitioning (just some of the many ways to solve this)
Instance Provisioning Pro Apps don’t need to be multi-tenant Apps don’t even have to be the same You can do some neat tricks with git Con $caling Management Provisioning and spin-up times The Infrastructure Engineer’s solution
RBAC Partitioning Consider tenancy as just RBAC to another degree Pro Able to address complex shared/private data requirements Con Easy to leak data with bugs and oversights Perhaps too clever by half? User Model Query scoped by permissions The “I just groked CanCan and it’s awesome” solution
RBAC Partitioning e.g. with CanCan CanCan + InheritedResources But beware .. you are still responsible for validations scoping options in forms (e.g. select lists) class Ability include CanCan::Ability def initialize(user) … # include nil so we can handle :new can :manage, Project,  :tenant_id => [user.tenant_id, nil] … end end # then we can.. Project.accessible_by(Ability.new(current_user)) class ProjectsController < InheritedResources::Base prepend_before_filter :authenticate_user! load_and_authorize_resource # that’s all folks! end Proxy data access thru one model (user/tenant)
Model Partitioning Probably the most common pattern Pro No special deployment or infra requirements Con Relies on application query logic Does not inherently prevent data leakage Scaling with performance tenant_id (or similar) present in every table .. and every query users projects The Software Architect’s solution id tenant_id name … id tenant_id name …
Model Partitioning github.com/wireframe/multitenant   Handles model aspects well Released gem (0.2.0) WIP controller support etc github.com/penguincoder/acts_as_restricted_subdomain   Comprehensive, but internals need overhaul for Rails 3 github.com/mconnell/multi_tenant   Partial implementation only Software as a Service Rails Kit railskits.com/saas/ Model partitioning: single-database scoped access Not free, but it solves other problems like recurring billing and payment gateway integration
gem install multitenant Uses dynamic default scopes to partition model access to Multitenant.current_tenant NB: belongs_to_tenant (as used in released gem 0.2.0) renamed to belongs_to_multitenant in current master class Account < ActiveRecord::Base has_many :users has_many :projects end class User < ActiveRecord::Base belongs_to :account belongs_to_tenant :account end class Project < ActiveRecord::Base belongs_to :account belongs_to_tenant :account end def belongs_to_tenant(association = :tenant) include DynamicDefaultScoping […] default_scope :scoped_to_tenant, lambda { return {} unless Multitenant.current_tenant where({reflection.primary_key_name => Multitenant.current_tenant.id}) } end I think I’d prefer this to return where('1=0')
gem install multitenant Pro Transparent enforcement unless you bypass e.g. Model.unscoped  Con Work-in-progress You must figure out how you want to set the current tenant Defaults to unscoped if tenant not set  Doesn’t address validation requirements # get a user user = User.find(1) => #<User id: 1, name: &quot;a1-u1&quot;, account_id: 1 ..> # set the current tenant Multitenant.current_tenant = user.account => #<Account id: 1, name: &quot;one” ..> # now model access is scoped Project.count => 2  # but we can bypass if we do so explicitly Project.unscoped.count => 4 # also, scoping bypassed if tenant not set Multitenant.current_tenant = nil Project.count => 4
gem install multitenant Scoping validations needs your attention – it is not automatic! class Project < ActiveRecord::Base belongs_to :account belongs_to_tenant :account validates_uniqueness_of :name, :scope => :account_id end
gem install multitenant class User < ActiveRecord::Base belongs_to :account belongs_to_tenant :account belongs_to :project end = simple_form_for [user] do |f| = f.error_messages = f.input :name = f.association :project user = User.find(1) => #<User id: 1, name: &quot;a1-u1&quot;, account_id: 1 ..> Account.find(2).projects.first.id => 3 # set the current tenant Multitenant.current_tenant = user.account Project.where(:id => 3).present? => false  # but we can still assign an inaccessible project: user.update_attributes(:project_id => 3).valid? => true # we can’t access the association user.project => nil  # but the invalid key is persisted user.project_id => 3  Since all tenant-related models are scoped, our app will display valid options to the user But what if someone sneaks in with some form-injection?
gem install multitenant Necessary to validate associations to ensure only accessible values are persisted class User < ActiveRecord::Base belongs_to :account belongs_to_tenant :account belongs_to :project   validates_each :project_id do |record,attr,value| record.errors.add attr, &quot;is invalid&quot; unless Project.where(:id => value).present? end end
Schema Partitioning Namespace access by schema Pro Complete segregation Can share selected tables (e.g. users) Con Database-specific Non-std migration See Guy Naor’s definitive presentation: confreaks.net/videos/111-aac2009-writing-multi-tenant-applications-in-rails   Model AR::Connection PUBLIC TENANT1 TENANT2 TENANT n SET search_path TO TENANT2,PUBLIC; The “Now I’m using a real database let’s see what it can do” solution
Schema Partitioning User Queries Account Customer Project -- default search path: ‘$user’,PUBLIC PUBLIC
Schema Partitioning User Queries Account Customer Project PUBLIC User Account Customer Project TENANT1 SET search_path TO TENANT1, ‘$user’, PUBLIC;
Schema Partitioning User Queries Account Customer Project PUBLIC Customer Project TENANT1 SET search_path TO TENANT1, ‘$user’, PUBLIC;
Schema Partitioning It just works.. # default search path conn=ActiveRecord::Base.connection conn.schema_search_path => &quot;\&quot;$user\&quot;,public&quot;  Project.count => 4 # but if we change the search path conn.schema_search_path = ”tenant1,public&quot;  Project.count => 2  public tenant1
Schema Partitioning But how do you setup the schemas? Can’t use rails migrations because Migrator does not qualify the schema when calling table_exists? (supplied by the postgres adapter) So the standard answer is Normal rake db:migrate to the public schema Setup any other schemas yourself
Warez: gem install vpd github.com/tardate/vpd   gemifies the schema partitioning solution “ experimental” Enables standard migrations into a selected schema To do: Abstract the solution for other databases? Migration syntax to include/exclude objects for selected schema? gem 'vpd' class ApplicationController < ActionController::Base protect_from_forgery before_filter :set_tenant def set_tenant schema_to_use = request.subdomains.last if schema_to_use.present? # now activate the schema (and make sure # it is migrated up to date) Vpd.activate(schema_to_use,true) else # ensure we are running with the default Vpd.activate_default end end end
LESSONS LEARNT AND TTD
Think about the End-user Experience My SaaS Site Tenant 1 Tenant 2 Tenant 3 Users Users Users
Think about the End-user Experience My SaaS Site Tenant 1 Tenant 2 Tenant 3 Users Users Users Less niche => more likely user groups overlap Should I have “single sign-on”? Can a user see consolidated data? How to handle different permissions/profiles?
Disambiguation Deciding how to establish the relationship between a site visitor and a tenant Sub-domain (tenant1.example.net) URL path (www.example.net/tenant1) User login (current_user.tenant) Do you need non-tenant landing pages?
Refactoring towards multi-tenancy Tenancy fundamentally shifts the goal posts Adding RBAC or model partitioning to a lovingly crafted single-tenant app built with the best test-first principles.. Great way to see your tests break en-masse! Get sucked into the “code is (probably) right but test is broken” tar pit Expect to revisit all tests .. or take another approach
TAKEAWAYS
Decide early It pays to decide early if multi-tenancy is known to be required (now/planned) Don’t be bullied into avoiding the issue! Selecting a specific multi-tenant architecture from the start can save a whole lot of pain, minimize rework, and avoid coding yourself into a corner  YAGNI! Do the simplest thing possible!
‘ Productize’ If you ever hear this in a meeting, or dream about it at night .. .. then chances are you need to be multi-tenant!
The Good News Schema partitioning is a good default solution Relatively easy to implement Most of your app can remain tenant-agnostic Offers the flexibility to selectively break tenant boundaries (& hard to do by accident) Ideal for migrating single-tenant apps
We need a ‘Rails Way’ Isn’t it about time Rails had an opinionated, out-of-the-box, solution for multi-tenancy? Perhaps schema partitioning with scoped migrations..? create_table :widgets,  scope => :all  do |t| t.string :name t.timestamps end scope :only => :instance do add_column :projects, :private_attrib, :string end scope :except => :public do add_column :projects, :private_attrib2, :string  end
Thanks!

Multi-tenancy with Rails

  • 1.
    MULTI-TENANCY AND RAILSHouse party, slumlord, or mogul? RedDotRubyConf – Singapore 22-Apr-2011
  • 2.
    Why do Icare about this? The product: 360º engagement between you and your customers Unify and socialize support, brand management, product management, feedback & innovation Launched privately April 1 st Objective is a public SaaS launch within 6 months Many customers Customers of customers Single, cohesive (rails) application ..hence we’ve been thinking a lot about multi-tenancy ;-) Paul Gallagher [email_address] evendis.com personal: tardate.com twitter.com/tardate
  • 3.
    A long timeago in a galaxy far, far away…. My first encounter with “multi-tenancy” Take a single-tenant on-premise application and implement for 8 related entities (fiduciary requirement for separation) Sure, no problem! Physical separation of the app tier Shared database cluster hosting separate database instances Shared identity management service Custom perl-based provisioning system OMG the hardware 4-node db cluster (9 db instances) 9x2 app tier servers Another 4 infrastructure service nodes These days we’d call this virtualization at best, and cloud-washing otherwise! .. and they say rails can’t scale!
  • 4.
  • 5.
  • 6.
    Multi-tenancy Simplest definitionpossible: a principle in software architecture where a single instance of the software serves multiple client organizations
  • 7.
  • 8.
    Tenancy Models Socialnetworks, LBS, Web 2.0 MSP/ASP, PaaS, IaaS Business, Enterprise 2.0, SaaS The same idea can manifest itself across the continuum – depending how you approach it
  • 9.
    Tenancy Models Socialnetworks, LBS, Web 2.0 MSP/ASP, PaaS, IaaS Business, Enterprise 2.0, SaaS Hardware/infrastructure challenge Application architecture challenge Explicit Rails support.. ..but our usage increasingly extends here
  • 10.
  • 11.
    Key Concern: DataPartitioning
  • 12.
  • 13.
    Four Data PartioningTechniques.. Instance Partitioning RBAC Partitioning Model Partitioning Schema Partitioning (just some of the many ways to solve this)
  • 14.
    Instance Provisioning ProApps don’t need to be multi-tenant Apps don’t even have to be the same You can do some neat tricks with git Con $caling Management Provisioning and spin-up times The Infrastructure Engineer’s solution
  • 15.
    RBAC Partitioning Considertenancy as just RBAC to another degree Pro Able to address complex shared/private data requirements Con Easy to leak data with bugs and oversights Perhaps too clever by half? User Model Query scoped by permissions The “I just groked CanCan and it’s awesome” solution
  • 16.
    RBAC Partitioning e.g.with CanCan CanCan + InheritedResources But beware .. you are still responsible for validations scoping options in forms (e.g. select lists) class Ability include CanCan::Ability def initialize(user) … # include nil so we can handle :new can :manage, Project, :tenant_id => [user.tenant_id, nil] … end end # then we can.. Project.accessible_by(Ability.new(current_user)) class ProjectsController < InheritedResources::Base prepend_before_filter :authenticate_user! load_and_authorize_resource # that’s all folks! end Proxy data access thru one model (user/tenant)
  • 17.
    Model Partitioning Probablythe most common pattern Pro No special deployment or infra requirements Con Relies on application query logic Does not inherently prevent data leakage Scaling with performance tenant_id (or similar) present in every table .. and every query users projects The Software Architect’s solution id tenant_id name … id tenant_id name …
  • 18.
    Model Partitioning github.com/wireframe/multitenant Handles model aspects well Released gem (0.2.0) WIP controller support etc github.com/penguincoder/acts_as_restricted_subdomain Comprehensive, but internals need overhaul for Rails 3 github.com/mconnell/multi_tenant Partial implementation only Software as a Service Rails Kit railskits.com/saas/ Model partitioning: single-database scoped access Not free, but it solves other problems like recurring billing and payment gateway integration
  • 19.
    gem install multitenantUses dynamic default scopes to partition model access to Multitenant.current_tenant NB: belongs_to_tenant (as used in released gem 0.2.0) renamed to belongs_to_multitenant in current master class Account < ActiveRecord::Base has_many :users has_many :projects end class User < ActiveRecord::Base belongs_to :account belongs_to_tenant :account end class Project < ActiveRecord::Base belongs_to :account belongs_to_tenant :account end def belongs_to_tenant(association = :tenant) include DynamicDefaultScoping […] default_scope :scoped_to_tenant, lambda { return {} unless Multitenant.current_tenant where({reflection.primary_key_name => Multitenant.current_tenant.id}) } end I think I’d prefer this to return where('1=0')
  • 20.
    gem install multitenantPro Transparent enforcement unless you bypass e.g. Model.unscoped Con Work-in-progress You must figure out how you want to set the current tenant Defaults to unscoped if tenant not set Doesn’t address validation requirements # get a user user = User.find(1) => #<User id: 1, name: &quot;a1-u1&quot;, account_id: 1 ..> # set the current tenant Multitenant.current_tenant = user.account => #<Account id: 1, name: &quot;one” ..> # now model access is scoped Project.count => 2 # but we can bypass if we do so explicitly Project.unscoped.count => 4 # also, scoping bypassed if tenant not set Multitenant.current_tenant = nil Project.count => 4
  • 21.
    gem install multitenantScoping validations needs your attention – it is not automatic! class Project < ActiveRecord::Base belongs_to :account belongs_to_tenant :account validates_uniqueness_of :name, :scope => :account_id end
  • 22.
    gem install multitenantclass User < ActiveRecord::Base belongs_to :account belongs_to_tenant :account belongs_to :project end = simple_form_for [user] do |f| = f.error_messages = f.input :name = f.association :project user = User.find(1) => #<User id: 1, name: &quot;a1-u1&quot;, account_id: 1 ..> Account.find(2).projects.first.id => 3 # set the current tenant Multitenant.current_tenant = user.account Project.where(:id => 3).present? => false # but we can still assign an inaccessible project: user.update_attributes(:project_id => 3).valid? => true # we can’t access the association user.project => nil # but the invalid key is persisted user.project_id => 3 Since all tenant-related models are scoped, our app will display valid options to the user But what if someone sneaks in with some form-injection?
  • 23.
    gem install multitenantNecessary to validate associations to ensure only accessible values are persisted class User < ActiveRecord::Base belongs_to :account belongs_to_tenant :account belongs_to :project validates_each :project_id do |record,attr,value| record.errors.add attr, &quot;is invalid&quot; unless Project.where(:id => value).present? end end
  • 24.
    Schema Partitioning Namespaceaccess by schema Pro Complete segregation Can share selected tables (e.g. users) Con Database-specific Non-std migration See Guy Naor’s definitive presentation: confreaks.net/videos/111-aac2009-writing-multi-tenant-applications-in-rails Model AR::Connection PUBLIC TENANT1 TENANT2 TENANT n SET search_path TO TENANT2,PUBLIC; The “Now I’m using a real database let’s see what it can do” solution
  • 25.
    Schema Partitioning UserQueries Account Customer Project -- default search path: ‘$user’,PUBLIC PUBLIC
  • 26.
    Schema Partitioning UserQueries Account Customer Project PUBLIC User Account Customer Project TENANT1 SET search_path TO TENANT1, ‘$user’, PUBLIC;
  • 27.
    Schema Partitioning UserQueries Account Customer Project PUBLIC Customer Project TENANT1 SET search_path TO TENANT1, ‘$user’, PUBLIC;
  • 28.
    Schema Partitioning Itjust works.. # default search path conn=ActiveRecord::Base.connection conn.schema_search_path => &quot;\&quot;$user\&quot;,public&quot; Project.count => 4 # but if we change the search path conn.schema_search_path = ”tenant1,public&quot; Project.count => 2 public tenant1
  • 29.
    Schema Partitioning Buthow do you setup the schemas? Can’t use rails migrations because Migrator does not qualify the schema when calling table_exists? (supplied by the postgres adapter) So the standard answer is Normal rake db:migrate to the public schema Setup any other schemas yourself
  • 30.
    Warez: gem installvpd github.com/tardate/vpd gemifies the schema partitioning solution “ experimental” Enables standard migrations into a selected schema To do: Abstract the solution for other databases? Migration syntax to include/exclude objects for selected schema? gem 'vpd' class ApplicationController < ActionController::Base protect_from_forgery before_filter :set_tenant def set_tenant schema_to_use = request.subdomains.last if schema_to_use.present? # now activate the schema (and make sure # it is migrated up to date) Vpd.activate(schema_to_use,true) else # ensure we are running with the default Vpd.activate_default end end end
  • 31.
  • 32.
    Think about theEnd-user Experience My SaaS Site Tenant 1 Tenant 2 Tenant 3 Users Users Users
  • 33.
    Think about theEnd-user Experience My SaaS Site Tenant 1 Tenant 2 Tenant 3 Users Users Users Less niche => more likely user groups overlap Should I have “single sign-on”? Can a user see consolidated data? How to handle different permissions/profiles?
  • 34.
    Disambiguation Deciding howto establish the relationship between a site visitor and a tenant Sub-domain (tenant1.example.net) URL path (www.example.net/tenant1) User login (current_user.tenant) Do you need non-tenant landing pages?
  • 35.
    Refactoring towards multi-tenancyTenancy fundamentally shifts the goal posts Adding RBAC or model partitioning to a lovingly crafted single-tenant app built with the best test-first principles.. Great way to see your tests break en-masse! Get sucked into the “code is (probably) right but test is broken” tar pit Expect to revisit all tests .. or take another approach
  • 36.
  • 37.
    Decide early Itpays to decide early if multi-tenancy is known to be required (now/planned) Don’t be bullied into avoiding the issue! Selecting a specific multi-tenant architecture from the start can save a whole lot of pain, minimize rework, and avoid coding yourself into a corner YAGNI! Do the simplest thing possible!
  • 38.
    ‘ Productize’ Ifyou ever hear this in a meeting, or dream about it at night .. .. then chances are you need to be multi-tenant!
  • 39.
    The Good NewsSchema partitioning is a good default solution Relatively easy to implement Most of your app can remain tenant-agnostic Offers the flexibility to selectively break tenant boundaries (& hard to do by accident) Ideal for migrating single-tenant apps
  • 40.
    We need a‘Rails Way’ Isn’t it about time Rails had an opinionated, out-of-the-box, solution for multi-tenancy? Perhaps schema partitioning with scoped migrations..? create_table :widgets, scope => :all do |t| t.string :name t.timestamps end scope :only => :instance do add_column :projects, :private_attrib, :string end scope :except => :public do add_column :projects, :private_attrib2, :string end
  • 41.

Editor's Notes

  • #12 The lower in the stack you can partition More robust and secure But greater scalability challenges
  • #15 http://www.whitelabelapps.com/