Successfully reported this slideshow.
Your SlideShare is downloading. ×

Scaling Multi-tenant Applications Using the Django ORM & Postgres | PyCon Canada 2018 | Sai Srirampur

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 48 Ad

Scaling Multi-tenant Applications Using the Django ORM & Postgres | PyCon Canada 2018 | Sai Srirampur

Download to read offline

In the real-world there are 10000s of B2B companies. Their app-stack fits the multi-tenant model - each tenant(customer) deals with it’s own data. It is super critical to build scalable applications which gives the company leeway to grow as more customers get on-boarded. Let’s learn how to do that!

In the real-world there are 10000s of B2B companies. Their app-stack fits the multi-tenant model - each tenant(customer) deals with it’s own data. It is super critical to build scalable applications which gives the company leeway to grow as more customers get on-boarded. Let’s learn how to do that!

Advertisement
Advertisement

More Related Content

Slideshows for you (19)

Similar to Scaling Multi-tenant Applications Using the Django ORM & Postgres | PyCon Canada 2018 | Sai Srirampur (20)

Advertisement

More from Citus Data (20)

Recently uploaded (20)

Advertisement

Scaling Multi-tenant Applications Using the Django ORM & Postgres | PyCon Canada 2018 | Sai Srirampur

  1. 1. Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018 Scaling Multi-Tenant Applications Using the Django ORM & Postgres Sai Srirampur PyCon Canada | Toronto | Nov 2018
  2. 2. Sai Srirampur | PyConCA 2018 • Sai Srirampur a.k.a Sai • Engineer at Citus Data • Joined Citus to make it so developers never have to worry about scaling their database • Creator django multi-tenant • Follow me @saisrirampur @citusdata
  3. 3. Sai Srirampur | PyConCA 2018
  4. 4. Sai Srirampur | PyConCA 2018 Talk about my favorite things Sai Srirampur | PyConCA 2018
  5. 5. Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018 PYTHON & DJANGO POSTGRES SCALING MULTI-TENANT APPLICATIONS
  6. 6. Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018 Why 💙 Postgres? TLDR; Open source Constraints Extensions PostGIS / Geospatial HLL, TopN, Citus Foreign data wrappers Rich SQL CTEs Window functions Full text search Datatypes JSONB
  7. 7. Sai Srirampur | PyConCA 2018 Today… Scaling Multi-Tenant Apps Using the Django ORM & Postgres 3 architectures to build Multi-tenant apps django_multitenant Scaling multi-tenant apps with distributed postgres
  8. 8. Sai Srirampur | PyConCA 2018 • Multiple customers (“tenants”) • Each with own data • SaaS • Shopify, Salesforce Multi-Tenant Apps Sai Srirampur | PyConCA 2018
  9. 9. Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018 Why talk about scalable multi-tenant applications? tens 100’s 1000’s
  10. 10. Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018 Architectures To Build Multi-Tenant Applications3
  11. 11. Sai Srirampur | PyConCA 2018 [1] One database per tenant Sai Srirampur | PyConCA 2018
  12. 12. Sai Srirampur | PyConCA 2018 What defines a Database? • Organized collection of interrelated data • Don’t share resources. • Username and password • Connections • Memory
  13. 13. Sai Srirampur | PyConCA 2018 One database per tenant Tenant 1251 Tenant 1252 Tenant 5 [1]
  14. 14. Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018 CREATE DATABASE tenant_100; ./manage.py migrate --database=tenant_100; One database per tenant / Onboarding database_routing: db_for_read, db_for_write etc.
  15. 15. Sai Srirampur | PyConCA 2018 ● Start quickly ● Isolate customer (tenant) data ● Compliance is a bit easier ● Time for DBA/developer to manage ● Maintain consistency (ex: create index across all databases) ● Longer running migrations ● Performance degrades as # customers (tenants) goes up PROS CONS [1] One Database Per Tenant Sai Srirampur | PyConCA 2018
  16. 16. Sai Srirampur | PyConCA 2018 [2] One schema per tenant Sai Srirampur | PyConCA 2018
  17. 17. Sai Srirampur | PyConCA 2018 What defines a database Schema? • Logical namespaces to hold a set of tables • Share resources: • Username and password • Connections • Memory
  18. 18. Sai Srirampur | PyConCA 2018 One schema per tenant Tenant 5 Tenant 1251 Tenant 1252 Database [2]
  19. 19. Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018 CREATE schema tenant_100; ./manage.py migrate --schema=tenant_100; app_code for schema routing based on tenant One schema per tenant / Onboarding
  20. 20. Sai Srirampur | PyConCA 2018 ● Better resource utilization vs. one database per tenant ● Start quickly ● Logical isolation ● Hard to manage (ex: add column across all schemas) ● Longer running migrations ● Performance degrades as # customers (tenants) goes up PROS CONS [2] One Schema Per Tenant Sai Srirampur | PyConCA 2018
  21. 21. Sai Srirampur | PyConCA 2018 [3] Shared table architecture Sai Srirampur | PyConCA 2018
  22. 22. Sai Srirampur | PyConCA 2018 Accounts Shared table architecture Campaigns Leads TenantId 5 5 5 1251 Database [3]
  23. 23. Sai Srirampur | PyConCA 2018 Example
  24. 24. Sai Srirampur | PyConCA 2018 ● Easy maintenance ● Faster running migrations ● Best resource utilization ● Faster performance ● Scales to 1k-100k tenants ● Application code to guarantee isolation ● Make sure ORM calls are always scoped to a single tenant PROS CONS [3] Shared Table Architecture :-) Sai Srirampur | PyConCA 2018
  25. 25. Sai Srirampur | PyConCA 2018 Comparing the 3 architectures for scaling multi-tenant Django applications ONE DATABASE PER TENANT ONE SCHEMA PER TENANT SHARED TABLE ARCHITECTURE Database Database [1] [2] [3] start quickly & isolation guarantees, bad resource utilization & not scalable better resource util, logical isolation not scalable Scales to over 100K tenants! explicitly handle isolation
  26. 26. Sai Srirampur | PyConCA 2018
  27. 27. Sai Srirampur | PyConCA 2018 django_multitenant to the rescue Sai Srirampur | PyConCA 2018
  28. 28. Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018 django_multitenant Automates all ORM calls to be scoped to a single tenant
  29. 29. Sai Srirampur | PyConCA 2018 Today... Purchase.objects.filter(id=1) <=> "SELECT* from purchase where id=1"
  30. 30. Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018 With django_multitenant Purchase.objects.filter(id=1) <=> "SELECT* from purchase where id=1 and store_id=<current_tenant>"
  31. 31. Sai Srirampur | PyConCA 2018 Components of django_multitenant Sai Srirampur | PyConCA 2018
  32. 32. Sai Srirampur | PyConCA 2018 django_multitenant usage — 3 steps 1. Inherit all models with TenantModel 2. Change ForeignKey to TenantForeignKey 3. Define tenant scoping: set_current_tenant(t)
  33. 33. Sai Srirampur | PyConCA 2018 TenantModel defines new Manager Sai Srirampur | PyConCA 2018
  34. 34. Sai Srirampur | PyConCA 2018 TenantForeignKey mimics composite foreign key behavior • Adds tenant_id filter to referenced model. Handles: • Reference lookups — ex: (Purchase.product.name) • select_related() / prefetch_related() • Explicit Joins (product__name)
  35. 35. Sai Srirampur | PyConCA 2018 set_current_tenant(t) • Specifies which tenant the APIs should be scoped to • Set at authentication logic via middleware • Set explicitly at top of function (ex. view, external tasks/jobs)
  36. 36. Sai Srirampur | PyConCA 2018 Example Code Sai Srirampur | PyConCA 2018
  37. 37. Sai Srirampur | PyConCA 2018 Models without django_multitenant class Purchase(models.Model): store = models.ForeignKey(Store) product_purchased = models.ForeignKey(Product) ordered_at = models.DateTimeField(default=timezone.now) billing_address = models.TextField() Sai Srirampur | PyConCA 2018
  38. 38. Sai Srirampur | PyConCA 2018 Post django_multitenant class Purchase(TenantModel): store = models.ForeignKey(Store) product_purchased = TenantForeignKey(Product) ordered_at = models.DateTimeField(default=timezone.now) billing_address = models.TextField() Sai Srirampur | PyConCA 2018
  39. 39. Sai Srirampur | PyConCA 2018 Setting the tenant at authentication class SetCurrentTenantFromUser(object): def process_request(self, request): if not hasattr(self, 'authenticator'): from rest_framework_jwt.authentication import JSONWebTokenAuthentication self.authenticator = JSONWebTokenAuthentication() try: user, _ = self.authenticator.authenticate(request) except: return try: #Assuming your app has a function to get the tenant associated for a user current_tenant = get_tenant_for_user(user) except: # TODO: handle failure return set_current_tenant(current_tenant) Sai Srirampur | PyConCA 2018
  40. 40. Sai Srirampur | PyConCA 2018
  41. 41. Sai Srirampur | PyConCA 2018 Benefits of django_multitenant Drop-in implementation of shared tables architecture Guarantees isolation Ready to scale with distributed Postgres (Citus)
  42. 42. Sai Srirampur | PyConCA 2018
  43. 43. Sai Srirampur | PyConCA 2018 Infinite scale with Citus
  44. 44. Sai Srirampur | PyConCA 2018 django_multitenant with Postgres
  45. 45. Sai Srirampur | PyConCA 2018 django_multitenant with Citus
  46. 46. Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018 django_multitenant with Citus
  47. 47. Sai Srirampur | PyConCA 2018 also one of my favorites: Citus
  48. 48. Scale out Django! github.com/citusdata/django-multitenant @saisrirampur @citusdata sai@citusdata.com citusdata.com/newsletter Thank You

×