Salesforce’s Multitenant
Architecture
​ Doug Merrett
​ Principal Enterprise Architect – Asia Pacific
​ dmerrett@salesforce.com
How we do the magic we do…
Safe harbor statement under the Private Securities Litigation Reform Act of 1995:
This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties
materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed
or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-
looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any
statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new,
planned, or upgraded services or technology developments and customer contracts or use of our services.
The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new
functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our
operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any
litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our
relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our
service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger
enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in
our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter.
These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section
of our Web site.
Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available
and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features
that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.
Safe Harbor
Shared services across applications
The Customer Success Platform
APIs
2,700+ Partner Apps
Open Ecosystem
Workflow
Data &
Objects Identity
Fast App Dev & Customization
AnalyticsCollaborationMobile UI
Scalable Metadata Platform
Complete CRM
Trusted Multitenant Cloud
AnalyticsCommunityMarketingServiceSales Apps
Multitenancy
One Cloud with Many Customers
Shared Elastic Services
One Primary Data Store per Production Instance
8K+ Customers per Instance
50+ Production Instances
All data segregated by customer
All operations include Org ID
Disaster Recovery
Per Org encryption keys
What is in an Instance
Pivot
tables
Data tables
Metadata
tables
Shared Database Metadata Cache
Bulk data
processing
Multitenant aware
Query optimizer
Runtime App
Generator
Full text search
engine
Virtual Application Components
Common
Application
Screens
Tenant
Specific
Screens
Objects
(Tables)
What Multitenancy means for Salesforce R&D
No Legacy Teams
Bugs fixed for everyone
One Version
What Multitenancy means for Salesforce R&D
No Legacy Teams
Bugs fixed for everyone
One Version
260K+ of our Tests
Run your tests as well
Automation
What Multitenancy means for Salesforce R&D
No Legacy Teams
Bugs fixed for everyone
One Version
Instance Architecture
260K+ of our Tests
Run your tests as well
Automation
Staggered Releases
Scalability across all sizes
What Multitenancy means for Salesforce R&D
Three major releases per year
Bug fixes every week
Predictability
No Legacy Teams
Bugs fixed for everyone
One Version
Instance Architecture
260K+ of our Tests
Run your tests as well
Automation
Staggered Releases
Scalability across all sizes
What Multitenancy means for Salesforce R&D
Three major releases per year
Bug fixes every week
Predictability
No Legacy Teams
Bugs fixed for everyone
One Version
Instance Architecture
260K+ of our Tests
Run your tests as well
Automation
Staggered Releases
Scalability across all sizes
​ Stateless Appservers
​ Database system of record
​ No Database Definition Language (DDL) at Runtime
​ All tables partitioned by OrgId
​ Smart Primary Keys, Polymorphic Foreign Keys
​ Creative de-normalization and pivoting
​ Use every RDBMS feature & optimization
Key Architectural Principles
Metadata, data, and pivot table structures store data
corresponding to virtual data structures
The Objects table stores metadata about custom objects
(tables)‫‏‬
The Fields table stores metadata about custom fields
(columns)‫‏‬
The Data heap table stores all structured data corresponding
to custom objects
A single slot can store various types of data that originate
from different objects
The Indexes pivot table manages tenant-specific selective
indexes
The UniqueFields pivot table facilitates uniqueness for
custom fields
The Relationships pivot table facilitates referential integrity
and optimizes joins
​ Tables hash partitioned by OrgId
​ Separate connection pools point to physical hosts
​ App tier is also dynamically partitioned by OrgId
​ Distributed metadata cache with transactional invalidation
All data & metadata structures are partitioned to improve
performance and manageability
§  Native Declarative features
§  Bulk Processing
§  The Recycle Bin
§  Full Text Search
§  Smart Bulk Data Manipulation Language (DML)
§  Web Services APIs
Application Framework: a whole lot for free
Force.com’s native Application Framework provides
declarative development, no coding
Validation rules and simple formulas:
Business analysts can “code” these
Not so simple: Rollup-summary fields provide for easy cross-
object summaries
Force.com’s bulk processing optimizations reduce overhead
for data loads
​ Examples:
§  Sort all records by primary key before attempting DML
§  Operate on tables in deterministic order
§  Slot reallocation for field datatype change
§  Deferred calculation for new rollup-summary field
§  Background processing of mass changes
Data definition processing is optimized to avoid performance
hits or concurrency limits
The Recycle Bin: Smart Undeletes
Restore
§  Individual object instances
(records)
§  Related object instances (parent/
child records)
§  Entire fields and objects
(dropped columns and tables)
Secondary Instance
Multitenant Search, anything but simple
Index Backup
Replication
Primary Instance
Multitenancy delivers Blazing Performance
Transactions Per Quarter
234B Transactions in Q2FY16
79% YoY Growth
Average Page Time
208ms Latency in Q2FY16
4% YoY Improvement
API
Web
​ Consistent SQL generation across the application
​ Deep awareness of pivot table structure
•  Flex schema does impose a cost
​ Tenant, user, object, fields statistics are crucial
​ No runaway queries allowed
​ Deep integration with the sharing model
Multitenant Query Optimization Principles
Multitenant Query Optimizer
Check user
visibility
Check filer
selectivity
Dynamically
write query
based on pre-
queries
Run Pre-Queries
Execute
optimized
query
user visibility = number of rows user can access
filter selectivity = index corresponding to filter column
Search originates
from API or global
search
return results
The optimizer considers pre-query selectivity measurements
when writing a query
Pre-Query
Selectivity
Measurements
… use of index related to filter.HighHigh
… ordered hash join; drive using Data table.LowHigh
… use of index related to filter.HighLow
… nested loops join; drive using view of rows that the user can see.LowLow
Write final database access query, forcing …
FilterUser
Highly Scalable Multitenant
Architecture
Service isolation and redundant design ensures availability
Highly Scalable Multitenant Architecture
•  Instances
–  Consistent deployment size
–  Easy to scale
–  Repeatable
•  Instance Groups
–  Shared service isolation
within a Data Center
•  Data Centers
–  Consistent for Production
and Disaster Recovery
–  Shared services across data
centers
Data Centre
Shared Services
Instance Group
Shared
Services
Sandbox DR
Instances
Sandbox
Instances
Core DR
Instances
Core
Instances
Instance Group
Shared
Services
Sandbox DR
Instances
Sandbox
Instances
Core DR
Instances
Core
Instances
Data Centre
Shared Services
Instance Group
Shared
Services
Sandbox DR
Instances
Sandbox
Instances
Core DR
Instances
Core
Instances
Instance Group
Shared
Services
Sandbox DR
Instances
Sandbox
Instances
Core DR
Instances
Core
Instances
​ Production cluster capacity
•  7+1 (active + spare) nodes
•  Active Data Guard to local standby
(2 node cluster) for fast data
recovery
​ Data replication to secondary
instance
•  Encrypted block-level replication
•  Enables site switching for disaster
recovery
Redundant Site Design with 4 Online Copies of Data
Today: Reliable Core by Design
Encrypted
Async
ReplicationData Guard
Replication
Primary Instance
Application
Servers
Production
DB Cluster
Standby
DB Cluster
Data Guard
Replication
Application
Servers
Production
DB Cluster
Standby
DB Cluster
Secondary Instance
​ 6+2 Production RAC Cluster
•  2 spare nodes provides additional
DB cluster resilience
​ Data Guard to remote site
•  Hardens remote replication
•  Enables faster site switching
•  Shorter planned and unplanned
maintenance windows
Faster Site Switches
Tomorrow: Enhancing Core Reliability
Encrypted
Async DB
Replication
Data Guard
Replication
Primary Instance
Application
Servers
Production
DB Cluster
Standby
DB Cluster
Data Guard
Replication
Application
Servers
Production
DB Cluster
Standby
DB Cluster
Secondary Instance
4 key methods to enable performance at scale
Adaptive Capacity Planning
​ Horizontally
​ Scaling
​ Scale by adding
additional servers or
storage for each
application
​ Scale by using more
powerful servers
•  More CPU cores
•  Larger memory
​ Continuous Salesforce
application and
database optimizations
​ Regular customer and
partner application
tuning to improve
individual org
performance
​ Vertically
​ Scaling
​ Application and DB
​ Tuning
​ Customer Application
​ Tuning
45% Reduction in
Latency
NA7
Aug 20
NA7
Aug 27
360ms
250ms
140+ Hours
Loading Time
21 Hours
Loading Time
85% Reduction in
Load Time
1 2 3 4
Automatically Done For Customers
48c
48c
64c
64c
Force.com Extras
Apex: Force.com’s procedural frontier
Integer NUM = 10;
Account [] accs;
// Clean up old data
accs = [select id from account where name like 'test%'];
delete accs;
commit;
accs = new Account [NUM];
for (Integer i = 0; i < NUM; i++)
{
accs [i] = new Account (name='test ' + i, outstandingshares__c=i);
}
insert accs;
Contact [] cons = new Contact [0];
for (Account acc : accs)
{
cons.add (new Contact (lastName=acc.name + '1', accountid=acc.id));
cons.add (new Contact (lastName=acc.name + '2', accountid=acc.id));
}
insert cons;
SOQL
Query
Variable
Declaration
Control
Structure
Array
Data
Operation
Commit
Transaction
Apex code is stored as metadata, interpreted at runtime, and
cached for scalability
§  Bulk DML
§  Email and messaging
§  Asynchronous processing (@future)
§  XmlStream / HTTP (RESTful) services classes
§  Declarative exposure as new Web Services
§  Platform Encryption
Apex is deeply integrated with platform features
Share Your Feedback, and Win a GoPro!
3 Earn a GoPro prize entry for
each completed survey
Tap the bell to take a
survey2Enroll in a session1
Understanding the Salesforce Architecture: How We Do the Magic We Do

Understanding the Salesforce Architecture: How We Do the Magic We Do

  • 1.
    Salesforce’s Multitenant Architecture ​ Doug Merrett ​ PrincipalEnterprise Architect – Asia Pacific ​ dmerrett@salesforce.com How we do the magic we do…
  • 2.
    Safe harbor statementunder the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward- looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements. Safe Harbor
  • 3.
    Shared services acrossapplications The Customer Success Platform APIs 2,700+ Partner Apps Open Ecosystem Workflow Data & Objects Identity Fast App Dev & Customization AnalyticsCollaborationMobile UI Scalable Metadata Platform Complete CRM Trusted Multitenant Cloud AnalyticsCommunityMarketingServiceSales Apps
  • 4.
  • 5.
    One Cloud withMany Customers Shared Elastic Services One Primary Data Store per Production Instance 8K+ Customers per Instance 50+ Production Instances All data segregated by customer All operations include Org ID Disaster Recovery Per Org encryption keys
  • 6.
    What is inan Instance Pivot tables Data tables Metadata tables Shared Database Metadata Cache Bulk data processing Multitenant aware Query optimizer Runtime App Generator Full text search engine Virtual Application Components Common Application Screens Tenant Specific Screens Objects (Tables)
  • 7.
    What Multitenancy meansfor Salesforce R&D No Legacy Teams Bugs fixed for everyone One Version
  • 8.
    What Multitenancy meansfor Salesforce R&D No Legacy Teams Bugs fixed for everyone One Version 260K+ of our Tests Run your tests as well Automation
  • 9.
    What Multitenancy meansfor Salesforce R&D No Legacy Teams Bugs fixed for everyone One Version Instance Architecture 260K+ of our Tests Run your tests as well Automation Staggered Releases Scalability across all sizes
  • 10.
    What Multitenancy meansfor Salesforce R&D Three major releases per year Bug fixes every week Predictability No Legacy Teams Bugs fixed for everyone One Version Instance Architecture 260K+ of our Tests Run your tests as well Automation Staggered Releases Scalability across all sizes
  • 11.
    What Multitenancy meansfor Salesforce R&D Three major releases per year Bug fixes every week Predictability No Legacy Teams Bugs fixed for everyone One Version Instance Architecture 260K+ of our Tests Run your tests as well Automation Staggered Releases Scalability across all sizes
  • 12.
    ​ Stateless Appservers ​ Database systemof record ​ No Database Definition Language (DDL) at Runtime ​ All tables partitioned by OrgId ​ Smart Primary Keys, Polymorphic Foreign Keys ​ Creative de-normalization and pivoting ​ Use every RDBMS feature & optimization Key Architectural Principles
  • 13.
    Metadata, data, andpivot table structures store data corresponding to virtual data structures
  • 14.
    The Objects tablestores metadata about custom objects (tables)‫‏‬
  • 15.
    The Fields tablestores metadata about custom fields (columns)‫‏‬
  • 16.
    The Data heaptable stores all structured data corresponding to custom objects
  • 17.
    A single slotcan store various types of data that originate from different objects
  • 18.
    The Indexes pivottable manages tenant-specific selective indexes
  • 19.
    The UniqueFields pivottable facilitates uniqueness for custom fields
  • 20.
    The Relationships pivottable facilitates referential integrity and optimizes joins
  • 21.
    ​ Tables hash partitionedby OrgId ​ Separate connection pools point to physical hosts ​ App tier is also dynamically partitioned by OrgId ​ Distributed metadata cache with transactional invalidation All data & metadata structures are partitioned to improve performance and manageability
  • 22.
    §  Native Declarativefeatures §  Bulk Processing §  The Recycle Bin §  Full Text Search §  Smart Bulk Data Manipulation Language (DML) §  Web Services APIs Application Framework: a whole lot for free
  • 23.
    Force.com’s native ApplicationFramework provides declarative development, no coding
  • 24.
    Validation rules andsimple formulas: Business analysts can “code” these
  • 25.
    Not so simple:Rollup-summary fields provide for easy cross- object summaries
  • 26.
    Force.com’s bulk processingoptimizations reduce overhead for data loads
  • 27.
    ​ Examples: §  Sort allrecords by primary key before attempting DML §  Operate on tables in deterministic order §  Slot reallocation for field datatype change §  Deferred calculation for new rollup-summary field §  Background processing of mass changes Data definition processing is optimized to avoid performance hits or concurrency limits
  • 28.
    The Recycle Bin:Smart Undeletes Restore §  Individual object instances (records) §  Related object instances (parent/ child records) §  Entire fields and objects (dropped columns and tables)
  • 29.
    Secondary Instance Multitenant Search,anything but simple Index Backup Replication Primary Instance
  • 30.
    Multitenancy delivers BlazingPerformance Transactions Per Quarter 234B Transactions in Q2FY16 79% YoY Growth Average Page Time 208ms Latency in Q2FY16 4% YoY Improvement API Web
  • 31.
    ​ Consistent SQL generationacross the application ​ Deep awareness of pivot table structure •  Flex schema does impose a cost ​ Tenant, user, object, fields statistics are crucial ​ No runaway queries allowed ​ Deep integration with the sharing model Multitenant Query Optimization Principles
  • 32.
    Multitenant Query Optimizer Checkuser visibility Check filer selectivity Dynamically write query based on pre- queries Run Pre-Queries Execute optimized query user visibility = number of rows user can access filter selectivity = index corresponding to filter column Search originates from API or global search return results
  • 33.
    The optimizer considerspre-query selectivity measurements when writing a query Pre-Query Selectivity Measurements … use of index related to filter.HighHigh … ordered hash join; drive using Data table.LowHigh … use of index related to filter.HighLow … nested loops join; drive using view of rows that the user can see.LowLow Write final database access query, forcing … FilterUser
  • 34.
  • 35.
    Service isolation andredundant design ensures availability Highly Scalable Multitenant Architecture •  Instances –  Consistent deployment size –  Easy to scale –  Repeatable •  Instance Groups –  Shared service isolation within a Data Center •  Data Centers –  Consistent for Production and Disaster Recovery –  Shared services across data centers Data Centre Shared Services Instance Group Shared Services Sandbox DR Instances Sandbox Instances Core DR Instances Core Instances Instance Group Shared Services Sandbox DR Instances Sandbox Instances Core DR Instances Core Instances Data Centre Shared Services Instance Group Shared Services Sandbox DR Instances Sandbox Instances Core DR Instances Core Instances Instance Group Shared Services Sandbox DR Instances Sandbox Instances Core DR Instances Core Instances
  • 36.
    ​ Production cluster capacity • 7+1 (active + spare) nodes •  Active Data Guard to local standby (2 node cluster) for fast data recovery ​ Data replication to secondary instance •  Encrypted block-level replication •  Enables site switching for disaster recovery Redundant Site Design with 4 Online Copies of Data Today: Reliable Core by Design Encrypted Async ReplicationData Guard Replication Primary Instance Application Servers Production DB Cluster Standby DB Cluster Data Guard Replication Application Servers Production DB Cluster Standby DB Cluster Secondary Instance
  • 37.
    ​ 6+2 Production RACCluster •  2 spare nodes provides additional DB cluster resilience ​ Data Guard to remote site •  Hardens remote replication •  Enables faster site switching •  Shorter planned and unplanned maintenance windows Faster Site Switches Tomorrow: Enhancing Core Reliability Encrypted Async DB Replication Data Guard Replication Primary Instance Application Servers Production DB Cluster Standby DB Cluster Data Guard Replication Application Servers Production DB Cluster Standby DB Cluster Secondary Instance
  • 38.
    4 key methodsto enable performance at scale Adaptive Capacity Planning ​ Horizontally ​ Scaling ​ Scale by adding additional servers or storage for each application ​ Scale by using more powerful servers •  More CPU cores •  Larger memory ​ Continuous Salesforce application and database optimizations ​ Regular customer and partner application tuning to improve individual org performance ​ Vertically ​ Scaling ​ Application and DB ​ Tuning ​ Customer Application ​ Tuning 45% Reduction in Latency NA7 Aug 20 NA7 Aug 27 360ms 250ms 140+ Hours Loading Time 21 Hours Loading Time 85% Reduction in Load Time 1 2 3 4 Automatically Done For Customers 48c 48c 64c 64c
  • 39.
  • 40.
    Apex: Force.com’s proceduralfrontier Integer NUM = 10; Account [] accs; // Clean up old data accs = [select id from account where name like 'test%']; delete accs; commit; accs = new Account [NUM]; for (Integer i = 0; i < NUM; i++) { accs [i] = new Account (name='test ' + i, outstandingshares__c=i); } insert accs; Contact [] cons = new Contact [0]; for (Account acc : accs) { cons.add (new Contact (lastName=acc.name + '1', accountid=acc.id)); cons.add (new Contact (lastName=acc.name + '2', accountid=acc.id)); } insert cons; SOQL Query Variable Declaration Control Structure Array Data Operation Commit Transaction
  • 41.
    Apex code isstored as metadata, interpreted at runtime, and cached for scalability
  • 42.
    §  Bulk DML § Email and messaging §  Asynchronous processing (@future) §  XmlStream / HTTP (RESTful) services classes §  Declarative exposure as new Web Services §  Platform Encryption Apex is deeply integrated with platform features
  • 43.
    Share Your Feedback,and Win a GoPro! 3 Earn a GoPro prize entry for each completed survey Tap the bell to take a survey2Enroll in a session1