1) The document discusses various techniques for optimizing multi-tenant architectures on AWS including data partitioning, tenant-aware caching, and using tenant policies and profiles to customize access and optimize resource usage. 2) It emphasizes the importance of monitoring metrics at both the infrastructure and application levels to understand tenant usage patterns and identify opportunities for optimization. 3) The document also covers service selection considerations for multi-tenant workloads, strategies for optimizing serverless architectures on AWS, and using metering and billing to enable tiered tenant plans.

1. Building on AWS:
Optimizing & Delivering
Derek Ewell
Solutions Architect, ASEAN

2. The Foundation
On-Boarding &
Provisioning
Data
Partitioning
Identity
Deployment
Agility
CI/CD Distributed
Making it Sing
Billing Metering
Granular
Scale
Cost
Optimized
Policy Driven
Tenant
Experience
Thinking Beyond Agility & Delivery

3. Real-Time Tenant Experience Optimization
Idle Tenants
Active Tenants
Busy Tenants

4. Tenant Id Last Accessed
Tenant-3 1/1/2015 12:01:24
Tenant-1 1/1/2015 11:51:01
Tenant-5 1/1/2015 10:41:53
Building A Tenant-Optimized Cache
Tenant 1 Tenant 2 Tenant 3
Tenant Cache Status
Cache
Manager
1
Data Access
2
3 5
7
6
4

5. Sizing Your Cache
Tenant Id Weight Last Accessed
Tenant-4 9 1/1/2016 09:03:01
Tenant-2 6 1/1/2016 09:41:53
Tenant-1 1 1/1/2016 08:39:24
Tenant-9 6 1/1/2016 08:25:51
Tenant-1 4 1/1/2016 08:22:43
Tenant-1 9 1/1/2016 07:19:11
Top Tenant Cache
How many tenants should be
cached?
Cache Size # Busy Tenants

6. Tenant Optimization Considerations
• It’s all about making the “right” tenants successful
• Make optimization a real-time, dynamic mechanism
• Focus on improving experience while limiting the footprint
• Find the balance between complexity and value
• Expect to continually tune your “most active tenant”
algorithms
• Surface optimization metrics on your operational views

7. Data Partitioning and Storage Optimization
• Tenants come all shapes and sizes
• The data profile for each tenant can vary
significantly
• Treating all tenants the same can impact scale,
performance, and cost

8. Creating a Compute Bottleneck
Storage
Instance
CustomerID AccountID Name StartDate
8482994 429391818 Jake Young 12-02-2014
9782838 782839234 Mary Toms 11-14-2011
3948299 919499919 Lisa Lewis 03-01-2011
Tenant1-Customer
CustomerID AccountID Name StartDate
8482994 429391818 Jake Young 12-02-2014
9782838 782839234 Mary Toms 11-14-2011
3948299 919499919 Lisa Lewis 03-01-2011
Tenant2-Customer
Tenants
• CPU
• Memory
• IO

9. Sharding a Shard
Tenant-1
Shard-1
Shard-2
Shard-3
ShardID AccountID Name
Shard-3 429391818 Jake Young
Shard-6 782839234 Mary Toms
Shard-1 919499919 Lisa Lewis
Shard-3 434452444 Bob Jones
Shard-5 845341445 Lisa Smith
Shard-2 525323564 Tom Wall
Shard-5 525555321 Eva Walters
Shard-1 134567643 Kim Williams
Shard-6 356273627 Tim Hawk
Tenant-2
Shard-5
Shard-6
A more intelligent, tenant aware partitioning scheme
Shard Analytics

10. Sharding with DynamoDB
Partition Key Attributes
TenantID
Tenant1
CustomerTable
{
ShardCount: 4,
ShardSize: [4, 9, 4, 5],
ShardIds: [“93”, “932”, “21”, “36”]
}
Partition Key Attributes
ShardID
93
CustomerID
492300009
Name
Bob Jones
ShardID
221
CustomerID
983019391
Name
Jane Thomas
ShardID
21
CustomerID
3492098u7
Name
Sally Smith
ShardID
932
CustomerID
115830489
Name
Randy Hanson
ShardID
93
CustomerID
819492229
Name
Wendy Watts
ShardID
36
CustomerID
480002194
Name
Henry Hanks
• Let shard count grow dynamically
• Continually evaluate shard size
Tenant Lookup Table
Customer Table

11. Sharding With RDS
TenantID Name StartDate
Tenant-1 Bob Smith 01-12-2012
Tenant-4 Sue Jones 09-01-2015
Shard-1
Instance
Shard-2
Instance
TenantID Name StartDate
Tenant-2 Lisa Louis 11-21-2014
Tenant-3 Tom Tucker 06-09-2012
Customer Table
Customer Table
TenantID ShardID
Tenant-1 Shard-1
Tenant-2 Shard-2
Tenant-3 Shard-2
Tenant-4 Shard-1
Tenant-Shard Mapping

12. Partitioning Optimization Considerations
• Tenant data is not going to be evenly distributed
• Uneven data distribution can artificially increase costs
• Each storage service may require a unique solution
• Distributing data will add complexity to your solution
• Sharding should be a dynamic, metrics driven process
• Sharding strategies are often domain specific

13. Using Tenant Policies for Optimization
Tenant 1 Tenant 2
API’s
Tenant
Policies
Services
Data Access
• Tenant policies can be applied at
all layers in your architecture
• Policies can be used to alter and
optimize tenant experience
• Policies can enable more
granular tiering

14. Tier-Driven Optimization
Data Access
Catalog
Service
Order
Service
Tax
Service
Tenant Policies
{
“TenantID”: “29304”,
“Plan”: “Gold”
},
{
“TenantID”: “91039”,
“Plan”: “Platinum”
}
DynamoDB RDS S3
API
Tier StorageLimit
API
SLA
Gold 500 GB 5 ms
Platinum 2 TB 2 ms

15. Workflow-Driven Optimization
Tenant 1
DynamoDB RDS S3
1. Get orders from yesterday
2. Get orders for last 30 days
3. Get all orders
Data Access
Tenant 2
DynamoDB RDS
Data Access
Tenant
Policies
1
2
3 1,2 3

16. Tenant Policy Considerations
• Think more granularly about tenant loads
• Provide centralized management of policies
• Context can change how a request is processed
• Align the resource profile with the application need
• Leverage the diversity of AWS services
• Policies create new opportunities for tiering and cost
optimization

17. SaaS Load Optimization
• Aligning resource consumption
with multi-tenant load profile
• Achieving “just enough”
infrastructure
• Minimizing costs
• What does idle look like?
• Think about cost per tenant

18. Picking the Right Services
Checkout
Taxation
Order = Unit of Scale
Shipping
Order Service
Checkout
Taxation
Order = Unit of Scale
Shipping
Order Service
Single Tenant Load Multi-Tenant Load
Checkout Service
Taxation Service
Shipping Service
Independent Services

19. Serverless SaaS with AWS Lambda
Order REST API
Order Service
Update Create
Delete Find
Ship Cancel
Update Create Delete Find Ship Cancel
REST API
• Load & cost distributed across REST calls
• Cleaner model for agile deployment
• Management and scale offloaded to AWS
• Gateway to throttle tenants
• Larger failure surface area
• Scaling is your responsibility
• More dependencies

20. REST API
Severless SaaS: Optimization + Isolation
Web Tier
App Tier
Web Tier
App Tier
Tenant 1
REST API
Tenant 2Tenant 1 Tenant 2
Serverless IsolationEC2 Isolation
• Cost
• Management
• Deployment
• Provisioning
Tradeoffs

21. Load Optimization Considerations
• Multi-tenant loads are more difficult to profile
• Load optimization relies on granular metrics
• Live on the edge, but never impact scale or experience
• Serverless represents an opportunity to optimize both
load and cost

22. Monitoring & Management
• SaaS sets a higher bar for management & monitoring
• Views of both infrastructure and application health
• Ability to view health and activity with tenant context
• Instrumenting logs with application and tenant metrics
• Support for tenant policies for alerts/alarms

23. Management & Monitoring
S3
CloudWatch
AWS Config
CloudTrail
TenantContext
Splunk
Sumologic
Kibana
• Create custom views that aggregate
and summarize cross-tenant activity
• Proactively identify cross-tenant issues
• Introduce cross tenant policies
Application Services
Catalog
Service
Order
Service
Ratings
Service
Tax
Service

24. Resource Metrics
Application Activity
Aggregating Usage & Health Analytics
Storage MetricsApp Service Metrics Compute Metrics
• Analyze tenant flows
• Assess consumption
metrics
Page Load Metrics User Clicks REST Calls

25. Metering Enables Tiering
Sign Up
Silver
$49 / month
Sign Up
Bronze
$29 / month
Sign Up
Gold
$99 / month
Features
Users

26. Billing

27. SaaS Subscriptions for AWS Marketplace

28. In Summary
• Bring the best experience to those that need it
• Leverage tenet profiles to drive optimization
• Factor tenant data distribution into your partitioning
model
• Take advantage of AWS service diversity
• Metrics and monitoring essential to optimization
• Make cost part of your optimization strategy