This document discusses isolating tenants in an internal data distribution platform. It defines what a tenant is in this context, such as a destination, domain, or event type. It provides an example design where each tenant is isolated to their own Kafka topic based on routing rules. This design allows for elastic scaling, handles failures, and provides a simple starting point. However, it may not be suitable for situations requiring strict security, accurate cost control, or very large scale with diverse requirements. The future of this design could include hybrid solutions, more customization capabilities, and better isolation of failures between tenants.

1. How to isolate tenants in
a data distribution platform?
Joanna Eriksson

2. Source: Satellite image from Maxar Technologies

4. Domain

5. Speed

6. Destination

7. Multiple channels

9. Isolating tenants
The idea behind tenant isolation is that your SaaS architecture
introduces constructs that tightly control access to
resources, and block any attempt to access resources of
another tenant. Note that tenant isolation is separate from
general security mechanisms.
Source: Tenant isolation - SaaS Architecture Fundamentals

10. Isolating tenants in an internal
data platform
• Handling failures
• Cater for different needs
• Cost control
• …

11. What is a tenant in an
internal data platform?
• Destination
• Domain
• Business areas
• Event type
• Latency requirements
• Completeness
• …

12. Example of separating
tenants in practise

14. Realtime processing (SLO 5 sec)
Elasticity is essential
The traffic
≈ 90k/sec
00:00 00:00 00:00 00:00 00:00 00:00
00:00

16. Kafka topics
≈ 400 topics
30 brokers

17. Topic
• One-to-one mapping between topic and
destination of the data, also called data sink.
• Data sink definition contains routing rule and
configuration of destination.
BrandX-1:
eventType: PulseBase
filter: get-client(.) == "brand-x"
transform: transforms/brand-x.jslt
postFilter: .event_type and (.user_id or .device_id)
consumerId: BrandXAmplitude
amplitude:
apikey: 1234567890
batchSize: 50

18. Kafka topics

25. Scaling up

26. Scaling down

27. Fairness
• Thousands of threads
• JVM Thread Scheduling
• Completely Fair Scheduler
• Idle time

30. Fastforwarding to 2023

31. When is this a good design?
✅ Expandible
• new data and new destinations
✅ Elasticity
✅ Simple design
✅ Good starting point
✅ Main purpose is handling failing integrations

32. When is this design not suitable?
❌ Strict security requirements
❌ Accurate cost control
❌ Very big scale
❌ Big diversity in use cases and requirements

33. The future
• Hybrid solution
• More customization
• Isolating failures

34. Key points
• Thread-consumer model for elasticity
• Handle user errors and failing integrations
• A tenant can be many things - define what makes
sense based on your requirements