Single tenant software to multi-tenant SaaS using K8S

1. Single tenant software to
multi-tenant SaaS using K8S
By: Igor Seletskiy
i@cloudlinux.com

2. Back Story
§ Successful Linux distro for service providers
o That sell shared hosting
§ Started to work on platform to sell Docker
o Added multi-tenancy
o Network isolation
o Resource metering
o billing

3. Back Story
§ Realized smaller hosts cannot
out-compete AWS/Google

4. Application hosting?§ Decided to pivot to
Application Hosting
o Wordpress
o Joomla
o ElasticSearch
§ Each app would be a pod
o But we need persistence
• K8s Volumes
Our KD hat with apps
going out of it?

5. Where do we fit in?
§ We would simplify packaging
o integrate billing
• Prepare signup
o Do user management
o Use k8s to orchestrate

6. Can we have more fun?
§ Realized there is more interesting case
o Single tenant to SaaS
§ Or “new” way to do SaaS

7. New way? Huh?
Where are my
microservices?
§ More like really old way
o Wrong way
• Against all the best practices
way

8. So, what is the new way
§Single tenant
o Single instance of application
• Single or multiple containers (pods)
§K8S to orchestrate

9. SaaS burden
§ SaaS companies have an idea that they love to implement
o Yet, they also have to implement…
• registration
• customer management
• billing
• upgrade procedures

10. SaaS burden
o They also have to deal with
• scalability
• reliability
• QoS
• backups
Multi-tenancy
Subscription &
Billing
Customization &
Configurability
Scalability &
Availability
Integration
Security

11. § High availability
§ Upgrades
o Rollbacks
o Staged upgrades
§ A/B testing
§ Custom deployments
§ on demand scalability
§ caching proxy
§ CDN
SaaS: Nice To Have

12. § Single entity -> small amount of data (10mb?
10gb?)
o easy to deal with
o easy to process
o fast to access
§ Multiply by 10,000x customers
o 100GB to 100T
o Requires advanced database clustering
o costly to access
o waste of resources scanning other’s people data
§ 100,000 customers - petabytes...
Scalability / Data

13. Sharding at it best
§ Perfect sharding
o single shard / single customer
Shard A Shard B Shard C Shard D
Collection I
256GB 256GB 256GB 256GB
Collection I
1 TB

14. Large Scale Database HA
mm m m
Apps
§ Large clusters are hard
o requires advance knowledge
o requires advance tuning
• support contracts
• high end db admins

15. Small Scale Database HA
§ Trivial to setup
o easy to maintain
MySQL Master
Clients/Web apps
MySQL Replication
MySQL Slave
Reads and
writes
Only reads

16. Requests/Second
§ If one client generates 1 request per second at peak time
o 10,000 clients could easily generate 2,000 requests per second at peak time
It is a HARD problem to scale a single
application to 2,000 requests per second.
APP

17. The end of ‘Old’ SaaS
§ Hard to deploy
§ Difficult to update
§ Huge overhead
§ Impossible to manage

18. The end of ‘Old’ SaaS
Nah, Just Kidding

19. But we do solve problems
§ Easy to make existing single tenants apps into SaaS
oJust package as a pod (group of pods)
§ Easy to write new ones
oAll the plumbing done
oNo need to deal with lots of data
APP

20. Solve
§ scalability, reliability & high availability
§ customer signup & management
o billing integration
§ QoS
o on demand scalability
o customer separation
§ rolling upgrades
§ a/b testing
KuberDock
for ISV

21. But what about density?
§ OverlayFS is great
oSaves plenty of memory & IO
§ Container check pointing (future)
oNo tenant activity – checkpoint all the pods
• Bring the back up on first packet for the tenant
o In the right order

22. Why it wasn’t done before
§ Wrong technologies
o50k VMs are much scarier then 50k containers
oPuppet is great
• no atomic updates
§ OverlayFS is important – saves tons of
memory
§ Some hybrids were done before

23. But how much would it scale
§ Don’t know yet…
§ Not recomended design for your b2c venture
software with 100M free users

24. But how much would it scale
§ But it would work for:
oB2B software with deployment of 50k users
• Especially if each tenant needs a lot of data
o 100MB+

25. Did we ever…
§ Work in progress..
oLaunch in a month
oBilling software (PHP/MySQL) – 5+ years old
• Used by 10K+ companies
• Single tenant application, runs on client’s servers
oHad demand for hosted version
• No resources to re-design
• No resources to automate

26. What does it take?
§Timeline
oMinor software redesign:
• putting all customizable files into single folder
• Changing setup wizard
• Adjusting licensing process
• Packaging as pod
Total dev time so far < 40h

27. The End
Thank You
Igor Seletskiy
i@cloudlinux.com
More info:
https://kuberdock.com

28. EASY SCALABILITY
§ Scale to more customers just by adding servers
o Limited only k8s scalability
• support for multiple k8s clusters
§ Using native k8s replication & load balancing functionality to
scale for larger customers

29. Implementable QoS
§ Tenants isolation
o make sure that one tenant never affects other tenants,
no matter what they do
§ Native high availability
§ Issues affects subset of the customers - if any

30. Upgrades
§ Docker ‘image’ based upgrades
o Roll updates to X% of customer base
• multiple versions in production
• A/B testing
o Roll back updates
o Custom releases for those who need it
All managed using centralized UI

31. Easy Packaging Provisioning
§ Applications described as YAML
o multi-server, high availability, custom settings…
§ Instantiated using native KuberDock/k8s YAML provisioning

32. Complex Apps

33. BILLING MADE EASY
§ Integrate with common billing platforms
§ Let ISV bill per resources, customers, or other metrics

34. DevOps & Programmers friendly
§ Fully automated
§ micro-services or monolithic
§ any programming language
§ any DBMS
SIGNIFICANTLY SIMPLIFIED DEVELOPMENT

35. Resource Management
§ Dashboard on server utilization, peak usage
o allows for planing & scaling
§ Vertical scale out using public clouds (AWS) when additional computing needs
required

36. More to come...
§ Provide platform for in-app user management
o dual auth
o facebook/google auth
§ Built in help desk integration
§ build it app monitoring
§ cross customer jobs