From AWS to GCP, TABLEAPP Architecture Story

1. Migrate From AWS to GCP
TABLEAPP Architecture Story
Yen-Wen Chen
TABLEAPP Architect
29 May 2017

2. TABLEAPP AWS Architecture
TABLEAPP
Cloudfront CDN
Assets / Images
S3
DNS
Route53
Mail Server
SES
Notification
SNS
Load Balancer
Application
Clusters
EC2 / AutoScale
Database
RDS / HA

3. It’s good, but ...

4. AWS Architecture Resource Problem
Scaling!!
● Auto Scaling Benefit
○ Maintain availability
○ Maintain performance
○ Decrease capacity during lulls to
reduce costs
● But
○ Other instance resource is waste
○ EC2 requiring is slow
■ about 1~2min to activate a ec2
■ about 1~2min to update latest
source code
■ about 30s to run health check

5. Our solution is Container

6. Virturalized Infrastructure
Infrastructure
Host OS
Docker Engine
BINS/LIBS
APP
Container
BINS/LIBS
APP
Container
BINS/LIBS
APP
Container
We can ignore resource problem

7. Container Auto Scaling
Container A
Container B
Container C
Container A
Container B
Container C
Container A
Container C
Machine 1 Machine 2 Machine 3
If Container C causes CPU overload
Then put a new C into Machine 3

8. Service Auto Discovery
Container A
Container B
Container C
Container A
Container B
Container C
Container A
Container C
Machine 1 Machine 2 Machine 3
Service A Service B Service C
Service auto discovery / configuration / load balance

9. Other Benefits of Docker Moby
● Version Control
○ push code and build
○ every commit has its own images
● Lightweight
○ fast activation (~ 10s) and rollback
○ small image
● Isolation
○ applications and resources are isolated and segregated
● Protable
○ Multi-cloud platform support
○ BOSS asked that

10. We want to run moby on …?

11. Kubernetes vs EC2 Container Service
Kubernetes EC2 Container Service
Open Source NO
PaaS Google Compute Engine,
Rackspace, Microsoft Azure,
vSphere, ...
AWS
Community x
Management Kubectl / API AWS CLI / API
Protable Run anywhere AWS only
Service / Discovery built-in Elastic Load Balance

12. How about K8s on AWS?

13. Deploy Kubernetes on EC2
● Maintain Kubernetes yourself
○ Install
○ Testing
○ Update
○ and ….
● Handle auto-scale manually
○ remove pods before remove ec2
● Using CI to deploy image
● Set up logging and monitoring policy
● Integrate AWS resource and Kubernetes manually
○ It must be scalable
No, it sucks ...

14. Kubernetes on Google Cloud is awesome

15. Google Container Engine (GKE)
● Easy
● Full managed
○ even update kubernetes
● Logging and Monitoring support (stackdrive)
● Automatic and configurable cluster scaling
● Google Cloud Platform resource integration

16. Design new Architecture

17. TABLEAPP GCP Architecture
Cloud Load
Balancing
Database
Cloud SQL
EastAsia Cluster
Container Engine
TABLEAPP
Cloud CDN
TABLEAPP
Cloudfront CDN
Assets / Images
S3
DNS
Route53
Mail Server
SES
Notification
SNS

18. Architecture inside kubernetes
TABLEAPP Kubernetes Cluster
Front Service
API Service
Manage Service
Front
Front
Manage
Manage
API
CloudSQL Proxy
API
CloudSQL Proxy
API
Cloud Load Balancing
Manage
Cloud Load Balancing
WWW
Cloud Load Balancing
Database
Cloud SQL
Logger Service Logger
Logging

19. Deployment
EastAsia Cluster
Container Engine
BitBucket Build Trigger
Container Registry
Build Images
Container Registry
Base Images
Code Repository
ImageUpdate
Pub/Sub
Pods Update
Cloud Functions
Config Files
Cloud Storage
2
1 Base Images Dockerfile Updated
2 Web Source Code Updated
3 Build Images Process
4 Pull Latest Config File
5 Trigger Image Update
6 Notified Image Update
7 Pull Latest Docker Image
8 Pull Latest Config File From Pods
startup script
9 Migrate Database From Pods startup
script
Database
Cloud SQL
1
3
4 7
8
5
6
9

20. Log Collection
EastAsia Cluster
Container Engine
BitBucket Build Trigger
Container Registry
Build Images
Container Registry
Code Repository
Base Images
ImageUpdate
Pub/Sub
Pods Update
Cloud Functions
Config Files
Cloud Storage
Database
Cloud SQL
Cloud Load
Balancing
Tableapp
Cloud CDN
Logging
Build Log
Log Files
Cloud StorageBackup

21. Cost and Usage

22. Request History
Requests has grown
5 times in past year5x

23. Payment History
● cost increased
almost 2 times in six
month
● We start optimize
AWS architecture at
2016-11
Start
Optimize
2x

24. Payment History
● After reduce cost
● We start maintain
architecture in
Google at 2017-02
Start Test
Google

25. Payment History
Now
We redece cost successful
We can afford more request

26. Conclusion
● We still remain some service in AWS
○ SES
○ SNS
○ S3
○ Read replica Database
● CloudCDN is really fast
○ very low latency
○ optimize network
○ POP is stable
● Lower price
○ based on new arch, we save 40% cost