Graduation project for Gazi University Faculty of Technology Computer Engineering BS Degree. Tech stack: Kubernetes , Federation-v2, Knative, Istio, Elasticsearch, Grafana, AWS, Google Cloud

1. Multi Cloud Serverless Platform
Using Kubernetes
Fahri YARDIMCI

2. Problem
- Vendor Lock-in
- Multi and/or Hybrid Cloud Serverless Platform
- Same stack of serverless on all cloud
Most used serverless platforms:
- AWS Lambda
- Azure Functions
- Google Cloud Functions
- Cloudflare Workers ..

3. Kubernetes
- We need a common platform
Kubernetes:
- Open Source
- Available as PaaS on most cloud providers
- AWS EKS, GKE, Azure AKS, DigitalOcean Kubernetes

4. How to manage Kubernetes Clusters Across World?
- We need some kind of automated, reliable control plane.
Kubernetes Federation v2:
- Managed by Multicluster-SIG community.
- Selecting master cluster to control all.
- Currently at v0.0.4 version (Still new but awesome).

5. How to create/run Serverless apps on Kubernetes?
- Now we managed worldwide clusters using Fed-v2. But how to serve
Serverless ?
Knative :
- Released in July 2018 at Google Cloud Next ‘18
- Scales to zero ! So you don’t need to reserve resources.
- Uses Istio as Service Mesh to make requests and autoscale works.
- Currently at version v0.2.2

6. Istio
Istio is a service mesh which has a lot of functionality.
Istio :
- Makes it so much easier to create blue/green deployments.
- Gives tracing and monitoring information easily.
- Manage containers networks basically.
- Security
- Ingress/Egress management.
*Still it needs some functionalities like ip whitelist per service/ingress

7. How do we Federate Knative?
- Kubernetes Federation uses their own crd for deploy all clusters.
- So we need to make Knative Serving Service CRD to federated.
- Lately Fed-v2 released kubefed federate tool for this :)
We need to push propagate KnativeService to FederatedKnativeService for
Federation Controller sends all clusters.

8. Why just federate Knative ?
- I can federate Istio as well but some clusters need additional
configuration like:
Cluster A: /regionA service its own traffic management
Cluster B: /regionB service its own traffic management
- It’s still doable federated placements and overrides but it unnecessary for
my project

9. Monitor Federation
Monitoring one cluster is easy and we can store it inside that cluster. But what
should i do with multi clusters.
I separated this to two part: Logs/Traces and Metrics.
Logs/Traces : Sending this to specially created cluster for just for this. So Asia
and US clusters send their logs to same place. Use separated index names like
EU1,USW1 ...
Pros: Log management is easy and showing/querying doesn’t need much
effort.
Cons: Latency… But still maximum 1~2 min delay for logs and traces.

10. Logs/Traces
I used ElasticSearch for store this data and Fluentd for collect and send to ES
Cluster.
ElasticSearch:
- Created at AWS Frankfurt with 3 node.
- Used v6.4.3
Fluentd:
- Cloud Native Collector.
- Sends to ES using logstash_format.

11. Logs/Traces
OpenZipkin:
- I used OpenZipkin for Collecting Traces
- OpenZipkin sends to ES Cluster.

12. Logs/Traces
- Diagram:

13. Metrics
Kubernetes pods and resource metrics i used Kube-state-metrics and
Prometheus with Grafana.
Kube-state-metrics:
- Using API Server to generate metrics
Prometheus:
- Store metrics
Grafana:
- Show metrics / graphs

14. Metrics
Showing users serverless apps metrics can be doable in 2 ways:
1. Make Prometheus cluster separate
2. Use reverse-proxy for grafana with authentication access using that
region with Kubernetes Loadbalancer - Istio Gateway
I choose second way because metric sending could be slow and alerting would
be wrong.

15. Metrics
Diagram:
-Nginx reverse
proxy with auth
-Kube
configmaps and
secrets
-Istio ingress
gateway

16. AWS RDS and S3
User and functions information stored in RDS Mysql db using AWS.
Codes stored in S3.
RDS:
- Managed and highly available Mysql instance
- Low cost and fast(comparing to mysql inside cluster)
S3:
- Managed storage service
- Fast and stores anything, any size.

17. How to build and push users functions/apps ?
Using Kaniko to build images for users code and push these images to GCR
(Google Container Registry)
Kaniko:
- Image builder inside Kubernetes.
- Doesn’t depends Docker.

18. Kubernetes Ops Using Python Flask and KubeClient
I used Python Client for Kubernetes related operations. It’s reasonably simple
compared to Java Client.
Python Flask:
- WebUI sends function and username uuid. This service runs knative and
push image to gcr using this format : fahritez/useruuid:functionuuid
- Creates namespace on master node.
- Deletes services

19. Diagram of creating Functions

20. S3 and GCR
Using userUUID and functionUUID for sending/retrieving and dns.
S3
GCR

21. Java Spring WebUI
Using this webUI user can do : register,create or delete function,show logs and
traces ,show metrics.
WebUI is responsible for:
- Inserting function informations to RDS
- Uploading user code with .tar.gz format to AWS S3
- Creating DNS Record on Route53 with latency-based routing
- Querying Elasticsearch
- Accessing Grafana with auth.

22. WebUI
Function fqdn is : functionuuid.useruuid.app.fahri.pw
Example :
http://f1cfca943-4807-470d-8e1c-1befddb437d9.285c5af8-3a7f-4a1a-bccd-4cf
b0ffe2c82.app.fahri.pw/

23. WebUI Create Function
- Taking env variables (these aren't stored to db or storage)

24. WebUI Traces

25. WebUI Metrics

26. Scale to zero works
Creating function using web ui and waiting to be scaling to zero.
Then making another request

27. WebUI Logs

28. Who am I
Fahri YARDIMCI
twitter.com/FahriYardimci
linkedin.com/in/fahriyardimci
Gazi University
Computer Engineering Student

29. Multi Cloud Serverless Platform
Using Kubernetes
Fahri YARDIMCI