More Related Content Similar to Sf bay area Kubernetes meetup dec8 2016 - deployment models (20) Sf bay area Kubernetes meetup dec8 2016 - deployment models1. Copyright © 2015 Mirantis, Inc. All rights reserved
www.mirantis.com
Kubernetes deployment models
(modelling complex applications in K8S)
Dec8 2016
Piotr Siwczak
(https://www.linkedin.com/in/psiwczak)
2. Copyright © 2015 Mirantis, Inc. All rights reserved
Agenda
● Challenges in managing complex microservice architectures
● What’s missing in K8S to manage complex microservice architectures
efficiently
● K8S AppController as an enhancement to handle complex architectures
● Demo of AppController
● Q&A about AppController
3. Copyright © 2015 Mirantis, Inc. All rights reserved
Challenges in managing
microservice architectures
4. Copyright © 2015 Mirantis, Inc. All rights reserved
Linux as a microservice apps platform
Collection of small, independent programs acting together to
form larger systems.
Programs communicate over standardized protocols/API-s
Abstracts computing resources (kernel)
5. Copyright © 2015 Mirantis, Inc. All rights reserved
Unix/Linux simplified architecture
Kernel
Libraries
Init system Interactive shell
Userspace apps
Resource access
Orchestration
User functionality
6. Copyright © 2015 Mirantis, Inc. All rights reserved
The role of init system
SysVInit … Upstart … Systemd
Init makes sure that apps start in proper order and deps for
them are handled
e.g.
Network subsystem -> Iptables -> ssh
7. Copyright © 2015 Mirantis, Inc. All rights reserved
Apps - combined K8S resources
Kubernetes clients (kubectl...)
Kubernetes resources (pod, service…)
Unix/Linux vs K8S
Kernel
Libraries
Init system Interactive shell
Userspace apps
?
8. Copyright © 2015 Mirantis, Inc. All rights reserved
Do we have init equivalent in k8s?
Original photo by https://www.flickr.com/people/waferboard/ License: CC BY v2.0, Title: “wholesome stirfry”
“Containers start in
parallel after volumes are
mounted, leaving no
opportunity for
coordination between
containers...”
https://github.com/kubernetes/kubernetes/blob/master/
docs/proposals/container-init.md
9. Copyright © 2015 Mirantis, Inc. All rights reserved
...to make a good dish one needs to follow steps
Original photo by https://www.flickr.com/people/waferboard/ License: CC BY v2.0, Title: “wholesome stirfry”
● heat oil
● add garlic
● add veggies and sauce
● add meat
10. Copyright © 2015 Mirantis, Inc. All rights reserved
K8S challenge for complex apps
(T3) wordpress depl/service
(T2) mysql depl/service
(T1) mysql password
password
db dns name
& password
kubectl create -f mysql-pass.yaml
kubectl create -f mysql-deployment.yaml
kubectl create -f wordpress-deplyment.yaml
11. Copyright © 2015 Mirantis, Inc. All rights reserved
Is lack of deps really a problem for K8S?
We all know microservices are supposed to orchestrate
themselves and tolerate failures
...but…
12. Copyright © 2015 Mirantis, Inc. All rights reserved
...deps seem to be needed
https://blog.xebialabs.com/2015/04/13/before-you-go-over-the-container-cliff-with-docker-mesos-etc-po
ints-to-consider/
“"A common definition for a microservice we often hear mentioned is an
“independently-deployable unit”, and indeed it is good practice to design your
microservices so they can start up successfully without requiring all kinds of other
components to be available. But in the vast majority of cases, “no microservice is an
island”...
13. Copyright © 2015 Mirantis, Inc. All rights reserved
...deps seem to be needed
Docker-compose and Mesos application groups are here and being used:
https://docs.docker.com/compose/gettingstarted/
https://mesosphere.github.io/marathon/docs/application-groups.html
14. Copyright © 2015 Mirantis, Inc. All rights reserved
...deps seem to be needed
https://github.com/vishnubob/wait-for-it
15. Copyright © 2015 Mirantis, Inc. All rights reserved
...deps seem to be needed
https://github.com/mesosphere/kubernetes-mesos/issues/119
http://stackoverflow.com/questions/27701994/specify-order-dockers-run-on-kube
rnetes-pod
https://github.com/kubernetes/kubernetes/issues/29804
16. Copyright © 2015 Mirantis, Inc. All rights reserved
https://github.com/Mirantis/k8s-AppController
AppController
17. Copyright © 2015 Mirantis, Inc. All rights reserved
● way to express dependencies between K8S objects,
● thus allowing to deploy complex, multi-tier applications
in fully automated fashion
● k8s object dependency graph:
● definitions (nodes)
● dependencies (edges)
AppController - really short summary...
18. Copyright © 2015 Mirantis, Inc. All rights reserved
Before…
● kubectl create -f t1.yaml
● check status…
● kubectl create -f t2.yaml
● check status…
● kubectl create -f t3.yaml
● ….
After…
● kubectl create -f
graph.yaml
● k8s-appcontroller ac-run
19. Copyright © 2015 Mirantis, Inc. All rights reserved
AppController architecture
Kubernetes
k8s-appcontroller pod
kubeac binary k8s API
extensions
3rd party resources:
dependency
definition
21. Copyright © 2015 Mirantis, Inc. All rights reserved
AppController workflow - definitions
Definition
Standard K8S
resource
(deployment, pod,
service…)
Definition
Standard K8S
resource
(deployment, pod,
service…)
Definition
Standard K8S
resource
(deployment, pod,
service…)
Definition
Standard K8S
resource
(deployment, pod,
service…)
kubectl create -f definitions.yaml
22. Copyright © 2015 Mirantis, Inc. All rights reserved
AppController workflow - definitions
apiVersion: appcontroller.k8s/v1alpha1
kind: Definition
metadata:
name: secret-mysql-pass
secret:
apiVersion: v1
data:
password.txt: cXdxd3F3
kind: Secret
metadata:
creationTimestamp: 2016-12-06T16:56:02Z
name: mysql-pass
namespace: default
secret/mysql-pass
Standard K8S
resource
(secret)
Objects are not created in k8s until triggered by
AppController!
23. Copyright © 2015 Mirantis, Inc. All rights reserved
Definitions - summary
Definition:
● “node” in the graph
● wrapper over regular k8s resource
● defers the creation of the resource until triggered (in
contrary to “kubectl create -f” which creates the resource
immediately
24. Copyright © 2015 Mirantis, Inc. All rights reserved
AppController - dependencies
Definition
Standard K8S
resource
(deployment, pod,
service…)
Definition
Standard K8S
resource
(deployment, pod,
service…)
Definition
Standard K8S
resource
(deployment, pod,
service…)
Definition
Standard K8S
resource
(deployment, pod,
service…)
start end
kubectl create -f deps.yaml
25. Copyright © 2015 Mirantis, Inc. All rights reserved
AppController - dependencies
apiVersion: appcontroller.k8s/v1alpha1
kind: Dependency
metadata:
name: mysql-pass--to--mysql-deployment
parent: secret/mysql-pass
child: deployment/mysql
secret/mysql-pass
Standard K8S
resource
(secret)
deployment/mysql
Standard K8S
resource
(deployment)
26. Copyright © 2015 Mirantis, Inc. All rights reserved
Dependencies - summary
Dependency:
● “edge” in the graph
● links definitions together
● provides the sense of dependency between definitions
27. Copyright © 2015 Mirantis, Inc. All rights reserved
AppController - application rollout
Definition
Standard K8S
resource
(deployment, pod,
service…)
Definition
Standard K8S
resource
(deployment, pod,
service…)
Definition
Standard K8S
resource
(deployment, pod,
service…)
Definition
Standard K8S
resource
(deployment, pod,
service…)
start end
kubectl exec k8s-appcontroller ac-run
kubectl exec k8s-appcontroller kubeac
get-status
28. Copyright © 2015 Mirantis, Inc. All rights reserved
AppController workflow - summary
● “wrap” regular k8s resources into definitions > defs.yaml
● load defs.yaml to k8s
● create dependencies between resources > deps.yaml
● load deps.yaml into k8s
● trigger the app deployment from AppController
application (kubectl exec k8s-appcontroller ac-run)
30. Copyright © 2015 Mirantis, Inc. All rights reserved
def_db_password
AppController - wordpress deployment
secret:
db_password
def_db_deploymt
deployment:
mysql-deploym
ent
def_db_service
service:
db_service
def_wp_deploymt
deployment:
wordpress-dep
loyment
def_wp_service
service:
wordpress-ser
vice
DB_PASS
DB_HOSTNAME, DB_PASS
32. Copyright © 2015 Mirantis, Inc. All rights reserved
Q&A
Q:
What if I abort the deployment in the middle - how does AppController recover
from partially provisioned graph?
A:
AppController will check the status of already provisioned resources. Will only
provision the ones which are absent
33. Copyright © 2015 Mirantis, Inc. All rights reserved
Q&A
Q:
Can I run multiple AppControllers on a single K8S ?
A:
You can run 1 AppController per namespace
34. Copyright © 2015 Mirantis, Inc. All rights reserved
Q&A
Q:
What K8S resources can be currently wrapped into definitions?
A:
- Daemonset
- Job
- Petset
- Pod
- Replicaset
- Service
- ConfigMap
- Secrets
- Deployments
35. Copyright © 2015 Mirantis, Inc. All rights reserved
Q&A
Q:
How is provisioning of resources validated?
A:
Status of the k8s resource is checked.
AppController implements also some checks of its own (e.g. for replicasets
readiness probe is based on “success factor” or all resources ready. Success
factor is a part of appcontroller and for services we are checking service selector
and see if the backends are ready - e.g. replica sets)
36. Copyright © 2015 Mirantis, Inc. All rights reserved
Q&A
Q:
What’s on the roadmap?
A:
Graph notifications, reactions, error handling
More resources supported
Usability improvements
Better documentation (incl. real-life complex examples)
37. Copyright © 2015 Mirantis, Inc. All rights reserved
Q&A
Q:
How AppController is different from Init Containers?
A:
Supports more complex deployments (complex graphs)
Handles deps not only between containers
Checks resource states (no need to implement custom probes in the container)
Keeps debug logs in one place
Can react to changes in the graph
38. Copyright © 2015 Mirantis, Inc. All rights reserved
Recording
https://www.youtube.com/watch?v=7GSwSTtBAYo&utm_cont
ent=38600000