kubernetes on aws

1. Kubernetes on aws
Yousun Jeong
jerryjung@apach.org

2. Agenda
• Docker
• Kubernetes
• Kubernetes on AWS
• Demo

3. Docker
Automate the deployment of any application as a
lightweight, portable, self-sufficient container
that will run virtually anywhere
https://www.docker.com/what-docker#Why-Should-I-Care-(For-Developers)

4. Docker For Developers
Docker automates the repetitive tasks of setting up and
configuring development environments so that developers
can focus on what matters: building great software.

5. Docker for Ops
• Develop and deploy bug fixes and new features
without roadblocks and scale applications in real
time.

6. Diff VM
• Containers have similar resource isolation and
allocation benefits as virtual machines but a
different architectural approach allows them to
be much more portable and efficient.

7. Docker Engine
• It uses a shared kernel and independent user
mode sessions to provide isolated run-times for
multiple applications on the same host.

8. Docker Container
• It is a packaged application running in an
isolated mode on a Docker host.

9. Docker Client
• It provides a command line interface to control
the Docker engine.

10. Docker Image
• It is an application template ready to be
deployed multiple times.

11. Docker image
http://blog.octo.com/en/docker-registry-first-steps/

12. Docker Architecture
http://southworks.com/blog/tag/docker/

13. Docker Compose
• Docker Compose allows you to define your
multi-container application with all of its
dependencies in a single file, then spin your
application up in a single command

14. Docker Orchestration
https://www.linkedin.com/pulse/docker-redefining-cloud-computing-landscape-akash-
goswami

15. Kubernetes
• Open Source Automation Framework for
deploying, managing, and scaling applications.

16. Why Kubernetes
• Kubernetes provides a common API and self-
healing framework which automatically
handles machine failures and application
deployments, logging, and monitoring.

17. Kubernetes Concept
• Clusters - set of compute, storage, network
resource
• Pods - colocated group of application containers
that share volumes and a networking stack
• Replication Controllers - ensure a specific
number of pods, manage pods, status updates
• Services - cluster wide service discovery

18. Architecture

19. Components
etcd flannel
kubernetes
master
kubernetes
node
kubernetes
node

20. kubectl
• Pod
• Replication controller
• Deployment
• Service
• Volumes
https://kubernetes.io/docs/user-guide/deployments/

21. Kubernetes on AWS
https://kubernetes.io/docs/getting-started-guides/aws/

22. Kubernetes on AWS
export AWS_DEFAULT_PROFILE=default
export KUBERNETES_PROVIDER=aws; wget -q -O - https://get.k8s.io | bash
export AWS_ACCESS_KEY_ID=XXXX
export AWS_SECRET_ACCESS_KEY=XXX
export KUBERNETES_PROVIDER=aws
export KUBE_AWS_ZONE=us-east-1c
export NUM_NODES=3
export MASTER_SIZE=t2.medium
export NODE_SIZE=m3.large
export AWS_S3_REGION=us-east-1
export AWS_S3_BUCKET=XXX
export INSTANCE_PREFIX=k8s
export NODE_ROOT_DISK_SIZE=10
export MASTER_ROOT_DISK_SIZE=10
export AWS_SSH_KEY=/Users/dummy/.ssh/metatron
kube-up.sh

23. Amazon Container Service
• Amazon ECS makes it easy to deploy, manage,
and scale Docker containers running
applications, services, and batch processes.

24. Amazon Container Servie
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html

25. Amazon Container Servie

26. Screenshot

27. Screenshot

28. Episode
• Pod restart
• Failed Containers that are restarted by the
kubelet are restarted with an exponential
back-off delay (10s, 20s, 40s …) capped at
five minutes, and is reset after ten minutes of
successful execution

29. PersistentVolume
• awsElasticBlockStore
kind: PersistentVolume
apiVersion: v1
metadata:
name: metatron-pv
labels:
type: amazonEBS
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteMany
awsElasticBlockStore:
volumeID: vol-XXXXXXXXXXXXXXXX
fsType: ext4

30. PersistentVolumeClaim
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: metatron-pvc
labels:
type: amazonEBS
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi

31. Deployment
spec:
volumes:
- name: ebs-volume
awsElasticBlockStore:
volumeID: vol-XXXXXXXXXXXXXXXX
fsType: ext4
containers:
- name: jupyter
image: jerryjung/jupyter-notebook-gfs:latest
volumeMounts:
- mountPath: "/home/jupyter/work"
name: ebs-volume

32. Kubernetes meets Hadoop
• Persistent Volume Issue by permission
• Shared Nothing Architecture - hadoop
• DaemonSet or StatefulSet
for i in 0 1; do kubectl exec yarn-nm-$i -- sh -c 'hostname'; done
yarn-nm-0
yarn-nm-1

33. DEMO

34. Q&A

35. Thank you