Kubernetes offers a powerful abstraction layer for managing containerized infrastructure. Amazon EKSmakes it easy to run Kubernetes on AWS without having to manage the Kubernetes Control Plane. In this session, see how Amazon EKS makes deploying Kubernetes on AWS simple and scalable, including networking, security, monitoring, and logging. Learn what we’re doing to make AWS an even better place to run Kubernetes and watch a demo of AWS customers starting to use Amazon EKS.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Peter Dalbhanjan
Sr. Specialist Solutions Architect
Aug 2nd, 2018
Running Kubernetes with Amazon EKS

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
Kubernetes cluster setup
CI/CD with applications deployed on Kubernetes
Networking
Visibility
Security – Authentication and Authorization

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Open source container
management platform
Helps you run
containers at scale
Gives you primitives
for building
modern applications
What is Kubernetes?

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
57% of Kubernetes workloads
run on AWS today
— Cloud Native Computing Foundation

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubernetes cluster
setup — choices

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubernetes cluster setup—choices
Install, operate, upgrade, delete a Kubernetes cluster
Development—Minikube
Enterprise
• Elastic Container Service for Kubernetes (EKS)
• CoreOS Tectonic
• Red Hat OpenShift
Community—Kops
• List: kubernetes-aws.io
AWS partners: Heptio, Docker
Custom
• CloudFormation
• Terraform

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Manage a Kubernetes cluster: Kops
Community supported
• SIG AWS
• Kops office hours and Slack channel
github.com/kubernetes/kops
Generate CloudFormation or Terraform scripts

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
O p erat i on al
E xc el l en c e
Sec u r i t y
Rel i a b i l i t yPer fo r m a nc e
Effi c i en c y
C o st
O pt i m i zat i on
M a sters
etc d

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Elastic Container Service for Kubernetes
Managed K8s control plane — highly available master and etcd
Bring your own worker nodes, like ECS
Core tenets
• Platform for enterprises to run production-grade workloads
• Provides a native and upstream Kubernetes experience – Kubernetes certified
• Not forced to use additional AWS services, but offer seamless integration
• Actively contributes to the Kubernetes project
APIs

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EKS architecture
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
AWS Managed
Customer
Managed

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
Kubectl

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EKS Customers
C r e a t e E K S c l u s t e r
P r o v i s i o n w o r k e r n o d e s
L a u n c h a d d - o n s
L a u n c h w o r k l o a d s

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EKS – Kubernetes masters
C r e a t e H A m a s t e r s
C e r t i f i c a t e
m a n a g e m e n t
I A M i n t e g r a t i o n
S e t u p L BC r e a t e H A e t c d
A u t o s c a l e
C r e a t e c l u s t e r

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Networking

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“Every pod should have it’s own IP address,
and all pods should be able to talk to one
and other”
Node Node
Pod Pod
Networking with Kubernetes

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is CNI
Network
Plugin
Runtime
Network
• A way for Kubernetes to tell an underlying
SDN that it wants to connect a container to a
network.
• Standards based pluggable architecture for
container networking.
• API for writing plugins to configure network
interfaces for containers.
• CNCF Project

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC CNI plugin
• Bridge between the K8s land – AWS VPC
• AWS Routable IPs
• Thin layer – no performance impact
• Pod IP ENI secondary IP
• Github – https://github.com/aws/amazon-vpc-cni-k8s

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How do I use it?
• Any K8s cluster on AWS.
• EKS
• BYOK8s
• Daemonset deployment.
kubectl create –f eks-cni.yaml

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC CNI networking internals
K u b e l e t
V P C C N I
p l u g i n
1 . C N I A d d / D e l e t e
E C 2
E N I E N I E N I
P o d P o d P o d P o d
V P C
N e t w o r k
.........
0 . C r e a t e E N I
2 . S e t u p v e t h

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC CNI plugin architecture
K u b e l e t
V P C C N I
p l u g i n
N e t w o r k l o c a l
c o n t r o l p l a n e
E N I s /
S e c o n d a r y I P s
C N I A d d / D e l e t e
g R P C
E C 2

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Packet flow : pod - to - pod
E C 2
Default namespace
Pod namespace
veth veth
Main RT
E C 2
Default namespace
Pod namespace
veth
Route
Table
Main RT
ENI RT
veth
VPC
fabric
ENI RT
Route
Table

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Packet flow : pod - to external
E C 2
Default namespace
Pod namespace
veth
Route Table
Main RT
ENI RT
veth
External
Network
IPTables

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
That looks good. But, what about isolation boundaries for my
workloads?

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Consider network policies for segmentation and namespace for
isolation

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Frontend
Cats Dogs
kind: NetworkPolicy
apiVersion: extensions/v1beta1
metadata:
name: default-deny
spec:
podSelector:
matchLabels: {}
dev-namespace
Kubernetes Network policies

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Frontend
Cats Dogs
kind: NetworkPolicy
apiVersion: extensions/v1beta1
metadata:
name: default-deny
spec:
podSelector:
matchLabels: {}
dev-namespace
Kubernetes Network policies

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Frontend
Cats Dogs
dev-namespace
apiVersion: extensions/v1beta1
kind: NetworkPolicy
metadata:
name: public-to-frontend
spec:
podSelector:
matchLabels:
role: frontend
ingress:
- from:
- ipBlock:
cidr: "0.0.0.0/0"
ports:
- protocol: TCP
port: 80
Kubernetes Network policies

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Frontend
Cats Dogs
dev-namespace
apiVersion: extensions/v1beta1
kind: NetworkPolicy
metadata:
name: public-to-frontend
spec:
podSelector:
matchLabels:
role: frontend
ingress:
- from:
- ipBlock:
cidr: "0.0.0.0/0"
ports:
- protocol: TCP
port: 80
Kubernetes Network policies

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Frontend
Cats Dogs
dev-namespace
apiVersion: extensions/v1beta1
kind: NetworkPolicy
metadata:
name: frontend-to-cats
spec:
podSelector:
matchLabels:
role: cats
ingress:
- from:
- podSelector:
matchLabels:
role: “frontend”
ports:
- protocol: TCP
port: 80
Kubernetes Network policies

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Frontend
Cats Dogs
dev-namespace
apiVersion: extensions/v1beta1
kind: NetworkPolicy
metadata:
name: frontend-to-cats
spec:
podSelector:
matchLabels:
role: cats
ingress:
- from:
- podSelector:
matchLabels:
role: “frontend”
ports:
- protocol: TCP
port: 80
Kubernetes Network policies

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
kubectl create namespace prod-namespace

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Frontend
Cats Dogs
dev-namespace
Frontend
Cats Dogs
prod-namespace
Kubernetes Namespaces

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Frontend
Cats Dogs
prod-namespace
apiVersion: extensions/v1beta1
kind: NetworkPolicy
metadata:
name: frontend-to-cats-and-dogs
namespace: prod-namespace
spec:
podSelector:
matchLabels:
role: cats-and-dogs
ingress:
- from:
- podSelector:
matchLabels:
role: “frontend”
ports:
- protocol: TCP
port: 80
Kubernetes Namespaces

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Security – Authentication and
Authorization

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubernetes + AWS IAM
• AWS native access management
• In collaboration with Heptio
• Kubectl and worker nodes
• Works with Kubernetes RBAC

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
K8s action allowed/denied
Kubernetes + AWS IAM
Authorizes AWS Identity with RBAC
K8s API
Passes AWS Identity
Verifies AWS Identity
AWS Auth
1
2
3
4
Kubectl
Github – htt ps : / / github.com/heptiolabs/kubernetes-aws-authenticator

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Worker provisioning
k u b e c t l
AW S A u t h
c o n f i g m a p & R B A C
W o r k e r s
R o l e
R o l e

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
RBAC : namespace
Users
Service
Account
pod-reader
RoleBinding
pod-reader
Role
get
list
pod

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
RBAC : cluster-wide
Users
Service
Account
cluster-reader
ClusterRoleBin
ding
cluster-reader
ClusterRole
get
list
Cluster

40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Visibility in cluster

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Visibility throughout your Kubernetes cluster
TracingAlertsEventsMetricsLogs
ApplicationContainerNodeCluster

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
WorkerWorkerMaster
WorkerWorkerMaster
ASG
AZ1
Region
AZ2
ASG
CloudWatch
Logs
Elasticsearch
Kibana
Fluentd
DaemonSet
Kubectl logs
Elasticsearch (index),
Fluentd (store), and
Kibana (visualize)
Logs

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Metrics

44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CICD for applications deployed
on Kubernetes

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CI/CD of apps on Kubernetes—choices
Jenkins
AWS partners
• GitLab
• Shippable
• CircleCI
• Codeship
AWS CodePipeline, AWS CodeCommit, AWS CodeBuild

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Jenkins

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubernetes continuous deployment
AWS CodePipeline
AWS CodeCommit AWS CodeBuild AWS Lambda
Amazon ECR
1 2 4
3 5
6
1
Developers continuously integrate
changes into a main branch hosted
within a repo
2
Triggers an execution of the pipeline
when a new version is found, builds
a new image with build id
3
Pushes the newly built image
tagged with build id to ECR repo
4
Invokes a Lambda function to
trigger application deployment
5
Leverages Kubernetes Python SDK
to update a deployment
6
Fetches new container image
and performs a rolling update
of deployment
Developer

48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s Next?
Learn more: https://aws.amazon.com/eks
https://github.com/aws-samples/aws-workshop-for-kubernetes

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!
dalbhanj@amazon.com
T: @pdalbhan