Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
jetstack.io
From Rollercoasters
to Meerkats:
3 Generations of
Production
Kubernetes Clusters
Presented by
Matthew Bates / ...
jetstack.io
WHO ARE WE?
Matt Bates
Co-founder
twitter.com/mattbates25
github.com/mattbates
Christian Simon
Senior Solution...
jetstack.io
KUBERNETES
What is Kubernetes?
Kubernetes
Master
Node Node Node
Declarative description of
application workloa...
jetstack.io
KUBERNETES
Exploring the cluster map
kubectl
storage (etcd) API server
scheduler controller-manager
Node
kubel...
jetstack.io
REPLICA SETS
Actual vs. Desired state
apiVersion: core/v1
kind: ReplicaSet
metadata:
name: nginx
spec:
replica...
jetstack.io
REPLICA SETS
Actual vs. Desired state
apiVersion: core/v1
kind: ReplicaSet
metadata:
name: nginx
spec:
replica...
jetstack.io
REPLICA SETS
Actual vs. Desired state
apiVersion: core/v1
kind: ReplicaSet
metadata:
name: nginx
spec:
replica...
jetstack.io
REPLICA SETS
Actual vs. Desired state
apiVersion: core/v1
kind: ReplicaSet
metadata:
name: nginx
spec:
replica...
jetstack.io
REPLICA SETS
Actual vs. Desired state
apiVersion: core/v1
kind: ReplicaSet
metadata:
name: nginx
spec:
replica...
jetstack.io
REPLICA SETS
Actual vs. Desired state
apiVersion: core/v1
kind: ReplicaSet
metadata:
name: nginx
spec:
replica...
jetstack.io
CLUSTER PROVISIONING
Provisioning Kubernetes clusters
storage (etcd) API server
scheduler controller-manager
N...
jetstack.io
FIRST GENERATION
● CoreOS
● Infrastructure: CloudFormation
● Config: Cloud-Init Bash
● Ruby scripts to drive
C...
jetstack.io
SECOND GENERATION
● CoreOS
● Infrastructure: Terraform
● Config: Cloud-Init Bash
● Shell script/Makefile to wr...
jetstack.io
LESSONS LEARNED
● Immutable infrastructure isn’t always that desireable
● Testing and debugging ability is cri...
jetstack.io
NEXT-GEN MOTIVATIONS
● Quick feedback loop for developers
○ Testability (unit, integration)
● Reusability of c...
jetstack.io
NEXT-GEN MOTIVATIONS
● Immutable infrastructure updates can be expensive and slow
● Desired vs actual state fo...
jetstack.io
NEXT-GEN MOTIVATIONS
● All Kubernetes solutions behave differently
● Abstractions
○ Provider/Infrastructure
○ ...
jetstack.io
NEXT-GEN MOTIVATIONS
Consistency across different Infrastructures
jetstack.io
NEXT-GEN MOTIVATIONS
Consistency across different Infrastructures
jetstack.io
NEXT-GEN MOTIVATIONS
Consistency across different Infrastructures
jetstack.io
“Tarmak is an open-source toolkit for Kubernetes cluster lifecycle management.
It focuses on best-practice clu...
jetstack.io
● Build (cloud)
infrastructure with
Terraform
● Instances types have
roles
● Puppet manifests
artifact uploade...
jetstack.io
● Instance Cloud-Init
runs Wing agent
on startup
● Wing downloads
Puppet artifact
● Wing runs the
puppet-agent...
jetstack.io
WING
Deep-dive
● Using puppet apply means we run masterless
○ reduces complexity
○ reduces insights
● Wing API...
jetstack.io
WING
Architecture
tarmak
storage (etcd) wing-server
instance-a
wing
puppet
instance-b
wing
puppet
Wing server
jetstack.io
WING
Actual vs. Desired state
apiVersion: wing/v1alpha1
kind: Instance
metadata:
name: instance-a
spec:
conver...
jetstack.io
WING
Actual vs. Desired state
apiVersion: wing/v1alpha1
kind: Instance
metadata:
name: instance-a
spec:
conver...
jetstack.io
WING
Actual vs. Desired state
apiVersion: wing/v1alpha1
kind: Instance
metadata:
name: instance-a
spec:
conver...
Demo time
jetstack.io
jetstack.io
PUPPET MODULES
Open source modules on PuppetForge
● aws_ebs attach, format, mount EBS volumes
● calico calico ...
jetstack.io
MEERKATS
What about them?
github.com/jetstack/tarmak
docs.tarmak.io
@JetstackHQ
Thanks!
Get involved
jetstack.io
jetstack.io
TARMAK RESOURCES
Providers, Environments and Clusters
WHO ARE JETSTACK?
jetstack.io
We are a UK-based company that help enterprises in their path to modern cloud-native
infrast...
Upcoming SlideShare
Loading in …5
×

of

From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 1 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 2 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 3 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 4 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 5 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 6 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 7 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 8 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 9 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 10 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 11 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 12 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 13 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 14 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 15 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 16 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 17 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 18 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 19 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 20 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 21 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 22 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 23 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 24 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 25 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 26 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 27 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 28 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 29 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 30 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 31 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 32 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 33 From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Slide 34
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

1 Like

Share

Download to read offline

From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters

Download to read offline

A walk through the various generations of Kubernetes deployments by Jetstack, and a reasoning for the development and launch of Tarmak the open-source toolkit for Kubernetes cluster lifecycle management.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters

  1. 1. jetstack.io From Rollercoasters to Meerkats: 3 Generations of Production Kubernetes Clusters Presented by Matthew Bates / Christian Simon
  2. 2. jetstack.io WHO ARE WE? Matt Bates Co-founder twitter.com/mattbates25 github.com/mattbates Christian Simon Senior Solution Engineer twitter.com/simonswine github.com/simonswine
  3. 3. jetstack.io KUBERNETES What is Kubernetes? Kubernetes Master Node Node Node Declarative description of application workloads Workloads scheduled onto nodes and actively managed ● Declarative system description using application abstractions ● Abstracts away the servers so developers can concentrate on code ● Infrastructure-agnostic ● Pro-actively monitors, scales, auto-heals and updates
  4. 4. jetstack.io KUBERNETES Exploring the cluster map kubectl storage (etcd) API server scheduler controller-manager Node kubelet proxy docker Node kubelet proxy docker Control Plane Other clients
  5. 5. jetstack.io REPLICA SETS Actual vs. Desired state apiVersion: core/v1 kind: ReplicaSet metadata: name: nginx spec: replicas: 1 template: [...] API server kubectl Desired state Actual state nginx
  6. 6. jetstack.io REPLICA SETS Actual vs. Desired state apiVersion: core/v1 kind: ReplicaSet metadata: name: nginx spec: replicas: 1 template: [...] status: replicas: 0 API server controllers Desired state Actual state nginx nginx
  7. 7. jetstack.io REPLICA SETS Actual vs. Desired state apiVersion: core/v1 kind: ReplicaSet metadata: name: nginx spec: replicas: 1 template: [...] status: replicas: 1 Desired state Actual state nginx nginx API server controllers
  8. 8. jetstack.io REPLICA SETS Actual vs. Desired state apiVersion: core/v1 kind: ReplicaSet metadata: name: nginx spec: replicas: 3 template: [...] status: replicas: 1 API server kubectl Desired state Actual state nginx nginx nginx nginx
  9. 9. jetstack.io REPLICA SETS Actual vs. Desired state apiVersion: core/v1 kind: ReplicaSet metadata: name: nginx spec: replicas: 3 template: [...] status: replicas: 1 API server controllers Desired state Actual state nginx nginx nginx nginx nginx nginx
  10. 10. jetstack.io REPLICA SETS Actual vs. Desired state apiVersion: core/v1 kind: ReplicaSet metadata: name: nginx spec: replicas: 3 template: [...] status: replicas: 3 API server controllers Desired state Actual state nginx nginx nginx nginx nginx nginx
  11. 11. jetstack.io CLUSTER PROVISIONING Provisioning Kubernetes clusters storage (etcd) API server scheduler controller-manager Node kubelet proxy docker Node kubelet proxy docker Control Plane
  12. 12. jetstack.io FIRST GENERATION ● CoreOS ● Infrastructure: CloudFormation ● Config: Cloud-Init Bash ● Ruby scripts to drive CloudFormation “Rollercoasters”
  13. 13. jetstack.io SECOND GENERATION ● CoreOS ● Infrastructure: Terraform ● Config: Cloud-Init Bash ● Shell script/Makefile to wrap Terraform ● All wrapped-up in Docker “Elevation”
  14. 14. jetstack.io LESSONS LEARNED ● Immutable infrastructure isn’t always that desireable ● Testing and debugging ability is critical for development - and operations ● All dependencies need to be versioned ● Cluster PKI management is not easy ● Advantages/disadvantages of declarative infrastructure (Terraform et al) abstraction Lessons learned through the first generations of cluster deployment
  15. 15. jetstack.io NEXT-GEN MOTIVATIONS ● Quick feedback loop for developers ○ Testability (unit, integration) ● Reusability of code ○ Follow DRY, KISS principles ● Continuous roll-out of changes ● Dry-run capability and meaningful output ● Tried and tested, well-understood tools Developer/operator experience
  16. 16. jetstack.io NEXT-GEN MOTIVATIONS ● Immutable infrastructure updates can be expensive and slow ● Desired vs actual state for configuration updates ● Stateful applications (caches, risks of migration) Less disruptive upgrades
  17. 17. jetstack.io NEXT-GEN MOTIVATIONS ● All Kubernetes solutions behave differently ● Abstractions ○ Provider/Infrastructure ○ Operating system ○ Version of the components Consistency across different infrastructure
  18. 18. jetstack.io NEXT-GEN MOTIVATIONS Consistency across different Infrastructures
  19. 19. jetstack.io NEXT-GEN MOTIVATIONS Consistency across different Infrastructures
  20. 20. jetstack.io NEXT-GEN MOTIVATIONS Consistency across different Infrastructures
  21. 21. jetstack.io “Tarmak is an open-source toolkit for Kubernetes cluster lifecycle management. It focuses on best-practice cluster security and operations. It’s built from the ground-up to be cloud provider-agnostic, and give consistent and reliable cluster deployment and management, across cloud and on-premises environments.”
  22. 22. jetstack.io ● Build (cloud) infrastructure with Terraform ● Instances types have roles ● Puppet manifests artifact uploaded to object storage TARMAK Infrastructure
  23. 23. jetstack.io ● Instance Cloud-Init runs Wing agent on startup ● Wing downloads Puppet artifact ● Wing runs the puppet-agent TARMAK Configuration management
  24. 24. jetstack.io WING Deep-dive ● Using puppet apply means we run masterless ○ reduces complexity ○ reduces insights ● Wing API Server as single source of truth, stores actual and desired state ● Wing Agent on every instance, evaluates states and acts accordingly
  25. 25. jetstack.io WING Architecture tarmak storage (etcd) wing-server instance-a wing puppet instance-b wing puppet Wing server
  26. 26. jetstack.io WING Actual vs. Desired state apiVersion: wing/v1alpha1 kind: Instance metadata: name: instance-a spec: converge: hash: sha16:ffaa tarmak
  27. 27. jetstack.io WING Actual vs. Desired state apiVersion: wing/v1alpha1 kind: Instance metadata: name: instance-a spec: converge: hash: sha16:ffaa status: converge: hash: sha16:ffaa state: converging wing (instance-a) puppet apply
  28. 28. jetstack.io WING Actual vs. Desired state apiVersion: wing/v1alpha1 kind: Instance metadata: name: instance-a spec: converge: hash: sha16:ffaa status: converge: hash: sha16:ffaa state: converged wing (instance-a)
  29. 29. Demo time jetstack.io
  30. 30. jetstack.io PUPPET MODULES Open source modules on PuppetForge ● aws_ebs attach, format, mount EBS volumes ● calico calico overlay network ● etcd etcd kv-store ● kubernetes kubernetes core components ● kubernetes_addons additional kubernetes components ● prometheus monitoring using prometheus ● tarmak tarmak kubernetes cluster ● vault_client get secrets, TLS credentials from vault
  31. 31. jetstack.io MEERKATS What about them?
  32. 32. github.com/jetstack/tarmak docs.tarmak.io @JetstackHQ Thanks! Get involved jetstack.io
  33. 33. jetstack.io TARMAK RESOURCES Providers, Environments and Clusters
  34. 34. WHO ARE JETSTACK? jetstack.io We are a UK-based company that help enterprises in their path to modern cloud-native infrastructure. We also develop tooling and integrations for Kubernetes to improve user experience and operations for customers and end-users alike. UK/EU customers include major e-commerce websites, telcos, media and publishers and government departments.
  • Alshaari

    Oct. 26, 2017

A walk through the various generations of Kubernetes deployments by Jetstack, and a reasoning for the development and launch of Tarmak the open-source toolkit for Kubernetes cluster lifecycle management.

Views

Total views

649

On Slideshare

0

From embeds

0

Number of embeds

14

Actions

Downloads

7

Shares

0

Comments

0

Likes

1

×