This document discusses managing Kubernetes clusters using BOSH. It begins with an introduction to Kubernetes and some of its key capabilities and pain points in managing distributed systems at scale. It then introduces BOSH and how it can help solve problems around infrastructure provisioning, rolling upgrades, release engineering, self-healing, and storage management. The document explains the basic principles of BOSH around identifiability, reproducibility, consistency and agility. It argues that BOSH is well-suited for Kubernetes because it enables high availability, scaling, virtual machine self-healing and avoids vendor lock-in. It provides an example BOSH deployment manifest and discusses how to get started with BOSH and Kubo deployments.

1. Managing Kubernetes cluster using BOSH
Ronak Banka
DevOps Days India 2017

2. $whoami
Ronak Banka
Senior Platform Engineer @ Rakuten
@ronakbanka

3. Rakuten
13k+ employee, +3k engineers, 40+ subsidiaries

4. FCB Global Partner
Partners Golden State WARRIORS

5. Rakuten - Five Principles for
Success
• Always Improve, Always Advance
• Passionately Professional
• Hypothesize → Practice → Validate → Shikumika
• Maximize Customer Satisfaction
• Speed!! Speed!! Speed!!

6. What’s in pipeline for today
• What is Kubernetes?
• Let’s take a look at pain points!
• What is BOSH?
• Why Kubernetes with BOSH?
• Q&A

7. What is Kubernetes?
“Kubernetes is an open-source platform for
automating deployment, scaling, and operations of
application containers across clusters of hosts,
providing container-centric infrastructure.”

8. What does Kubernetes do?
• Provide a runtime environment for Docker
containers
• Scale and load balance docker containers
• Abstract away the infrastructure containers run on
• Monitor/health check containers
• Declarative definition for running containers
• Update containers (also rolling updates)
• Storage mounting (allow abstracting infrastructure)
• Service discovery and exposure

9. Pain Points!
With growing number of automation and deployment tooling,
it is getting harder to choose right one & maintain distributed
system deployments across different environments.
• High availability
• Scaling
• Self healing infrastructure
• Rolling upgrades
• Vendor lock-in

10. ”BOSH is an open source tool for release
engineering, deployment, lifecycle management,
and monitoring of distributed systems.”

11. What problems does BOSH solve?
• Infrastructure provisioning
• Rolling upgrades via canaries
• Release Engineering(packaging,deployments)
• Self-healing w/ Resurrector
• OS updates / Security
• Storage management

13. What does a BOSH deployment
look like?
• Stemcell – Versioned OS image packed with IaaS
specific packaging, bare minimum OS with no
knowledge of software it will run.
• Releases – versioned collection of configuration
properties, configuration templates, start up scripts,
source code, binary artifacts, and anything else
required to build and deploy software in a reproducible
way
• Deployment – A YAML file that maps various
releases/jobs along with stemcell and specifies how to
orchestrate and configure them for a deployment.

14. Maria Shaldibina of Pivotal presents "Building Your First BOSH Release.“

15. How does BOSH achieve
this??
With the help of declarative infrastructure, BOSH ensures system consistency
across different deployment environments by following some basic principles of
release engineering.
• Identifiability
• Reproducibility
• Consistency
• Agility

16. Identifiability
• BOSH has a concept of a software release which
packages up all related source code, binary assets,
configuration etc. This allows users to easily track
contents of a particular release.
• In addition to releases BOSH provides a way to
capture all Operating System dependencies as one
image.

17. Reproducibility
• BOSH tool chain provides a centralized server that
manages software releases, Operating System
images, persistent data, and system configuration
• It provides a clear and simple way of operating a
deployed system

18. Consistency
• BOSH software releases workflows are used
throughout the development of the software and
when the system needs to be deployed
• BOSH centralized server allows users to see and
track changes made to the deployed system.

19. Agility
• BOSH tool chain integrates well with current best
practices of software engineering (including
Continuous Delivery).
• software releases can be created in an automated
way and to update complex deployed systems with
simple commands

20. Why Kubernetes using BOSH?
• HA : BOSH can provision components across different AZ’s to make platform
stable and resilient.
• Scaling: BOSH allows cluster operator to upscale/downscale the # instances
simply by changing #instances in the bosh manifest.
• VM Self-Healing: BOSH will continuously monitor the health of all VM instances
and recreate VMs
• No vendor lock-in: BOSH is capable of provisioning systems across various clouds
including - Aws - Azure - Google cloud platform - Vsphere - Openstack

21. Deployment info
releases
Networks
Disk pools
compilation
update
Instance_groups
---
instance_groups:
- azs:
- z1
instances: 3
jobs:
- name: etcd
properties:
etcd:
peer_require_ssl: false
require_ssl: false
release: kubo-etcd
name: etcd
networks:
- name: my-kubokubo-network
persistent_disk_type: 5120
stemcell: trusty
vm_type: common

22. Availability zones
azs:
- name: z1
cloud_properties:
zone: us-east1-b
- name: z2
cloud_properties:
zone: us-east1-c
- name: z3
cloud_properties:
zone: us-east1-d
instance_groups:
- name: etcd
instances: 3
jobs:
- name: etcd
release: kubo-etcd
vm_type: common
stemcell: default
networks:
- name: my-kubokubo-network
azs: [z1, z2,z3]
Cloud config manifest

23. Deployment Hooks
install pre-start start post-start post-deploy
drain

26. Kubernetes Deployment with
BOSH

27. Getting started with BOSH and
Kubo
• Bosh Deployment Repo
• https://github.com/cloudfoundry/bosh-deployment
• Kubo Deployment Repo
• https://github.com/cloudfoundry-inclubator/kubo-
deployment
• Documentation
• bosh.io

28. Q&A?