Modern DevOps practices involve deploying applications to platforms. From basic IaaS to PaaS to serverless functions. But who runs those platforms and how? At Pivotal we build and operate platforms, and we run those platforms on a platform designed to run complex distributed systems called Bosh which was inspired by google borg. Paul will talk through a couple of successful patterns for deploying and operating platforms as well as how to help your business determine which platform[s] are right for them and how to successfully get the business to adopt those platforms.

1. © Copyright 2018 Pivotal Software, Inc. All rights Reserved. Version 1.0
Paul Czarkowski / @pczarkowski
Principal Technologist
Pivotal software
BOSH: running platforms so you
can run platforms on your
platforms

2. © Copyright 2018 Pivotal Software, Inc. All rights Reserved. Version 1.0
Paul Czarkowski / @pczarkowski
Principal Technologist
Pivotal software
Sucessful Patterns for
Deploying [and Operating]
Platforms.

3. Do you know Cloud Foundry?

4. Do you know Kubernetes ?

5. Do you know BOSH ?

6. What is a platform ?
https://en.wikipedia.org/wiki/Computing_platform

7. A modern software
platform provides
API driven compute
resources.

8. “Every IT team
is a
Platform Team.”

10. http://slides.eightypercent.net/platform-platform/index.html#1

11. https://twitter.com/kelseyhightower/status/935252923721793536

12. Generic Platform
API
Users
Storage Compute NetworkDatabase AccessArtifacts

13. Enterprise IT
API
Users
Systems
Admin
Network
Engineer
SecurityDBA QA
Storage
Admin

15. “You don’t have a platform
problem, you have a culture
problem.”

16. Infra
Services
App
Platform
Evolve your IT teams!
Platform
Team
Application
Team
Build common services
for App Teams
Take business
requirements and turn
them into features
IaaS
Virtual Infrastructure
Physical Infrastructure
Abstract infrastructure
complexity with easy
consumption
DBaaSELK
App2App1 App3
Middleware
ML
Creds/CertsMessaging
???
Container Services
Container Hosts | Kubernetes
Infrastructure
Team

17. People build Platforms
People build Apps
Apps run on Platforms
People are the most
important component
of any platform.

18. Traditional
Ticket
Based
Human
Toil
Build App
Artifact
Build App
Container(s)
App → to the
Platform
Container Runtime
Container Hosts
CaaS
Container
Orchestrator
PaaS
Application Platform
IaaS
CaaS
PaaS
Infrastructure
As
Code
What Abstraction is the Right One for You ?
More Control
Less Complexity
IaaS API
CF API
K8s API
Config
Management
Build
Application
Dependencies
IaaSHardware
PXE boot ?
18

19. Hardware
IaaS
CaaS
PaaS
FaaS
Strategic goal: Push as many workloads as technically
feasible to the top of the platform hierarchy
Higher flexibility and
less enforcement of
standards
Lower development
complexity and higher
operational efficiency

20. Hardware
IaaS
CaaS
PaaS
FaaS
Infrastructure
Team
Infrastructure Team
Platform Team
Platform Team
Infrastructure Team
App Team
App Team
Platform Team
App Team
Platform Team
Infrastructure Team
Consumers of the abstraction

21. “In general, taking something that’s
already working somewhere and
expanding its usage (capabilities) is far
more likely to succeed than building
these capabilities from scratch”

22. Embedded OS
(Windows & Linux)
NSX-T
CPI (15 methods)
v1
v2
v3
...
CVEs
Product Updates
Java | .NET | NodeJS
Pivotal Application
Service (PAS)
Application Code & Frameworks
Buildpacks | Spring Boot | Spring Cloud |
Steeltoe
Elastic | Packaged Software | Spark
Pivotal Container
Service (PKS)
>cf push >kubectl run
YOU build the containerWE build the container
vSphere
Azure &
Azure StackGoogle CloudAWSOpenstack
Pivotal
Network
“3Rs”
Github
Concourse
Concourse
Pivotal Services
Marketplace
Pivotal and
Partner Products
Continuous
delivery
Public Cloud
Services
Customer
Managed
Services
Repair
— CVEs
Repave Rotate
— Credhub

24. The Google Problem
x 1,000,000

28. API Server
Kube
Scheduler
K8s
MasterContr
oller
Mana
ger
Etcd
C
Kubelet
Kube-
proxy
K8s
Worker
Pod
Pod
Pod
K8s
Worker
Pod
Pod
Pod
K8s
Worker
Pod
Pod
Pod
CNI CNI CNI
Docker
Kubelet
Kube-
proxy
Docker
Kubelet
Kube-
proxy
Docker

30. API Server
Kube Scheduler
K8s Master
Controller
Manager
Etcd
C
Kubelet
Kube-proxy
K8s Worker
Pod
Pod
Pod
K8s Worker
Pod
Pod
Pod
K8s Worker
Pod
Pod
Pod
CNI CNI CNI
Docker
Kubelet
Kube-proxy
Docker
Kubelet
Kube-proxy
Docker
Google Compute Engine
(hidden from user)
Google Compute Engine

32. Traditional
Ticket
Based
Human
Toil
Build App
Artifact
Build App
Container(s)
App → to the
Platform
Container Runtime
Container Hosts
CaaS
Container
Orchestrator
PaaS
Application Platform
IaaS
CaaS
PaaS
Infrastructure
As
Code
What Abstraction is the Right One for You ?
More Control
Less Complexity
IaaS API
CF API
K8s API
Config
Management
Build
Application
Dependencies
IaaSHardware
PXE boot ?
32

34. BOSH - Component Architecture
Director
Postgres
DB
NATS
CLI
Health
Monitor
Blob
Store
IaaS
API
Cloud
Provider
Interface
IaaS
BOSH provides the means to go
from deployment configuration to VM
creation and management. It
includes interfaces for Azure,
vSphere, AWS, GCP, and
OpenStack. Additional CPI can be
written for alternative IaaS providers.
Registry

35. w
BOSH - Service Deployment
Manifest
Release
Stemcell
Deployment

36. BOSH - Stemcell
Stemcell
▪ Secured, Hardened, and Versioned Operating System
image wrapped with IaaS specific packaging
▪ Contains a bare minimum OS skeleton with a few
common utilities pre-installed, a BOSH Agent, and a
few configuration files to securely configure the OS by
default.
▪ Images come in two flavors Ubuntu 14.04 and CentOS
7 for all IaaS’ supported
▪ Maintained by BOSH team and available at
http://bosh.io/stemcells

37. BOSH - Release
Job(s)
Package(s)
Monit
Packaging
Spec
Spec
Src
Blobs
Dependency
Tree:
Organization:Elements:
Jobs - Pieces of the service or application you are releasing,
including how to compile & run them
Packages - Provide source code and dependencies to jobs
Src - Non-binary files which is provided to packages
Blobs - Provide binary files (other than those checked into a
source code repository) to packages
Monit - Script utilized to start/stop/restart the job
Packaging - Script utilized to compile the source
needed by a job
Spec - Key/Value file which stores all configuration
properties which can be set externally
Green - Script (erb or bash)
Orange - Properties (yml)

38. BOSH - Manifest
▪ Deployment Identification: A name for the deployment and the UUID of the Director managing the deployment
▪ Releases Block: Name and version of each release in a deployment
▪ Stemcells Block: Name and version of each stemcell in a deployment
▪ Update Block: Defines how BOSH updates instances during deployment
▪ Instance Groups Block: Configuration and resource information for instance groups (Jobs)
▪ Properties Block: Describes global properties and generalized configuration information
Required Blocks:
YAML - Primer found at end of presentation
Provides the ability to customize BOSH releases (your service)

39. BOSH - Service Deployment
Director
Postgres
DB
NATS
CLI
Health
Monitor
Blob
Store
IaaS
API
Cloud
Provider
Interface
Registry
Stemcel
l
*
Release
*
Manifest Config

40. BOSH - Process High Availability
Director
NATS
IaaS
Restart!Health
Monitor
Alert Sent!

41. BOSH - VM High Availability
Manifest - Desired
State
Director
NATS
IaaS
Health
Monitor
Alert Sent!
=
IaaS
API
Cloud
Provider
Interface

42. BOSH - Canary Upgrades
Manifest -
Code Update 4.04!
Any update error causes the deployment
to stop. Since only canaries are affected
before an update stops, problem jobs and
packages are prevented from taking over
all instances.

43. BOSH - Day 2 Ops
Consistent, Reliable, Scalable, Secure
▪ Checks against “desired state to return
consistency
▪ No ad hoc automation burden
▪ Manage services, not servers
▪ 4 layers of Self Healing
Provision
Deploy
and
Configure
Monitor
and
Detect
Re-mediate
Current
State
Desired
State

47. IaaS
Node
Node
Kubernetes
Cluster Services
API
Cluster3
NSX-T
vSphere
PKS includes:
• PKS Control Plane, CFCR
• NSX-T, Harbor, GCP Broker
• BOSH Release for Kubernetes
• Configures Day 1 of
- CFCR
- vSphere
- NSX Integration
- Harbor
• Manages Day 2 of
Kubernetes Clusters
- Scaling
- Patching
- Upgrades
- Failures
Kubo CFCR
Kubernetes
(As a Bosh Release)
BOSH
(Deploys/Manages VMs)
CPI
CNI
Harbor
Private Container
Registry
PKS “How it Works”
The value of BOSH
Node
Node
Node
Kubernetes
Cluster Services
API
Node
Node
Node
Kubernetes
Cluster Services
API
Node
Cluster1
Cluster2
GCP
Service
Broker
API
#pks create-cluster K8s-1 -n 3#pks create-cluster K8s-2 -n 3#pks create-cluster K8s-3 -n 3# pks create-cluster --plan small Cluster3
PKS Control Plane
V
M
V
M
V
M
V
M
V
M
V
M
V
M
V
M
V
M
Node
Node
Kubernetes
Cluster Services
API
Cluster3 Node
Node
Node
V
M
V
M
V
M
V
M
V
M

49. Hardware
IaaS
CaaS
PaaS
FaaS
HPE, Dell,
IBM, Lenovo
AWS, Azure,
GCP, Openstack,
VMWare
PKS, GKE,
OpenShift, AWS
Fargate, Kubernetes
PCF, Azure App
Service, Heroku,
Deis
AWS Lambda, Azure
Functions, OpenWhisk,
kubeless, PFS

53. http://www.bsielearning.com.au/keep-simple-stupid/

54. Full Opensource DIY
● ??? pxe
● Openstack Ansible
● Kubespray
● Ansible Hardening
https://github.com/openstack/openstack-ansible
https://github.com/openstack/ansible-hardening
https://github.com/kubernetes-incubator/kubespray

55. !!!

56. PLATFORM VALUE STREAM AND METRICS
REPLATFORM > MODERNIZE > OPTIMIZE
ESTABLISH, MEASURE AND UPDATE
KEY OBJECTIVES AND RESULTS (OKRs)
SPEED & AGILITY STABILITY
SCALABILITY SAVINGS
$SECURITY
40-60%*
More Projects With
Same Staff
Millions
Annual Savings on
HW, SW and
Support
25-50%*
Fewer Support
Incidents
40%*
Faster Patching
Delivery @ Zero
Downtime
-90%*
Time to Scale
$
$
%
How We Think about the Business Case

57. Questions ?
THE END