Learn how BOSH deploys distributed systems. Plus, Discover Diego, the flexible container engine, inside Cloud Foundry… or as the standalone Lattice engine!

1. BOSH
&
Diego
Benjamin Gandon
@BenjGa
2016-04-04

2. @BenjGa
Founder & CEO @ Gstack.io
Formerly: CTO @ Alvarum.com
Benjamin Gandon
BOSH
At:
We do:
And:

3. BOSH
Deploys Distributed Systems
Argh! It’s again those Americans! Don’t worry, Herr General, we will find out soon!

4. What is BOSH
BOSH is a flexible and resilient
technology that ………………………
complete ecosystems of distributed
software.
1. Package
2. Deploy
3. Monitor
4. Manage
(start/stop, scale, resurrect, update, …)
5. Upgrade

5. Instance in a BOSH Deployment
Container (or VM)
Filesystem + BOSH Agent
NginxHAproxy Pcre
HAproxy Nginx
Stemcell
Packages
Jobs
Instance Stemcell
+ Jobs
+ Packages
= Instance
Equation
/var/vcap/jobs/*
/var/vcap/packages/*

6. Availability Zone #3
AZ2AZ1
What we do with those “instances”
HAproxy
Stemcell
Pkg1 Pkg2
Nginx
Stemcell
Pkg3 Pkg2
HAproxy
Stemcell
Pkg1 Pkg2
Nginx
Stemcell
Pkg3 Pkg2
Nginx
Stemcell
Pkg3 Pkg2

7. ● Garden containers (in a VitrualBox VM)
● Docker (experimental)
● AWS
● OpenStack
● CloudStack
● Azure
● vSphere
BOSH secret? → Use abstract Cloud Provider Interfaces (CPIs)
Deploy where ?

8. Diego
Flexible Container Engine
From

9. Try at home! Because…
● BOSH
● Cloud Foundry
● Diego
Open
Source

10. Why “diego”?
● Since 2019, intuition for stateless “not-yet-containers” → the Droplets
● Their engine: the Droplet Execution Agent
→ DEA
● When rewritten in Go: DEA-GO
→ Diego!

11. Diego architecture
(logs)
(controller)
(http routes)
Source: https://github.com/cloudfoundry-incubator/diego-design-notes

12. Not just about containers
● Long Running Processes
→ LRPs
● Tasks
→ Servers
→ Cron jobs

13. Two ways to use Diego
● Multi-tenant Diego
→ as Cloud Foundry central container engine
● Single-tenant Diego
→ as Lattice, a single-tenant Diego cluster
Dude, go to http://lattice.cf

14. Lattice is definitely cool
● Containerized workloads
○ Long running, or temporary tasks
○ Dynamically scaled
○ Dynamically balanced across cells (but no live re-balancing)
● Cluster scheduler
● HTTP load balancing
● Log aggregation
● Health management

15. Some code!

16. Diego Auctions Scoring
// See: github.com/cloudfoundry-incubator/rep/…/resources.go#L15-L24
type CellState struct {
RootFSProviders RootFSProviders
AvailableResources Resources
TotalResources Resources
LRPs []LRP
Tasks []Task
StartingContainerCount int
Zone string
Evacuating bool
VolumeDrivers []string
}
// See: github.com/cloudfoundry-incubator/rep/…/resources.go#L74-L79
func (c CellState) ComputeScore(res *Resource, startingContainerWeight float64) float64 {
remainingResources := c.AvailableResources.Copy()
remainingResources.Subtract(res)
startingContainerScore := float64(c.StartingContainerCount) * startingContainerWeight
return remainingResources.ComputeScore(&c.TotalResources) + startingContainerScore
}
// See: github.com/cloudfoundry-incubator/rep/…/resources.go#L90-L93
type Resources struct {
MemoryMB int32
DiskMB int32
Containers int
}
// See: github.com/cloudfoundry-incubator/rep/…/resources.go#L110-L115
func (r *Resources) ComputeScore(total *Resources) float64 {
fractionUsedMemory := 1.0 - float64(r.MemoryMB)/float64(total.MemoryMB)
fractionUsedDisk := 1.0 - float64(r.DiskMB)/float64(total.DiskMB)
fractionUsedContainers := 1.0 - float64(r.Containers)/float64(total.Containers)
return (fractionUsedMemory + fractionUsedDisk + fractionUsedContainers) / 3.0
}

17. // See: github.com/cloudfoundry-incubator/auction/…/auctionrunner/scheduler.go#L206-L258
func (s *Scheduler) scheduleLRPAuction(lrpAuction *auctiontypes.LRPAuction) (*auctiontypes.LRPAuction, error) {
var winnerCell *Cell
winnerScore := 1e20
// Lists all zones, and count how many times the LRP is already in each of them
zones := accumulateZonesByInstances(s.zones, lrpAuction.ProcessGuid)
// Retain only zones with compatible filesystem (e.g. Linux or Windows)
filteredZones := filterZones(zones, lrpAuction)
if len(filteredZones) == 0 {
return nil, auctiontypes.ErrorCellMismatch
}
sortedZones := sortZonesByInstances(filteredZones)
// …
}
Diego Auctions Evaluation (1/3)

18. // See: github.com/cloudfoundry-incubator/auction/…/auctionrunner/scheduler.go#L206-L258
func (s *Scheduler) scheduleLRPAuction(lrpAuction *auctiontypes.LRPAuction) (*auctiontypes.LRPAuction, error) {
// …
for zoneIndex, lrpByZone := range sortedZones {
for _, cell := range lrpByZone.zone {
score, err := cell.ScoreForLRP(&lrpAuction.LRP, s.startingContainerWeight)
if err != nil {
continue
}
// Look for the Diego cell with minimal overall utilization
if score < winnerScore {
winnerScore = score
winnerCell = cell
}
}
// if (not last zone) && (this zone has the same # of instances as the next sorted zone)
// acts as a tie breaker
if zoneIndex+1 < len(sortedZones) &&
lrpByZone.instances == sortedZones[zoneIndex+1].instances {
continue
}
if winnerCell != nil {
break
}
}
// …
}
Diego Auctions Evaluation (2/3)

19. // See: github.com/cloudfoundry-incubator/auction/…/auctionrunner/scheduler.go#L206-L258
func (s *Scheduler) scheduleLRPAuction(lrpAuction *auctiontypes.LRPAuction) (*auctiontypes.LRPAuction, error) {
// …
if winnerCell == nil {
return nil, rep.ErrorInsufficientResources
}
err := winnerCell.ReserveLRP(&lrpAuction.LRP)
if err != nil {
s.logger.Error("lrp-failed-to-reserve-cell", err, lager.Data{"cell-guid": winnerCell.Guid, "lrp-guid": lrpAuction.Identifier()})
return nil, err
}
winningAuction := lrpAuction.Copy()
// Return a copy of the auction containing the winning cell
winningAuction.Winner = winnerCell.Guid
return &winningAuction, nil
}
Diego Auctions Evaluation (3/3)
→ Simple as that!

20. Thank you!

21. www.gstack.io (not yet in Google)
www.bosh.io
www.lattice.cf
www.cloudfoundry.org
Questions?
↘ ↙
→ @BenjGa ←
↗ ↖