Dealing with high-load services of all kinds makes us to seek for new generation tools to build reliable, scalable, and 100% available systems. At this workshop, you will have chance to dive deep into how Cloud Foundry solves the issues of portability, scalability, reliability and extensibility. Hands-on agenda: - Application lifecycle: from development to production - Deep dive into Cloud Foundry architecture - Where to deploy Cloud Foundry - How to Deploy Cloud Foundry: from small evaluation to hundreds VMs High Availability production environments - Scale up and down your infrastructure. Can you auto scale? - Zero downtime upgrades - Auto Healing deployments - Cloud Foundry system logging and monitoring - Services: types, current restrictions and expectations

1. Hands-on Workshopfor Operators
Manuel Garcia / altoros.com / @rmgarciap

2. * Whythis training?
•SysAdmins and DevOps requested it in several
meetup Altoros talks
•Evolution from a minimalistic CF local installation
workshop to a full CF deployment done with BOSH

3. * Goals
•Understand how Cloud Foundry is deployed
•Get to know how Cloud Foundry internally works
from an PaaS Operator perspective

4. Hands on?
1, 2, 3… go!

5. * How do I deploy Cloudfoundry?
Nise Bosh, a lightweight BOSH emulator. Virtual and bare metal
Altoros Vagrant Installer, developer oriented deployment
BOSH, tool chain for release engineering
Bosh Lite, a lite development environment for BOSH.
Conteinerized VMs
Canonical Juju Charms, cloud infrastructure automation

6. * It is so easy
•Install Bosh
•Deploy Something (ex.: ElasticSearch)
•Upload stemcell
•Upload release
•Configure deployment manifest
•Deploy !

7. * Bosh Lite – From Local to theCloud
• Prerequisites
• GIT
• Ruby 1.9.3 (latest. 2.0.X not supported)
• RubyGems and Bundler
• VirtualBox
• Vagrant
• Clone repo and deploy bosh lite (preferable local)
• Lower RAM if needed (Vagrantfile)
• $ vagrant up

8. * Bosh Lite – From Local to theCloud
• Upload Stemcell
• $ bosh public stemcells
• $ bosh download public stemcell bosh….tgz
• Download Elasticsearch bosh release
• $ git clone https://github.com/bonzofenix/elasticsearch-boshrelease
• Upload it to Bosh
• $ bosh upload release releases/<version>.yml
• Deploy ElasticSeach
• $ bosh manifest <elasticsearch manifest file>
• $ bosh deploy

9. * What is Bosh? Why BOSH?
Designed for large scale, distributed services
Tool chain for release engineering, deployment and lifecycle
management
Already Supports AWS, OpenStack & VMware vSphere (Cloudstack)
Two floors up from Chef/Puppet. Multi-cloud, IaaS Provider independent
Updates & Operates Deployments
Deploys & Manages Clusters of Cloud Foundry, Databases, etc

10. BOSH Overview

12. BOSH Releases

13. * What is a release?
• A collection of configuration:
– files,
– job definitions
– source code
– package definitions
– and accompanying information needed to make a
software component deployable by BOSH.
• A release should have no dependencies that need
to be fetched from the internet.

14. * What is a release?
• Source directories
– jobs: start and stop commands for each of the jobs
(processes) running on Cloud Foundry nodes.
– packages: packaging instructions used by BOSH to build
each of the dependencies.
– src: the source code for the components in Cloud Foundry.
Note that each of the components is a submodule with a
pointer to a specific SHA.

15. * What is a release?
• Releases directories
– releases: yml files containing the references to blobs for
each package in a given release; these are solved within
.final_builds
– .final_builds: references into the public blostore for final
jobs & packages (each referenced by one or more releases)
– config: URLs and access credentials to the bosh blobstore
for storing final releases

16. * Example BOSH Release
• ElasticSearch release
•  Take me to the repo…

17. * Cloud Foundry BOSH Release
• Around 20 jobs
• Open Source: github.com/cloudfoundry/cf-release
• Weekly releses (releases directory)
• Fully tested (CAT)

18. * Bosh Lite (lets continue)
• Upload Warden Stemcell
• $ bosh public stemcells
• $ bosh upload stemcell latest-bosh-stemcell-warden.tgz

19. * Stemcells
• A minimal VM image that can convert into anything
• Contains a BOSH Agent: A process that runs continuously on
each VM that BOSH deploys (one Agent process per VM).
The BOSH Agent executes tasks in response to messages it
receives from the BOSH Director

20. * In the meantime.. What have we done?
• MicroBosh in a local VM
• Director, public API
• Blobstore, to store and retrieve
precompiled packages
• Health Manager, to track the state of
deployed systems
• Internal DNS, called PowerDNS, for
internal unique naming of servers
within bosh deployments
• Bosh Database, desired state of a
BOSH deployment
• Message Bus (NATS)
• Registry, for tracking the
infrastructure that has been
provisioned (servers, persistent disks)
• Resurrector
• Task Queue (requires Redis), async
queue used by the BOSH Director and
Workers to manage tasks

21. Example Component Interaction

23. Lets Deploy Cloud Foundry

24. * CF deployment steps
• Download Release from repo and upload it to
BOSH
– $ git clone https://github.com/cloudfoundry/cf-release
– $ bosh upload release releases/cf-169.yml
– Check it is there: $ bosh releases
• Build deployment manifest and tell BOSH to use it
– $ ./scripts/make_manifest_spiff
– $ bosh deployment manifests/cf-manifest.yml
• Deploy: $ bosh deploy

25. Test your Cloud Foundry Deployment

26. * Target and deploy
• $ cf login -a https://api.10.0.244.34.xip.io
• Download, build and deploy the app
– $ git clone https://github.com/mgarciap/cf-ruby-example.git
– $ cd cf-ruby-example
– $ cf push
• App metadata?
– Manifests

27. Runtimes and Frameworks
Buildpacks

28. *
• Java
– Java, Grails, Play, Spring or any other JVM-based language or
framework
• Node.js
– Node or JavaScript
• Ruby
– Ruby, Rack, Rails or Sinatra
• Go Lang
Cloud FoundrySystem Buildpacks

29. *
> cf push = deploy
CLI Cloud
Controller
CCDB
(MySQL
)
Blob
Store
(S3,
etc.)
Executor
Stager
W
Build
packs
A2
A2 A3
A3
A1
A1
Pkg
Metadata
PkgMetadata Pkg
Droplet
Droplet
Users
R
o
u
t
e
r
A1.yourdomain.com
Frontend Backend
Stage A1
Deploy A1
DEA Nodes

30. *
• $ cf app [app]
• $ cf logs [app]
• Logs are streamed. CC API, Staging, DEA, Router – HTTP and finally
your app
• Dump: cf logs [app] –recent
• $ cf env [app name]
• $ cf events [app name]
• $ cf files [app]
Something wrong deploying theapp?

31. Services
( *aaS )

32. *
• They can be anything external resource as far as they provide
an API and they are registered with the CC
• Actions
• Provision/deprovision
• Bind/unbind
Services

33. No downtime deployments
Blue-green deployment release technique

34. *
• One production domain and two apps
• Blue  prod
• Green  next release
• $ cf push Blue -n demo-time
Blue-GreenDeployment

35. *
• Update App and Push
• $ cf push Green -n demo-time-
temp
Blue-GreenDeployment

36. *
• Map Original Route to Green
• $ cf map-route Green
example.com -n demo-time
Binding demo-time.example.com to
Green... OK
Blue-GreenDeployment

37. *
• Unmap Route to Blue
• cf unmap-route Blue example.com
-n demo-time-temp
Unbinding demo-time.example.com
from blue... OK
Blue-GreenDeployment

38. *
• Remove Temporary Route to Green
• $ cf unmap-route Green
example.com -n demo-time-temp
Blue-GreenDeployment

39. Key architectural characteristics

40. *
Portable
Key architecturalcharacteristics

41. *
Portable
Key architecturalcharacteristics
https://github.com/cloudfoundry
Open Source

42. *
Scalable
• From few servers to thousands
• Horizontally and Vertically
Key architecturalcharacteristics

43. *
Reliable
Very few Single Points of Failure which are been improved
(Message Bus -NAT S server-, Collector)
Key architecturalcharacteristics

44. *
Extensible
Loosely Coupled Components with specific responsibilities and
technology agnostic intercommunication through a message
bus.
Ruby? Rewrite in GO lang? No problem
Key architecturalcharacteristics

45. Cloud Foundry Architecture
Overview

46. * Core components

47. * What have we done?
• Install BOSH (with bosh lite)
• Install BOSH CLI
• Upload stemcell
• Upload Release
• Create and configure Deployment Manifest
• Deploy CF

48. * What have we done?
• Install Cloud Foundry CLI (cf)
• Target and log into CF
• Create organization and space
• Push an application

49. * What is next? Deploy CF into a IaaS
• Small/Medium deployment for demos/testing
• Microbosh
• Medium to large production deployments
• Deploy Microbosh
• With Microbosh deploy BOSH
• From BOSH deploy CF

50. Thank you
manuel.garcia@altoros.com
@rmgarciap
(650) 395-7002