Your SlideShare is downloading. ×
0
BUILD YOUR VERY OWN
PRIVATE CLOUD FOUNDRY

MATT STINE
COMMUNITY ENGINEER, CLOUD FOUNDRY
MSTINE@GOPIVOTAL.COM
HTTP://CLOUDF...
Agenda
•CF Architecture - Quick Overview
•What is BOSH? Why BOSH?
•BOSH Architecture
•BOSH Releases
•BOSH Deployments
•CF ...
Cloud Foundry Architecture
(in FIVE minutes or less!)
Cloud Foundry Architecture
Router
User	
  Authen=ca=on	
  and	
  Authoriza=on
Health	
  Manager
Cloud	
  Controller
Apps
S...
Router
e.
co
m
kt
g.
ac
m
ap
p1
.m

DEV
A

S

TEST
S

m
co
e.
m
ac
g.

MARKETING ORG

n
.e
p2
ap

•Load balancing
•Maintai...
UAA
• Token Server
• ID Server (User management)
• OAuth Scopes (Groups)
• Login Server
• UAA Database
• SAML support (for...
Cloud Controller
• App expected state
• Permissions/Auth
• Orgs/Spaces/Users
• Services management
• App placement
• App d...
Messaging (NATS)
•Non-Persistent messaging
•Pub/Sub
•Queues (app events)
•Directed messages (INBOX)
Droplet Execution Agent (DEA)
•Manage Linux containers (Warden)
• Process
• File system
• Network
• Memory
•Manage app lif...
Health Manager
HealthManager
SCHEDULER

•Maintains the actual state of
apps

•
•Sends suggestions to make

Compares to exp...
Service Broker
•Advertising service catalog
•Makes create/delete/bind/unbind calls to service
nodes

•Requests inventory o...
CF Architecture: Why?
•Component Isolation
•Scalability
•Fault Tolerance
•Pre-provisioned Capacity (Containers / Warden)
How do we manage this thing?
We need a toolchain that can manage
a large distributed system through:
•Deployment
•Configuration Changes
•Updates/Upgrade...
What about?
BOSH!
(Bosh Outer SHell)
What is BOSH?
Why BOSH?
•Provision services, not machines
•Eliminate bespoke automation on top of configuration
management

•Enable conti...
BOSH Architecture
BOSH Architecture
Director
• Core orchestrating component
• Controls:
• creation/deletion of VMs, Disks, Networks
• software deployment
• li...
Agent
•Carries out instructions from the Director
•Fetches packages/jobs from Blobstore
•Installs packages/jobs onto VM
•S...
Message Bus (NATS)
•Non-Persistent messaging
•Pub/Sub
•Queues (app events)
•Directed messages (INBOX)
Health Monitor
•Receives from Agent:
• VM health status
• Agent lifecycle events
•Triggers:
• Alerts (Email, CloudWatch, D...
Stemcells
•“Pleuripotent” VMs
• Base VM filesystem image
• Never booted
• Currently ~Ubuntu 10.04.4 LTS
• Embedded BOSH Age...
Blobstore
•
•
•
•
•

Stores the content of BOSH Releases
Uploaded via BOSH CLI
Stored via Director
Also stores:

•
•

pack...
Cloud Provider Interface (CPI)
• current_vm_id
• create_stemcell
• delete_stemcell
• create_vm
• delete_vm
• has_vm?
• reb...
Great! So how do I deploy my stuff
with BOSH?
BOSH Releases
Hello BOSH Release!

Redis
Anatomy of a BOSH Release
• Release Manifest
• Package
• spec
• packaging
• Job
• spec
• monit
• templates
Release Manifest

redis-boshrelease/releases/redis-1.yml
Package

redis-boshrelease/packages/redis/spec
Packages

redis-boshrelease/packages/redis/packaging
Jobs

redis-boshrelease/jobs/redis/spec
Jobs
http://mmonit.com/monit

redis-boshrelease/jobs/redis/monit
Jobs

redis-boshrelease/jobs/redis/templates/bin/redis_ctl
Jobs

redis-boshrelease/jobs/redis/templates/conf/redis.conf.erb
OK. So what happens when I
deploy?
BOSH Agent
Director creates VM from Stemcell

Director

VM

CPI
Agent Boots
Director

VM
Director Pings Agent
Director

NATS

VM
Director Assigns Job to Agent
Director

NATS

VM
Agent Fetches Blobs from Blobstore

VM

HTTP

Blobstore
Agent Starts Jobs
http://mmonit.com/monit

VM
Now let’s go outside-in!
BOSH Deployments
Deployment Manifests
• The mapping of a BOSH release to infrastructure
• Specifies:
• Release
• Compilation VMs
• Update (c...
redis-boshrelease/examples/aws-solo.yml
redis-boshrelease/examples/aws-solo.yml
redis-boshrelease/examples/aws-solo.yml
redis-boshrelease/examples/aws-solo.yml
cf-release
Let’s follow one of our
components...
• GoRouter
• Release Manifest
• Package
• spec
• packaging
• Job
• spec
• monit
• te...
Release Manifest

cf-release/releases/cf-release-134.yml
Package

cf-release/packages/gorouter/spec
Package

cf-release/packages/gorouter/packaging
Job

cf-release/jobs/gorouter/spec
Jobs
http://mmonit.com/monit

cf-release/jobs/gorouter/monit
Jobs

cf-release/jobs/gorouter/templates/gorouter_ctl.erb
Jobs

cf-release/jobs/gorouter/templates/gorouter_yml.erb
Cloud Foundry on vSphere
vSphere 101
• Minimum Lab HW:
• 6 Cores (12 threads)
• 64 GB RAM
• 4 TB Disk
• vSphere 5.1 (2 ESXi hosts, 1 vCenter)
• can...
BOSH Bootstrap
•Provide vCenter Credentials (via http://fog.io/)
•Provide Network Settings
•Provide vCenter Settings (Data...
BOSH Bootstrap Deploy
BOSH Prepare CF
vSphere Deployment Manifest
vSphere Deployment Manifest
vSphere Deployment Manifest
vSphere Deployment Manifest
http://xip.io
BOSH Create CF
Cloud Foundry on AWS
AWS 101
•Requirements:
• AWS Credentials
• Capacity to provision 8 servers and 3 elastic IPs
• Approximately $0.42/hour yo...
BOSH Bootstrap
•Provide AWS Credentials (via http://fog.io/)
•Provide AWS Region
•Provision MicroBOSH!
BOSH Bootstrap Deploy
BOSH Prepare CF
AWS Deployment Manifest
AWS Deployment Manifest
AWS Deployment Manifest
AWS Deployment Manifest
BOSH Create CF
Create ‘cf’ Security Group
Allocate Elastic IP for CF Router
Cloud Foundry on OpenStack
OpenStack 101
•Requirements:
• OpenStack Credentials
• Capacity to provision 8 servers and 3 floating IPs
• Manually configu...
BOSH Bootstrap
•Provide OpenStack Credentials (via http://fog.io/)
•Provide OpenStack Region (optional)
•Provision MicroBO...
BOSH Bootstrap Deploy
BOSH Prepare CF
OpenStack Deployment Manifest
OpenStack Deployment Manifest
OpenStack Deployment Manifest
OpenStack Deployment Manifest
BOSH Create CF
Create ‘cf’ Security Group
Allocate Floating IP for CF Router
Web Console UI
Edit src/main/resources/
styx.properties
mvn clean package
cf push!
THANK YOU!

MATT STINE
COMMUNITY ENGINEER, CLOUD FOUNDRY
MSTINE@GOPIVOTAL.COM
HTTP://WWW.CLOUDFOUNDRY.COM
TWITTER: @MSTINE
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Build Your Very Own Private Cloud Foundry
Upcoming SlideShare
Loading in...5
×

Build Your Very Own Private Cloud Foundry

4,199

Published on

Speaker: Matt Stine
This session will focus on how you can build your very own Cloud Foundry private PaaS running in your own data center or on AWS or even on OpenStack on your own Mac mini. You will learn how the Cloud Foundry BOSH tool constructs a full Cloud Foundry instance from a bare bones virtual machine and continues to coordinate and manage the entire PaaS cloud once it is operational. If you want the convenience of developing against your own private custom PaaS within your company, then this session will give you all the steps you need to get started.

Published in: Technology

Transcript of "Build Your Very Own Private Cloud Foundry"

  1. 1. BUILD YOUR VERY OWN PRIVATE CLOUD FOUNDRY MATT STINE COMMUNITY ENGINEER, CLOUD FOUNDRY MSTINE@GOPIVOTAL.COM HTTP://CLOUDFOUNDRY.ORG TWITTER: @MSTINE
  2. 2. Agenda •CF Architecture - Quick Overview •What is BOSH? Why BOSH? •BOSH Architecture •BOSH Releases •BOSH Deployments •CF to BOSH: cf-release •[:vsphere, :openstack, :aws].each { |iaas| iaas.deploy_cf }
  3. 3. Cloud Foundry Architecture (in FIVE minutes or less!)
  4. 4. Cloud Foundry Architecture Router User  Authen=ca=on  and  Authoriza=on Health  Manager Cloud  Controller Apps Service  Broker Service  Nodes Messaging Build  Packs DEA  Pool    
  5. 5. Router e. co m kt g. ac m ap p1 .m DEV A S TEST S m co e. m ac g. MARKETING ORG n .e p2 ap •Load balancing •Maintain routing table •Access logs app3.finance.acme.co m ROUTER ENGINEERING ORG FINANCE ORG PROD S DEV A S TEST A S PROD DEV A PLATFORM A TEST PROD A
  6. 6. UAA • Token Server • ID Server (User management) • OAuth Scopes (Groups) • Login Server • UAA Database • SAML support (for SSO integration) and Active Directory support with the VMWare SSO Appliance • Access auditing
  7. 7. Cloud Controller • App expected state • Permissions/Auth • Orgs/Spaces/Users • Services management • App placement • App desired state convergence • Auditing/Journaling • Billing events • Blob storage
  8. 8. Messaging (NATS) •Non-Persistent messaging •Pub/Sub •Queues (app events) •Directed messages (INBOX)
  9. 9. Droplet Execution Agent (DEA) •Manage Linux containers (Warden) • Process • File system • Network • Memory •Manage app lifecycle •App log and file streaming •DEA heartbeats (NATS to CC, HM) Router App App App App DEA/Warden DEA/Warden
  10. 10. Health Manager HealthManager SCHEDULER •Maintains the actual state of apps • •Sends suggestions to make Compares to expected state actual match expected App App Schedule Observation App App HARMONIZER Observes Observes App Trigger Harmonization Desired State NUDGER Bulk API App Actual State NATS NATS Cloud Controller NATS DEA
  11. 11. Service Broker •Advertising service catalog •Makes create/delete/bind/unbind calls to service nodes •Requests inventory of existing instances and bindings from cloud controller for caching, orphan management •SaaS marketplace gateway
  12. 12. CF Architecture: Why? •Component Isolation •Scalability •Fault Tolerance •Pre-provisioned Capacity (Containers / Warden)
  13. 13. How do we manage this thing?
  14. 14. We need a toolchain that can manage a large distributed system through: •Deployment •Configuration Changes •Updates/Upgrades (w/ minimal - zero - downtime!) •Component Failure / Restoration of Service •Scale Out / Scale In •Across multiple IaaS providers: vSphere, OpenStack, AWS, and beyond...
  15. 15. What about?
  16. 16. BOSH! (Bosh Outer SHell)
  17. 17. What is BOSH?
  18. 18. Why BOSH? •Provision services, not machines •Eliminate bespoke automation on top of configuration management •Enable continuous delivery of platform services •Cloud-agnostic view of platform operations •Holistic toolchain to “rule them all” •How we manage Cloud Foundry in production!
  19. 19. BOSH Architecture
  20. 20. BOSH Architecture
  21. 21. Director • Core orchestrating component • Controls: • creation/deletion of VMs, Disks, Networks • software deployment • lifecycle events for software and services • Via: • Cloud Provider Interface • Director-Agent Interaction
  22. 22. Agent •Carries out instructions from the Director •Fetches packages/jobs from Blobstore •Installs packages/jobs onto VM •Starts/monitors jobs via Monit
  23. 23. Message Bus (NATS) •Non-Persistent messaging •Pub/Sub •Queues (app events) •Directed messages (INBOX)
  24. 24. Health Monitor •Receives from Agent: • VM health status • Agent lifecycle events •Triggers: • Alerts (Email, CloudWatch, DataDog, PagerDuty, ...) • Resurrection!
  25. 25. Stemcells •“Pleuripotent” VMs • Base VM filesystem image • Never booted • Currently ~Ubuntu 10.04.4 LTS • Embedded BOSH Agent
  26. 26. Blobstore • • • • • Stores the content of BOSH Releases Uploaded via BOSH CLI Stored via Director Also stores: • • packages compiled by BOSH intermediate location for large payloads Options: • • • • EMC Atmos S3 OpenStack Swift Simple/Local
  27. 27. Cloud Provider Interface (CPI) • current_vm_id • create_stemcell • delete_stemcell • create_vm • delete_vm • has_vm? • reboot_vm • set_vm_metadata • configure_networks • create_disk • delete_disk • attach_disk • snapshot_disk • delete_snapshot • detach_disk • get_disks
  28. 28. Great! So how do I deploy my stuff with BOSH?
  29. 29. BOSH Releases
  30. 30. Hello BOSH Release! Redis
  31. 31. Anatomy of a BOSH Release • Release Manifest • Package • spec • packaging • Job • spec • monit • templates
  32. 32. Release Manifest redis-boshrelease/releases/redis-1.yml
  33. 33. Package redis-boshrelease/packages/redis/spec
  34. 34. Packages redis-boshrelease/packages/redis/packaging
  35. 35. Jobs redis-boshrelease/jobs/redis/spec
  36. 36. Jobs http://mmonit.com/monit redis-boshrelease/jobs/redis/monit
  37. 37. Jobs redis-boshrelease/jobs/redis/templates/bin/redis_ctl
  38. 38. Jobs redis-boshrelease/jobs/redis/templates/conf/redis.conf.erb
  39. 39. OK. So what happens when I deploy?
  40. 40. BOSH Agent
  41. 41. Director creates VM from Stemcell Director VM CPI
  42. 42. Agent Boots Director VM
  43. 43. Director Pings Agent Director NATS VM
  44. 44. Director Assigns Job to Agent Director NATS VM
  45. 45. Agent Fetches Blobs from Blobstore VM HTTP Blobstore
  46. 46. Agent Starts Jobs http://mmonit.com/monit VM
  47. 47. Now let’s go outside-in!
  48. 48. BOSH Deployments
  49. 49. Deployment Manifests • The mapping of a BOSH release to infrastructure • Specifies: • Release • Compilation VMs • Update (canary) settings • Network configuration • Resource pools (VM “templates”) • Jobs • Job Properties
  50. 50. redis-boshrelease/examples/aws-solo.yml
  51. 51. redis-boshrelease/examples/aws-solo.yml
  52. 52. redis-boshrelease/examples/aws-solo.yml
  53. 53. redis-boshrelease/examples/aws-solo.yml
  54. 54. cf-release
  55. 55. Let’s follow one of our components... • GoRouter • Release Manifest • Package • spec • packaging • Job • spec • monit • templates
  56. 56. Release Manifest cf-release/releases/cf-release-134.yml
  57. 57. Package cf-release/packages/gorouter/spec
  58. 58. Package cf-release/packages/gorouter/packaging
  59. 59. Job cf-release/jobs/gorouter/spec
  60. 60. Jobs http://mmonit.com/monit cf-release/jobs/gorouter/monit
  61. 61. Jobs cf-release/jobs/gorouter/templates/gorouter_ctl.erb
  62. 62. Jobs cf-release/jobs/gorouter/templates/gorouter_yml.erb
  63. 63. Cloud Foundry on vSphere
  64. 64. vSphere 101 • Minimum Lab HW: • 6 Cores (12 threads) • 64 GB RAM • 4 TB Disk • vSphere 5.1 (2 ESXi hosts, 1 vCenter) • can get 30 day evals! • iSCSI Data Store (http://www.openfiler.com/) • Available IP addresses = 2X number of VMs • https://github.com/cloudfoundry-community/vsphere-home-lab
  65. 65. BOSH Bootstrap •Provide vCenter Credentials (via http://fog.io/) •Provide Network Settings •Provide vCenter Settings (Datacenter, Cluster, ...) •Provide NTP Settings •Provide MicroBOSH VM Sizing •Provision MicroBOSH!
  66. 66. BOSH Bootstrap Deploy
  67. 67. BOSH Prepare CF
  68. 68. vSphere Deployment Manifest
  69. 69. vSphere Deployment Manifest
  70. 70. vSphere Deployment Manifest
  71. 71. vSphere Deployment Manifest
  72. 72. http://xip.io
  73. 73. BOSH Create CF
  74. 74. Cloud Foundry on AWS
  75. 75. AWS 101 •Requirements: • AWS Credentials • Capacity to provision 8 servers and 3 elastic IPs • Approximately $0.42/hour you keep it running (useast-1/us-west-1) • Manually configured security group for CF
  76. 76. BOSH Bootstrap •Provide AWS Credentials (via http://fog.io/) •Provide AWS Region •Provision MicroBOSH!
  77. 77. BOSH Bootstrap Deploy
  78. 78. BOSH Prepare CF
  79. 79. AWS Deployment Manifest
  80. 80. AWS Deployment Manifest
  81. 81. AWS Deployment Manifest
  82. 82. AWS Deployment Manifest
  83. 83. BOSH Create CF
  84. 84. Create ‘cf’ Security Group
  85. 85. Allocate Elastic IP for CF Router
  86. 86. Cloud Foundry on OpenStack
  87. 87. OpenStack 101 •Requirements: • OpenStack Credentials • Capacity to provision 8 servers and 3 floating IPs • Manually configured security group for CF
  88. 88. BOSH Bootstrap •Provide OpenStack Credentials (via http://fog.io/) •Provide OpenStack Region (optional) •Provision MicroBOSH!
  89. 89. BOSH Bootstrap Deploy
  90. 90. BOSH Prepare CF
  91. 91. OpenStack Deployment Manifest
  92. 92. OpenStack Deployment Manifest
  93. 93. OpenStack Deployment Manifest
  94. 94. OpenStack Deployment Manifest
  95. 95. BOSH Create CF
  96. 96. Create ‘cf’ Security Group
  97. 97. Allocate Floating IP for CF Router
  98. 98. Web Console UI
  99. 99. Edit src/main/resources/ styx.properties
  100. 100. mvn clean package
  101. 101. cf push!
  102. 102. THANK YOU! MATT STINE COMMUNITY ENGINEER, CLOUD FOUNDRY MSTINE@GOPIVOTAL.COM HTTP://WWW.CLOUDFOUNDRY.COM TWITTER: @MSTINE
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×