Build Your Very Own Private Cloud Foundry

7,744 views

Published on

Speaker: Matt Stine
This session will focus on how you can build your very own Cloud Foundry private PaaS running in your own data center or on AWS or even on OpenStack on your own Mac mini. You will learn how the Cloud Foundry BOSH tool constructs a full Cloud Foundry instance from a bare bones virtual machine and continues to coordinate and manage the entire PaaS cloud once it is operational. If you want the convenience of developing against your own private custom PaaS within your company, then this session will give you all the steps you need to get started.

Published in: Technology

Build Your Very Own Private Cloud Foundry

  1. 1. BUILD YOUR VERY OWN PRIVATE CLOUD FOUNDRY MATT STINE COMMUNITY ENGINEER, CLOUD FOUNDRY MSTINE@GOPIVOTAL.COM HTTP://CLOUDFOUNDRY.ORG TWITTER: @MSTINE
  2. 2. Agenda •CF Architecture - Quick Overview •What is BOSH? Why BOSH? •BOSH Architecture •BOSH Releases •BOSH Deployments •CF to BOSH: cf-release •[:vsphere, :openstack, :aws].each { |iaas| iaas.deploy_cf }
  3. 3. Cloud Foundry Architecture (in FIVE minutes or less!)
  4. 4. Cloud Foundry Architecture Router User  Authen=ca=on  and  Authoriza=on Health  Manager Cloud  Controller Apps Service  Broker Service  Nodes Messaging Build  Packs DEA  Pool    
  5. 5. Router e. co m kt g. ac m ap p1 .m DEV A S TEST S m co e. m ac g. MARKETING ORG n .e p2 ap •Load balancing •Maintain routing table •Access logs app3.finance.acme.co m ROUTER ENGINEERING ORG FINANCE ORG PROD S DEV A S TEST A S PROD DEV A PLATFORM A TEST PROD A
  6. 6. UAA • Token Server • ID Server (User management) • OAuth Scopes (Groups) • Login Server • UAA Database • SAML support (for SSO integration) and Active Directory support with the VMWare SSO Appliance • Access auditing
  7. 7. Cloud Controller • App expected state • Permissions/Auth • Orgs/Spaces/Users • Services management • App placement • App desired state convergence • Auditing/Journaling • Billing events • Blob storage
  8. 8. Messaging (NATS) •Non-Persistent messaging •Pub/Sub •Queues (app events) •Directed messages (INBOX)
  9. 9. Droplet Execution Agent (DEA) •Manage Linux containers (Warden) • Process • File system • Network • Memory •Manage app lifecycle •App log and file streaming •DEA heartbeats (NATS to CC, HM) Router App App App App DEA/Warden DEA/Warden
  10. 10. Health Manager HealthManager SCHEDULER •Maintains the actual state of apps • •Sends suggestions to make Compares to expected state actual match expected App App Schedule Observation App App HARMONIZER Observes Observes App Trigger Harmonization Desired State NUDGER Bulk API App Actual State NATS NATS Cloud Controller NATS DEA
  11. 11. Service Broker •Advertising service catalog •Makes create/delete/bind/unbind calls to service nodes •Requests inventory of existing instances and bindings from cloud controller for caching, orphan management •SaaS marketplace gateway
  12. 12. CF Architecture: Why? •Component Isolation •Scalability •Fault Tolerance •Pre-provisioned Capacity (Containers / Warden)
  13. 13. How do we manage this thing?
  14. 14. We need a toolchain that can manage a large distributed system through: •Deployment •Configuration Changes •Updates/Upgrades (w/ minimal - zero - downtime!) •Component Failure / Restoration of Service •Scale Out / Scale In •Across multiple IaaS providers: vSphere, OpenStack, AWS, and beyond...
  15. 15. What about?
  16. 16. BOSH! (Bosh Outer SHell)
  17. 17. What is BOSH?
  18. 18. Why BOSH? •Provision services, not machines •Eliminate bespoke automation on top of configuration management •Enable continuous delivery of platform services •Cloud-agnostic view of platform operations •Holistic toolchain to “rule them all” •How we manage Cloud Foundry in production!
  19. 19. BOSH Architecture
  20. 20. BOSH Architecture
  21. 21. Director • Core orchestrating component • Controls: • creation/deletion of VMs, Disks, Networks • software deployment • lifecycle events for software and services • Via: • Cloud Provider Interface • Director-Agent Interaction
  22. 22. Agent •Carries out instructions from the Director •Fetches packages/jobs from Blobstore •Installs packages/jobs onto VM •Starts/monitors jobs via Monit
  23. 23. Message Bus (NATS) •Non-Persistent messaging •Pub/Sub •Queues (app events) •Directed messages (INBOX)
  24. 24. Health Monitor •Receives from Agent: • VM health status • Agent lifecycle events •Triggers: • Alerts (Email, CloudWatch, DataDog, PagerDuty, ...) • Resurrection!
  25. 25. Stemcells •“Pleuripotent” VMs • Base VM filesystem image • Never booted • Currently ~Ubuntu 10.04.4 LTS • Embedded BOSH Agent
  26. 26. Blobstore • • • • • Stores the content of BOSH Releases Uploaded via BOSH CLI Stored via Director Also stores: • • packages compiled by BOSH intermediate location for large payloads Options: • • • • EMC Atmos S3 OpenStack Swift Simple/Local
  27. 27. Cloud Provider Interface (CPI) • current_vm_id • create_stemcell • delete_stemcell • create_vm • delete_vm • has_vm? • reboot_vm • set_vm_metadata • configure_networks • create_disk • delete_disk • attach_disk • snapshot_disk • delete_snapshot • detach_disk • get_disks
  28. 28. Great! So how do I deploy my stuff with BOSH?
  29. 29. BOSH Releases
  30. 30. Hello BOSH Release! Redis
  31. 31. Anatomy of a BOSH Release • Release Manifest • Package • spec • packaging • Job • spec • monit • templates
  32. 32. Release Manifest redis-boshrelease/releases/redis-1.yml
  33. 33. Package redis-boshrelease/packages/redis/spec
  34. 34. Packages redis-boshrelease/packages/redis/packaging
  35. 35. Jobs redis-boshrelease/jobs/redis/spec
  36. 36. Jobs http://mmonit.com/monit redis-boshrelease/jobs/redis/monit
  37. 37. Jobs redis-boshrelease/jobs/redis/templates/bin/redis_ctl
  38. 38. Jobs redis-boshrelease/jobs/redis/templates/conf/redis.conf.erb
  39. 39. OK. So what happens when I deploy?
  40. 40. BOSH Agent
  41. 41. Director creates VM from Stemcell Director VM CPI
  42. 42. Agent Boots Director VM
  43. 43. Director Pings Agent Director NATS VM
  44. 44. Director Assigns Job to Agent Director NATS VM
  45. 45. Agent Fetches Blobs from Blobstore VM HTTP Blobstore
  46. 46. Agent Starts Jobs http://mmonit.com/monit VM
  47. 47. Now let’s go outside-in!
  48. 48. BOSH Deployments
  49. 49. Deployment Manifests • The mapping of a BOSH release to infrastructure • Specifies: • Release • Compilation VMs • Update (canary) settings • Network configuration • Resource pools (VM “templates”) • Jobs • Job Properties
  50. 50. redis-boshrelease/examples/aws-solo.yml
  51. 51. redis-boshrelease/examples/aws-solo.yml
  52. 52. redis-boshrelease/examples/aws-solo.yml
  53. 53. redis-boshrelease/examples/aws-solo.yml
  54. 54. cf-release
  55. 55. Let’s follow one of our components... • GoRouter • Release Manifest • Package • spec • packaging • Job • spec • monit • templates
  56. 56. Release Manifest cf-release/releases/cf-release-134.yml
  57. 57. Package cf-release/packages/gorouter/spec
  58. 58. Package cf-release/packages/gorouter/packaging
  59. 59. Job cf-release/jobs/gorouter/spec
  60. 60. Jobs http://mmonit.com/monit cf-release/jobs/gorouter/monit
  61. 61. Jobs cf-release/jobs/gorouter/templates/gorouter_ctl.erb
  62. 62. Jobs cf-release/jobs/gorouter/templates/gorouter_yml.erb
  63. 63. Cloud Foundry on vSphere
  64. 64. vSphere 101 • Minimum Lab HW: • 6 Cores (12 threads) • 64 GB RAM • 4 TB Disk • vSphere 5.1 (2 ESXi hosts, 1 vCenter) • can get 30 day evals! • iSCSI Data Store (http://www.openfiler.com/) • Available IP addresses = 2X number of VMs • https://github.com/cloudfoundry-community/vsphere-home-lab
  65. 65. BOSH Bootstrap •Provide vCenter Credentials (via http://fog.io/) •Provide Network Settings •Provide vCenter Settings (Datacenter, Cluster, ...) •Provide NTP Settings •Provide MicroBOSH VM Sizing •Provision MicroBOSH!
  66. 66. BOSH Bootstrap Deploy
  67. 67. BOSH Prepare CF
  68. 68. vSphere Deployment Manifest
  69. 69. vSphere Deployment Manifest
  70. 70. vSphere Deployment Manifest
  71. 71. vSphere Deployment Manifest
  72. 72. http://xip.io
  73. 73. BOSH Create CF
  74. 74. Cloud Foundry on AWS
  75. 75. AWS 101 •Requirements: • AWS Credentials • Capacity to provision 8 servers and 3 elastic IPs • Approximately $0.42/hour you keep it running (useast-1/us-west-1) • Manually configured security group for CF
  76. 76. BOSH Bootstrap •Provide AWS Credentials (via http://fog.io/) •Provide AWS Region •Provision MicroBOSH!
  77. 77. BOSH Bootstrap Deploy
  78. 78. BOSH Prepare CF
  79. 79. AWS Deployment Manifest
  80. 80. AWS Deployment Manifest
  81. 81. AWS Deployment Manifest
  82. 82. AWS Deployment Manifest
  83. 83. BOSH Create CF
  84. 84. Create ‘cf’ Security Group
  85. 85. Allocate Elastic IP for CF Router
  86. 86. Cloud Foundry on OpenStack
  87. 87. OpenStack 101 •Requirements: • OpenStack Credentials • Capacity to provision 8 servers and 3 floating IPs • Manually configured security group for CF
  88. 88. BOSH Bootstrap •Provide OpenStack Credentials (via http://fog.io/) •Provide OpenStack Region (optional) •Provision MicroBOSH!
  89. 89. BOSH Bootstrap Deploy
  90. 90. BOSH Prepare CF
  91. 91. OpenStack Deployment Manifest
  92. 92. OpenStack Deployment Manifest
  93. 93. OpenStack Deployment Manifest
  94. 94. OpenStack Deployment Manifest
  95. 95. BOSH Create CF
  96. 96. Create ‘cf’ Security Group
  97. 97. Allocate Floating IP for CF Router
  98. 98. Web Console UI
  99. 99. Edit src/main/resources/ styx.properties
  100. 100. mvn clean package
  101. 101. cf push!
  102. 102. THANK YOU! MATT STINE COMMUNITY ENGINEER, CLOUD FOUNDRY MSTINE@GOPIVOTAL.COM HTTP://WWW.CLOUDFOUNDRY.COM TWITTER: @MSTINE

×