Cloud foundry elastic architecture and deploy based on openstack

1. CloudFoundry: Elastic Architecture Design and Deploy Based on OpenStack 郭峰 (Golfen Guo) @教授也是浮云 golfen.guo@emc.com Cloud Platform and Application Lab, EMC Labs China © Copyright 2012 EMC Corporation. All rights reserved. 1

2. EMC Labs China Advanced Technology Research and Development University Collaboration Vision Big Data Lab Become an elite research and advanced technology institute Industry Cloud Infrastructure in China Standards Office and System Lab - Become the model for future EMC Labs Cloud Platform and worldwide Applications Lab IP Portfolio Development © Copyright 2012 EMC Corporation. All rights reserved. 2

3. Agenda • What is PaaS • CloudFoundry Overview • Elastic Architecture in CloudFoundry • Introduce to BOSH • CPI and OpenStack • Deploy PaaS with BOSH © Copyright 2012 EMC Corporation. All rights reserved. 3

4. What is PaaS? (from Search Cloud Computing) • Platform as a Service (PaaS) is a way to rent hardware, operating systems, storage and network capacity over the Internet. • The service delivery model allows the customer to rent virtualized servers and associated services for running existing applications or developing and testing new ones. RENT, not BUILD one! Virtual Server and Associated Service © Copyright 2012 EMC Corporation. All rights reserved. 4

6. How PaaS Benefit Us? (Cont.) From Accelerating your Journey to Application Transformation, EMC World 2012 PaaS powered You Manage Traditional IT IaaS PaaS You Manage Applications Applications Applications Agility and Data Data Data Cost Savings Managed by Platform Managed by Platform Runtime You Manage Runtime Runtime Middleware Middleware Middleware O/S O/S O/S Virtualization Virtualization Virtualization Servers Servers Servers Storage Storage Storage Networking Networking Networking © Copyright 2012 EMC Corporation. All rights reserved. 6

8. PaaS in Action… Another analogy from Mark Lucovsky, Every one knows, “Hello World”: $ cat hw.c #include <stdio.h> main() { printf(“Hello Worldn”); } $ cc hw.c; ./a.out © Copyright 2012 EMC Corporation. All rights reserved. 8

9. PaaS in Action... (Cont. ) “Hello World” in the Cloud: $ cat hw.rb require 'rubygems' require 'sinatra' $hits = 0 get '/' do $hits = $hits + 1 "Hello World - #{$hits}" end $ vmc push hw © Copyright 2012 EMC Corporation. All rights reserved. 9

10. Scale App On Demand Implement a Scalability “Hello World” in Cloud: Step 1: Scale instance to 10: $ vmc instances hw 10 Step 2: Change the codes: get '/' do $hits = $redis.incr(‘hits’) "Hello World - #{$hits}" Redis end Step 3: bind Redis service to App: $ vmc create-service redis –bind hw © Copyright 2012 EMC Corporation. All rights reserved. 10

12. Simplify Deployment [mysqld] user = foobar port = 3306 Before: basedir = /usr bind-address = 172.58.77.101 key_buffer = 16M thread_stack = 128K thread_cache_size = 8 … [nginx] http.include mime.types; default_type: application/octet-stream; Web App log_format: main ‘$remote_addr - $remote_user []…’ keepalive_timeout 65; [tomcat] <Connector redirectPort=‚8443‛ emptySessionPath…/> <bean id=‚sessionFactory‛ class=‚org.springframework…/> [frontend] dependencies: - mysqlclient - ruby files: - core/app/fe/**/* - core/common/**/* [blah] - blah blah blah © Copyright 2012 EMC Corporation. All rights reserved. 12

13. Simplify Deployment (Cont.) Now: # to target and login to cloud foundry vmc target http://api.cloudfoundry.com vmc login Web App # to create and boot the app for the first time vmc push myapp –instances 2 –mem 64M –path ../code # to create the database and bind it to the app vmc create-service mysql –name mydb –bind myapp # update live app with new code vmc update myapp –path ../code © Copyright 2012 EMC Corporation. All rights reserved. 13

14. First Touch CloudFoundry prompt> gem install vmc prompt> vmc target api.cloudfoundry.com prompt> vmc login prompt> vmc push Would you like to deploy from the current directory? [Yn] Yes Application Name: hello Application Deployed URL: 'hello.cloudfoundry.com'? hello-bob.cloudfoundry.com Detected a Sinatra Application, is this correct? [Yn] Yes Memory Reservation [Default:128M] (64M, 128M, 256M, 512M or 1G) (Press Enter to take default) Would you like to bind any services to 'hello'? [yN]: No Uploading Application: Checking for available resources: OK Packing application: OK Uploading (0K): OK Push Status: OK Staging Application: OK Starting Application: OK © Copyright 2012 EMC Corporation. All rights reserved. 14

15. First Touch CloudFoundry(Cont.) Upload your app Memory of each instance Change served instances Your apps deployed Start, stop, update, restart Services of current app Your all created services Information of app © Copyright 2012 EMC Corporation. All rights reserved. 15

19. Conclusion Abstract to three layers: • Routers for finding right endpoint of Apps • Runtime for Apps • Services provided by platform, consumed by Apps © Copyright 2012 EMC Corporation. All rights reserved. 19

21. Elastic Architecture in CloudFoundry Sends droplet Design principals: heart beats and exit messages 1. Each components can be run standalone; Router 2. Each components can be scale-out, and notify the peersand with Router Registers message; Registers and unregisters unregisters 3. The components communicate only with message or RESTful API. Routes droplet Routes REST API requests requests Droplet change notiﬁcations Self-government and LooseController Cloud Couples : Stop, Find) Droplet Cloud Orchestrates Health Manager start/stop Controller (Start, Droplet Execution requests • Easy to add new components. eg Stager, UAA, ACM…; Agent (DEA) • Easy to evolve each component. eg. CC_ng, Router v2… Periodically scans Persists droplets for consistency and provisioned Guest applications Advertise services consume Cloud Service Controller Provision and Database unprovision Service "A" Provision Service and "A" Provisioning Agent unprovision © Copyright 2012 EMC Corporation. All rights reserved. 21

22. Elastic Architecture in CloudFoundry Sends droplet A request comes… heart beats and exit messages Router Router Registers and Registers and unregisters unregisters Routes droplet Routes REST API requests requests Droplet change notiﬁcations Droplet Cloud Controller Orchestrates Health Manager start/stop Cloud Controller (Start, Stop, Find) Droplet Execution requests Agent (DEA) Periodically scans Persists droplets for consistency and provisioned Guest applications Advertise services consume Cloud Service Controller Provision and Database unprovision Service "A" Provision Service and "A" Provisioning Agent unprovision © Copyright 2012 EMC Corporation. All rights reserved. 22

23. Pluggable Runtime Support Refers to https://github.com/cloudfoundry/vcap-staging • stager -> vcap-staging klass = StagingPlugin.load_plugin_for(plugin_name) plugin = klass.from_file(config_path) plugin.stage_application App Stager StagingPlugin Start Stop Script Script Server PHP Rails3 Java_Web Play … extends StagingPlugin © Copyright 2012 EMC Corporation. All rights reserved. 23

24. Pluggable Runtime Support (Cont.) So what we need to do is… • Extends Class StagingPlugin in Common.rb Line3 ~ Line 62, 59 lines of codes to support PHP.  © Copyright 2012 EMC Corporation. All rights reserved. 24

25. Pluggable Runtime Support (Cont.) Key methods to rewrite: • stage_application • start_command • startup_script • stop_command • stop_script © Copyright 2012 EMC Corporation. All rights reserved. 25

26. Pluggable Services Support Refers to a nice presentation by Nicholas Kushmerick Cloud Foundry Services in last forum: • Service advertisement • Service Gateway -> Cloud Controller • POST /services/v1/offerings • DELETE /services/v1/offerings/:label • Instance management • Cloud Controller -> Service Gateway • Provision: POST /gateway/v1/configurations • Bind: POST /gateway/v1/configurations/:id/handles • Unbind: DELETE /gateway/v1/configurations/:id/handles/:handle • Unprovision: DELETE /gateway/v1/configurations/:id © Copyright 2012 EMC Corporation. All rights reserved. 26

27. But … • Modular Design • Version Tolerance • Flexible Runtime/Service • Elastic Architecture Trade Off • Many kinds of nodes • Many nodes each kind Complex deployment process like other distribution system © Copyright 2012 EMC Corporation. All rights reserved. 27

28. Practical problem at CloudFoundry.com 40+ unique node types 75+ unique software packages 500-5,000 VMs 2x/week cf.com updates 24*7*365 non-stop operation No-downtime deployments Reliable ,robust ,repeatable deployments ,updates , capacity adjustments Small teams manage many instances © Copyright 2012 EMC Corporation. All rights reserved. 28

30. CloudFoundry BOSH CloudFoundry BOSH is an open source tool-chain for release engineering, deployment, and lifecycle management of large scale distributed services – Prescriptive way of creating releases and managing systems and services – It is not a collection of shell scripts, not a pile of Perl Built to deploy and manage production-class, large scale clusters – Production grade Cloud Foundry clusters: 500+ VMs, 40+ jobs, 75+ packages – Multi-node, multi-tier, complex clusters Built for DevOps usage and scale by a crack team of veterans – A project, not a product: command line interface, YAML, etc. – Continuous improvement, iterative development, rough edges https://github.com/cloudfoundry/bosh © Copyright 2012 EMC Corporation. All rights reserved. 30

31. CloudFoundry BOSH (Cont.) Built from the need to operate cloudfoundry.com – Over 40 different types of VMs – Hundreds of running instances of VMs – Pushing at least 2 changes per week End-to-end management – Release engineering – Deployment – Lifecycle management Generic solution – Any service – Any cloud – Any operating system © Copyright 2012 EMC Corporation. All rights reserved. 31

32. BOSH Architecture 1. Upload Stemcell 1. CLI -> Director 2. Director -> Blobstore 2. Upload Release 1. CLI -> Director 2. Director -> Blobstore 3. Deploy 1. CLI -> Director 2. Director -> A 3. Agent -> B 4. Agent -> C © Copyright 2012 EMC Corporation. All rights reserved. 32

33. BOSH Concepts Stemcell – VM template – BOSH Agent – IaaS Plugin Release – Jobs Job – Packages – Templates (scripts, confs) – Monitoring Package – Source/blobs – Dependencies – Packaging (scripts) © Copyright 2012 EMC Corporation. All rights reserved. 33

34. IaaS Neutral vSphere: battle tested implement AWS: code complete Cloud Foundry BOSH Cloud Provider Interface(CPI) OpenStack: testable release https://github.com/piston/ openstack-bosh-cpi © Copyright 2012 EMC Corporation. All rights reserved. 34

35. Cloud Provider Interface Stemcell – create_stemcell (image, cloud_properties) – delete_stemcell (stemcell) VM – create_vm (agent_id, stemcell, resource_pool, networks, disk_locality, env) – delete_vm (vm) – reboot_vm (vm) – configure_networks (vm, networks) Disk – create_disk (size, vm_locality) – delete_disk (disk) – attach_disk (vm, disk) – detach_disk (vm, disk) © Copyright 2012 EMC Corporation. All rights reserved. 35

36. Cloud Provider Interface (Cont.) For OpenStack Stemcell – create_stemcell … – delete_stemcell Cloud Provider Interface VM – create_vm – delete_vm – reboot_vm OpenStack VM VM Image Service VM – configure_networks VM VM Disk – create_disk OpenStack – delete_disk Compute – attach_disk – detach_disk © Copyright 2012 EMC Corporation. All rights reserved. 36

37. Deploy PaaS with BOSH Sends droplet heart beats and exit messages Router Router Registers and Registers and unregisters unregisters CLI Routes REST API Routes droplet requests requests Droplet change notiﬁcations Droplet Cloud Controller Orchestrates Health Manager start/stop Cloud Controller (Start, Stop, Find) Droplet Execution requests Agent (DEA) Periodically scans Persists droplets for consistency and provisioned Guest applications Advertise services consume Cloud Service Controller Provision and Database unprovision Service "A" Provision Service and "A" Provisioning Agent unprovision BOSH IaaS © Copyright 2012 EMC Corporation. All rights reserved. 37

39. Deployments for CloudFoundy Cloudfoundry.yml name: cloudfoundry release: name: cloudfoundry version: 89.1-dev compilation: workers: 4 network: default cloud_properties: ram: 1024 disk: 2048 cpu: 2 update: canaries: 1 canary_watch_time: 3000-90000 update_watch_time: 3000-90000 max_in_flight: 2 max_errors: 1 © Copyright 2012 EMC Corporation. All rights reserved. 39

40. Deployments for CloudFoundy (Cont.) Cloudfoundry.yml networks: - name: default subnets: - static: - 192.168.2.50 - 192.168.2.89 range: 192.168.2.0/24 gateway: 192.168.2.1 dns: - 10.254.174.10 cloud_properties: name: PrivateNetwork - name: lb subnets: - static: - 192.168.2.90 - 192.168.2.99 range: 192.168.2.0/24 gateway: 192.168.2.1 dns: - 10.254.174.10 cloud_properties: name: PrivateNetwork © Copyright 2012 EMC Corporation. All rights reserved. 40

41. Deployments for CloudFoundy (Cont.) Cloudfoundry.yml resource_pools: - name: infrastructure network: default size: 29 stemcell: name: bosh-stemcell version: 0.6.2 cloud_properties: ram: 256 disk: 2048 cpu: 1 env: bosh: password: © Copyright 2012 EMC Corporation. All rights reserved. 41

42. Deployments for CloudFoundy (Cont.) Cloudfoundry.yml jobs: - name: cloud_controller template: cloud_controller instances: 1 resource_pool: infrastructure networks: - name: default static_ips: - 192.168.2.60 - name: nats template: nats instances: 1 resource_pool: infrastructure networks: - name: default static_ips: - 192.168.2.52 © Copyright 2012 EMC Corporation. All rights reserved. 42

43. Deployments(CloudFoundy) Cloudfoundry.yml properties: domain: cflocal.com env: {} networks: apps: default management: default nats: user: nats password: aaa3ij3122 address: 192.168.2.52 port: 4222 © Copyright 2012 EMC Corporation. All rights reserved. 43

45. Acknowledgments VMware China R&D Center Network & Information Center, Shanghai Jiao Tong Univ. CloudFoundry Community Piston Community http://www.cloudfoundry.org https://github.com/piston/open stack-bosh-cpi © Copyright 2012 EMC Corporation. All rights reserved. 45

51. Evolving Cloud Foundry - Kernel uaa stager db Package cloud controller cache v2 v2 nats health manager v2 v2 v2 router v2 v2 warden dea caldecott service app app © Copyright 2012 EMC Corporation. All rights reserved. 51

52. How to write a custom service? • Public API for advertising services and managing service instances Cloud • Endpoints: Controll – Cloud Controller er – Service Gateway (one per service) • REST / JSON / HTTP(S) MySQL Gatewa MongoD y B Gatewa y © Copyright 2012 EMC Corporation. All rights reserved. 52

53. How to write a custom service? (Cont.) Step 1: Service advertisement Service Gateway -> Cloud Controller, two methods:  POST /services/v1/offerings  DELETE /services/v1/offerings/:label :label => 'mysql-5.1' :url => 'http://12.34.56.78:4321', :plans => ['free', 'bronze', 'silver', 'gold'] :tags => ['mysql', 'mysql-5.1', 'database'] :active => true, :description => 'MySQL database service' :acls => { :wildcards => '*@vmware.com', :plans => { :bronze => { :users => ['joe@vmware.com', 'jane@vmware.com'] } } } © Copyright 2012 EMC Corporation. All rights reserved. 53

54. How to write a custom service? (Cont.) Step 2: Instance management Cloud Controller -> Service Gateway, four methods:  Provision: POST /gateway/v1/configs  Bind: POST /gateway/v1/configs/:id/handles  Unbind: DELETE /gateway/v1/configs/:id/handles/:handle  Unprovision: DELETE /gateway/v1/configs/:id © Copyright 2012 EMC Corporation. All rights reserved. 54

55. Reference implementation Develope A vmc create-service postgresql r vmc, STS B POST /services/v1/configurations Cloud Controller C G H {host: 12.34.56.78, port: 3456, POST /gateway/v1/configurations VCAP_SERVICES = [postgres: db: abc123, user: pqr456} {host: 12.34.56.78, port: 3456, db: abc123, user: pqr456}] Service Gateway F D Private API {host: 12.34.56.78, port: 3456, App NATS: provision SELECT * db: abc123, user: pqr456} E FROM FOO CREATE DATABASE abc123 CREATE ROLE pqr456 I GET /foo/bar Service Node (eg, PostgreSQL) Service (eg, PostgreSQL) host 12.34.56.78 End Users © Copyright 2012 EMC Corporation. All rights reserved. 55

56. For legacy service – Service Broker • Cloud Foundry users may already have existing services • Legacy database that hasn’t / won’t / can’t be moved CloudFoundry.com • Services not yet supported • Service Broker exposes an arbitrary legacy service instance to Cloud Foundry so that it can be used just like any other instance © Copyright 2012 EMC Corporation. All rights reserved. 56

57. For legacy service – Service Broker (Cont.) B D register hrdb {host: 87.65.43.21, port: 6543, db: def123, user: stu456} vmc create-service hrdb vmc, STS Cloud Controller Service Broker Admin Tools E H POST /services/v1/configurations VCAP_SERVICES = [hrdb: POST /gateway/v1/configurations G {host: 87.65.43.21, port:6543, db: def123, user: stu456}] port: 6543, {host: 87.65.43.21, C F POST /broker/v1/configuration db: def123, user: stu456} Service Gateway App Connecti on Details Service Broker A Core Service CREATE DATABASE def123 (eg, PostgreSQL) CREATE USER stu456 SELECT * FROM FOO Brokered Service host 12.34.56.78 (eg, Oracle DB) GET /foo/bar I © Copyright 2012 EMC Corporation. All rights reserved. 57

Cloud foundry elastic architecture and deploy based on openstack

You are reading a preview.

Check these out next

Cloud foundry elastic architecture and deploy based on openstack

More Related Content

Slideshows for you (20)

Similar to Cloud foundry elastic architecture and deploy based on openstack (20)

More from OpenCity Community (20)

Recently uploaded (20)