彭—Elastic architecture in cloud foundry and deploy with openstack
Upcoming SlideShare
Loading in...5
×
 

彭—Elastic architecture in cloud foundry and deploy with openstack

on

  • 1,742 views

 

Statistics

Views

Total Views
1,742
Views on SlideShare
1,740
Embed Views
2

Actions

Likes
1
Downloads
58
Comments
0

1 Embed 2

http://www.linkedin.com 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • 我们是如何部署CouldFoundry。CouldFoundry是一个PaaS平台。PaaS就是一个功能强大的分布式系统,既可以运行在物理机上,也可以运行在虚拟机上,更可以运行在OpenStack上。使用脚本,CloudFoundry提供了一个安装脚本。使用部署工具,CloudFoundry以前是使用Chef使用这些工具有一些不便的地方,就是要写脚本对于CloudFoundry来说,写脚本太繁琐了现有的部署工具和IaaS平台并没有很好的继承。不能完成像自动创建虚拟机,对虚拟机配置的问题。
  • CloudFoundry是一个功能非常强大的PaaS平台,cloudfoundry.com是世界上最大的部署。前面关心的CloudFoundry的架构,我关心的是其在OpenStack上的结合。在.com上,遇到了一些实际的问题。CloudFoundry 。更多的node type和software packages.CloudFoundry本身就很强大,最微小的集群也很麻烦。节点很多。需要管理VM。雪上加霜。更新频繁,不间断服务。写脚本的时候不能出问题。测试环境的脚本和线上的还不一样。不断的调整,调整容量。纵向扩展和软件关系比较小,横向比较大。团队很小CloudFoundry的理想是成为PaaS平台的Linux。可以在IaaS上方便的部署运行。突出OpenStack的兼容性。部署,运维成为麻烦阻碍CloudFoundry成为PaaS平台的Linux绊脚石,自己吃苦就算了,不能让用户也写繁琐的脚本,必须要解决这个问题。
  • 前面说了 写脚本的方式太繁琐,有没有一个办法来解决上面的问题?BOSHBOSH是CloudFoundry的部署运维工具,在今年4月满周岁的正式发布。有什么问题随时打断。BOSH是胡扯。是BOSH Outer Shell的首字母缩写。BOSH不是一个CloudFoundry的组件。和IaaS集成很好的一站式部署工具Initially the B stood for a code name which we deprecated soon after. For the compleetepictur, the Cloud Foundry PaaS (vcap) is the inner shell. Bosh是一个工具。可以打包,部署,运维 分布式系统。不仅仅是部署,还包括管理。不仅仅是CloudFoundry,所有的分布式系统。使用他,发布工程师打一个包,部署工程师写一个配置文件,一条命令部署到分布式系统,运维也是改改配置,一条命令不是简单的一组脚本,他有很多功能是脚本很难完成的,比如跨Cloud。Repeatable。能力强:环境复杂。大集群。多节点。多软件。多任务。使用简单:使用简单。只要会简单的CLI,YAML配置就可以不断持续的更新应对CloudFoundry的实际需要。真刀真枪的线上系统系统架构中模块很多。很多模块,各有关系,写脚本乱规模很大。执行比较困难。出错什么的处理起来费力。变化很快。不是一劳永逸,扩容更新很频繁。功能,解决方案。对软件提供一个从头到脚的运维工具发布。打包成一个Release。部署。使用Bosh deploy。运维。使用修改配置。Bosh log,ssh,deploy之类。通用的不限于CloudFoundry, 任何分布式系统都可以。最适合的是系统平台类。比如一个云服务。不限于vmware。现在Vmware,AWS,OpenStack都支持。尤其是OpenStack.OpenStack兼容。不限操作系统,任意操作系统都是支持的。现在主要是Linux,版本是Ubuntu10.4
  • 对 BOSH 有了一个大致的印象后。看一下BOSH的架构。架构解说。1.看圈。圈内可以有OpenStack。2. 连线靠得是CPI。但我们部署一个应用的时候。上传一个模板。(模板就是虚拟机模板,里面有Agent和虚拟机的插件)通过CLI告诉Director说我要上传一个模板Director把模板送到BlobStore里面上传一个Release通过CLI告诉Director说我要上传一个ReleaseRelease会下载一些依赖包。Director把Release送到BlobStore里面部署通过CLI告诉Director说我要部署Director将包从BlobStore取出编译编译完放回去通过A创建一个VM。使用Stemcell作为模板通过B。VM上的Agent部署这个应用其他流程
  • 刚刚提到了一些的概念。StemCell,Release解释 StemCell是集成了BOSH_Agent的虚拟机模板。给CloudFoundry用的stemcell是一个ubunutu的版本。可以使用CLI来上传。BOSH Director用他来创建VM。当创建Vm的时候,要传入Network和存储的配置,还有Blobstroe的位置。Release是Job的集合。可以理解为一个文件夹,里面什么都有。这个文件夹是发布工程师搞的。Job就是一个服务。比如Mysql,可能有多个实例。Job包含Package+运维的脚本. 比如我一个Myql的Job里面可以有两个package,一个是MysqlServer还有一个是Mysql的Client.Job还有配置文件和启动的脚本。可以让一个Job跑多个进程。PackageSource/blobsPackaging Script编译期
  • BOSH在SHELL外面,VM 在 SHELL里面。BOSH是如何来实现跨虚拟机平台的呢?是通过CPI。将一些常用的虚拟机操作抽象出来,形成一套简单的接口,然后再分别加以实现。vSphere:真刀真枪的实现,支持4.2和5.0两个版本。有成千上万的部署使用到了。AWS:逐步完善,已经可以使用。在论坛上看到。有很多人已经部署过了。OpenStack:积极的进展。有Vmware和合作伙伴piston cloud开发。这个公司是个专门做OpenStack的公司。现在关于OpenStack有越来越多的资料。已经有了可以测试的版本。piston cloud希望将这部被代码集成在自己的企业版本里面,作为新项目提交到OpenStack的官方外围。可以再延伸一下,https://github.com/piston/openstack-bosh-cpihttps://github.com/drnic/bosh-getting-started/blob/master/create-a-bosh/creating-a-micro-bosh-from-stemcell-openstack.mdPiston简介:Piston Cloud is the enterprise OpenStack™ company. 他的创始人是NASA出来的,在OpenStachk里面是一家很重要的创业公司。Piston是利用开源的工具,给用户搭建一完整的一站式的计算和存储云。CouldFoundry的社区是非常的开放的,起步的时候使用Vmware
  • https://github.com/cloudfoundry/bosh/blob/master/cpi/lib/cloud.rb.CPI非常的精简。上面列出的就是全部,没有省略。当配置一台虚拟机的时候。Stemcell是模板相关的。可以create来上传一个模板。VM是VM相关的。可以create_vm来create一个vm.需要传入一个stemcell作为参数,虚拟机就是从那个模板clone出来的通过configure_network配置网卡。Disk是存储相关的。通过create创建,attach来连接到VM上。Create disk的时候在哪个存储上?最后可以通过reboot_vm来重启动vm。
  • 刚刚说的是CPI的接口声明,现在主要是说CPI在OpenStack上面的实现。看图:CloudFondry东西运行在OpenStack虚拟机之上。OpenStack Compute 提供了操作VM的接口。而ImageService给OpenStack增加了虚拟机模板的功能。Bosh通过CPI接口来操作OpenStack.而CPI通过有一个名为FOG的接口,可以操作OpenStack。Stemcell调用create和delete的时候。会使用glance来创建删除Image.操作器ImageServiceVM调用Vm的接口的时候,实际会操作OpenStack Compute 也就是NOVA.让他使用Glance中的Image来Create一个。其中Configure-network也是使用的NOVA,来配置虚拟网络参数。Disk也是一样的使用ova-volume来分配足够的存储空间。
  • 眼见为实,耳听为虚,直接来实践。部署一个CloudFoundry集群CloudFoundry本身很强大,组件比较多。就不深入结构。有哪些package ,哪些Jobs,哪些release灌入概念
  • 看视频,一步一步说。Depoy配置文件一会讲Deploy在做什么然后有时间再演示Scale out.Upload sc:检查,上传。保存。bosh create release指定配置文件。编译,创虚拟机,配置成功了,测试http://www.youtube.com/watch?v=RogBrWaW1Ww
  • Stemcells and Packages是静态的,可以是别人已经发布好的。需要使用Deployment配置文件将他们和我们自己的环境结合在一起。配置文件 主要 是这样的。Release:名称版本Network:网络环境。网卡,ip访问Resource pool: VM的池,默认的CPU和内存,磁盘大小。Job:希望使用的Job:实例的个数,网络和存储,机器配置 自定义。部署的时候替换成Properties。每一个Job可以定义一种组件,如Mysql,NighxProperties:填充模板的变量

彭—Elastic architecture in cloud foundry and deploy with openstack 彭—Elastic architecture in cloud foundry and deploy with openstack Presentation Transcript

  • Elastic Architecture in CloudFoundry and Deploy with OpenStack Layne Peng Kay Yan @Layne_Peng @yankay layne.peng@emc.com kay.yan@emc.com Cloud Platform and Application Lab, EMC Labs China© Copyright 2012 EMC Corporation. All rights reserved. 1
  • About Us Technologist from Cloud Platform and Application, EMC Labs China – Our work: EMC Labs China ▪ Research topics related to cloud architecture – Lab focus areas: Advanced Technology Research and Development ▪ PaaS/IaaS ▪ NGDC automation management Big Data Lab ▪ Cloud platform middleware Cloud Infrastructure ▪ Multi-tenant management and System Lab Cloud Platform and Application Lab© Copyright 2012 EMC Corporation. All rights reserved. 2
  • Agenda • Why We Here? • First Touch CloudFoundry • Elastic Architecture in CloudFoundry • Introduce to BOSH • CPI and OpenStack • Deploy PaaS with BOSH© Copyright 2012 EMC Corporation. All rights reserved. 3 View slide
  • Why We Here? From Accelerating your Journey to Application Transformation, EMC World 2012 PaaS powered You Manage Traditional IT IaaS PaaS You Manage Applications Applications Applications Agility and Data Data DataCost Savings Managed by Platform Managed by Platform Runtime You Manage Runtime Runtime Middleware Middleware Middleware O/S O/S O/S Virtualization Virtualization Virtualization Servers Servers Servers Storage Storage Storage Networking Networking Networking © Copyright 2012 EMC Corporation. All rights reserved. 4 View slide
  • First Touch… prompt> gem install vmc prompt> vmc target api.cloudfoundry.com prompt> vmc login prompt> vmc push Would you like to deploy from the current directory? [Yn] Yes Application Name: hello Application Deployed URL: hello.cloudfoundry.com? hello-bob.cloudfoundry.com Detected a Sinatra Application, is this correct? [Yn] Yes Memory Reservation [Default:128M] (64M, 128M, 256M, 512M or 1G) (Press Enter to take default) Would you like to bind any services to hello? [yN]: No Uploading Application: Checking for available resources: OK Packing application: OK Uploading (0K): OK Push Status: OK Staging Application: OK Starting Application: OK© Copyright 2012 EMC Corporation. All rights reserved. 5
  • First Touch…(Cont.) Upload your app Memory of each instance Change served instances Your apps deployed Start, stop, update, restart Services of current app Your all created services Information of app© Copyright 2012 EMC Corporation. All rights reserved. 6
  • What CloudFoundry Offer?© Copyright 2012 EMC Corporation. All rights reserved. 7
  • What CloudFoundry Offer? (Cont.)From Cloud Foundry Launch Event, April, 2011 PaaS powered© Copyright 2012 EMC Corporation. All rights reserved. 8
  • PaaS Architecture Pattern MySQL RabbitMQ MongoDB Redis© Copyright 2012 EMC Corporation. All rights reserved. 9
  • PaaS Architecture Pattern (Cont. )© Copyright 2012 EMC Corporation. All rights reserved. 10
  • PaaS Architecture Pattern (Cont. )© Copyright 2012 EMC Corporation. All rights reserved. 11
  • Conclusion Simplify to three layers: • Routers for finding right endpoint of Apps • Nodes of runtime for Apps • Nodes of services provided by platform, consumed by Apps© Copyright 2012 EMC Corporation. All rights reserved. 12
  • The Keys of Design… • Failover/System Robust • Scalable Elastic • Resource Recycling© Copyright 2012 EMC Corporation. All rights reserved. 13
  • Elastic Architecture in CloudFoundry Sends droplet A request comes… Design principals: heart beats and exit messages 1. Each components can be run standalone; Router 2. Each components can be scale-out, and notify the peersand with Router Registers message; Registers and unregisters unregisters 3. The components communicate only with message or Restful API; 4. Platform works above the infrastructures, no IaaS droplet Routes REST API Routes locking. requests requests Droplet change notifications Droplet Cloud Controller Orchestrates Health Manager start/stop Cloud Controller (Start, Stop, Find) Droplet Execution requests Self-government and Loose Couples : Agent (DEA) • Easy toPeriodically scans components. eg Stager, UAA, ACM…; add new Persists droplets • Easy to evolve each component. eg. CC_ng, Router v2…; for consistency and provisioned Guest applications Advertise services consume Cloud Service • Can be run aboveDatabase different IaaS. eg. OpenStack , AWS, vSphere Controller Provision and unprovision Service "A" Provision Service and "A" Provisioning Agent unprovision© Copyright 2012 EMC Corporation. All rights reserved. 14
  • Open Ecosystem • Open Dev Proc • Partners & Communities© Copyright 2012 EMC Corporation. All rights reserved. 15
  • Elastic Runtime Support Refers to https://github.com/cloudfoundry/vcap-staging • stager -> vcap-staging klass = StagingPlugin.load_plugin_for(plugin_name) plugin = klass.from_file(config_path) plugin.stage_application App Stager StagingPlugin Start Stop Script Script Server PHP Rails3 Java_Web Play … extends StagingPlugin© Copyright 2012 EMC Corporation. All rights reserved. 16
  • Elastic Runtime Support (Cont.) So what we need to do is… • Extends Class StagingPlugin in Common.rb Line3 ~ Line 62, 59 lines of codes to support PHP. © Copyright 2012 EMC Corporation. All rights reserved. 17
  • Elastic Runtime Support (Cont.) Key methods to rewrite: • stage_application • start_command • startup_script • stop_command • stop_script© Copyright 2012 EMC Corporation. All rights reserved. 18
  • Elastic Services Support Refers to a nice presentation by Nicholas Kushmerick Cloud Foundry Services in last forum: • Service advertisement - Service Gateway -> Cloud Controller • POST /services/v1/offerings • DELETE /services/v1/offerings/:label • Instance management - Cloud Controller -> Service Gateway • Provision: POST /gateway/v1/configurations • Bind: POST /gateway/v1/configurations/:id/handles • Unbind: DELETE /gateway/v1/configurations/:id/handles/:handle • Unprovision: DELETE /gateway/v1/configurations/:id© Copyright 2012 EMC Corporation. All rights reserved. 19
  • Tradeoffs • Modular Design • Version Tolerance • Flexible Runtime/Service • Elastic Architecture Cause • Many kinds of nodes • Many nodes each kind Complex deployment process like other distribution system© Copyright 2012 EMC Corporation. All rights reserved. 20
  • How we deployed CloudFoundry?© Copyright 2012 EMC Corporation. All rights reserved. 21
  • Practical problem at CloudFoundry.com 40+ unique node types 75+ unique software packages 500-5,000 VMs 2x/week cf.com updates Small teams manage many instances© Copyright 2012 EMC Corporation. All rights reserved. 22
  • CloudFoundry BOSH CloudFoundry BOSH is an open source tool-chain for release engineering, deployment, and lifecycle management of large scale distributed services – Prescriptive way of creating releases and managing systems and services – It is not a collection of shell scripts, not a pile of Perl Built to deploy and manage production-class, large scale clusters Built for DevOps usage and scale by a crack team of veterans – A project, not a product: command line interface, YAML, etc. Built from the need to operate cloudfoundry.com End-to-end management Generic solution - Any IaaS, Any Service https://github.com/cloudfoundry/bosh© Copyright 2012 EMC Corporation. All rights reserved. 23
  • BOSH Architecture Upload Stemcell Upload Release Deploy – CLI -> Director – Director -> A – Agent -> B – Agent -> C© Copyright 2012 EMC Corporation. All rights reserved. 24
  • BOSH Concepts Stemcell – VM template – BOSH Agent – IaaS Plugin Release – Jobs Job – Packages – Templates (scripts, confs) – Monitoring Package – Source/blobs – Dependencies – Packaging (scripts)© Copyright 2012 EMC Corporation. All rights reserved. 25
  • IaaS Neutral vSphere: battle tested implement AWS: code complete Cloud Foundry BOSH Cloud Provider Interface(CPI) OpenStack: testable release https://github.com/piston/ openstack-bosh-cpi© Copyright 2012 EMC Corporation. All rights reserved. 26
  • Cloud Provider Interface Stemcell – create_stemcell (image, cloud_properties) – delete_stemcell (stemcell) VM – create_vm (agent_id, stemcell, resource_pool, networks, disk_locality, env) – delete_vm (vm) – reboot_vm (vm) – configure_networks (vm, networks) Disk – create_disk (size, vm_locality) – delete_disk (disk) – attach_disk (vm, disk) – detach_disk (vm, disk)© Copyright 2012 EMC Corporation. All rights reserved. 27
  • Cloud Provider Interface (Impl.)For OpenStack Stemcell – OpenStack Image Service … VM, Disk & Network – OpenStack Compute Cloud Provider Interface OpenStack VM VM Image Service VM VM VM OpenStack Compute© Copyright 2012 EMC Corporation. All rights reserved. 28
  • Deploy PaaS with BOSH Sends droplet heart beats and exit messages Router Router Registers and Registers and unregisters unregisters CLI Routes REST API Routes droplet requests requests Droplet change notifications Droplet Cloud Controller Orchestrates Health Manager start/stop Cloud Controller (Start, Stop, Find) Droplet Execution requests Agent (DEA) Periodically scans Persists droplets for consistency and provisioned Guest applications Advertise services consume Cloud Service Controller Provision and Database unprovision Service "A" Provision Service and "A" Provisioning Agent unprovision BOSH IaaS© Copyright 2012 EMC Corporation. All rights reserved. 29
  • Demo Deploy CloudFoundry using BOSH – Upload Stemcell – Upload Release ▪ bosh create release ▪ bosh upload release – Write deployment file – Deploy CloudFoundry HelloWorld – Login – Push Application© Copyright 2012 EMC Corporation. All rights reserved. 30
  • Deployments Release Network Resource pools Jobs Properties Update concurrency Compilation workers Cloud properties© Copyright 2012 EMC Corporation. All rights reserved. 31
  • Deployments for CloudFoundyCloudfoundry.ymlname: cloudfoundryrelease: name: cloudfoundry version: 89.1-devcompilation: workers: 4 network: default cloud_properties: ram: 1024 disk: 2048 cpu: 2update: canaries: 1 canary_watch_time: 3000-90000 update_watch_time: 3000-90000 max_in_flight: 2 max_errors: 1© Copyright 2012 EMC Corporation. All rights reserved. 32
  • Deployments for CloudFoundy (Cont.)Cloudfoundry.ymlnetworks: - name: default subnets: - static: - 192.168.2.50 - 192.168.2.89 range: 192.168.2.0/24 gateway: 192.168.2.1 dns: - 10.254.174.10 cloud_properties: name: PrivateNetwork- name: lb subnets: - static: - 192.168.2.90 - 192.168.2.99 range: 192.168.2.0/24 gateway: 192.168.2.1 dns: - 10.254.174.10 cloud_properties: name: PrivateNetwork© Copyright 2012 EMC Corporation. All rights reserved. 33
  • Deployments for CloudFoundy (Cont.)Cloudfoundry.ymlresource_pools: - name: infrastructure network: default size: 29 stemcell: name: bosh-stemcell version: 0.6.2 cloud_properties: ram: 256 disk: 2048 cpu: 1 env: bosh: password:© Copyright 2012 EMC Corporation. All rights reserved. 34
  • Deployments for CloudFoundy (Cont.)Cloudfoundry.ymljobs: - name: cloud_controller template: cloud_controller instances: 1 resource_pool: infrastructure networks: - name: default static_ips: - 192.168.2.60 - name: nats template: nats instances: 1 resource_pool: infrastructure networks: - name: default static_ips: - 192.168.2.52© Copyright 2012 EMC Corporation. All rights reserved. 35
  • Deployments(CloudFoundy)Cloudfoundry.ymlproperties: domain: cflocal.com env: {} networks: apps: default management: default nats: user: nats password: aaa3ij3122 address: 192.168.2.52 port: 4222router: status: port: 8080 user: aaaUxXlS0pc71wVef password: aaamaIf9vPV4mJyBe© Copyright 2012 EMC Corporation. All rights reserved. 36
  • User Case bosh deployment bosh target cf cf.yml Bug report to Dev git pull Run bosh upload release Done! bosh deploy© Copyright 2012 EMC Corporation. All rights reserved. 37
  • Acknowledgments VMware China R&D Center Network & Information Center, Shanghai Jiao Tong Univ. CloudFoundry Community Piston Community Sina Weibo: @CloudFoundry https://github.com/piston/open http://www.cloudfoundry.org stack-bosh-cpi© Copyright 2012 EMC Corporation. All rights reserved. 38