Cloud Foundry @ RubyConf China 2001


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Cloud Foundry @ RubyConf China 2001

  1. 1. `the industrys first open platform as a service © 2011 VMware Inc. All rights reserved
  2. 2. introduction2 RubyConf China 2011
  3. 3. who am I Haofei Wang Twitter @haofei Weibo @haofei Email wangh@vmare.com3 RubyConf China 2011
  4. 4. what is cloud foundry radically simplify the development and operation of applications and services across public and private clouds. completely written in pure ruby.4 RubyConf China 2011
  5. 5. ruby creator “Matz" matsumoto joins heroku as chief architect5 RubyConf China 2011
  6. 6. cloud foundry ecosystem .js framework and runtime interface data private services clouds msg public services clouds other micro services clouds6 RubyConf China 2011
  7. 7. two key initiatives• • OSS project, Apache License, Version 2.0 •• • the live service • operated by VMware, powered by VMware vSphere7 RubyConf China 2011
  8. 8. applications8 RubyConf China 2011
  9. 9. at a high level: app evolution• evolution step1: start with a great idea for an app • build web app as a prototype, major refactor it into v1 • written using spring, rails, or sinatra with scripting around the edges • scale, learn by doing, experiment with new approaches, etc.• evolution step2: at scale, tons of traffic, pushing limits • need to extend app with a backend processing tier • use some services that are shared between my front end and backend components e.g., messaging, kv store, etc. • use some services that are private to each tier e.g., kv store, document store, sql database etc. • leverage cloudfoundry scalability and self healing9 RubyConf China 2011
  10. 10. at a high level: my expectations• expectation1: write code not tickets… • the application is my unit of currency • expect friction free deployment, the system is the architect • I manage to the boundaries of my code, no further • don’t force me to learn how to cobble together a middleware stack, and then service it for life • I write code because its fun: configuring a kernel, installing packages, writing nginx configs is not fun• expectation2: choose my own cloud • develop and test on a low cost cloud • deploy into a high SLA cloud • don’t want to learn a new model each time I go to a new cloud10 RubyConf China 2011
  11. 11. typical app• spring web app, rails, sinatra, node.js, etc. • elastic pool of app instances, easy to scale • database accessible by all instances • most apps start out looking something like this system load balancer elastic pool app app database instance instance11 RubyConf China 2011
  12. 12. deploying typical app the old way [mysqld] user = foobar port = 3306 basedir = /usr bind-address = key_buffer = 16M thread_stack = 128K thread_cache_size = 8 … [nginx] http.include mime.types; default_type: application/octet-stream; mvc web app log_format: main ‘$remote_addr - $remote_user []…’ keepalive_timeout 65; [tomcat] <Connector redirectPort=‚8443‛ emptySessionPath…/> <bean id=‚sessionFactory‛ class=‚org.springframework…/> [frontend] dependencies: - mysqlclient - ruby files: - core/app/fe/**/* - core/common/**/* [blah] - blah blah blah12 RubyConf China 2011
  13. 13. deploying typical app on cloudfoundry # to create and boot the app for the first time vmc target mvc web app vmc push myapp –instances 2 –mem 64M –path ../code vmc create-service mysql –name mydb –bind myapp # update live app with new code vmc update myapp –path ../code13 RubyConf China 2011
  14. 14. quick summary• cloudfoundry lets me start small • learn new approaches, frameworks, and services • develop on my cloud or yours• cloudfoundry lets me grow my app • multi node distributed systems • built in scaling at the node level• cloudfoundry lets me deploy/run with no friction • there is no learning curve. 0 to cloud in 3 commands • cloudfoundry is my architect, F$#@ IT!• cloudfoundry lets me choose my own cloud14 RubyConf China 2011
  15. 15. logical architecture15 RubyConf China 2011
  16. 16. applications, instances, services, tools application concepts all of the code, libraries, and data that are my code needed to run on a system supplied stack instances make my app scale. the more instances, the more load the app can handle apps are url addressable, can have multiple urls, allow custom domains on some clouds services are used to extend an app with higher level functions kv store, email, etc. application tools $ vmc update myapp $ vmc apps the command line tool: vmc, and sts plugin $ vm are the primary tools used by developers16 RubyConf China 2011
  17. 17. cloudfoundry logical view• infrastructure abstraction: servers, networks, storage delivered as software • no more wires, boxes, configuring, cooling• cloudfoundry abstraction • applications, instances, and services • manage to the boundaries of your code • cloudfoundry is your architect client tools user apps user apps cloudfoundry infrastructure17 RubyConf China 2011
  18. 18. api surface area• core app lifecycle api • the services api • create, start, stop, update • enumerate system serves • set url(s), instance count, • select and create service memory instance • get stats, logs, crashes, files • bind/unbind service & apps • miscellaneous • REST api with JSON payloads, full function api • info about for both system and account space • account management api • vmc command line app excercises the entire api18 RubyConf China 2011
  19. 19. vmc command line toolingCreate app, update app, control appvmc push [appname] [--path] [--url] [--instances N] [--mem] [--no-start]vmc update <appname> [--path PATH]vmc stop <appname>vmc start <appname>vmc target [url]Update app settings, get app informationvmc mem <appname> [memsize]vmc map <appname> <url>vmc instances <appname> <num | delta>vmc {crashes, crashlogs, logs} <appname>vmc files <appname> [path]Deal with services, users, and informationvmc create-service <service> [--name servicename] [--bind appname]vmc bind-service <servicename> <appname>vmc unbind-service <servicename> <appname>vmc delete-service <servicename>vmc user, vmc passwd, vmc login, vmc logout, vmc add-uservmc services, vmc apps, vmc info19 RubyConf China 2011
  20. 20. system architecture20 RubyConf China 2011
  21. 21. architectural principles• dynamic discovery and binding • no persistent configuration of components • all components discover their surroundings automatically via messaging • no prescribed boot order• self healing • applications and system components auto start and auto config on failure • flap detection and prevention built in• horizontal scaling • each core component can run as 1-N instances • components are peers, no explicit sharding21 RubyConf China 2011
  22. 22. cloud foundry kernel (OSS) auth/ router authz app app lifecycle nats execution management (deas) database service apps lifecycle redis management service blobstore instances22 RubyConf China 2011
  23. 23. app lifecycle management start/stop instances• cloud controller manages all aspects of lifecycle • CRUD operations for apps cloud controller health • staging apps (gathering all manager 3rd party components, creating start scripts, cc rewriting app environment actual etc) database get expected state state • fetching, building and caching gems NFS dea • serving droplets to DEAs resources, droplets, packages• securing and scaling the cloud controller is challenging (and dangerous) fetch droplets23 RubyConf China 2011
  24. 24. router router http request • all data flows from nginx to the router.rb nginx • built on proxied eventmachine – EM request has no flow control • bloats the ruby vm router.rb • 2x the number of syscalls • latency on every io nats • sticky session support <app> -> node:port dea24 RubyConf China 2011
  25. 25. app execution • apps run in separate processes protected with unix security start/stop instances • all see the same resources: ports, file system, etc • can talk to entire dea.rb service network by fetch droplets design • apps can launch direct attacks against other communication apps deas, services and the with services cloud controller • rooting a dea compromises the entire dea, including the nats message bus25 RubyConf China 2011
  26. 26. service provider cloud controller NATS cloud foundry gateway/node services API private protocol service gateway vm container service node service instance service instance service components26 RubyConf China 2011
  27. 27. at scale: multi-node, distributed system27 RubyConf China 2011
  28. 28. auto scaling producer/consumer front end front end producer producer redis rabbitMQ mongodb autoscaler back end back end back end consumer consumer consumer28 RubyConf China 2011
  29. 29. deploying app on cloudfoundry # create the front end and backend apps # front end is small but multi-instance vmc push fe –instances 8 –mem 64M –path ../fe_codemulti-node app vmc push be –instances 2 –mem 256M –path ../be_code # create the services and bind per spec vmc create-service mongodb –name mongo –bind fe vmc create-service redis –name redis –bind fe vmc bind-service redis be # to perform a rolling update of new code vmc update fe –path ../fe_code vmc update be –path ../be_code 29 RubyConf China 2011
  30. 30. hacking cloud foundry bash < <(curl -s -k –B cap/master/setup/install)30 RubyConf China 2011
  31. 31. we’re hiring in china• SRE – Platform/Delivery Engineer, JD: p001• Core Engineer, JD: k001, k001-ncg• Delivery Engineer, JD: dm001, dm002• QA Engineer, JD: qa001• Project Manager, JD: pgm00131 RubyConf China 2011