Agile Development at GoGrid with Pallet and JClouds

2,121 views
2,064 views

Published on

This is a presentation to JClouds Meetup @ Twitter's HQ. Presented by Toni Batchelli. It discusses the methodologies that the GoGrid engineering team is employing to quickly and accurately produce timely builds.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,121
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
29
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Agile Development at GoGrid with Pallet and JClouds

  1. 1. Complex Infrastructure Made Easy™Agile Development at GoGrid with Pallet and JClouds Presented to JClouds Meetup @ Twitter’s HQ Toni Batchelli, Consultant and Entrepreneur wannabe tbatchelli@gmail.com, @tbatchelli, @disclojure February 2, 2011 02/01/11
  2. 2. GoGrid’s Challenges GoGrid is a cloud provider, building their own software Developing and testing their cloud management software is complex. They need to create and maintain realistic environments in order to load them with their latest software builds ~10 different servers types ~20 different services
  3. 3. The problem? Current time to setup a development/QA environment is 2 days • Mostly manual work Building the environments manually is very error prone A high level of interconnectivity between services
  4. 4. Goal To be able to create development andtesting environments dynamically, in a customizable way, and on cloud infrastructure A Cloud in a Cloud
  5. 5. Additional Challenges Environments need to be verified as fully functional before deploying software on them Must support multiple locations Must be able to scale dynamically Must support concurrent development • Different feature branches Must manage/version non-software components in a lockstep with the software: • Network configuration • System-level configurations • Application-level configurations
  6. 6. tools 02/01/11
  7. 7. Abstraction layer on resource managementIf you code your infrastructure managementcode to the jclouds API, you can port yourcode to all cloud providersJVM-basedProvides access to GoGrid’s infrastructure byabstracting the communication GoGrid’sCloud API
  8. 8. Uses JClouds to create and manage nodesProvisions nodes with software andconfigurationVerifies node and service correct operationOperates the services in the nodesNodes are grouped by groups (formerlyknown as ‘tags’). All nodes in a group areequivalent
  9. 9. ClojurePallet is built on ClojureClojure is a JVM-based lisp with superbintegration with Java and concurrencyCan use any existing Java library (Tomcat, jdbc,etc...)Provides a high level of abstraction, which iswhat is needed in this caseGrowing community: there are 7 books on Clojureeither published or in preparation
  10. 10. how? 02/01/11
  11. 11. GoGrid Cloud Infrastructure Service ServiceGoGrid Cloud API Service Service Service Service Service Service Service Service Service
  12. 12. GoGrid Cloud Infrastructure Service Service GoGrid Cloud APIjclouds Service Service Service Service Service Service Service Service Service
  13. 13. GoGrid Cloud Infrastructuregimme Service Servicenodes! GoGrid Cloud API jclouds Service Service Service Service Service Service Service Service Service
  14. 14. GoGrid Cloud Infrastructuregimme Service Servicenodes! GoGrid Cloud API jclouds Service Service Service Service Service Service Service Service Service
  15. 15. GoGrid Cloud Infrastructure Service ServiceGoGrid Cloud API Service Service Service Service Service Service Service Service Service
  16. 16. GoGrid Cloud Infrastructure Service Service et GoGrid Cloud APIpall Service Service Servicepallet Service Service Service Service Service Service
  17. 17. GoGrid Cloud Infrastructuredo stuff! Service Service et GoGrid Cloud API pall Service Service Service pallet Service Service Service Service Service Service
  18. 18. GoGrid Cloud Infrastructuredo stuff! Service Service et GoGrid Cloud API pall Service Service Service pallet Service Service Service Service Service Service
  19. 19. GoGrid Cloud InfrastructureGoGrid Cloud API
  20. 20. GoGrid Cloud Infrastructure GoGrid Cloud APIjclouds
  21. 21. GoGrid Cloud Infrastructure make mea dev cloud! GoGrid Cloud API jclouds
  22. 22. GoGrid Cloud Infrastructure make me Service Servicea dev cloud! GoGrid Cloud API jclouds Service Service Service Service Service Service Service Service Service
  23. 23. GoGrid Cloud Infrastructure Service Service et GoGrid Cloud APIpall Service Service Servicepallet Service Service Service Service Service Service
  24. 24. GoGrid Cloud Infrastructure install cloud Service Serviceinfrastructure! et GoGrid Cloud API pall Service Service Service pallet Service Service Service Service Service Service
  25. 25. GoGrid Cloud Infrastructure install cloud Service Serviceinfrastructure! et GoGrid Cloud API pall Service Service Service pallet Service Service Service Service Service Service
  26. 26. GoGrid Cloud Infrastructure Service Service et GoGrid Cloud APIpall Service Service Servicepallet Service Service Service Service Service Service
  27. 27. GoGrid Cloud Infrastructureis all working Service Service fine? et GoGrid Cloud API pall Service Service Service pallet Service Service Service Service Service Service
  28. 28. GoGrid Cloud Infrastructureis all working Service Service fine? et GoGrid Cloud API pall Service Service Service pallet Service Service Service OK! Service Service Service
  29. 29. GoGrid Cloud Infrastructure Service Service et GoGrid Cloud APIpall Service Service Servicepallet Service Service Service Service Service Service
  30. 30. GoGrid Cloud Infrastructure deploy Service Servicenew build! et GoGrid Cloud API pall Service Service Service pallet Service Service Service Service Service Service
  31. 31. GoGrid Cloud Infrastructure deploy Service Servicenew build! et GoGrid Cloud API pall Service Service Service pallet Service Service Service Service Service Service
  32. 32. GoGrid Cloud Infrastructure deploy GoGrid Cloud Infrastructure Service Servicenew build! et GoGrid Cloud API pall GoGrid Cloud API Service Service Service pallet Service Service Service Service Service Service
  33. 33. GoGrid Cloud Infrastructure GoGrid Cloud Infrastructure Service ServiceGoGrid Cloud API GoGrid Cloud API Service Service Service Service Service Service Service Service Service
  34. 34. GoGrid Cloud Infrastructureawesome! GoGrid Cloud Infrastructure Service Service GoGrid Cloud API GoGrid Cloud API Service Service Service Service Service Service Service Service Service
  35. 35. GoGrid Cloud Infrastructure GoGrid Cloud Infrastructure Service ServiceGoGrid Cloud API GoGrid Cloud API Service Service Service Service Service Service Service Service Service
  36. 36. GoGrid Cloud Infrastructure GoGrid Cloud Infrastructure Service Service GoGrid Cloud API GoGrid Cloud API Service Service Servicejclouds Service Service Service Service Service Service
  37. 37. GoGrid Cloud Infrastructure let’s test: GoGrid Cloud Infrastructure Service Servicegimme nodes! GoGrid Cloud API GoGrid Cloud API Service Service Service jclouds Service Service Service Service Service Service
  38. 38. GoGrid Cloud Infrastructure let’s test: GoGrid Cloud Infrastructure Service Servicegimme nodes! GoGrid Cloud API GoGrid Cloud API Service Service Service jclouds Service Service Service Service Service Service
  39. 39. GoGrid Cloud Infrastructure GoGrid Cloud Infrastructure Service ServiceGoGrid Cloud API GoGrid Cloud API Service Service Service Service Service Service Service Service Service
  40. 40. GoGrid Cloud Infrastructure GoGrid Cloud Infrastructure Service Service GoGrid Cloud API GoGrid Cloud API Service Service Servicejclouds Service Service Service Service Service Service
  41. 41. GoGrid Cloud InfrastructureI’m done, GoGrid Cloud Infrastructure Service Service thanks! GoGrid Cloud API GoGrid Cloud API Service Service Service jclouds Service Service Service Service Service Service
  42. 42. GoGrid Cloud InfrastructureI’m done, thanks! GoGrid Cloud API jclouds
  43. 43. GoGrid Cloud InfrastructureGoGrid Cloud API
  44. 44. environment GoGrid Cloud Infrastructurefor branch A? GoGrid Cloud API
  45. 45. environment GoGrid Cloud Infrastructurefor branch A? GoGrid Cloud Infrastructure branch A GoGrid Cloud API GoGrid Cloud API
  46. 46. environment GoGrid Cloud Infrastructurefor branch A? GoGrid Cloud Infrastructure branch A environment GoGrid Cloud API GoGrid Cloud APIfor branch B?
  47. 47. environment GoGrid Cloud Infrastructurefor branch A? GoGrid Cloud Infrastructure branch A GoGrid Cloud Infrastructure branch B environment GoGrid Cloud API GoGrid Cloud APIfor branch B? GoGrid Cloud API
  48. 48. environment GoGrid Cloud Infrastructurefor branch A? GoGrid Cloud Infrastructure branch A GoGrid Cloud Infrastructure branch B environment GoGrid Cloud API GoGrid Cloud APIfor branch B? GoGrid Cloud API environmentfor QA please?
  49. 49. environment GoGrid Cloud Infrastructurefor branch A? GoGrid Cloud Infrastructure branch A GoGrid Cloud Infrastructure branch B GoGrid Cloud Infrastructure QA environment GoGrid Cloud API GoGrid Cloud APIfor branch B? GoGrid Cloud API GoGrid Cloud API environmentfor QA please?
  50. 50. development 02/01/11
  51. 51. GoGrid’s Development Process Create a feature branch Create an environment (cloud infrastructure) for this branch Verify new environment (end-to-end tests) Automatically build and deploy the new software every time new code is committed to the branch. Destroy environment and free resources when done
  52. 52. An Environment Usually one environment for active development branch Each environment is specified by: • The development branch it tracks • Service types and node count for each service type • Network configuration • External services/DB provisioning
  53. 53. Development Environments Each development environment tracks a development feature branch Each environment has a node with Hudson CI, configured to track the environment’s branch When new code is pushed to the development branch, the environment’s Hudson server builds the software and updates the environment with it.
  54. 54. more on pallet 02/01/11
  55. 55. Why Pallet? There are well known solutions for node provisioning in the cloud, why pallet? • It integrates with infrastructure management • Service interconnections are first class entities in the framework • Scalable programming model • Integrates well with SCM and with other OPS tools • Comprehensive support for all development cycles
  56. 56. ✓ No central server✓ No agents on nodes✓ Works with standard and custom images✓ First-class support for complex multi-service configurations✓ All node configuration done remotely via generated scripts over SSH✓ DSL for generating scripts, resources✓ Testable
  57. 57. crates Pallet’s basic configuration unit A crate is a grouping of functions around a service Functions usually: • manipulate/create internal representations • result in resources deployed or scripts being run in the target nodes
  58. 58. proxiedCustom Crates tomcat reverse bootstrap tomcat deploy proxyStandard Crates automated haproxy admin user java tomcat
  59. 59. A Node(core/defnode  proxied    "Basic  web  app,  served  by  tomcat"    {:os-­‐family  :ubuntu        :os-­‐description-­‐matches  "10.04"      :inbound-­‐ports  [8080  22]}      :bootstrap  (resource/phase  (crates/bootstrap))    :configure  (resource/phase  (crates/tomcat))    :deploy  (resource/phase   (crates/tomcat-­‐deploy  "webapp.war"))    :restart-­‐tomcat  (resource/phase                                      (service/service   "tomcat6" :action  :restart)))
  60. 60. jclouds section of a node(core/defnode  proxied    "Basic  web  app,  served  by  tomcat"    {:os-­‐family  :ubuntu        :os-­‐description-­‐matches  "10.04"      :inbound-­‐ports  [8080  22]}      :bootstrap  (resource/phase  (crates/bootstrap))    :configure  (resource/phase  (crates/tomcat))    :deploy  (resource/phase   (crates/tomcat-­‐deploy  "webapp.war"))    :restart-­‐tomcat  (resource/phase                                      (service/service   "tomcat6" :action  :restart)))
  61. 61. Execution Phases(core/defnode  proxied    "Basic  web  app,  served  by  tomcat"    {:os-­‐family  :ubuntu        :os-­‐description-­‐matches  "10.04"      :inbound-­‐ports  [8080  22]}      :bootstrap  (resource/phase  (crates/bootstrap))    :configure  (resource/phase  (crates/tomcat))    :deploy  (resource/phase   (crates/tomcat-­‐deploy  "webapp.war"))    :restart-­‐tomcat  (resource/phase                                      (service/service "tomcat6"   :action  :restart)))
  62. 62. Cloud Infrastructure Crates proxy proxy Cloud API Pallet webapp webapp webapp Nodedefinitions DB DB
  63. 63. Cloud Infrastructure Crates proxy proxy Cloud API Pallet webapp webapp webapp Nodedefinitions DB DB
  64. 64. Cloud Infrastructure Crates proxy proxy Cloud API Pallet webapp webapp webapp Nodedefinitions DB DB
  65. 65. Cloud Infrastructure Crates proxy proxy jclouds Cloud API Pallet webapp webapp webapp Nodedefinitions DB DB
  66. 66. Cloud Infrastructure proxy proxy proxy proxy Crates jclouds Cloud API Pallet webapp webapp webapp Nodedefinitions DB DB
  67. 67. Cloud Infrastructure proxy proxy proxy proxy Crates jclouds Cloud API webapp webapp webapp Pallet webapp webapp webapp Nodedefinitions DB DB
  68. 68. Cloud Infrastructure proxy proxy proxy proxy Crates jclouds Cloud API webapp webapp webapp Pallet webapp webapp webapp Node DB DB DB DBdefinitions
  69. 69. Cloud Infrastructure proxy proxy proxy proxy Crates Cloud API webapp webapp webapp Pallet webapp webapp webapp Node DB DB DB DBdefinitions
  70. 70. Cloud Infrastructure proxy proxy proxy proxy Crates squid squid Cloud API webapp webapp webapp Pallet webapp webapp webapp Node DB DB DB DBdefinitions
  71. 71. Cloud Infrastructure proxy proxy proxy proxy Crates squid squid Cloud API webapp webapp webapp Pallet webapp webapp webapp Node DB DB DB DBdefinitions
  72. 72. Cloud Infrastructure proxy proxy proxy proxy Crates squid squid Cloud API webapp webapp webapp Pallet webapp webapp webapp tomcat tomcat tomcat Node DB DB DB DBdefinitions
  73. 73. Cloud Infrastructure proxy proxy proxy proxy Crates squid squid Cloud API webapp webapp webapp Pallet webapp webapp webapp tomcat tomcat tomcat Node DB DB DB DBdefinitions
  74. 74. Cloud Infrastructure proxy proxy proxy proxy Crates squid squid Cloud API webapp webapp webapp Pallet webapp webapp webapp tomcat tomcat tomcat Node DB DB DB DBdefinitions mysql mysql
  75. 75. How are systems interconnected? Phase 1 • Retrieve and/or create needed nodes • For each node in each group: run phases for crates and a) Collect configuration info b) Schedule configuration actions to be performed on node Phase 2 • Generate and execute scripts and resources
  76. 76. how does this look today? 02/01/11
  77. 77. Environment Definition(def branch-1-environment {:branch "http://svn....../branches/branch-1" :compute-service {:provider "gogrid" :identity "<username>" :credential "<password>" :endpoint "http://10...:8080/api/"} :db-name "branch-1-db" :db-properties env-network-properties :pre-bootstrap-fn #setup-environment-db :bootstrap-tags {com.gogrid.nodes.services/LDAP 1 com.gogrid.nodes.services/Cache 1} :bootstrap-phases [:restart-cache :restart-LDAP] :verify-phases [:verify]})
  78. 78. Operation user => (start branch-1-environment) user => (verify branch-1-environment) user => (destroy branch-1-environment)
  79. 79. Challenge with Pallet It’s a newcomer in the field. • Documentation needs work • Untested support for some combinations of flavors and versions of Linux Steep learning curve, a new language OMG! It’s A Lisp!
  80. 80. Some pointers... http://jclouds.org Adrian Cole et al. http://palletops.com Hugo Duncan et al. http://gogrid.com For more info on GoGrid services please contact: Paul Lappas VP Engineering & Co-Founder of GoGrid paul@gogrid.com GoGrid is hiring devops and SW engineers for their SF office (contact Paul too)
  81. 81. Questions? Toni Batchelli tbatchelli@gmail.com @tbatchelli @disclojure 02/01/11

×