Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Successful PaaS and CI in the Cloud - EclipseCon 2012

2,181 views

Published on

Session at AgileALM/EclipseCon 2012 presented by Steve Harris of CloudBees

Published in: Technology, Business
  • Be the first to comment

Successful PaaS and CI in the Cloud - EclipseCon 2012

  1. 1. Successful PaaS and CI in the CloudSteven G. Harrissteven.g.harris@cloudbees.com@stevengharrisAgileALM/EclipseCon 2012 ©2012 CloudBees, Inc. All Rights Reserved
  2. 2. Platform as a Service ©2012 CloudBees, Inc. All Rights Reserved
  3. 3. As-a-Service Examples TodaySaaS "Cloud computing is on- demand access to virtualized IT resources that are housed outside of your own data center, shared by others, simple to use, paid for via PaaS subscription, and accessed over the Web.“ - John Foley, Information Week IaaS ©2012 CloudBees, Inc. 3 All Rights Reserved
  4. 4. Traditional Software Stack Validate Setup Update Monitor Patch ©2012 CloudBees, Inc. 4 All Rights Reserved
  5. 5. Cloud Computing: How to do it? Who doeswhat? Validate Setup Cloud Provider? Update Monitor Patch ©2012 CloudBees, Inc. 5 All Rights Reserved
  6. 6. PaaS in Summary• Applications and data are at the center of your world! – Forget about servers, VMs, load-balancers, etc.• Cloud concepts are applied to applications as first class citizens – On-demand, pay-as-you-go, elasticity, etc. – No need to handle updates, patches, scalability, failover, etc. Delivered as a Service, not as Packaged Software! ©2012 CloudBees, Inc. 6 All Rights Reserved
  7. 7. External Overview of CloudBees PaaS cloudbees.com Partner Test Ecosystem Forge Repositories Git Grand Session Central Clustering Web Jenkins SVN Code Stage Console Master SDK mvn Router CloudBees Build API Application Jenkins Executor MySQL Developer and Development Runtime End User Operations Services Services Interaction Interaction ©2012 CloudBees, Inc. 7 All Rights Reserved
  8. 8. What’s So Hard About Building PaaS, Anyway?• Technology investment to operate at scale• Intersection of app development/deployment and complex infrastructure• Cultural shift can be more challenging than technology• Efficient use of infrastructure resource pools ©2012 CloudBees, Inc. 8 All Rights Reserved
  9. 9. Internal Overview of CloudBees PaaS cloudbees.com Partner Test Ecosystem Forge Repositories Git Grand Session Central Clustering Web Jenkins SVN Code Stage Console Master SDK mvn Router CloudBees Build API Application Jenkins Executor MySQL Identity Provisioning AS Agent DB Agent … Scaling Monitoring AnyCloud Message Bus Alerting Auditing Shared Services Agents ©2012 CloudBees, Inc. 9 All Rights Reserved
  10. 10. Example of What PaaS and Cloud Enable ©2012 CloudBees, Inc. All Rights Reserved
  11. 11. Easier Deployment Model Load balancer App v7 Building Cloud Tools for NetFlix: http://www.slideshare.net/joesondow/building-cloudtoolsfornetflix-9419504 ©2012 CloudBees, Inc. 11 All Rights Reserved
  12. 12. Easier Deployment Model Load balancer App v7 App v8 ©2012 CloudBees, Inc. 12 All Rights Reserved
  13. 13. Easier Deployment Model Load balancer App v7 App v8 ©2012 CloudBees, Inc. 13 All Rights Reserved
  14. 14. Easier Deployment Model Load balancer App v7 App v8 ©2012 CloudBees, Inc. 14 All Rights Reserved
  15. 15. Easier Deployment Model Load balancer App v7 App v8 ©2012 CloudBees, Inc. 15 All Rights Reserved
  16. 16. Easier Deployment Model Load balancer App v7 App v8.1 ©2012 CloudBees, Inc. 16 All Rights Reserved
  17. 17. Easier Deployment Model Load balancer App v8.1 ©2012 CloudBees, Inc. 17 All Rights Reserved
  18. 18. Easier Deployment Model Load balancer • Doubled resources – briefly & cheaply! App v8.1 • Reduced risk dramatically ©2012 CloudBees, Inc. 18 All Rights Reserved
  19. 19. CI and the Cloud ©2012 CloudBees, Inc. All Rights Reserved
  20. 20. Development Tradeoffs for the Cloud Ecosystem Elasticity Managed Services Scale Latency Turnaround Local speed Local control Your environment Your IDE Your debugger ©2012 CloudBees, Inc. 2020 All Rights Reserved
  21. 21. Use the Right Tool for the Job Low Resource High Resource Machines High IT Integration Cloud Testing Advantages Unit X-Platform Testing Functional Testing Performance Testing Testing Stress Debugging Testing Regression Cutting Testing Acceptance Code! Testing Local Code People Low IT Advantages Quality High Touch Low Touch ©2012 CloudBees, Inc. 2121 All Rights Reserved
  22. 22. CI Resources Over a Few Days What you would need What you have (and pay for) What you consume ©2012 CloudBees, Inc. 22 All Rights Reserved
  23. 23. CI Resources Over a (Traditional) Product Lifecycle What you would need What you consumeProject Team Public start working Team Hollidays Release Bug working ! fix Maintenance ©2012 CloudBees, Inc. 23 All Rights Reserved
  24. 24. Your CI feels like… but looks like… ©2011 CloudBees, Inc. All Rights Reserved 24
  25. 25. Cloud & PaaS – An Abundance of ComputeResource• How do you get it when you need it?• Once you have it, how do you harness it?• Once you’ve harnessed it, how do you keep it control and running at peak efficiency? What you want What you want to avoid ©2012 CloudBees, Inc. 25 All Rights Reserved
  26. 26. Jenkins Has Been Doing Distributed Builds for 5Yrs• Easily able to control and manage 100+ computers from a single place• Plugin mechanism simplifies doing more ©2012 CloudBees, Inc. 26 All Rights Reserved
  27. 27. Validated Merge of CommitsTaking advantage of an abundance of computing: #1 ©2012 CloudBees, Inc. ©2010 CloudBees, Inc. All Rights Reserved All Rights Reserved
  28. 28. Is CI Really Helping You as Much as It Could?• Does your CI server shift work from laptops to servers? – You need to commit to have Jenkins test it – But if your commit is bad, it blocks others – You end up testing locally before committing – … #fail ©2012 CloudBees, Inc. 28 All Rights Reserved
  29. 29. Mathematics of Large Projects Every developer makes mistakes once in a while ↓ The more developers you add, the less stable the repository gets100% 80% 60% 40% 20% 0% 0 5 10 15 20 ©2012 CloudBees, Inc. 29 All Rights Reserved
  30. 30. Solution: Validated Merge• Dev commits to branches• Jenkins tests branches• If good, Jenkins merges to the trunkAliceBobMaster ©2012 CloudBees, Inc. 30 All Rights Reserved
  31. 31. It’s Not as Bad as It Might Look• More realistic commit graph is like this – Especially if devs remain close to the tip – Your coding is slower than “slow tests” ©2012 CloudBees, Inc. 31 All Rights Reserved
  32. 32. Advantages• Your mistake doesn’t impact others – Commit without worrying• Tests run on servers – Large environment-dependent tests are no longer a problem• Tests run asynchronously – Commit, then move on – You don’t wait for tests to complete• Works with other plugins… – Subversion Merge Plugin – Git Plugin – Gerrit PluginRef: http://blog.cloudbees.com/2012/03/dont-phunk-with-my-stable-branch.html ©2012 CloudBees, Inc. 32 All Rights Reserved
  33. 33. Hierarchical Validated Merges• NetBeans team works like this master repo team team team repo repo repo ©2012 CloudBees, Inc. 33 All Rights Reserved
  34. 34. Automated DeploymentTaking advantage of an abundance of computing: #2 ©2012 CloudBees, Inc. ©2010 CloudBees, Inc. All Rights Reserved All Rights Reserved
  35. 35. 35
  36. 36. But You Need To Test Before Deploying• It compiled ≠ it’s ready to ship• Hence pipeline – Progressively run more expensive tests – A failure, and it’s out – Avoids wasting computer cycles and improves feedback speed Unit Build Int. Test Staging UAT Test ©2012 CloudBees, Inc. 36 All Rights Reserved
  37. 37. Build Pipeline: #1 Build Integration• Waterfall model Code• Connect jobs via upstream/ Quality downstream• Copy binaries from upstream Deploy ©2012 CloudBees, Inc. 37 All Rights Reserved
  38. 38. Same Model from Different Angle TimeBuild Build Build Build Build Build Integration Integration Integration Deploy Deploy ©2012 CloudBees, Inc. 38 All Rights Reserved
  39. 39. Build Pipeline Plugin ©2012 CloudBees, Inc. 39 All Rights Reserved
  40. 40. Promoted Builds Plugin• Think about state flow, not process flow – What conditions trigger a transition? Good build QC Pass Stability • Compiled OK • Coverage > 60% • Ran > 3d in • Unit test passed • Int. tests passed UAT env ©2012 CloudBees, Inc. 40 All Rights Reserved
  41. 41. Combining Pipeline and Promotions Developer Org Developer Tests Developer Job Functional Tests Security Tests X-Team Tests Access Tests QA Org Database Tests SQE Job Compliance Tests Stress Tests Production Org Backup Production Production Job Deploy Monitor ©2012 CloudBees, Inc. 4141 All Rights Reserved
  42. 42. Advantages of Promotion• Asynchronous• Retryable• Flexibility in shape• More stable – Conditions can be tweaked – Human teams often interface by such states• Break free from triggers – More flexible conditions• More opportunity to take advantage of resources ©2012 CloudBees, Inc. 42 All Rights Reserved
  43. 43. TraceabilityFinding the needle in the compute haystack ©2012 CloudBees, Inc. ©2010 CloudBees, Inc. All Rights Reserved All Rights Reserved
  44. 44. 44
  45. 45. Machine-Assisted Traceability• Stitch together information in siloed systems – Dev: commit ID – QA: test execution log – Ops: deployment log, running version check ©2012 CloudBees, Inc. 45 All Rights Reserved
  46. 46. CI Server Is Big Brother• Source : commit ID = Binary : checksum – In Jenkins, we call it “fingerprints”• Liberally record fingerprints every time you “use” binaries – When you run tests – When you deploy – When you integrate with something else• Then cross check where they showed up ©2012 CloudBees, Inc. 46 All Rights Reserved
  47. 47. Traceability and Pipeline Time 1a5d a820 83ad 2f03 02d9 ecdaBuild Build Build Build Build Build 1a5d 83ad 2f03 Integration Integration Integration 1a5d 2f03 Deploy Deploy ©2012 CloudBees, Inc. 47 All Rights Reserved
  48. 48. Use Case ©2012 CloudBees, Inc. All Rights Reserved
  49. 49. Major Retailer (courtesy Marco Vermeulen)• Directive from top management to move to cloud• Assessment of various solutions – Committed to CI – Did not want to manage infrastructure, monitoring – Grails app – Wanted to use standard .war• Continuous delivery in Dev and Test environments• Release after each 2-week sprint• Source code on GitHub• Plugins… – Git, Cobertura, Codenarc, Violations, Grails, Pipeline, Twitter ©2012 CloudBees, Inc. 49 All Rights Reserved
  50. 50. Production Pipeline View • Exploits parallelism • Mirrored DEV and PRODUCTION pipeline • Manual trigger to push to production ©2012 CloudBees, Inc. 50 All Rights Reserved
  51. 51. Enforcing Your Team’s Values • Code coverage < 80 triggers FAILURE • Excessive Codenarc violations trigger FAILURE ©2012 CloudBees, Inc. 51 All Rights Reserved
  52. 52. Summary• PaaS is serving up platform capabilities to you without the headaches of install, update, patch, and ongoing infrastructure management• Removing roadblocks to dev/test/production resource availability changes the way we have to work• Extensible CI server like Jenkins provides hooks to harness the abundance of resources ©2012 CloudBees, Inc. 52 All Rights Reserved
  53. 53. Thank You!steven.g.harris@cloudbees.com@stevengharris ©2012 CloudBees, Inc. 53 All Rights Reserved
  54. 54. ©2012 CloudBees, Inc. All Rights Reserved

×