Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Keeping Your CI/CD Pipeline as Fast as It Needs to Be

274 views

Published on

Slides from my presentation in JavaOne 2016 on the topic of how to keep your CI/CD pipeline under control. Don't let it grow to unmanageable build times! Learn to find out when your pipeline is too slow and you need to do something about it, and when it's fine and you can just carry on with your life.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Keeping Your CI/CD Pipeline as Fast as It Needs to Be

  1. 1. Abraham Marin-Perez @AbrahamMarin fromfragiletoagile.com Keeping Your CI / CD Pipeline as Fast as It Needs to Be #JavaOne @AbrahamMarin @EqualExperts
  2. 2. About Me
  3. 3. About Me
  4. 4. About Me
  5. 5. About Me
  6. 6. About Me
  7. 7. About Me
  8. 8. About Me
  9. 9. About Me
  10. 10. About Me https://goo.gl/I0lbhi
  11. 11.  Continuous Integration: check everything is still working after every commit  Continuous Deployment: every successful commit turns into a release What is CI / CD?
  12. 12. About This Talk
  13. 13. About This Talk
  14. 14. About This Talk
  15. 15. About This Talk
  16. 16. SUPER APP # Files: 75 # Tests: 800 Build Time: 4 min Output: superapp.war
  17. 17. SUPER APP # Files: 113 # Tests: 1200 Build Time: 6 min Output: superapp.war
  18. 18. SUPER APP # Files: 169 # Tests: 1800 Build Time: 9 min Output: superapp.war
  19. 19. Slow feedback Broken builds mask issues Development paralysis Impact on ability to meet our SLAs Missed business opportunities The Problems Of Size
  20. 20. Live with it Partial CD: only quick tests Phased CD: split into components Test Deprecation Policy Microservices How Organisations Manage Size
  21. 21. Microservices
  22. 22. Microservices
  23. 23. Microservices
  24. 24. Microservices
  25. 25. Microservices
  26. 26. Microservices
  27. 27. Microservices
  28. 28. Microservices
  29. 29. Microservices
  30. 30. SUPER APP # Files: 169 # Tests: 1800 Build Time: 9 min Output: superapp.war APP BACKEN D SUPER APP # Files: 115 # Tests: 1200 Build Time: 6 min Output: superapp.war # Files: 72 # Tests: 800 Build Time: 4 min Output: appbackend.jar
  31. 31. https://goo.gl/LvkkRq Scalable Continuous Deployment With Maven
  32. 32. A real case scenario
  33. 33. WAR file WAR file WAR file Parent POM Logging
  34. 34. WAR file WAR file WAR file Parent POM Logging 28%
  35. 35. WAR file WAR file WAR file Parent POM Logging 28% 28% 28%
  36. 36. WAR file WAR file WAR file Parent POM Logging 28% 28% 28%
  37. 37. WAR file WAR file WAR file Parent POM Logging 28% 28% 28% 20%
  38. 38. WAR file WAR file WAR file Parent POM Logging 48% 28% 28% 20%
  39. 39. WAR file WAR file WAR file Parent POM Logging
  40. 40. WAR file WAR file WAR file
  41. 41.  Build Time (BT): time an individual build takes to run  Change Rate (CR): percentage of commits upon an individual build with respect to the whole system Useful Metrics
  42. 42. WAR file WAR file WAR file
  43. 43. WAR file WAR file WAR file Parent POM Logging 28%
  44. 44.  Impact Time (IT): total time to run a build and all the builds that will be triggered as a result Useful Metrics
  45. 45. No dependants  IT(A) = BT(A) A Useful Metrics
  46. 46. Serial execution  IT(A) = BT(A) + IT(B) + IT(C) B A C Useful Metrics
  47. 47. Parallel execution  IT(A) = BT(A) + max(IT(B), IT(C)) B A C Useful Metrics
  48. 48. WAR file WAR file WAR file
  49. 49. Weighted Impact Time (WIT): impact time of a build weighted according to its change rage WIT(A) = IT(A) * CR(A) Useful Metrics
  50. 50. Average Impact Time (AIT): total time needed, on average, to execute all necessary builds after any given commit anywhere in the system AIT = WIT(A) + WIT(B) + ... + WIT(Z) Useful Metrics
  51. 51. Sample Thresholds
  52. 52. Average Impact Time Average Impact Time is what indicates how well you have scaled your system Sample Thresholds
  53. 53. Maximum Impact Time In a worst-case scenario, a build won’t take longer than this. Sample Thresholds
  54. 54. Maximum Impact Time for Critical Components The same, but only for your most sensitive modules (log-in, payment gateway, etc.) Beware of dependencies! Sample Thresholds
  55. 55. WAR file WAR file WAR file
  56. 56. Manual processing takes time...
  57. 57.  Most CI systems provide an API  Calculations aren’t complex  Multiple graphical tools available Automating Build Analysis
  58. 58. github.com/quiram/build-hotspots Build Hotspots
  59. 59. https://commons.wikimedia.org/wiki/File:2012_Italian_GP_-_Lotus_wheel.jpg
  60. 60. Thank You @EqualExper ts equal- experts equalexperts.com Thank You fromfragiletoagile.com @AbrahamMarin #FastCI #JavaOne

×