Downloaded 45 times
Uploaded byDevOps Enterprise Summit
Tanay Nagjee - Electric Cloud - Better Continuous Integration with Test Acceleration
The document discusses strategies to improve continuous integration through test acceleration and emphasizes the financial impact of software bugs. It highlights the use of ElectricAccelerator for parallelizing builds and tests to reduce cycle times significantly, demonstrating successful case studies where test duration was drastically decreased. The overall goal is to achieve faster, more efficient software delivery without sacrificing testing quality.
More Related Content
Similar to Tanay Nagjee - Electric Cloud - Better Continuous Integration with Test Acceleration
![[India Merge World Tour] Electric Cloud](https://cdn.slidesharecdn.com/ss_thumbnails/india-mergeworldtourelectriccloud-130819053645-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...](https://cdn.slidesharecdn.com/ss_thumbnails/dpesummitseptember20-212023-howimprovingthetestingexperiencegoesbeyondqualityadeveloperproductivityp-230924000523-453e8af8-thumbnail.jpg?width=640&height=640&fit=bounds)
Tanay Nagjee - Electric Cloud - Better Continuous Integration with Test Acceleration
- 1. © Electric Cloud| electric-cloud.com Better Continuous Integration with Test Acceleration Tanay Nagjee Solutions Architect & Manager, Electric Cloud @tanayciousd
- 2. © Electric Cloud| electric-cloud.com
- 3. © Electric Cloud| electric-cloud.com High stakes “Ford recalls 433,000 cars: software bug breaks engine off-switch” The Register, 2015 “To keep a Boeing Dreamliner flying, reboot every 248 days” Engadget, 2015 Healthcare.gov: “hundreds of software bugs had to be found and fixed” Forbes, 2013 “One bad algorithm cost traders $440m” The Register, 2012 Software Bugs Cost Economy $312 Billion Per Year Cambridge University, 2013
- 4. © Electric Cloud| electric-cloud.com Business pressures that impact testing Get-it-out-the-door mentality Will test more later New feature commitments Fast feedback
- 5. © Electric Cloud| electric-cloud.com Try to speed up the commit build. Continuous Integration on a build of a few hours is better than nothing, but getting down to that magic ten minute number is much better. This usually requires some pretty serious surgery on your code base to do as you break dependencies on slow parts of the system. - Martin Fowler
- 6. © Electric Cloud| electric-cloud.com CI testing toolbox Unit Test Tools Static Analysis Tools Code Coverage Tools
- 7. © Electric Cloud| electric-cloud.com Typical fixes for long test suites Manually parallelize test jobs to do more in the same amount of time Buy faster hardware Accept long CI cycles Cut down on tests and analyses that are run during CI
- 8. © Electric Cloud| electric-cloud.com
- 9. © Electric Cloud| electric-cloud.com
- 10. © Electric Cloud| electric-cloud.com
- 11. © Electric Cloud| electric-cloud.com
- 12. © Electric Cloud| electric-cloud.com ElectricAccelerator dramatically accelerates software builds and tests by safely parallelizing jobs across shared clusters of physical or cloud CPUs.
- 13. © Electric Cloud| electric-cloud.com ElectricAccelerator Massive parallelization and distribution for fast builds and tests Actionable visualization and reports for optimal parallelization Automatic dependency detection Efficient resource utilization; create build/test clouds to lower costs
- 14. © Electric Cloud| electric-cloud.com
- 15. © Electric Cloud| electric-cloud.com Use case #1 • Need to reduce cycle time to 10 minutes to enable gated check-ins and continuous delivery • Sticklers for test-driven development, suites will grow over time • Tools: Boost, MS Code Coverage, Bullseye • Time to run tests & coverage: Before: 7m30s After: 3m11s on 2 machines & 39s best case 2.4x2 Hosts 11.5xBest Case
- 16. © Electric Cloud| electric-cloud.com Use case #2 • Browser tests not previously run during CI because they take too long • Tool: Selenium • Time to run test suite (~100 tests): Before: 27m30s After: 3m49s on 2 machines & 27s best case 7.2x2 Hosts 61xBest Case
- 17. © Electric Cloud| electric-cloud.com Demonstration
- 18. © Electric Cloud| electric-cloud.com How does parallelizing tests help? Shorter CI cycles without sacrificing tests/coverage/analysis Comprehensive testing Scale no matter how large the code base grows Minimize new hardware investment Maximize current hardware investment Continue to use existing tools Learn what tests take the longest and any dependencies
- 19. © Electric Cloud| electric-cloud.com The journey Build Compile, Package Test Functional, non-functional Deploy Provision, Configure, Deploy weeks days+ automation minutes+ acceleration How long does it take to get a software update through a delivery pipeline?
- 20. © Electric Cloud| electric-cloud.com “If it hurts, do it more frequently, and bring the pain forward.” Martin Fowler
- 21. © Electric Cloud| electric-cloud.com Thank you! http://electric-cloud.com/downloads @tanayciousd
Editor's Notes
- #2 Feedback from last year’s DOES: focus more on testing Introduce myself (history at EC, dev for EF not EA, then SE, used EA in the field) What we’re going to cover today: Why testing is so important Problems with long test suites Step by step approach to speeding up tests Quick overview of ElectricAccelerator Some successful EA use cases Live demonstration How to get EA and try out the free offering, Huddle Questions & answers
- #3 #1 observation when it comes to the realm of automation/DevOps/whatever… …this is a real thing that needs to be treated as a first class citizen and invested in!
- #4 Software is EVERYWHERE, which makes the stakes of poor quality very high. And it will only get higher.
- #5 Let’s agree that testing is important, and talk about some business pressures that affect testing We live in a competitive landscape, and getting features to market is critical Testing is not glamorous, so it is almost natural to want to punt on it When you’re frantically trying to check off a list of features, testing falls through the cracks In orgs where automation is advanced, CI cycles can get so long that testing is moved out
- #6 There are lots of types of testing done at various stages of the delivery lifecycle Today, I’d like to focus on continuous integration (CI) builds POLL: How many people have CI cycles at 10 minutes or less?
- #7 What are some of the basic groups of CI testing tools?
- #8 Not as efficient as it could be, and manual=yuck Faster hardware is expensive, existing hardware is underutilized Accepting long cycles means losing a competitive edge in the long run (e.g. developer context switch penalty) Running tests/analyses at night or weekend means bugs linger, context switch penalty, teasing apart lots of commits to debug POLL: how many people have experienced 2 or more of these?
- #9 Let’s take a step back and talk about the journey to test acceleration Step 0 is to have tests to accelerate; again, we are not here to talk about the merits of testing SE story: I’ve been asked whether EC products can help generate unit tests
- #10 The number 1 place to start, not just for CI and testing but for everything. But for today, we are talking about CI. SE story: a script is not always a script (movie script vs. shell script)
- #11 Coarse grained parallelism means running more things at once. This is a win no doubt, especially if it is automated and not manual. It is not difficult to do this with automation tools, you just have to know how and spend the time SE story: manually estimating and slicing test suites up into batches
- #12 Fine grained parallelism means getting down to the lowest possible level and distributing it across compute nodes. Compute is cheap compared to developer time today. SE story: this can sometimes take creativity, but it is time well spent.
- #13 The truth is EA can accelerate pretty much anything, and tests are a natural fit!
- #14 A quick overview of the key benefits of ElectricAccelerator. The graph on the right shows something we had running at DOES last year
- #16 Let’s talk about a couple of success cases using EA to speed up testing
- #19 Now that you’ve seen it in action, let’s review the benefits as related to the problems we pointed out earlier.
- #20 Where does accelerating your tests fall into the grand scheme of things?