Build And Test Automation - Shortening the Feedback Loop


Published on

Agile teams embrace build and test automation to shorten the time between coding and testing feedback, gain productivity leverage, and lower the cost of change.

Explore the automation practices that Agile teams adopt.

Published in: Technology

Build And Test Automation - Shortening the Feedback Loop

  1. 1. © 2009 Rally Software Development, Inc.<br />1<br />Build and Test Automation<br />Shortening theFeedback Loop<br />
  2. 2. Agile Teams Embrace Build and Test Automationfor Shorter Feedback Loops and Leverage<br />© 2009 Rally Software Development, Inc.<br />2<br /><ul><li>Shorter Feedback LoopsAutomation shortens the timebetween implementation and feedback,while recent changes are top of mind.
  3. 3. LeverageShort, time-boxed iterations require usto be highly productive. Automationprovides us with this leverage.
  4. 4. Lowering the Cost of ChangeA key principle of Agile is to embrace change.Automated tests give us a safety net to refactor the system.</li></li></ul><li>Best, Least Cost AutomationMike Cohn’s Testing Pyramid<br />© 2009 Rally Software Development, Inc.<br />3<br /><ul><li>Mike Cohn devised the notion of a testing pyramid to show how Agile teams best allocate their automation resources.
  5. 5. At the top of the pyramid we have tests that are the most expensive to maintain. Agile teams put the least investment here.
  6. 6. At the bottom of the pyramid, we maximize use of automated tests with the lowest Total Cost of Ownership (TCO).
  7. 7. So what are the three levels?...</li></li></ul><li>Unit Tests have the highest ROI<br />© 2009 Rally Software Development, Inc.<br />4<br /><ul><li>Agile teams have found that Unit Tests are the single most valuable approach to automated testing.
  8. 8. Unit tests have low TCO, because they minimize the intervening code, which is the major source of test death (when product changes break the test).</li></ul>From Brian Marick’s“When Should a Test be Automated” –<br />
  9. 9. Unit Testing frees Resourcesfor Acceptance and Exploratory Testing<br />© 2009 Rally Software Development, Inc.<br />5<br /><ul><li>Unit tests are written by programmers, in the same language they develop in.
  10. 10. Unit tests take the smallest piece of testable code, isolate it from the rest of the system, and determine if the results are as expected.
  11. 11. Since unit tests isolate the code under test, they remain stable over time, with very lowTotal Cost of Ownership (TCO).
  12. 12. Unit tests serve as the base of the regression suite. They lower cost of change as the team can modify code knowing there is a safety net.
  13. 13. Unit Tests free Testers to do exploratory testing, automate acceptance tests and testnon-functional requirements.</li></li></ul><li>Unit Tests are used in conjunction with Continuous Integration (CI)<br />© 2009 Rally Software Development, Inc.<br />6<br /><ul><li>CI is a software development practice, where…
  14. 14. Team members integrate their work frequently,usually with each check-in to the version controlsystem…
  15. 15. An automated server builds the system,runs all the unit testsand reports “green” or “red”to indicate if the build is stable/broken.
  16. 16. Building and Testing are done on a dedicatedmachine to eliminate “it works on my machine”dependencies.</li></ul>Jacob<br />
  17. 17. How to Get Startedwith Continuous Integration<br />Very High ROI. One of the first engineeringpractices an Agile teams should adopt.<br />Many open source and commercial tools(/<br />Keep builds and tests fast(move slow running tests to a nightly build)<br />Adopt a “Stop the Line” mentality(all work stops until a broken build is fixed)<br />Test in a clone of production(Every environmental difference results in risk)<br />© 2009 Rally Software Development, Inc.<br />7<br />CI, IDE & VCS<br />Rally seamlessly integrates with Build Automation Solutions<br />
  18. 18. Functional/Acceptance Tests <br />© 2009 Rally Software Development, Inc.<br />8<br /><ul><li>Automated Functional Tests(end-to-end, not through the GUI) form the middle tier.
  19. 19. Agile teams typically have at least one per user story to automate the Acceptance Criteria established by the Product Owner.
  20. 20. Tests are developed iteratively
  21. 21. Start with the happy path
  22. 22. Automate the Acceptance Criteria
  23. 23. Add details as a result of exploratory learning
  24. 24. System Testability (e.g., services APIs, database migration, etc.) impacts cost.An Agile benefit is that testers raise testability issues while the team can do something about it.</li></li></ul><li>Getting Started with Functional Testing<br />Many commercial, open source tools.Rally integrates with most includingHP Quality Center<br />Start small and iterate – easy to get overwhelmed here. Use context-driven approaches to guide investmentdecisions<br />© 2009 Rally Software Development, Inc.<br />9<br /><ul><li>Emphasize Tester/Developer collaboration to support Testability
  25. 25. Most teams run these tests in a nightly build as they are too slowto run at check-in. Your CI server should support nightly builds.</li></li></ul><li>GUI Tests are Used for Smoke Tests<br />© 2009 Rally Software Development, Inc.<br />10<br /><ul><li>Many commercial and open source tools. Rallyintegrates with most, including HP Quality Center.
  26. 26. Agile teams have the fewest tests here. Theyoften use GUI tests to create a “smoke test” suite.
  27. 27. Smoke tests validate the basic functionality of the system. They exercise the system from end to end. They are not exhaustive tests, but are capable of detecting major problems.
  28. 28. Smoke tests are typically run as part of the nightly build Continuous Integration cycle.</li></li></ul><li>Build Artifact Repository<br />© 2009 Rally Software Development, Inc.<br />11<br /><ul><li>Advanced Agile Teams put buildautomation in place so that Testersand Product Owners can pull anybuild at any time
  29. 29. A “Test this Build” capabilityempowers Testers and POsto be completely in control of theirtesting environment, eliminatingdelays and a common source offrustration.
  30. 30. A “Build Artifact Repository” canbe as simple as a folder with zip files.There are also commercial tools andmany Continuous Integration solutionsinclude this capability (e.g., Maven).</li></ul>Jacob<br />
  31. 31. Test Driven Development<br />Test Driven Development (TDD) andAcceptance Test Driven Development(ATDD) are technical practices thatmany Agile teams adopt.<br />TDD and ATDD are specification anddesign approaches as opposed to thetraditional validation function of testing. They are useful emergent design approaches for producing higher quality, more maintainable code.<br />In TDD, a programmer writes a test before they write any production code. The test fails and the programmer writes just enough code to makeit pass. This cycle is repeated on timelines of every few minutes.<br />© 2009 Rally Software Development, Inc.<br />12<br />
  32. 32. Summary and Next Steps<br />Summary<br />Automate for shorter feedback loops…<br />Leverage and Productivity…<br />…And a lower cost of change<br />Continuous Integration + Unit Testingpays huge dividends<br />Use Cohn’s Testing Pyramid to Guide Automation Investment Decisions<br />Next Steps<br />© 2009 Rally Software Development, Inc.<br />13<br />View the “Leading an Agile Transition”presentation in this series to see how testing fits in a 1-4 week time-box.<br />Explore the Implementing Agile Teams, Agile Test & Engineering Practices, and Rally JumpStart Service Offerings<br />Request a free 30-day Rally Enterprise Edition trial.<br />