Automated Testing with Agile


Published on

This is a presentation given at the Hangzhou Scrum Forum 2009, sponsored by Perficient, China. The topic is how to incorporate automated functional testing into an agile project, and also some best practices, tips, and warnings.

Published in: Technology

Automated Testing with Agile

  1. 1. Automated Testing with Agile Ken McCorkell May 16th, 2009
  2. 2. Automated Testing with Agile <ul><li>What is Automated Testing? </li></ul><ul><li>Why use automated testing in Agile projects? </li></ul><ul><li>Traditional testing vs. Agile testing </li></ul><ul><li>How to best use Automated Testing? </li></ul><ul><li>Best Practices for writing automated tests </li></ul><ul><li>Agile, Automation, and Continuous Integration </li></ul><ul><li>Test Automation common mistakes </li></ul><ul><li>Testing Tools </li></ul>
  3. 3. What is Automated Testing? <ul><li>Automated testing is also called “Auto-Test”, “Auto-Testing” or “Automatic testing”, “Scripting” or just “Automation” </li></ul><ul><li>Automated testing is any testing that is done without human interaction. Tests can be fully automated, or partially automated. </li></ul><ul><li>Automated tests can take the following forms: </li></ul><ul><ul><li>Unit Testing </li></ul></ul><ul><ul><li>Data Setup </li></ul></ul><ul><ul><li>installation </li></ul></ul><ul><ul><li>Test Execution of any type of test – db, web, UI, webservice </li></ul></ul><ul><ul><li>Validation of Test Results </li></ul></ul><ul><ul><li>Checkpoints of System Test environment </li></ul></ul><ul><ul><li>Integration with Build System </li></ul></ul><ul><ul><li>Automatic test results reporting </li></ul></ul>
  4. 4. Why use auto-tests in agile projects? <ul><li>Automation is essential to ensure constant communication between testing and development efforts </li></ul><ul><li>Quick visibility into code quality </li></ul><ul><li>Not defect driven but test driven </li></ul><ul><li>Automation is difficult – but if solved and invested in early, will make changes much easier later in the project </li></ul><ul><li>Developers and testers can own test scripts together, problems with automation details can be solved as a team </li></ul><ul><li>Make testing part of definition of done </li></ul><ul><li>Main reason: Automation is often the only way to generate the velocity for testing needed to release fully tested builds and potential shippable projects in short iterations </li></ul>
  5. 5. Traditional Testing <ul><li>Separate test group from development </li></ul><ul><li>Tests are derived from detailed requirements instead of being part of requirements </li></ul><ul><li>Testing is conducted after development is complete </li></ul><ul><li>Defects are written to communicate with developers </li></ul><ul><li>Developers and Testers are working against each other </li></ul>
  6. 6. Agile Testing <ul><li>Testers are part of the development team ( pure agilists will say there are no testers, only teams covering all tasks) </li></ul><ul><li>Team works closely with customers to define acceptance tests for each requirement </li></ul><ul><li>Testers are included from the beginning of the requirements and design </li></ul><ul><li>Test each feature early as it is completed </li></ul><ul><li>Pair testing between testers and developers </li></ul><ul><li>Provides continuous feedback to development </li></ul>
  7. 7. How to best use Automated Testing <ul><li>Automated tests can be used on most projects </li></ul><ul><li>Automated testing for UIs can be done at any phase of Testing, but is usually done as UI is developed </li></ul><ul><li>Should plan to automate most repeated tests first </li></ul><ul><li>Automation should replace very important tests that are tedious or take time to execute manually </li></ul><ul><li>For a legacy application without automated tests, plan to implement some automated tests each iteration to make gradual progress </li></ul><ul><li>Use automation to test on multiple OS or environment configurations </li></ul><ul><li>The goal is usually not to automate 100% of tests </li></ul>
  8. 8. Best Practices for writing Automated Tests <ul><li>Best Practices for UI tests: </li></ul><ul><ul><li>Do not simply record and playback tests </li></ul></ul><ul><ul><li>Parameterize data use for tests – Data-Driven Testing </li></ul></ul><ul><ul><li>Parameterize field names for easy maintenance </li></ul></ul><ul><ul><li>Write reusable modules for common test functions </li></ul></ul><ul><ul><li>Make tests atomic – tests should not depend on other tests </li></ul></ul><ul><ul><li>Remember: automated tests are code, and should be planned just like coding for an application </li></ul></ul>
  9. 9. Agile, Automation, and Continuous Integration <ul><li>Steps for continuous integration in an agile project: </li></ul><ul><li>Automate the build </li></ul><ul><li>create tests against local build </li></ul><ul><li>Refactor tests until they pass with others’ code </li></ul><ul><li>Commit code along with tests to code repository </li></ul><ul><li>Tests are executed on your local build before check-ins, and after each build by the integration server. </li></ul><ul><li>If tests fail on the integration server – the build fails </li></ul><ul><li>Results are sent by email and shows who checked in since the latest check-in and the broken tests </li></ul><ul><li>Fix the build so that all tests pass </li></ul>
  10. 10. Test Automation common mistakes <ul><li>These are common mistakes teams make about auto-tests: </li></ul><ul><li>Automated tests can take a lot of time – plan ahead and only automate important tests </li></ul><ul><li>Make sure you have the right resources with the correct skills to implement automated test code </li></ul><ul><li>Do not require all automated testing to be done by testers – developers should also help </li></ul><ul><li>Not all testers need to write automated tests </li></ul>
  11. 11. Testing Tools <ul><li>Some automated test tools are: </li></ul><ul><ul><ul><li>TestNG –open source unit and integration testing for java </li></ul></ul></ul><ul><ul><ul><li>Selenium IDE/RC – open source automated web testing framework </li></ul></ul></ul><ul><ul><ul><li>soapUI – open source webservices/SOA automated testing </li></ul></ul></ul><ul><ul><ul><li>Jmeter/BadBoy – open source load testing for web applications </li></ul></ul></ul><ul><ul><ul><li>Fitnesse – wiki-based automation framework </li></ul></ul></ul><ul><ul><ul><li>HP QuickTest Pro, HP WinRunner, HP LoadRunner, HP Performance Center, Rational – commercial tools </li></ul></ul></ul><ul><ul><ul><li>Write your own! </li></ul></ul></ul>