Automated Testing with Agile Ken McCorkell May 16th, 2009

Automated Testing with Agile What is Automated Testing? Why use automated testing in Agile projects? Traditional testing vs. Agile testing How to best use Automated Testing? Best Practices for writing automated tests Agile, Automation, and Continuous Integration Test Automation common mistakes Testing Tools

What is Automated Testing?

Why use automated testing in Agile projects?

Traditional testing vs. Agile testing

How to best use Automated Testing?

Best Practices for writing automated tests

Agile, Automation, and Continuous Integration

Test Automation common mistakes

Testing Tools

What is Automated Testing? Automated testing is also called “Auto-Test”, “Auto-Testing” or “Automatic testing”, “Scripting” or just “Automation” Automated testing is any testing that is done without human interaction. Tests can be fully automated, or partially automated. Automated tests can take the following forms: Unit Testing Data Setup installation Test Execution of any type of test – db, web, UI, webservice Validation of Test Results Checkpoints of System Test environment Integration with Build System Automatic test results reporting

Automated testing is also called “Auto-Test”, “Auto-Testing” or “Automatic testing”, “Scripting” or just “Automation”

Automated testing is any testing that is done without human interaction. Tests can be fully automated, or partially automated.

Automated tests can take the following forms:

Unit Testing

Data Setup

installation

Test Execution of any type of test – db, web, UI, webservice

Validation of Test Results

Checkpoints of System Test environment

Integration with Build System

Automatic test results reporting

Why use auto-tests in agile projects? Automation is essential to ensure constant communication between testing and development efforts Quick visibility into code quality Not defect driven but test driven Automation is difficult – but if solved and invested in early, will make changes much easier later in the project Developers and testers can own test scripts together, problems with automation details can be solved as a team Make testing part of definition of done Main reason: Automation is often the only way to generate the velocity for testing needed to release fully tested builds and potential shippable projects in short iterations

Automation is essential to ensure constant communication between testing and development efforts

Quick visibility into code quality

Not defect driven but test driven

Automation is difficult – but if solved and invested in early, will make changes much easier later in the project

Developers and testers can own test scripts together, problems with automation details can be solved as a team

Make testing part of definition of done

Main reason: Automation is often the only way to generate the velocity for testing needed to release fully tested builds and potential shippable projects in short iterations

Traditional Testing Separate test group from development Tests are derived from detailed requirements instead of being part of requirements Testing is conducted after development is complete Defects are written to communicate with developers Developers and Testers are working against each other

Separate test group from development

Tests are derived from detailed requirements instead of being part of requirements

Testing is conducted after development is complete

Defects are written to communicate with developers

Developers and Testers are working against each other

Agile Testing Testers are part of the development team ( pure agilists will say there are no testers, only teams covering all tasks) Team works closely with customers to define acceptance tests for each requirement Testers are included from the beginning of the requirements and design Test each feature early as it is completed Pair testing between testers and developers Provides continuous feedback to development

Testers are part of the development team ( pure agilists will say there are no testers, only teams covering all tasks)

Team works closely with customers to define acceptance tests for each requirement

Testers are included from the beginning of the requirements and design

Test each feature early as it is completed

Pair testing between testers and developers

Provides continuous feedback to development

How to best use Automated Testing Automated tests can be used on most projects Automated testing for UIs can be done at any phase of Testing, but is usually done as UI is developed Should plan to automate most repeated tests first Automation should replace very important tests that are tedious or take time to execute manually For a legacy application without automated tests, plan to implement some automated tests each iteration to make gradual progress Use automation to test on multiple OS or environment configurations The goal is usually not to automate 100% of tests

Automated tests can be used on most projects

Automated testing for UIs can be done at any phase of Testing, but is usually done as UI is developed

Should plan to automate most repeated tests first

Automation should replace very important tests that are tedious or take time to execute manually

For a legacy application without automated tests, plan to implement some automated tests each iteration to make gradual progress

Use automation to test on multiple OS or environment configurations

The goal is usually not to automate 100% of tests

Best Practices for writing Automated Tests Best Practices for UI tests: Do not simply record and playback tests Parameterize data use for tests – Data-Driven Testing Parameterize field names for easy maintenance Write reusable modules for common test functions Make tests atomic – tests should not depend on other tests Remember: automated tests are code, and should be planned just like coding for an application

Best Practices for UI tests:

Do not simply record and playback tests

Parameterize data use for tests – Data-Driven Testing

Parameterize field names for easy maintenance

Write reusable modules for common test functions

Make tests atomic – tests should not depend on other tests

Remember: automated tests are code, and should be planned just like coding for an application

Agile, Automation, and Continuous Integration Steps for continuous integration in an agile project: Automate the build create tests against local build Refactor tests until they pass with others’ code Commit code along with tests to code repository Tests are executed on your local build before check-ins, and after each build by the integration server. If tests fail on the integration server – the build fails Results are sent by email and shows who checked in since the latest check-in and the broken tests Fix the build so that all tests pass

Steps for continuous integration in an agile project:

Automate the build

create tests against local build

Refactor tests until they pass with others’ code

Commit code along with tests to code repository

Tests are executed on your local build before check-ins, and after each build by the integration server.

If tests fail on the integration server – the build fails

Results are sent by email and shows who checked in since the latest check-in and the broken tests

Fix the build so that all tests pass

Test Automation common mistakes These are common mistakes teams make about auto-tests: Automated tests can take a lot of time – plan ahead and only automate important tests Make sure you have the right resources with the correct skills to implement automated test code Do not require all automated testing to be done by testers – developers should also help Not all testers need to write automated tests

These are common mistakes teams make about auto-tests:

Automated tests can take a lot of time – plan ahead and only automate important tests

Make sure you have the right resources with the correct skills to implement automated test code

Do not require all automated testing to be done by testers – developers should also help

Not all testers need to write automated tests

Testing Tools Some automated test tools are: TestNG –open source unit and integration testing for java Selenium IDE/RC – open source automated web testing framework soapUI – open source webservices/SOA automated testing Jmeter/BadBoy – open source load testing for web applications Fitnesse – wiki-based automation framework HP QuickTest Pro, HP WinRunner, HP LoadRunner, HP Performance Center, Rational – commercial tools Write your own!

Some automated test tools are:

TestNG –open source unit and integration testing for java

Selenium IDE/RC – open source automated web testing framework

soapUI – open source webservices/SOA automated testing

Jmeter/BadBoy – open source load testing for web applications

Fitnesse – wiki-based automation framework

HP QuickTest Pro, HP WinRunner, HP LoadRunner, HP Performance Center, Rational – commercial tools

Write your own!