Agile Testing ( Its all about early feedback ) Pankaj Nakhat [email_address]

[email_address]

What’s different? Nothing (Well… not quite) 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

What’s different?

Nothing

(Well… not quite)

Traditional Testing Approach 06/09/09 © 2008 ThoughtWorks, Inc. ® slide Release Release

Testing in “V – Model” 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Traditional Testing Approach Contd.. The Result: Practices Intended to Control Chaos Traditional testing practices attempt to manage the chaos (or at least avoid the blame): “ Last Defender of Quality” stance Strict change management Detailed preparation and up front planning Heavy weight documentation. Strict entrance and exit criteria with signoffs Heavyweight test automation focused on regression Attempts at process enforcement 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

The Result: Practices Intended to Control Chaos

Traditional testing practices attempt to manage the chaos (or at least avoid the blame):

“ Last Defender of Quality” stance

Strict change management

Detailed preparation and up front planning

Heavy weight documentation.

Strict entrance and exit criteria with signoffs

Heavyweight test automation focused on regression

Attempts at process enforcement

Agile – Continuous Stream of Value 06/09/09 © 2008 ThoughtWorks, Inc. ® slide Release Release Release Release Release Completed , Releasable And Tested code

Agile Practices XP Rigorous set of practices designed to keep both the code and team agile. Scrum Lightweight management framework . Lean Lean manufacturing concepts applied to software development. And more.. 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

XP

Rigorous set of practices designed to keep both the code and team agile.

Scrum

Lightweight management framework .

Lean

Lean manufacturing concepts applied to software development.

And more..

XP Practices Practices Planning Games Small Releases. Simple Design. TDD Refactoring. Pair Programming. Continuous Integration. Collective Ownership Onsite Customers Values Communication Simplicity Feedback Courage 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Practices

Planning Games

Small Releases.

Simple Design.

TDD

Refactoring.

Pair Programming.

Continuous Integration.

Collective Ownership

Onsite Customers

Values

Communication

Simplicity

Feedback

Courage

Scrum 06/09/09 © 2008 ThoughtWorks, Inc. ® slide Adapted from the original at http://www.controlchaos.com/about/

Agile practices – Iterative Development A working application is delivered regularly throughout development Functionality becomes available to test in small discrete pieces Test coverage can be built up incrementally as the application is developed Tests are run when developers are writing or have just written the code, making bugs easier to track down and fix as they are still fresh in a developer’s mind Testing specific pieces of functionality within each iteration increases visibility into the testers’ activities

A working application is delivered regularly throughout development

Functionality becomes available to test in small discrete pieces

Test coverage can be built up incrementally as the application is developed

Tests are run when developers are writing or have just written the code, making bugs easier to track down and fix as they are still fresh in a developer’s mind

Testing specific pieces of functionality within each iteration increases visibility into the testers’ activities

06/09/09 © 2008 ThoughtWorks, Inc. ® slide Traditional Agile Change Manage & control it (Hard to accommodate) Change is inevitable Planning Comprehensive up front design Plan as you go Collaboration Work alone Work with Devs and BAs Hand Offs Formal entrance and exit criteria It’s not a relay race: collaborate Test Automation System-level, built by tool specialists, created after the code is “done” All levels, built by everyone, an integral part of the project

Agile Testing – Few facts In an agile team Quality is “everyone's responsibility”. QA team continuously measures the Quality and provide the feedback. Role of QA changes many times within the life cycle of the Iteration. Test Automation is used to eliminate the time waste and Increase the Test coverage for “Any time Regression”. Automation helps QA’s to focus on exploratory testing, as one can focus on other important stuffs. Quality is not added later, it evolves gradually.

In an agile team Quality is “everyone's responsibility”.

QA team continuously measures the Quality and provide the feedback.

Role of QA changes many times within the life cycle of the Iteration.

Test Automation is used to eliminate the time waste and Increase the Test coverage for “Any time Regression”.

Automation helps QA’s to focus on exploratory testing, as one can focus on other important stuffs.

Quality is not added later, it evolves gradually.

Recommended Test Lifecycle 06/09/09 © 2008 ThoughtWorks, Inc. ® slide Iterations Automated Regression Test Performance Profiling Test Full Regression Test Integration Test Acceptance Testing I 1 Exploratory, Additional Automation I 1 Test Analysis I 1 I0 I1 I4 … In I2 I3 Acceptance Testing I 2 Exploratory, Additional Automation I 2 Test Analysis I 2 Acceptance Testing I 3 Exploratory, Additional Automation I 3 Test Analysis I 3 Pre-production UAT ‘ iIities operations &Compliance Deployment Data Migration End-To-End Test Load/Stress Load/Stress End-To-End Test Integration Test

Acceptance Test Driven Development Story Planning Session

Acceptance Test Driven Development Story Writing

Acceptance Test Driven Development Automate Acceptance Tests

Acceptance Test Driven Development Implement Functionality

Acceptance Test Driven Development First BA / QA Signoff

Acceptance Test Driven Development Story Testing & Showcase

System Testing

Repeat Do it again and again and again and again and again …

Do it again

and again

and again

and again

and again …

Announce the results (ping) Done!

(ping)

Done!

Agile QA Roles 06/09/09 © 2008 ThoughtWorks, Inc. ® slide Pre Iteration In Iteration Post - Iteration Review Stories – Talk to BAs Collaborate with Developers to help them understand the Stories/Acceptance Tests Do additional tests for stories from the previous iterations. Get Understanding of the Stories Write Automated Acceptance Tests Exploratory Testing Collaborate with BAs and define Acceptance criteria for the Tests Execute the Acceptance Tests Performance Testing Involvement in Planning Report the “Bugs/Gaps” and Retest them. Add more automated tests for the bugs found during exploratory testing.

More About Acceptance Testing What is Acceptance Test? Its not a Test. It should help in Driving a story to completion with all the requirements tested. Why? Acceptance tests verify the completion of user stories. How? Ideally written before the development Iteration Ideally Automated. Acceptance tests make sure code is doing exactly what you had expected it to do. (Can be called Exit criteria for the stories) 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

What is Acceptance Test?

Its not a Test.

It should help in Driving a story to completion with all the requirements tested.

Why?

Acceptance tests verify the completion of user stories.

How?

Ideally written before the development Iteration

Ideally Automated.

Acceptance tests make sure code is doing exactly what you had expected it to do.

(Can be called Exit criteria for the stories)

User Story and Tests 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Acceptance Criteria 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Acceptance Tests Positive Tests Test that cookie is stored up on clicking ‘Remembering me’ and logs in the user next time. Basic negative Test Test that no cookies are stored on not selecting ‘Remember me’. Additional Tests Test that not more then one cookies is stored upon selecting ‘Remember me’. Test that invalid login does not store the cookie. Test that ...................... 06/09/09 © 2008 ThoughtWorks, Inc. ® slide Success -1.a – valid user logged in and referred to home page. a) ‘Remember me’ tricked – store cookie/automatic login next time.

Positive Tests

Test that cookie is stored up on clicking ‘Remembering me’ and logs in the user next time.

Basic negative Test

Test that no cookies are stored on not

selecting ‘Remember me’.

Additional Tests

Test that not more then one cookies is stored upon selecting ‘Remember me’.

Test that invalid login does not store the cookie.

Test that ......................

Collaboration with Developers Why? Testing early on their machines provides early feedback. To make sure acceptance tests are well communicated . Developers gain insight into potential errors Together can succeed with automated testing Automate Acceptance Testing Write acceptance tests using the same programming environment used for development Reuse unit testing frameworks Make the software more testable Grey Box Testing Analyze the impact of changes Understand relationships between parts of the system Understand root cause of the bug/Vice versa help developers giving the business context. Learn to diagnose problems. 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Why?

Testing early on their machines provides early feedback.

To make sure acceptance tests are well communicated .

Developers gain insight into potential errors

Together can succeed with automated testing

Automate Acceptance Testing

Write acceptance tests using the same programming environment used for development

Reuse unit testing frameworks

Make the software more testable

Grey Box Testing

Analyze the impact of changes

Understand relationships between parts of the system

Understand root cause of the bug/Vice versa help developers giving the business context.

Learn to diagnose problems.

Collaboration with BAs Why? Testers need to understand the business There are always hidden requirements Analysts need to understand how to test their requirements Involve testers earlier Acceptance Test Have Business Context May help uncovering hidden requirements 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Why?

Testers need to understand the business

There are always hidden requirements

Analysts need to understand how to test their requirements

Involve testers earlier

Acceptance Test

Have Business Context

May help uncovering hidden requirements

How defects are Managed? Defects within the Iteration. On the fly Defect on a story, after the story is signed off. Prioritize and play the story. Add a test Significant misunderstanding in how the functionality was intended to work . Create a new story and prioritize How does Defect management happens? 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Defects within the Iteration.

On the fly

Defect on a story, after the story is signed off.

Prioritize and play the story.

Add a test

Significant misunderstanding in how the functionality was intended to work .

Create a new story and prioritize

How does Defect management happens?

06/09/09 © 2008 ThoughtWorks, Inc. ® slide Issue found in Testing Demonstrate issue The issue is demonstrated to the developer(s)/BA involved with the particular feature. A severity is agreed upon by all present. Triage Defect The defect is sent to the project owner. They review the severity and assign the defect to a release Is it a Defect? yes Create New Requirement no Verify Defect Fix The person who raised the defect, or a tester, verify the defect is fixed. Release Fix for this release? yes Place defect in Backlog no Defect Fixed? yes Fix Defect Developer fixes the defect, first they create an automated test(s) that covers the defect no Defect with story being developed? no Fix now? Create defect Defect is created in Mingle and assigned a severity, and it is put in backlog. no yes yes Create defect card Defect is created in Mingle, and linked to the story it belongs to. Move the defect in “ Ready for dev ” stack.

Automation in Agile projects. Why Automated? We do it when it is appropriate. Test Driven Development. Early feedback. Regression reduced. System Test phase reduced. Time to deployment reduced. Automated the defects found during additional testing. Coverage. 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Why Automated?

We do it when it is appropriate.

Test Driven Development.

Early feedback.

Regression reduced.

System Test phase reduced.

Time to deployment reduced.

Automated the defects found during additional testing.

Coverage.

Automation is not a SILVER BULLET 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Automation Tools Unit Testing Xunit, Junit, Nunit Acceptance Testing Fitness, Selenium, HTTP Unit, Sahi 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Unit Testing

Xunit, Junit, Nunit

Acceptance Testing

Fitness, Selenium, HTTP Unit, Sahi

Testing Practices – Exploratory Testing Unscripted, manual testing Simultaneously develop and execute test cases whilst learning an application’s functionality Supplements scripted automated tests

Unscripted, manual testing

Simultaneously develop and execute test cases whilst learning an application’s functionality

Supplements scripted automated tests

Testing Practices – Release End Testing As is the case with waterfall projects the following testing activities need to occur once development is complete Final pass of end-to-end Systems Integration Test Final performance test Final security test User Acceptance Test (UAT) Production Verification Test However the time required is dramatically shorter and the likelihood of any surprises is greatly reduced

As is the case with waterfall projects the following testing activities need to occur once development is complete

Final pass of end-to-end Systems Integration Test

Final performance test

Final security test

User Acceptance Test (UAT)

Production Verification Test

However the time required is dramatically shorter and the likelihood of any surprises is greatly reduced

Build Process 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Build Process

Continually integrate and test throughout the project Continuous Integration Server Source Repository Developers

Developers check out code into their private workspaces

Continuous Integration When done, they commit changes to the repository

The CI server monitors the repository and checks out changes when they occur

The CI server builds the system and runs tests

The CI server releases deployable artifacts for QA testing

The CI server assigns a build label to the version of the code that it just built

The CI server informs the team of the successful build and generates useful metrics

If the build or tests fail, the CI server alerts the team

The team fix the issue at the earliest opportunity

Continually integrate and test throughout the project

Agile practices – Team co-location Testers, Business Analysts, Developers, Iteration Managers and Project Managers all sit together in the same area Improves communication between team members Testers develop a better understanding of what the developers and business analysts are working on

Testers, Business Analysts, Developers, Iteration Managers and Project Managers all sit together in the same area

Improves communication between team members

Testers develop a better understanding of what the developers and business analysts are working on

Building coverage In order to ensure the quality of your application, it is important to build up good test coverage with different types of testing Unit tests Acceptance tests Automated functional tests Exploratory tests End-to-end integration tests Non-functional tests

In order to ensure the quality of your application, it is important to build up good test coverage with different types of testing

Unit tests

Acceptance tests

Automated functional tests

Exploratory tests

End-to-end integration tests

Non-functional tests

Adding Value Early tester involvement issues found early reduces the time required for UAT and other release end activities Difficult questions at the story-writing-stage Automated regression suite integral part of the development process fixed bugs not creep back into the system Rapid feedback issues can be resolved quickly Tests are specified with business input Testing what really matters

Early tester involvement

issues found early

reduces the time required for UAT and other release end activities

Difficult questions

at the story-writing-stage

Automated regression suite

integral part of the development process

fixed bugs not creep back into the system

Rapid feedback

issues can be resolved quickly

Tests are specified with business input

Testing what really matters

Summary Collaborate 06/09/09 © 2008 Pankaj Nakhat slide

No Blame 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Give/Receive Early Feedback 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Help 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Contact me at [email_address] 06/09/09 © 2008 ThoughtWorks, Inc. ® slide

Contact me at

[email_address]