Chapter 2 - Testing in Agile

Testing in Agile
1 Requirement
Engineering
2 Testing in Agile 3 Test Automation
Software Testing - ISTQB Advance
Agile Technical Tester Exam Preparation
Chapter 2
Neeraj Kumar Singh
4 Deployment &
Delivery

Testing in Agile
Contents
2.1 Agile development and test techniques
2.2 Experience-based testing in Agile
2.3 Aspects of code quality
Neeraj Kumar Singh

Testing in Agile
Contents
2.1.1 Apply test-driven development
2.1.2 Apply behavior-driven development
2.1.3 Analyze a product backlog to introduce
acceptance test-driven development
Neeraj Kumar Singh

Agile Development and Test Techniques
Applying Behavior Driven Development
According to the ISTQB® Agile Foundation syllabus, behavior-driven development (BDD) is a technique in which
developers, testers, and business representatives work together to analyze the requirements of a software system,
formulate them using a shared language, and verify them automatically.
BDD is strongly influenced by two separate disciplines: test-driven development (TDD) and domain driven design
(DDD). It incorporates many of the core principles of both disciplines, including collaboration, design, ubiquitous
language, automated test execution, short feedback cycles, and more.
TDD relies on unit tests to verify implementation details whereas BDD relies on executable scenarios to verify
behaviors and more. BDD generally follows a prescribed path:
 Create user stories collaboratively
 Formulate user stories as executable scenarios and verifiable behaviors
 Implement behaviors and execute the scenarios to verify them
Teams that apply BDD extract one or more scenario from each user story and then formulate them as automated
tests. A scenario represents a single behavior under specific conditions. Scenarios are typically based on user
story’s acceptance criteria, examples, and use cases. BDD scenarios should be written using language that can be
understood by all team members, whether technical or non-technical.
Neeraj Kumar Singh

BDD scenarios are typically composed of three main sections :
1. Given – describes the state of the environment (preconditions) before the behavior is triggered
2. When – describes the actions that trigger the behavior
3. Then – describes the expected results of the behavior
Extracting scenarios from user stories involves:
 Identifying all acceptance criteria specified by a user story and writing scenarios for each criterion.
 Identifying functional use cases and examples and writing scenarios for each of them.
 Creating scenarios while testing exploratory, which can help identifying related existing behaviors, potential conflicting
behaviors, minimal states, alternative flows, etc.
 Looking for repeated use of steps or groups of steps to avoid repetitive work.
 Identifying areas that require random or synthetic data.
 Identifying steps that require mocks, stubs or drivers to maintain isolation and to avoid executing integrations or possibly
costly processes.
 Ensuring scenarios are atomic and do not affect each other’s state (isolated).
 Deciding whether to limit the When sections to one step in accordance with the principle that every test checks just one
thing, or to optimize for other considerations, such as test execution speed.
Neeraj Kumar Singh

Recommended guidelines for formulating scenarios include:
 The scenario should describe a specific behavior that the system supports from the perspective of a specific user.
 The scenario should use the third person when describing the steps (Given, When, Then) to describe the state and
interactions from the perspective of the user.
 The scenarios should be isolated and atomic so that they can be run in any order and not affect each other or rely on
each other.
 The When steps should describe the semantic actions that a user performs rather than the specific technical actions,
unless there is a particular need to test a specific action. For example, “The user confirms the order” (a semantic
action) is generally preferable to “The user clicks the Confirm button” (a technical action) unless the button itself
has to be tested.
 The Then steps should describe specific observations or states.
Neeraj Kumar Singh

Applying Acceptance Test Driven Development
Acceptance test-driven development (ATDD) supports the coordination of software projects to address the delivery
based on the customer needs. Acceptance tests are specifications for the desired behavior and functionality of a
system.
A user story represents a piece of customer-valued functionality. Acceptance tests verify that this functionality is
implemented correctly.
This user story is split by the developers into a set of tasks required to implement that functionality. Developers
can implement these tasks using TDD. When a given task is completed, the developers move on to the next task,
and so forth, until the user story is completed, which is indicated by successfully executed acceptance tests.
Both BDD and ATDD are customer focused whereas TDD is developer focused. BDD and ATDD are similar in that they
both produce the same result: a shared understanding of what is to be built and how to build the thing right. BDD
is a structured way of writing test cases
Neeraj Kumar Singh

ATDD separates the test intention, which is expressed in a human readable format such as plain text, from the test
implementation, which is preferably automated. This important separation means that business-facing and other
non-technical team members such as product owners, analysts, and testers can take an active role in describing
testable examples (or acceptance tests) to drive development.
Stakeholders with different roles and perspectives, such as customer, developer, and tester come together to
collaboratively discuss a potential user story (or other form of requirement) to reach a shared understanding of the
problem to be solved, to ask each other open questions regarding the functionality and to explore concrete,
testable examples of required behavior.
The agreed examples are valuable outputs in themselves, so it is important to remember that although they may
also be automated as acceptance tests, it is not always necessary or cost-effective to do so.
Neeraj Kumar Singh

This illustrates an important role for the Agile tester in this situation. During the discussions, the tester may be
thinking in terms of test techniques, such as equivalence partitioning, boundary value analysis, and so on. They can
use this analytical approach to encourage questions to be asked to the product owner about where the interesting
behaviors and edge cases can be found.
Specification by example (or SBE) refers to a collection of useful patterns that allow an Agile team to discover,
discuss, and confirm the important outcomes required by business stakeholders and the software behaviors that
are needed to reach those outcomes.
The term has been widely used to include and broaden the meaning of the term acceptance test-driven
development (ATDD).
Neeraj Kumar Singh

Experience Based Techniques in Agile
Contents
2.2.1 Combining experience-based techniques
and black-box tests
2.2.2 Creating test charters and
interpreting their results
Neeraj Kumar Singh

Combining Experience based and Black box Techniques
The various characteristics of Agile projects such as the approach, length of iteration, applicable test level(s), risk
level of project and product, quality of requirements, level of experience/expertise of the team members, project
organization, etc. will influence the balance of automated tests, exploratory tests, and manual black-box tests in
an Agile project.
During risk analysis, the risk levels (e.g. high, medium, low) are determined for the individual system features and
functionality. The next step is to find the right mix and balance of automated tests, exploratory tests, and manual
black-box tests for a specific risk level.
Neeraj Kumar Singh

Below is a table describing this idea. In this table, the risk levels are listed vertically and the three test approaches
horizontally.
The table below is an example of a mix of different testing techniques (exploratory, automated, and manual) that
can be used in a safety critical system. This table can be adapted to other specific projects.
++ (highly recommended) + (recommended) o (neutral) - (not recommended) -- (not to be used)
When looking at the first row, it suggests that in this situation, a combination of automated test and black-box test
would be highly recommended in addition to an exploratory testing approach. The decision to automate (or not)
will also be influenced by many other factors.
Neeraj Kumar Singh
Safety Critical
System

The following is an example of a combination of different testing techniques when used in a non-safety critical
system.
++ (highly recommended) + (recommended) o (neutral) - (not recommended) -- (not to be used)
When looking at the last row, it suggests that in this situation, an exploratory testing approach is highly
recommended, while other approaches might not be used. In any situation (safety critical or not) the specific
mixture will depend on the project characteristics
Neeraj Kumar Singh
Non-Safety
Critical System

Creating Test Charters and Interpreting Their Results
Before an appropriate test charter can be created, the existing epics and user stories should be evaluated first.
When analyzing epics or user stories to create a test charter the following things should be considered:
 Who are the users in this epic or user story?
 What is the main functionality of the epic or user story?
 What are the actions a user can perform? (This can be obtained from the acceptance criteria list, which is
defined for a user story)
 Is the goal of the user story realized once the feature or functionality is complete? (Or are there other testing
tasks affecting the Definition of Done)
The granularity of a test charter is important. It should not be too small, as it must explore an area around an
identified problem (re-active, regression) or an area around a user story or an epic (proactive, revealing).
It shouldn’t be too big, as it should fit within a 60 to 120 minutes time box. The goal of running an exploratory
testing session is to help us make a good quality decision, regarding a phenomenon, bug clustered area, etc..
Neeraj Kumar Singh

Creating Test Charters and Interpreting Their Results
 Test charters can be created using flipcharts, spreadsheets, documents, existing test management system,
personas, mind mapping and a whole team approach. Exploratory testers use heuristics to drive their
creativity in writing and performing exploratory testing sessions. These can also be used for creating test
charters and in thinking creatively when analyzing user stories and epics.
 All the findings found during exploratory testing should be documented. The results of exploratory testing
should provide insights into better test design, ideas for testing the product, and ideas for any further
improvement. Findings that should be documented during exploratory testing include defects, ideas,
questions, improvement suggestions, etc.
 Tools can be used for documenting the exploratory testing sessions. These include video capture and logging
tools, planning tools, etc. The documentation should include the expected result. In some cases, pen and
paper is sufficient, depending on the volume of information to collect.
 When summarizing the exploratory testing session, during the debriefing meeting the information is collected
and aggregated to present a status of progress, coverage, and efficiency of the session. This summary
information can be used as management report, or used in retrospective meetings on any level and any scale.
 However, it can be quite challenging to determine accurate test metrics related to exploratory testing
sessions.
Neeraj Kumar Singh

Aspects of Code Quality
Contents
2.3.1 Refactoring
2.3.2 Code reviews and static code analysis
to identify defects and technical debt
Neeraj Kumar Singh

Refactoring
Refactoring is a way to clean up code in an efficient and controlled manner, by clarifying and simplifying the design
of existing code and test cases, without changing its behavior.
In Agile projects, the iterations are short, which creates a short feedback loop for all team members. Short
iterations also create a challenge for testers trying to achieve adequate coverage.
Due to the nature of the iterations, the fact that functionality is growing, and features are added and enhanced
over time, test cases that have been written for a feature in an earlier iteration, often need maintenance or even
complete redesign in later iterations.
After user stories are understood and acceptance criteria are written for each of them, the impact of the
functionality of the current iteration on the existing regression tests (manual and automated) can be analyzed, and
refactoring and/or enhancing of the tests may be required.
Teams are maintaining and extending their code extensively from iteration to iteration, and without continuous
refactoring, this is hard to do.
Neeraj Kumar Singh

Refactoring
The refactoring of the test cases can be done as follows:
 Identification: Identifying existing tests that require refactoring by reviewing or causal analysis.
 Analysis: Analyzing the impact of the changed tests on the overall regression test set.
 Refactor: Making changes to the internal structure of the tests to make it easier to understand and cheaper to
modify without changing its observable behavior.
 Re-run: Re-running the tests, checking their results and documenting defects where relevant. The refactoring
should not have influenced the result of test execution.
 Evaluate: Checking the results of the re-run tests, ending this phase when the tests have passed a quality
threshold defined and accepted by the team.
Neeraj Kumar Singh

Code Review and Static Code Analysis
A code review is a systematic examination of code by two or more people (one of whom is usually the author).
Static code analysis is the systematic examination of code by a tool. Both are effective and widespread practices
for exposing and identifying issues that affect code quality. Code reviews and static code analysis provide
constructive feedback that helps identify defects and manage technical debt.
Impediments such as resource constraints, higher technical complexity than expected, rapidly changing priorities,
and technical limitations may hinder efforts to write quality code and force programmers to make compromises
that will decrease the quality of the code in favour of more immediate results. These compromises can introduce
defects and incur technical debt.
Technical debt refers to the increased effort that will be required to implement a better solution (including
removing latent defects) in the future as a consequence of choosing an inferior but easierto-implement solution
now. Technical debt is often incurred unintentionally, by subtle compromises or a gradual accumulation of small or
unnoticed changes as the software evolves.
Besides helping identify defects and manage technical debt, code analysis and reviews offer additional benefits:
 Training and sharing knowledge
 Improving the robustness, maintainability, and readability of the code
 Providing oversight and maintaining consistent coding standards
Neeraj Kumar Singh

Code Review
By participating in code reviews, testers can use their unique perspective to make valuable contributions to code
quality by collaboratively working with programmers to identify potential defects and avoid technical debt at a
very early stage.
They can bring their expertise to bear in many ways, such as by asking questions about the behavior of the code,
suggesting use cases that may not have been considered, or monitoring code metrics that could indicate quality
issues.
Code reviews are manual activities, possibly supported by tools, that are performed by or with other people (in
addition to the author).
Code reviews may be performed with the reviewer and the author/developer sitting side-by-side. This mode,
which is common in ad-hoc reviews and pair programming, facilitates excellent communication and encourages
deeper analyses and better knowledge sharing.
On distributed teams or teams that prefer a more disconnected approach, the code review process is facilitated by
the configuration management system. The process is usually partially automated as part of the continuous
integration process. These processes may support code reviews with a single reviewer in each round of review, or
collaborative team reviews.
Neeraj Kumar Singh

Static Code Analysis
In static code analysis, a tool analyzes the code and searches for specific issues without executing the code. The
results of static code analysis may point at clear issues in the code or provide indirect indicators that require
further assessment.
Many development tools, especially integrated development environments (IDEs), can perform static code analysis
while writing the code. This provides the benefit of immediate feedback, though it may only be possible to apply
just a subset of the analysis performed during continuous integration.
Neeraj Kumar Singh

Testing in Agile
Sample Questions Pattern
Neeraj Kumar Singh
Source : www.istqb.org

Testing in Agile
Sample Questions
1. Summarize the characteristics of performing exploratory tests with test charters.
Select ONE options.
Answer Set
a. In contradiction to black-box testing, the expected result is documented after a defect is found and not as part
of the test design.
b. Test charters are a useful tool to be used for testing, when a detailed specification to the system under test is
given.
c. The result of performing exploratory testing by using test charters is finding defects and specification error.
d. Exploratory testing and black-box testing use the same metrics, when it comes to measure the test coverage.
Neeraj Kumar Singh

Testing in Agile
Sample Questions
2. You do have to analyze the following Bubble Sort function written in Java:
Analyzation should be done according to the following checklist:
1. All Variables should start with a Capital letter, except they are used only
as counting variable
2. If the name of the variable consists of more than one word, all words
have to start with Capital letters (e.g. FirstName)
3. All constants should be written in CAPITAL letters, except they are only
used as start or end point (e.g. in loops)
Which one of the above mentioned checkpoint items is not fulfilled? .
Select ONE options.
Answer Set
a. 2
b. 1
c. 3
d. None
Neeraj Kumar Singh

Testing in Agile
Sample Questions
3. You, as a tester in a BDD environment, know that the following user story has to be implemented and tested:
As a Scrum Master
I want to see Lead/Cycle time progress
So that I know whether we are improving our development process or not.
Which one of the following BDD scenarios in Gherkin format is at least partially, but best fitting to this user
story?
Select ONE options.
Answer Set
a. Given Reports section in Project area and Bug Tracking practice is DISABLED When I navigate to Lead and Cycle
Time Report Then I see Lead Time chart And chart contains 1 line for stories.
b. Given Reports section in Project area and Bug Tracking practice is ENABLED If I navigate to Lead and Cycle Time
Report Then I see Cycle Time And chart contains 2 lines (for stories and bugs) Else Error message is popping up.
c. Given Reports section in Project area and Bug Tracking practice is DISABLED When I navigate to Lead and Cycle
Time Report Then I see the chart And chart contains information for all stories.
d. When Reports section in Project area and Bug Tracking practice is ENABLED Then I navigate to Report section And
I see Cycle Time with chart containing 2 lines (for stories and bugs).
Neeraj Kumar Singh

Chapter 2 - Testing in Agile

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Chapter 2 - Testing in Agile

Similar to Chapter 2 - Testing in Agile (20)

More from Neeraj Kumar Singh

More from Neeraj Kumar Singh (17)

Recently uploaded

Recently uploaded (20)

Chapter 2 - Testing in Agile