Maria Teryokhina presented on testing artifacts in agile projects. She discussed common testing artifacts like test plans, test cases, defects, and reports/metrics. She outlined the pros and cons of having these artifacts, noting they provide assurance and understanding but can also take time. She suggested not writing certain artifacts for small teams/projects or those with dynamic products where risks are not a priority. The presentation aimed to provide solutions to decrease effort on testing documentation in agile while still maintaining quality.

1. Core Systems Transformation Solutions
Agile-Bureaucracy
Maria Teryokhina,
N. Novgorod, 2014
Agenda

2. Confidential 2
About Myself
Education:
• NNSU, 2010 – master’s degree, “Mathematics and
Mechanics” department
Work Experience:
• 2012- “Return on Intelligence”: web-applications (HR Management,
Benefits Management systems)
• 2011-2012 “Tecom”: Windows applications for automation of the digital
broadcasting system
• 2008-2011 “Symphony Teleca”: desktop applications (PC Sync for
Android), mobile phones (Win platforms), applications for mobile phones
(Symbian);
• 2014 (Part Time) “Freemake”: IPhone applications, desktop applications

3. Confidential 3
Goals
• To provide examples and problems related to cases when Testing
Artefacts were lost in Agile Projects.
• To provide solutions how to decrease effort for Testing Documentation
support in Agile

4. Confidential 4
Agenda
• Agile Manifesto Overview
• Testing Artefacts Overview
• Pros and Cons of having Testing Artefacts

5. Confidential 5
Agile Manifesto

6. Confidential 6
Your Opinion?
Working Software OVER Comprehensive Documentation

7. Confidential 7
Different Opinions
• Documentation is not necessary at all
• Only Customer’s required Documentation is necessary
• Minimum Documentation is necessary

8. Principles behind the Agile Manifesto
• Our highest priority is to satisfy the customer through early and
continuous delivery of valuable software.
• Welcome changing requirements, even late in development. Agile
processes harness change for the customer's competitive advantage.
• Working software is the primary measure of progress.
Confidential 8

9. Confidential 9
Testing Artefacts
IEEE Std 829-1998, the "IEEE Standard for Software Test
Documentation.“:
• Test Plan
• Test Design Specification
• Test Case Specification
• Test Procedure Specification
• Test Item Transmittal Report
• Test Log
• Test Incident Report
• Test Summary Report

10. Confidential 10
Testing Artefacts
• Test Plan
• Test Cases
• Defects
• Reports/Metrics

11. Confidential 11
Test Plan (IEEE 829)
Test Plan is a document which describes the scope, approach, resources, and schedule of
the testing activities
– Test plan identifier - A unique identifier so that this document can be distinguished from all other documents.
– Introduction - A summary of the software to be tested. A brief description and history may be included to set the
context. References to other relevant documents useful for understanding the test plan are appropriate. Definitions
of unfamiliar terms may be included.
– Test items - Identifies the software items that are to be tested. The word "item" is purposely vague. It is a "chunk"
of software that is the object of testing.
– Features to be tested - Identifies the characteristics of the items to be tested. These include functionality,
performance, security, portability, usability, etc.
– Features not to be tested - Identifies characteristics of the items that will not be tested and the reasons why.
– Approach - The overall approach to testing that will ensure that all items and their features will be adequately
tested.
– Item pass/fail criteria - The criteria used to determine whether each test item has passed or failed testing.
– Suspension criteria and resumption requirements - The conditions under which testing will be suspended and the
subsequent conditions under which testing will be resumed.
– Test deliverables - Identifies the documents that will be created as a part of the testing process.
– Testing tasks - Identifies the tasks necessary to perform the testing.
– Environmental needs - Specifies the environment required to perform the testing including hardware, software,
communications, facilities, tools, people, etc.
– Responsibilities - Identifies the people/groups responsible for executing the testing tasks.
– Staffing and training needs - Specifies the number and types of people required to perform the testing, including
the skills needed.
– Schedule - Defines the important key milestones and dates in the testing process.
– Risks and contingencies - Identifies high-risk assumptions of the testing plan. Specifies prevention and mitigation
plans for each.
– Approvals - Specifies the names and titles of each person who must approve the plan.

12. Confidential 12
Test Plan (Agile)
• Test Strategy, Objectives
• Types of Testing
• Priorities
• Test Engineers Roles and Responsibilities
• Test Environment Requirements (Configuration)
• Testing Tools (Tools for Tests Automation, etc.)
• Product Transition Procedure
• Test Reporting Procedure
• Bug Reporting Procedure
• Acceptance Testing Procedure

13. Confidential 13
Test Plan: Pros and Cons
• All team members understand
their goals, responsibilities and
steps
• The critical exceptions are
prevented
• New team members
understand the process quickly
• Time is spent on changes
• Time Saving
• “What are you doing?”
• “Why should I do it?”
• “Why should I do it this way?”

14. Confidential 14
Test Plan: Don’t Write
• No process at all
• Small Team (1-2 developers only)
• Small Project (1-2 months)

15. Confidential 15
Test Cases
• Test Cases
• Check-Lists
• Exploratory Testing

16. Test Case/Check-List/Exploratory Testing:
Pros and Cons
• Understanding the Requirements
• Finding Risk areas
• Assurance in the tested areas
• Defining Failed version
• Measuring Performance of Test
Confidential 16
Engineers
• Time spent on support
• Pesticide Effect
• Time Saving
• “What was tested?”
• “Who tested it?”
• “How was it tested?”

17. Confidential 17
Test Cases: Don’t Write
• No process at all
• Small Team (1-2 developers only)
• Small Project (1-2 months)
• 1 Test Engineer with good experience
• At the end of a Project
• High trust of the customer to team
• Product Dynamic, Risks are not interested for a customer

18. Confidential 18
Defects
• Formalized
• Not formalized

19. Confidential 19
Defects: Pros and Cons
• Finding Risk areas
• Quality assurance
• Metrics collection (measurement
of quality)
• Defining Stable versions
• Understanding the Priorities
• Time spent on defect life cycle
support
• Time Saving
• “Why don’t we know about this
defect?”
• “Was it fixed?”

20. Confidential 20
Defects: Don’t Track
• No process at all
• Small Team (1-2 developers only)
• Small Project (1-2 months)
• Product Dynamic, Risks are not interested for a
customer

21. Confidential 21
Report/Metrics
• Daily Standup Meetings
• Weekly Reports
• Sprint Reports
• …
• Pass Rate
• TC/TS Dynamics
• Defect Density
• Defect Backlog Statistic
• Defect Containment Statistic
• Weekly PR Dynamics
• Code Coverage
• Review Metrics
• …

22. Report/Metrics: Pros and Cons
Confidential 22
• Risk Management
• Quality Assurance
• Project Dynamic
• Problems can be Prevented
• No critical “surprises”
• Time spent on support
• Time Saving
• “I think that our product is
good”
• “We have problems …”
• “All right”

23. Confidential 23
Metrics: Don’t Track
• No process at all
• Small Team (1-2 developers only)
• Small Project (1-2 months)
• Product Dynamic, Risks are not interested for a
customer/Managers

24. Confidential 24
Time Saving

25. Confidential 25
Literature
• IEEE Std 829-1998, the "IEEE Standard for Software Test
Documentation.“
• www.google.ru

26. Confidential 26
Questions

27. Confidential 27
Thank you
Maria Teryokhina
QA Lead
Nizhniy Novgorod
Skype: mariateryokhina
maria.teryokhina@returnonintelligence.com