A practical guide to implementing Agile QA practices for Agile teams. It tries to answer the questions most teams ask when implementing QA practices in an Agile environment. If you would like, I will be happy to do this talk for your local Agile user group or at your company. Contact me by email srayhan@code71.com
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Practical Agile QA
1. Agile QA
A practical guide to implementing
Agile QA process on Agile Projects
Syed Rayhan
Co-founder, Code71, Inc.
Contact: srayhan@code71.com
Blog: http://blog.syedrayhan.com
Company: http://www.code71.com
Product: http://www.scrumpad.com
2. My Background
Co-founder, Code71, Inc.
Career
14+ years of total experience
Co-author of “Enterprise Java with UML”
Iterative incremental development
Ex pertise Technology planning and architecture
On-shore/Off-shore software development using
Agile/Scrum
Cultural aspect of self-organizing team
Scrum for projects delivered remotely
Int erest s
Agile engineering practices
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
2
3. Agenda
Section 1 Introduction
Section 2 Holistic View of QA
Section 3 Individual Practices
Section 4 A Case Study
Section 5 Recap
Section 6 Q&A
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
3
4. What to Expect
Teams and organizations are adopting Agile/Scrum
Contex t
Teams struggle with making the transition from waterfall to
Agile/Scrum
Build common base of understanding
Focus Develop a set of guidelines- process, roles, and team
composition
Address typical questions asked
How to perform QA on an Agile/Scrum project
K ey
T akeaways
Agile/QA best practices
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
4
5. Agenda
Section 1 Introduction
Section 2 Holistic View of QA
Section 3 Individual Practices
Section 4 A Case Study
Section 5 Recap
Section 6 Q&A
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
5
6. The challenges?
Is QA part of the development team?
Can we fit QA in the same iteration as development?
Who does QA?
Does QA costs more in Agile as product seems to change from
sprint to sprint?
How can we scale Agile QA?
Do we need “test plan”?
Who defines test cases?
Are story acceptance tests enough?
When do we know testing is done?
Do we need to track bugs?
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
6
7. What is QA (Quality Assurance)?
To ensure Software is working right
How?
Test, Test, Test
We will primarily focus on single-team model for our discussion
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
7
8. Types of Testing? White
Box
Unit Testing Regression Testing
Black
Box
Integration Testing Acceptance Testing
Load Testing
Functional Testing
System Testing Smoke Testing
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
8
9. Who Performs What?
What? Who? When? Automation?
Unit Testing Developer Coding Always
Integration Testing Developer Coding Always
System Testing Tester Test Possible
Regression Testing Developer/Tester Build/Test Possible
Acceptance Testing Client/Users Deployment/ Possible
Delivery
Smoke Testing Tester/Support Engineer Deployment Possible
Load Testing Performance Engineer Deployment Always
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
9
10. Right tools for right tests?
Test Tool
Unit Testing NUnit, JUnit, Mock, DBUnit
Integration Testing Unit test tools, HttpUnit, SoapUI,
RESTClient
System Testing Selenium, Fit, WET, Watir, WatiN
Regression Testing Unit test tools, System test tools
Acceptance Testing FIT, FitNesse
Smoke Testing Regression test tools
Load Testing JMeter, Httperf
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
10
11. What is missing?
1. Right Spec
Assumptions 2. Right Design
3. Right amount of Tests
4. Right Tests
Reality? Assumptions are farther from truth
Spec Spec review
Measures Design Design review
Code Code review
Test Test Coverage
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
11
12. Team Composition?
region iteration Out-of-Cycle
1 2 3 Separate Team
Dev
1
Test 1 2 3
Prod 1 2 …
region iteration
Dev 1 2 3
In-Cycle
2 Test 1 2 3 Integrated Team
Prod 1 2 …
Developer to tester ratio?
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
12
13. Quality Funnel
Unit & Int.
Backlog Design Code System
Test,
Review Review Review Test
CI
bugs
QG#1 QG#2 QG#3 QG#4 QG#5
bugs
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
13
14. Agenda
Section 1 Introduction
Section 2 Holistic View of QA
Section 3 Individual Practices
Section 4 A Case Study
Section 5 Recap
Section 6 Q&A
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
14
15. Test Coverage
“A measure of the proportion of a program
Definition exercised by a test suite, usually expressed as
a percentage.”
Measure Usually expressed as a percentage
• Function coverage
Types of coverage • Path coverage
• Statement coverage
Tests coverage metrics can tell you what code is not tested
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
15
16. Continuous Integration as the Glue
Report
Monitor Automated Regression Test
Source Code Build Test Coverage
Continuous Integration is Continuous QA
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
16
17. Test Case Prioritization
Frequency of use
high
normal 3 1 hot
Risk of having bugs
high
low
cold 4 2 warm
low
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
17
18. “Inspect and Adapt” through QA Lens
Discover Log bugs found by testers
Triage & fix Prioritize bugs over stories
Prevent Five “whys” of root cause analysis
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
18
19. Definition of “Done”
Is QA part of your definition of “Done?”
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
19
20. Tracking Quality
Quality Metrics
Defect Rate Bug count per iteration
Defect Density Bug count per module
Bug count per function point
What else should we track?
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
20
21. Technical Debt & QA
“Aged” open bugs can contribute to increasing “technical debt”
“Reoccurring” bugs may indicate hidden “technical debt”
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
21
22. Agenda
Section 1 Introduction
Section 2 Holistic View of QA
Section 3 Individual Practices
Section 4 A Case Study
Section 5 Recap
Section 6 Q&A
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
22
23. A Case Study
A large enterprise system that includes technologies like
Project ASP.Net, BizTalk, Workflow, Scanning, SQL Server, Data
Warehouse, and Mainframe
2 product owners, 1 scrum master, 1 architect, 5 developers, 1
Team QA tester
Sprint 2 weeks
Day 1, 2 Day 3, 4 Day 5-8 Day 9 Day 10
QA Process Refine Identify UI Write test scripts Final Demo
scope, elements, test data Test &
acceptance test data & Acceptanc
test cases QA schedule Fix e Test
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
23
24. Agenda
Section 1 Introduction
Section 2 Holistic View of QA
Section 3 Individual Practices
Section 4 A Case Study
Section 5 Recap
Section 6 Q&A
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
24
25. Recap
“In-cycle QA” or “Integrated Team” is critical to the success of a project
System testing is not the only “quality gate,” it includes all types of testing and
reviews
Test automation is critical to “in-cycle” QA
Target at least 90% test coverage
“Continuous Integration” is “Continuous QA”
Prioritize test cases based on risk and frequency of usage
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
25
26. Recap contd.
All known bugs should be fixed first
Right size story with well-thought out acceptance tests improves quality
Include all “Quality Gates” as part of definition of “Done”
Analyze each bug to understand where (Quality Gate) it should have been
caught and improve (Inspect and adapt)
QA is not a designated person’s responsibility, it is a team’s responsibility
(self-organizing team)
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
26
27. Q&A
“QA is making sure right software works right”
“QA is not an act, but a habit”
Please contact for on-site training or
Webinar:
Contact: srayhan@code71.com
Blog: http://blog.syedrayhan.com
Company: http://www.code71.com
Product: http://www.scrumpad.com
Copyright 2010, Code71, Inc.
www.Code71.com www.ScrumPad.com
27
Editor's Notes
Teams either skip QA, or Teams continue to do QA as they did on waterfall projects. Impedance mismatch results. We end up being not so Agile and QA becomes a sore point in the whole process.
Too many changes, not enough time By the end of the presentation, hopefully we all will be able to answer these questions.
“… planned and systematic production processes that provide confidence in a product's suitability for its intended purpose. It is a set of activities intended to ensure that products (goods and/or services) satisfy customer requirements in a systematic, reliable fashion.”
ACCEPTANCE TESTING. Testing to verify a product meets customer specified requirements. A customer usually does this type of testing on a product that is developed externally. FUNCTIONAL TESTING. Validating an application or Web site conforms to its specifications and correctly performs all its required functions. INTEGRATION TESTING. Testing in which modules are combined and tested as a group. LOAD TESTING. Load testing is a generic term covering Performance Testing and Stress Testing to verify that the system meets the scalability requirements. REGRESSION TESTING. Similar in scope to a functional test, a regression test allows a consistent, repeatable validation of each new release of a product or Web site. Such testing ensures reported product defects have been corrected for each new release and that no new quality problems were introduced in the maintenance process. Though regression testing can be performed manually an automated test suite is often used to reduce the time and resources needed to perform the required testing. SMOKE TESTING. A quick-and-dirty test that the major functions of a piece of software work without bothering with finer details. UNIT TESTING. tests for the behavior of components of a product to ensure their correct behavior prior to system integration. SYSTEM TESTING. Testing conducted on a complete, integrated system to evaluate the system's compliance with its specified requirements.
Who is using these tests? Who is automating which tests?
Who is using these tests? Who is automating which tests? Issues with commercial test tools- proprietary scripting, closed platform, vendor lock-in
What is issue with this approach? Technical Debt- developers’ focus change
This will typically involve collecting information about which parts of a program are actually executed when running the test suite in order to identify which branches of conditional statements which have been taken
“ Continuous Integration is a software development practice where members of a team integrate their work frequently , usually each person integrates at least daily - leading to multiple integrations per day . Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible ” - Martin Fowler
Aviation control system safety is of critical importance and hence you would test all and more Social Networking Software not as much, in fact, you would only focus on 1, 2 Financial transactions need to be tested more than other features Security features should be tested more thoroughly than all other features Administrative features probably more than user specific features
When a team says they are done with a story, what is the expectation? Is it ready for integration with other parts of the system? Is it ready for production?
QA was out numbered by the developers. We needed to add more QA persons. Best is 1:1. 1:2 also would work depending on the skill of the tester and the complexity of the domain.