Slides from a session presented by Fadi Stephan from Kaizenko at the 2019 Global Scrum Gathering in Austin, TX on 05/20/2019 DC. Also see the blog series on Agile Testing at https://www.kaizenko.com/agile-testing/
Abstract:
Many teams struggle with fitting in testing activities inside of a Sprint. They end up doing primarily development activities in a Sprint and push testing activities to run in dedicated testing Sprints following the coding Sprints or have a coding and testing Sprint running in parallel. However, in Scrum, the output of every Sprint is a potentially shippable product increment. This means the product increment should be well tested within the Sprint and ready to be delivered. Come to this presentation to learn how to tackle testing on an Agile team, what kind of tests to execute, what to automate and what not to automate, the different test responsibilities, and when to run which tests. Leave with a testing strategy that you can start applying the next day to gradually get a team to start testing from day 1 of the Sprint and deliver a true product increment at the end of each Sprint.
2. While we wait for the
session to start, meet your
neighbors and discuss …
AGILE TESTING @fadistephan | kaizenko.com
• Do you currently work on an Agile team?
• Does your team include testers?
• What testing challenges are you facing?
3. Technology Consultant and Certified Scrum Trainer
20 Years of Experience in Software Development
Co-organizer of the DC Scrum User Group (DCSUG)
Skier and Diver
www.kaizenko.com @KaizenkoLLC
@FadiStephan
FADI STEPHAN
AGILE TESTING @fadistephan | kaizenko.com
6. AGILE TESTING @fadistephan | kaizenko.com
Code 1 Code 2
Test 1 Test 2
2 Weeks 2 Weeks 2 Weeks 2 Weeks
Code 3 Code 4
Test 3
7. AGILE TESTING @fadistephan | kaizenko.com
2 Weeks
Designed
Coded
And
Tested Story 1 Story 1
Story 3
Story 4
Story 6
Story 2
Story 1
Story 2
Story 3
Story 5
Story
Story 4
Story 1
Story 2
Story 3
Story 5
Story 4
Story 1
Story 2
Story 3
Story 5
Story 6
Story 7
Day 1 Day 3 Day 6 Day 7 Day 10
11. “Our highest priority is to satisfy the
customer through early and continuous
delivery of valuable software.”
“Deliver working software frequently,
from a couple of weeks to a couple
of months, with a preference to the
shorter timescale.”
- First and third of the Twelve Principles behind the Agile Manifesto
A LEANER PMO IN THE FEDERAL GOVERNMENTAGILE TESTING @fadistephan | kaizenko.com
15. Brainstorm with your
neighbors …
AGILE TESTING @fadistephan | kaizenko.com
What are all the types of testing that
you currently do or should be doing?
16. Functional Tests
Acceptance Tests
Unit Tests
Integration Test
Component Tests
System Tests
Exploratory Tests
Usability Tests
Performance Tests
Load Tests
Stress Tests
Security Tests
Types of Testing
AGILE TESTING @fadistephan | kaizenko.com
17. Functional Tests
Acceptance Tests
Unit Tests
Integration Tests
Component Tests
System Tests
Exploratory Tests
Usability Tests
Performance Tests
Load Tests
Stress Tests
Security Tests
SupportProgramming
CritiqueProduct
Business Facing
Technology Facingfrom Brian Marick
Testing Quadrant
AGILE TESTING @fadistephan | kaizenko.com
18. WHICH TESTS DO WE
AUTOMATE?
AGILE TESTING @fadistephan | kaizenko.com
23. Testing Pyramid
AGILE TESTING @fadistephan | kaizenko.com
Unit Tests
Integration/System
Tests
Functional
& AT Tests
UI
Tests
Manual
24. WHEN DO WE START
TESTING?
AGILE TESTING @fadistephan | kaizenko.com
25. Task Board
AGILE TESTING @fadistephan | kaizenko.com
To Do In Progress Testing Done
User
Story
User
Story
User
Story
Task Task Task
Task Task
Task Task
Task Task
26. TESTING IS NOT A
PHASE
AGILE TESTING @fadistephan | kaizenko.com
27. Task Board
AGILE TESTING @fadistephan | kaizenko.com
To Do In Progress Done
User
Story
User
Story
User
Story
Task Task Task
Task Task
Task Task
Task Task
28. WHO IS DOING THE
TESTING?
AGILE TESTING @fadistephan | kaizenko.com
34. Functional Tests
Acceptance Tests
Unit Tests
Integration Tests
Component Tests
System Tests
Exploratory Tests
Usability Tests
Performance Tests
Load Tests
Stress Tests
Security Tests
SupportProgramming
CritiqueProduct
Business Facing
Technology Facingfrom Brian Marick
Testing Quadrant
AGILE TESTING @fadistephan | kaizenko.com
35. DO NOT DO
FUNCTIONAL TESTING
VIA THE UI
AGILE TESTING @fadistephan | kaizenko.com
36. AGILE TESTING @fadistephan | kaizenko.com
As a customer I want to see a running total of
all items in a shopping cart So that I can stay
within my budget.
Running Total
37. AGILE TESTING @fadistephan | kaizenko.com
Business Layer
Web Layer
Browser
Data Layer
Code
Code
Code
Code
Database
Code
Code
Code
Code
38. Functional Tests
Acceptance Tests
Unit Tests
Integration Tests
Component Tests
System Tests
Exploratory Tests
Usability Tests
Performance Tests
Load Tests
Stress Tests
Security Tests
SupportProgramming
CritiqueProduct
Business Facing
Technology Facingfrom Brian Marick
Testing Quadrant
AGILE TESTING @fadistephan | kaizenko.com
40. AGILE TESTING @fadistephan | kaizenko.com
As a customer I want to collect books in a shopping
cart So that I can order several books at once.
• Books can be added to the shopping basket
• Books can be removed from the shopping basket
• Shopping basket is initially empty
• The same book can be added multiple times to
the shopping basket
User Story
41. AGILE TESTING @fadistephan | kaizenko.com
• Requirements and tests are just two sides of
the same coin
– You can’t have a user story (requirement) without
acceptance criteria (test)
• Agile testing focuses on melding requirements
and tests into executable specifications
Executable Specifications
42. AGILE TESTING @fadistephan | kaizenko.com
As a customer I want to collect books in a shopping
cart So that I can order several books at once.
• Books can be added to the shopping basket
– Given my shopping basket is empty
– When I add the book “Harry Potter” to my shopping
basket
– Then my shopping basket should contain 1 copy of
“Harry Potter”
Specification by Example
43. AGILE TESTING @fadistephan | kaizenko.com
As a potential customer I want to collect books in a
shopping cart So that I can order several books at once.
• The same book can be added multiple times to the
shopping basket
– Given my shopping basket contains 1 copy of “Harry
Potter”
– When I add the book “Harry Potter” to my shopping
basket
– Then my shopping basket should contain 2 copies of
“Harry Potter”
Specification by Example
44. AGILE TESTING @fadistephan | kaizenko.com
Green Documents
• Use Specification by Example to create “green documents”
– Less Likely to rot
– Not write once documents
– Help people think and collaborate
– Communicate what the product really does
– Become automated acceptance tests
– Become part of regression test suite
45. AGILE TESTING @fadistephan | kaizenko.com
PASS
REFACTOR CODE
FAIL
FAIL
PASS
Automated Acceptance Test Automated Unit Test
User Story
Acceptance Criteria
Back
Acceptance Test Driven
Development (ATDD)
46. Functional Tests
Acceptance Tests
Unit Tests
Integration Tests
Component Tests
System Tests
Exploratory Tests
Usability Tests
Performance Tests
Load Tests
Stress Tests
Security Tests
SupportProgramming
CritiqueProduct
Business Facing
Technology Facingfrom Brian Marick
Testing Quadrant
AGILE TESTING @fadistephan | kaizenko.com
59. AGILE TESTING @fadistephan | kaizenko.com
Exploratory Testing
• A sophisticated, thoughtful approach to
testing without a script
• Enables going beyond the obvious variations
that have already been tested
• Combines learning, test design and test
execution into one test approach
• Applies heuristics and techniques in a
disciplined way
60. AGILE TESTING @fadistephan | kaizenko.com
Session Based Testing
• Time-boxed and structured
• Mission/charter based
• Look for patterns that do not look right
• Divert from the mission if necessary
• Report on your results
61. Functional Tests
Acceptance Tests
Unit Tests
Integration Tests
Component Tests
System Tests
Exploratory Tests
Usability Tests
Performance Tests
Load Tests
Stress Tests
Security Tests
SupportProgramming
CritiqueProduct
Business Facing
Technology Facingfrom Brian Marick
Testing Quadrant
AGILE TESTING @fadistephan | kaizenko.com
62. SUMMARY
1. Testing in not a phase
2. Testing is everyone’s responsibility
3. Testing starts on day 1
4. Do automate all scripted tests
5. Developers are the one’s doing most of the
automation
6. Do use executable documentation
7. Do not automate functional testing via the UI
8. Save most manual testing for usability and
exploratory testing
AGILE TESTING @fadistephan | kaizenko.com
64. Technology Consultant and Certified Scrum Trainer
20 Years of Experience in Software Development
Co-organizer of the DC Scrum User Group (DCSUG)
Skier and Diver
www.kaizenko.com @KaizenkoLLC
@FadiStephan
FADI STEPHAN
AGILE TESTING @fadistephan | kaizenko.com
Editor's Notes
TESTING IS NOT A PHASE
TESTING IS EVERYONE’S RESPONSIBILITY
TESTING STARTS ON DAY 1
DO AUTOMATE ALL SCRIPTED/FUNCTIONAL TESTS
DO NOT RUN AUTOMATED FUNCTIONAL TESTS VIA THE UI
DO MOST MANUAL TESTING IS FOR USABILITY and EXPLORATORY TESTING
TEST THAT THE CODE WORKS AS INTENEDED
TEST THAT THE PRODUCT WORKS AS EXPECTED