2. AGENDA
< 2
>
• The Essence
• Context for Agile Testing
• Testing in Agile Teams
• Organizational Challenges
• Keys for Success
3. Definition
Glen Myers
Testing is the process of executing a program with the intent
of finding errors
Objective explained
Paul Jorgensen
Testing is obviously concerned with errors, faults, failures and
incidents. A test is the act of exercising software with test
cases with an objective of
• Finding failure
• Demonstrate correct execution
6. AGILE INSIGHTS
< 6
>
Agile: iterative and
incremental
- Each story is expanded,
coded and tested
- Possible release after
each iteration
Context :- Agile Approach
7. Cost
of
Change
Time 7
Iteration 1
Requirements
Analysis & Design
Code
Test
Iteration 2
Requirements
Analysis & Design
Code
Test
Iteration 3
Requirements
Analysis & Design
Code
Test
Iteration 4
Requirements
Analysis & Design
Code
Test
Deploy
Context :- Agile Testing Approach
8. CONTEXT : AGILE TESTING SUCCESS FACTORS
•Be cathedral builders not stone cutters
•Collective ownership
Testers are part
of the team
•Drop the “Quality Police” mindset
•Focus on team goals & customer value
Agile testing
mindset
•Automate tests wherever practical
•Need rapid feedbackAutomate tests
•Balance against developer focus on
technical implementation
•Use agile test matrix as guide
Look at the big
picture
< 8
>
Source: http://www.agiletester.ca/
9. •Session-based testing, Agile test
environments
•Informative workspace
Foundation of
critical practices
•Collaborate with customers
•Collaborate within teamCollaborate
•Team retrospectives
•Personal training: reading, blogs, QAI, local
QA groups
Continually
improve
< 9
>
Source: http://www.agiletester.ca/
Context : Agile Testing Success Factors
10. < 10
>
Context :- Quarry
A quarry owner was walking through a quarry
and came across a worker and asked her
what she was doing. She said I am cutting
rocks.
She walked on came across a second worker
and asked her the same question. The
response was “I am supporting my family by
working here cutting rocks”. The owner was
impressed because it was more than just a
job; there was a deep important purpose to
the worker.
Finally, the owner came upon a third worker.
The third worker said that they were building a
cathedral.
We need to think of ourselves as cathedral
builders.
11. < 11
>
Context :- Traditional Engineering
Engineering is a mix of science, craft and
experiment
- experiments provide critical feedback
Engineering after 2nd world war heavily
shifted towards science at the expense of
craft and feedback.
Agile and lean are really a shift back to
traditional engineering
12. ›Testing in Agile Testing
• Agile Testing – Early Feedback
• Key Agile Testing practices
• Why Automate Tests
• Adaptations to handle these
• Outcome assessment
13. Collaborate with
Test early
Reduce # of open Focus on buggy or
defects critical areas
Testers‘ objective is to enable timely release
of quality software. They find defects and
verify the fixes as part of this objective
developers Automate
(reproducing alongside during
defects, bug Sprints
verification etc.)
Sensitise
developers
towards planned
tests
›Tester Role – In Agile
15. Testers are first class citizens on agile teams
and part of the “whole team” supporting
customers, business stakeholders, developers
and other team members
Testers support quality infusion through entire
team and product cycle
Test tasks and stories are planned and
executed like development tasks and stories
Automate where possible and use session-
based testing for exploratory testing
Communicate through information radiators
< 15
>
Key Agile Testing Practices
16. Agile Practice Benefits
Whole Team
• Quality is not just a tester responsibility
• Quality is more than just testing
• Testing role shifts to quality infusion
throughout project life cycle
Continuous
Integration
• Developers cannot check in code with
failing tests
Continuous Testing • Avoids long delays with “big-bang”
testing after the “final build”
• Bugs found closer to when they are
introduced making them easier to fix
16
›Key Agile Testing Practices
17. Continuous Integration and Automated test
Nightly builds followed by automated tests
Code to verification cycle will be less than a day.
Automated suite should grow with every Sprint.
Focus on Critical functional areas for automated System test
Image: http://www.agilenutshell.com/assets/continuous-integration/continuous-integration.png
›Key Agile Testing Practices
18. Test throughout the Sprint duration, not just at the end
The user stories are to be tested throughout the Sprint, not just when they are
completed (mini-waterfall) – and prevent clogging at the end.
›Key Agile Testing Practices
19. TEST AUTOMATION – KEY BENEFITS
Provides safety net
Supports rapid iterations
Provides footholds to keep notching upward
Provides rapid feedback
Focuses effort on what is valuable
Frees people to do their best work
< 19
>
Why Automate Tests?
Need to balance automation costs against delivered value
20. MIKE COHN’S TESTING PYRAMID
Why Automate Tests?
< 20
>
GUI
Acceptance
Tests
FitNesse
Unit Tests
• Small number
• Automate many
• Find the right ones
• Largest numbers
• Foster Test Driven Design
Start Stop Look
21. BREAK THE MANUAL TESTING PARADIGM
Why Automate Tests?
< 21
>
Manual GUI
Acceptance
Tests
Unit
Tests
Automated
GUI Tests
•Easy to Create
•Very familiar – what we always do
•Typically tedious
•How do we know coverage?
•Need Automation specialists
•Automation good for performance
•Seems like we always rewrite
•Sometimes fragile
•What is Dev testing?
•How do we know what these are?
•How do we know when they fail?
Start Stop Look
22. “You can never have too many manual
acceptance tests”
Manual tests are cute little bunnies, before you know it you have
hundreds or thousands in your regression suite
You inadvertently dig a hole you can never get out of
Whole team had to help run regression suite
Defect count typically is high
Most defects were found as manual tests were elaborated
Regression tests typically didn’t find many defects
Commonly found defects – things we didn’t think of
Adaptations to handle these
23. BETTER, BUT NOT PERFECT TESTING ARCHITECTURE
< 23
>
Unit Tests
Manual GUI
Acceptance
Tests
Automated
GUI Tests
& FitNesse
• Still too many here
• Add FitNesse
• Increase Coverage
• Increase Capability
Start Stop Look
Adaptations to handle these
24. TESTING TYPES AND SCHEDULING
Acceptance –
GUI?
•Minimize Manual
•Generate off of Use
Cases to get scenario
tests.
•During the Iteration
Acceptance -
Functional
•Combination of Unit
tests, FitNesse
•Build Verification and Run
Nightly
Load &
Performance
•Profiling and
Simulation
Automation
•Do it periodically
•Don’t wait till the end of
the Release cycle
Regression •Acceptance and
Functional tests from
previous Iterations
•Run Nightly
Exploratory •Manual Group
Explore
•Roles and Personas
•Before Releasing
Adaptations to handle these
25. UI Tests Through UI
Selenium
QC & QTP
Test Complete
Functional
Tests
In business
language
Fit/FitNesse
Unit Tests
In same
language as
system
xUnit
< 25
>
Types of Automated Tests
c
o
s
t
26. < 26
>
• Continuous Builds assures better quality
• > 50% of Acceptance Test Automated
• 90-95% Unit Test Case Coverage
• Feature Based Testing less defects
• Testers and Developer Co-ordination
Increased.
• Team Deliverables are high
• Early Feedback and Good Customer
Growth
• Cost of Quality reduced due early to Early
Testing.
Outcome Assessments
27. < 27
>
Organizational Challenges
• Dev as Testers and Testers as Dev – how does that
happen?
•Resistance to Change – how do we get the team
to welcome and embrace changes and not feel
threatened?
•Testers are an integral part of the team- do we
need to re-organize to make this happen?
28. KEYS TO OVERCOME THE ORGANIZATIONAL
CHALLENGES
< 28
>
›Have Dev help run manual Regression
tests
›Pair Dev and Test on Unit Testing
›Co-location of all the team
›Daily Standups
›Do Retrospectives
29. < 29
>
Summary
Agile Pulls Testing Forward
›You need to change your tools and
approaches to move it forward
›You might need to change the
model/structure of your team
›With Agile, you will create faster Release
cycles, shorter Iterations, more satisfied
customers, and team members that enjoy
what they are doing
30. Toyota plant in Cambridge near Waterloo builds 350,000 cars annually yet
carries no more than 1 day of inventory.
Kanban – there is no traditional sophisticate MRP show floor/inventory
system that “pushes” work through the plant. Instead Kanban cards are
used to “pull” new parts from upstream suppliers. The result is a highly
optimized inventory system that is responsive. When teams have nothing to
do they clean their work area or help other teams.
On agile teams our “inventory” are incomplete requirements. Or more
accurately any “unconfirmed decision” – such as a requirement,
specification, architecture, design, technical document or test case. We
want to minimize our inventory by pulling new requirements from concept
to working software within each of our “shifts” – which in agile terms is an
iteration or sprint.
At Toyota, each year the 67,000 employees submit approximately 700,000
improvement ideas. Each of these ideas saves money. Over 99% of the ideas area
implemented. The point is that it is the people closest to the work that know best
how to optimize and Toyota empowers them and rewards them for doing so.
< 30
>
Case Study – Toyota Motors
