Frank's Learning Summary - Evolution of agile testing
1. Evolution of Agile Testing
(A tester's view)
By Erik Petersen
Frank Lee’s Learning Summary
2. Methodologists meet in the snow
Mid February 2001, 17 people met in the
Utah mountains over a weekend
They represented styles of lightweight
software development
4. Methodologists meet in the snow
Only one, Brian Marick, had major
interest in testing (a quiet person)
Another tester invitee, James Bach, could
not attend (a loud person)
5. Where's testing?
Neither the agile manifesto or supporting
principles mention testing, focussing on
”sponsor, developer, user”
Principles do mention ”architecture,
requirements & designs” but no testing or
validation activity mentioned explicitly
6. Agile testing V1.0
Strong Extreme Programming slant
Developers do all tests, using TDD for design
& automated unit tests, Junit mostly
No need for system tester role, ”all tests
automated”
Developers to learn extra testing skills from
testers then replace them
Kent Beck tells conference of testers they
will all be out of jobs in a few years
8. A week of Ruby dev, test & talk
Mid June 2003, a group of agile devs and
exploratory testers spend a week together
talking and creating software at ”Agile
Fusion” (Erik watches via webcam!)
Early version of Ruby seen as immature by
devs (but great language for testing)
Some agile devs understand importance of
exploratory testing, e.g Ward Cunningham
9. Agile Testing manifesto?
”It’s very important to understand that we in
this Agile Fusion conference did not come to
agreement on *any* specifically phrased idea
about what agile testing is or isn’t. We tried to
discuss it, but it quickly became apparent that
there are important philosophical &
terminological differences among us.”
James Bach
10. Agile testing V2.0
Strong XP Programming slant weakening
Regression testing done by developers, using
automated unit tests, now Xunit
Some appreciation for system tester role, e,g
Thoughtworks, Mary Poppendieck
Some developers learn more about testing
and importance of exploratory testing
Agile testing champions, e.g Elisabeth
Hendrickson
11. Enter the test toolsmith
Developer explicitly supporting less technical
testers, or more busy devs
Creating tools to support testability
Automating data creation, configuration
table setup, security profiles, UI information
fields only visible in test mode or hidden in
html, etc
Focussed on manual & automated tests
Testability champion, Brett Pettichord
12. Test tools rule
Rapid expansion of free functional tools
Framework for Integrated Test (FIT)
2002, java based implementation of
customer examples as tests, lead dev
Ward Cunningham
Selenium 2004, web application driver
using javascript, lead dev Jason Huggins
Many other less popular tools
13. Dependency Injection
• This simplifies the design, and makes it more
flexible allowing different types of materials to
be provided in different cases.
• Another advantage is the ability to provide
fake materials, when we are only interested in
testing the function to check it still works
properly.
14. Agile testing V3.0 Flavors
Regression testing by developers, using TDD
& automated unit tests, with dependency
injection & testability
Testsmiths create custom tools for their
teams
Functional automation by developers or
technical testers using tools like Selenium
Customer examples linked by dev tester to
fixtures create simple tests, like FIT
15. ET in Agile Development
Jim Shore & Chromatic write ”Art of Agile
Development”, 2007 with community review
of each chapter
Lone voice of tester reviewer in dev cloud
(Erik), pushing tester role & viewpoint
Book expanded to include ET
Viewpoint: ET needed to find bugs missed by devs (to
improve process to find next time!)
Included Exploratory testing chapter written by Elisabeth
Hendrickson
16. Agile testing for testers & team
Agile Testing stage at Agile 2009 conference
Exploratory testing sessions at Agile
conference, Erik Petersen 2008, 2009, Jon
Bach 2010
”Agile Testing” 2009, by Crispin & Gregory
gets great reviews across all agile tribes
Strong agile tester community
Bughunts becoming popular in industry
18. The 2nd wave of test tools
Cucumber 2009, english-like fixture based
tool dominating functional test automation,
lead developer Aslak Hellesoy
Selenium and Selenium Grid almost industry
standard
Domain Specific Languages (e.g in Groovy),
Robot framework, etc
AA functional test tools group discussing and
leveraging tools, also GTAC (Google)
19. Key (system) testing skills
Test techniques: e.g. boundary analysis,
equivalence partitioning, test intuition
Test modelling: domain, application,
navigation, authorisation, etc.
Test case design and coverage
Defect management
Failure modelling (often innate skill)
Shadow architect & designer/ devils
advocate
20. Key skill: Exploratory testing
ET Framework: Three main parts
Investigation
Wide across app, or deep to feature
Exploration
Confirming app & other models, or trying
failure models
Reflection
Maintain focus or change to next priority
Investigate bug clusters
Attempt to make implicit skill visible
21. Agile testing today
Still no agile testing manifesto, just many
flavors (like with agile)
New tools can be adopted quickly
worldwide, e.g Cucumber
Many great developer testers but still
specialist tester role needed
Risk based exploratory testing preferred
technique for rapid testing
TDD ”checks” versus human ET ”tests”
22. Agile Tester
敏捷测试人员首先是专业化的测试者,适应变
化,与开发人员和业务人员开展良好的合作,贯彻
利用测试记录需求和驱动开发的精神,具有优秀的
技术能力,懂得合作以实现测试自动化,更擅长探
索性测试。了解客户在做什么,透彻的理解客户对
软件的需求。
– by Lisa Crispin