Learn how agile and test automation interact in the real world
To be presented at Aqua SW Test Automation Open Day at Hi-tech College (http://www.aquasw.com/services_training.php#an2)
4. Automation Before Aqua
WinRunner
Telnet
CLI
SSH
VS (Hardware)
JavaSwing GUI
StoragePro
SNMP
NMS
Client API
Windows Load
Command Line
Generator
Linux Load Command Line
Generator
5. What was broken
Bottom line
Late Delivery (6months late for a 9 months version)
Low customer satisfaction – Quality, Business Value
Root causes
Big bang integration – late discovery of issues
Scope fixed – schedule/quality paid the price.
Wasted work on detailed specs/architecture
QA Focus on manual scripted testing – got to advanced testing
late - big problems found in “release candidates”
Technical Debt slowed us down
12. Why Agile?
Early feedback/visibility - enable inspect and adapt
Smaller more manageable versions
Fix schedule + quality – manage scope
Better and earlier interaction between QA/DEV
teams/PM – Faster convergence on feature contents,
quality, usability
Leverage power/knowledge of the team
FASTER TIME TO MARKET
HIGHER CUSTOMER SATISFACTION
14. Our experience (Testing Aspects)
High points
Early tester involvement key to success
Much higher quality earlier
Much easier to successfully predict length of
hardening/testing period
Relatively easy to change functionality due to
external changes, QA input, etc.
Very stable Continuous Integration/Build
environment.
But…
15. Our experience (Testing Aspects)
But
Only partial test automation of main regression aspects
or new functionality
Testers late to the party from time to time (SPs, earlier
releases, etc.)
Swing GUI automation – very hard to achieve.
Technically works, pragmatically very slow development,
not worth it
So…
16. Our experience (Testing Aspects)
So…
Couldn’t safely refactor/rewrite
• hard to deal with Technical Debt
• velocity still limited
lack of full automation of main flows
Still surprises quite late in the game
Still hard to release frequently
17.
18. Lessons Learned
Agility requires comprehensive
hands-off acceptance test automation
Automated test automation ;-)
Test automate via APIs, even at the
cost of significantly reduced coverage
Start the project together, don’t let
DEV get too far ahead
23. Agile Testing
Cost per cycle should be minimal Automate Automate Automate!!!
Unit / Integration / Acceptance / Regression
Lean test documentation –
Consider Exploratory Testing where applicable.
Focus on outline, elaborate details verbally as you go, or trust testers to do
the right thing.
QA Measurements should drive Agile Testing
Minimize freeze/stabilization time
Aim for shorter length, and consider it a maturity sign
How?
• Maximize quality delivered for regular sprints
• Maximize efficiency of manual test work
• Did we mention Automate?
24. Test-Driven Development/Design
TDD is a method of designing software, not
merely an approach to testing.
Over a period of time, as a side effect, TDD
will lead to suite of automated unit tests.
25. Acceptance Test-Driven Development
Functional Implement
Functionality
Requirement
Execute
Acceptance Test
Draft Cases
Acceptance
Test Cases
Review
Results Verify Results
Accepted
Modify Acceptance
Test Cases
26
26. What do I do tomorrow?
Do you already have a significant part of your Sanity,
Regression, and new functionality automated?
Automate your nightly build – Create Continuous
Integration
Start the journey to Agility
Learn more about Agile/Scrum/Lean/XP.
Agile can jump-start your R&D engine,
• ESPECIALLY when includes effective test
automation and other engineering practices.
Inspect & Adapt – Learning Organization
27. For agilists without wings…
Are you “agile” without test automation?
You probably already know something’s
missing…
Choose an automation solution/partner
that will make test automation
Practical
Maintainable
Approachable for Testers, Developers
Scalable
28. Test Automation as a classic Agile Project
Backlog
Business Value ==
Coverage Priority and %
PO
QA Manager
Team
Core team focused on
automation
Short Sprints
Deploy to CI frequently
Adjust priorities
Adopt in delivery teams
Inspect and Adapt
29. Ask us how!
info@agilesparks.com
yuval@agilesparks.com
Get the slides at http://www.slideshare.net/yyeret/