Just few years ago most people didn’t know a thing about test automation. Today most companies either use test automation or plan to start doing it soon. However, even companies that use test automation regularly often have a long way to go in order to realize the full potential of test automation. In this talk I’ll try to outline a typical evolution of the test automation in an organization in the way to realizing its full potential.
6. QA Centric: Replace manual regressions
Dedicated Automation Team (part of QA)
Nightly
Helps find bugs
Based on manual tests of stable features
Challenges: stability and maintainability
8. Collaboration: Reduce the Feedback Cycle
Same team
Regressions are fixed on the next day
Tests are written for new features
Unit tests (CI) vs. System tests (Nightly)
10. Fusion – Shared Goals
Developers see the value
Refactoring made easy
Real collaboration
Design for testability
Faster tests (mainly unit and integration)
17. Complete Guide to Test Automation - Chapters
Part I:
1. The Value of Test Automation
2. From Manual to Automated Testing
3. People and Tools
4. Reaching Full Coverage
5. Business Processes
6. Test Automation and Architecture
7. Isolation and Test Environments
8. The Big Picture
18. Complete Guide to Test Automation - Chapters
Part II:
9. Preparing for the Tutorial
10. Designing the First Test Case
11. Start Coding the First Test
12. Completing the First Test
13. Investigating Failures
(continued on next slide)
19. Complete Guide to Test Automation - Chapters
14. Adding More Tests
15. Continuous Integration
16. Acceptance Test Driven
Development
17. Unit Tests and TDD
18. Other Types of Automated Tests
19. Where to Go from Here
20. Complete Guide to Test Automation - Chapters
Appendices:
A. Real World Examples
B. Cleanup Mechanism
C. Test Automation Essentials
D. Tips and Practices for
Programmer’s Productivity
I see many organizations. I quickly assess their level
It’s a generalization
The goal
The State of the industry
Characteristics:
Believe that it’s easy
Main Problems:
Lack of skills
Insufficient resources
Stability and maintainability vs. value
Characteristics:
Believe that it’s easy
Main Problems:
Lack of skills
Insufficient resources
Stability and maintainability vs. value
Characteristics:
Dedicated auto. dev. (part of QA)
Nightly builds
Investigation and opening bugs
Based on manual tests
Mainly stable features
Main Problems:
Hard to stabilize (struggle)
Hard to maintain
Feedback loop
Characteristics:
Dedicated auto. dev. (part of QA)
Nightly builds
Investigation and opening bugs
Based on manual tests
Mainly stable features
Main Problems:
Hard to stabilize (struggle)
Hard to maintain
Feedback loop
Characteristics:
Work in the same team (but not “as a team”)
Regressions fixed the same day (faster feedback loop)
Automation is developed 1-2 iterations after application
Mindset of DEV/QA (hard to break…)
Unit tests vs. System Tests
CI vs. Nightly
Main Problems:
Stability and maintainability, but less
Division of responsibilities (eg. vacations)
Bugs are still a fact of life
Characteristics:
Work in the same team (but not “as a team”)
Regressions fixed the same day (faster feedback loop)
Automation is developed 1-2 iterations after application
Mindset of DEV/QA (hard to break…)
Unit tests vs. System Tests
CI vs. Nightly
Main Problems:
Stability and maintainability, but less
Division of responsibilities (eg. vacations)
Bugs are still a fact of life
Characteristics:
Developers see the value
Working as a team (code reviews, etc.)
Continuum from Unit tests to system
More focus on Unit and Integration (better Testability, CI)
Refactoring
Striving towards Zero bugs
Problems:
Tests are still developed after product (towards the end of the sprint/user story).
Characteristics:
Developers see the value
Working as a team (code reviews, etc.)
Continuum from Unit tests to system
More focus on Unit and Integration (better Testability, CI)
Refactoring
Striving towards Zero bugs
Problems:
Tests are still developed after product (towards the end of the sprint/user story).
Characteristics:
ATDD
Team collaborates on defining the solution for user stories (explain)
Acceptance criteria as tests
Tests are developed before the application code
Drives the design
Tests act as documentation
Continuous Improvement
through Measuring and experimenting
Quality and Customer Mindset
Holistic Quality
Zero Defects
Characteristics:
ATDD
Team collaborates on defining the solution for user stories (explain)
Acceptance criteria as tests
Tests are developed before the application code
Drives the design
Tests act as documentation
Continuous Improvement
through Measuring and experimenting
Quality and Customer Mindset
Holistic Quality
Zero Defects
Retrospectives: Action items and Follow-up (try to measure)
YOU can drive the transformation
It can take time
Vision
Persistence
Gradually Convince your colleagues
Critical mass or Key person