Agile Testing Dilemmas

1,275 views

Published on

Based on hands-on Agile experience acquired over multiple delivery projects and Agile coaching/consulting assignments, Vatsala and Aman share common Agile Testing dilemmas, and possible solutions, tying them to the principle of "moving testing upstream".

Presented at Next Generation Testing conference in Bangalore, India (July 2014).

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,275
On SlideShare
0
From Embeds
0
Number of Embeds
164
Actions
Shares
0
Downloads
0
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide
  • Image credit: http://www.123rf.com/stock-photo/penalty.html
  • Image credit: http://www.coloring-pictures.net/drawings/MomumentsItaly/Leaning-Tower-of-Pisa.php
  • Image credit: http://sweetclipart.com/magnifying-glass-silhouette-877
  • Image credit: http://www.clker.com/clipart-164180.html
  • Agile Testing Dilemmas

    1. 1. Breaking Free of Agile Testing Dilemmas! Vatsala Singh Aman King
    2. 2. What’s a dilemma?
    3. 3. di·lem·ma noun də-ˈle-mə a situation in which you have to make a difficult choice http://www.m-w.com/dictionary/dilemma
    4. 4. What makes Agile Testing a dilemma?
    5. 5. Few Reasons “Continuous delivery of valuable software” “Welcome changing requirements” “Face-to-face conversation” “Working software is the primary measure of progress”
    6. 6. Agile Manifesto February 2001, Utah, USA http://agilemanifesto.org/principles.html
    7. 7. Lean Muda (無駄) 7 wastes “Whenever defects occur, extra costs are incurred… should not be passed on to the consumer and should be taken as a loss.” 2001 2003 http://en.wikipedia.org/wiki/The_Toyota_Way http://en.wikipedia.org/wiki/Lean_software_development
    8. 8. “Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.” W. Edwards Deming (1900 – 1993) https://www.deming.org/theman/theories/fourteenpoints
    9. 9. Cost of Change: In requirement phase x1 in analysis x10 in design x100 and so on 1981 http://www.academia.edu/1554649/Software_Engineering_Economics_by_Barry_W._Boehm
    10. 10. Dilemma! In essence… testing a developed product is already too late?!
    11. 11. 0 20 40 60 80 100 120 Requirements Design & Implementation Testing Cost of Defect Testing Upstream Move testing upstream!
    12. 12. Analyst Business Developer QA Requirements Testing Design & Implementation Testing $$$$$$ $$$$$$
    13. 13. Testing Upstream Testing Testing Analyst Business Developer QA Requirements Testing Testing Design & Implementation $$$ $$$$$$ $$$$$$ $$$$$$
    14. 14. Iterations Analyst Business Developer QA Requirements Design & Implementation Testing Testing Testing Testing $ $$ $$$$
    15. 15. Iterations Analyst Business Developer QA Requirements Design & Implementation Testing Testing Testing Testing How? How? How?
    16. 16. Business Facing Technology Facing
    17. 17. Business Facing Technology Facing SupportProgramming CritiqueProduct
    18. 18. Agile Testing Quadrants http://www.exampler.com/old-blog/2003/08/21/#agile-testing-project-1 Brian Marick
    19. 19. Quadrant Dilemmas •Automation? •Manual? What •Requirements? •Design & Implementation? •Testing? When •Business Analysts? •Developers? •QAs? Who
    20. 20. Agile Testing Quadrants Business Facing Technology Facing SupportProgramming CritiqueProduct Desk-check Showcase / UAT Exploratory Testing Usability Testing Performance Tests Security Tests Unit Tests Component Tests Integration Tests Paper Prototyping / UX Functional UI Tests Acceptance Tests Automated + Manual Manual Automated Automated + Manual What
    21. 21. Agile Testing Quadrants Business Facing Technology Facing SupportProgramming CritiqueProduct Design & Implementation Requirements Design & Implementation Design & Implementation Testing Post-Deployment Design & Implementation Testing Post-Deployment When
    22. 22. Agile Testing Quadrants Business Facing Technology Facing SupportProgramming CritiqueProduct Business Analysts QAs Developers QAs End-users Business Analysts QAs Developers QAs Who
    23. 23. http://agiletester.ca/ 2009
    24. 24. End to end – business facing Localized – technology facing Test Pyramid Unit Service UI http://martinfowler.com/bliki/TestPyramid.html
    25. 25. Test Pyramid Broaderscope Fasterfeedback Unit Service UI
    26. 26. Anti-pattern: Ice-cream Cone Unit Service UI
    27. 27. A closer look at common dilemmas
    28. 28. PracticePrinciple Moving testing upstream TDD ATDD Effective manual testing Timely feedback
    29. 29. Practice SolutionProblem ATDD
    30. 30. Practice SolutionProblem ATDD Needs close collaboration Slow tests Test maintenance Test data Flaky tests
    31. 31. Practice SolutionProblem ATDD Slow tests Test maintenance Test data Flaky tests Story kickoff Design tests before development Definition of “Done” No “stabilization” phase Needs close collaboration
    32. 32. Practice SolutionProblem ATDD Test maintenance Test data Flaky tests Needs close collaboration Test pyramid Parallelization Headless browser testing Slow tests
    33. 33. Practice SolutionProblem ATDD Slow tests Test data Flaky tests Needs close collaboration Test maintenance Test pyramid Tests consolidation Test reviewers/ champions
    34. 34. Practice SolutionProblem ATDD Slow tests Test maintenance Flaky tests Needs close collaboration Test data Subset of prod data Setup/ teardown scripts
    35. 35. Practice SolutionProblem ATDD Slow tests Test maintenance Test data Needs close collaboration Flaky tests Conditional waits Test data availability Dependent systems check Stubbed dependencies
    36. 36. Practice SolutionProblem Efficient manual testing
    37. 37. Practice SolutionProblem Efficient manual testing Balance automation vs manual Too many manual scenarios Lack of domain knowledge
    38. 38. Practice SolutionProblem Efficient manual testing Balance automation vs manual Too many manual scenarios Lack of domain knowledge Devs collaborate on automation Manual test what cannot be automated Automate in parallel to development Dev : QA ratio
    39. 39. Practice SolutionProblem Efficient manual testing Balance automation vs manual Too many manual scenarios Lack of domain knowledge Scope to current iteration Consolidate manual scenarios More automation
    40. 40. Practice SolutionProblem Efficient manual testing Balance automation vs manual Too many manual scenarios Lack of domain knowledge Pair with Business Analyst Run Client Showcases Training
    41. 41. Practice SolutionProblem Timely feedback
    42. 42. Practice SolutionProblem Missing CI practices Lack of CI environments “Scheduled” CI builds Timely feedback Delayed testing
    43. 43. Practice SolutionProblem Lack of CI environments “Scheduled” CI builds Timely feedback Delayed testing CI tool setup Training Automate in parallel to development Missing CI practices
    44. 44. Practice SolutionProblem “Scheduled” CI builds Timely feedback Delayed testing Provision dedicated VMs Infrastructure as Code Database schema on demand Missing CI practices Lack of CI environments
    45. 45. Practice SolutionProblem Timely feedback Delayed testing Trigger build per commit Pull out slow stages into parallel builds Composite builds Missing CI practices Lack of CI environments “Scheduled” CI builds
    46. 46. Practice SolutionProblem Timely feedback Manual desk-check testing CI deployment pipeline No “stabilization” phase Missing CI practices Lack of CI environments “Scheduled” CI builds Delayed testing
    47. 47. But how do we know we’re making progress?
    48. 48. Metrics Customer-reported defect count Average time between faulty commit and defect detection Runtime of tests Lead time of the entire process
    49. 49. What’s bothering you today? Thank you slideshare.net/amanking thoughtworks.com

    ×