Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Testing the limits of testing

542 views

Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Testing the limits of testing

  1. 1. Testing the limits of testing… Phil Robinson LonsdaleSystems.com LonsdaleSystems.com 1
  2. 2. Teaching software testing… Blah blah blah blah… In theory there is no difference between theory and practice. In practice there is. Yogi Berra LonsdaleSystems.com 2
  3. 3. The project… LonsdaleSystems.com 3
  4. 4. The project objective… Front End (ASP) Transaction Queue Back End (PL/SQL) Front End (Oracle ADF) LonsdaleSystems.com Database Transaction Queue (Oracle AQ) 4
  5. 5. The project plan… Testing Project Deadline LonsdaleSystems.com 5
  6. 6. The test strategy… LonsdaleSystems.com 6
  7. 7. Analysis Design The waterfall “mantra” Code Test Unit Integration System Acceptance LonsdaleSystems.com 7
  8. 8. Testing is actually three different activities… Test Design Test Execution Defect Repair LonsdaleSystems.com 8
  9. 9. Testing is about quality not project phases… • Fit for intended purpose • Conform to spec • Absence of Defects and other quality goals LonsdaleSystems.com 9
  10. 10. Testing objectives… • Validate that the software is fit for its intended purpose • Verify that the software conforms to its specification • Identify failures caused by defects • Measure non-functional attributes of the software (security, performance, reliability…) • Build confidence in the software LonsdaleSystems.com 10
  11. 11. Testing objectives… • • • • Test to pass… Validate Verify Measure Build confidence LonsdaleSystems.com Test to fail… • Identify failures • Measure (stress) 11
  12. 12. Vertical integration… Unit test LonsdaleSystems.com 12
  13. 13. Vertical integration… Integrate LonsdaleSystems.com 13
  14. 14. Vertical integration… Unit test LonsdaleSystems.com 14
  15. 15. Vertical integration… Integrate LonsdaleSystems.com 15
  16. 16. Vertical integration… Unit test LonsdaleSystems.com 16
  17. 17. Vertical integration… Integrate LonsdaleSystems.com 17
  18. 18. Horizontal integration… Database Feature LonsdaleSystems.com Feature Feature Feature 18
  19. 19. Horizontal integration… Database Feature LonsdaleSystems.com Feature Feature Feature 19
  20. 20. Testing should not be “black and white”… • “Black-box” • Functional • Specification based LonsdaleSystems.com • “White-box” • Structural • Code based 20
  21. 21. Designing test cases… Spec Code Test Basis Test Item Test Cases Other Docs Existing System LonsdaleSystems.com 21
  22. 22. A Better way to define test strategy… Test Item Test Basis • • • • Spec Code Other docs Existing systems Objective • • • • • LonsdaleSystems.com Fit for intended purpose Conform to spec Identify failures Measure nonfunctional attributes Build Confidence Who? • Developer • Technical specialist • Test Analyst • Business Analyst • Subject matter expert 22
  23. 23. Early and extensive unit testing… LonsdaleSystems.com 23
  24. 24. Minimal unit testing… System Coding LonsdaleSystems.com 24
  25. 25. Minimal unit testing… System Test Quality acceptable for user involvement Acceptance Test Project Deadline LonsdaleSystems.com 25
  26. 26. Extensive unit testing… System Coding LonsdaleSystems.com Unit Testing 26
  27. 27. Extensive unit testing… System Test Verify conforms to spec Acceptance Test Quality acceptable for user involvement LonsdaleSystems.com Validate fit for intended purpose Project Deadline 27
  28. 28. One in three developers dedicated to unit testing… 66% coding full time 33% testing full time LonsdaleSystems.com 28
  29. 29. Feature testing… LonsdaleSystems.com 29
  30. 30. Feature testing to build confidence… Feature Feature Feature Feature Testing Feature LonsdaleSystems.com 30
  31. 31. Modelling features… Feature Feature Test Case Test Case Feature Test Case Feature Test Case Feature LonsdaleSystems.com Test Case 31
  32. 32. End-to-end testing… LonsdaleSystems.com 32
  33. 33. End-to-end verification of business rules… Feature LonsdaleSystems.com Feature Feature Feature 33
  34. 34. End-to-end verification of business rules… Feature LonsdaleSystems.com Feature Feature Feature 34
  35. 35. Modelling business rules… e ur at Fe e ur at Fe e ur at Fe State State Choice State LonsdaleSystems.com State State 35
  36. 36. Generating scenarios… e ur at Fe e ur at Fe e ur at Fe Scenario State LonsdaleSystems.com 36
  37. 37. Date-based testing… LonsdaleSystems.com 37
  38. 38. Date-based testing… Scenario 1 Period 1 Period 2 Period 3 Scenario A Feature Feature Feature Feature Feature Feature 2 Scenario B Feature Feature Feature Feature 3 Scenario C LonsdaleSystems.com Feature Feature 38
  39. 39. Exploratory testing… LonsdaleSystems.com 39
  40. 40. Encouraging testers to get creative… 20% • Testing skills • Subject matter expertise 80% • Build confidence • Verify business rules Test Cases LonsdaleSystems.com 40
  41. 41. Triaging bugs… LonsdaleSystems.com 41
  42. 42. Triaging bugs… Test Execution Spreadsheet Spreadsheet JIRA Spreadsheet LonsdaleSystems.com 42
  43. 43. Triaging bugs… Bugs Re e ort p d • • • • Duplicates Vague descriptions Can’t duplicate Tester misunderstanding Resolved Time LonsdaleSystems.com 43
  44. 44. Still teaching software testing… Coming in 2014 Practical Software Testing LonsdaleSystems.com LonsdaleSystems.com 45
  45. 45. Questions LonsdaleSystems.com LonsdaleSystems.com 46

×