Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Defect localization

190 views

Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

Defect localization

  1. 1. . ...... Defect Localization Leo Arias Canonical September 24, 2013 Leo Arias (Canonical) Defect Localization September 24, 2013 1 / 26
  2. 2. Contents ...1 Goals ...2 Isolation ...3 Defect Localization problems ...4 Proposal ...5 Open discussion Leo Arias (Canonical) Defect Localization September 24, 2013 2 / 26
  3. 3. Goals Contents ...1 Goals ...2 Isolation ...3 Defect Localization problems ...4 Proposal ...5 Open discussion Leo Arias (Canonical) Defect Localization September 24, 2013 3 / 26
  4. 4. Goals Goals of Test Automation Improve quality. Understand the system under test. Reduce and not introduce risk. Leo Arias (Canonical) Defect Localization September 24, 2013 4 / 26
  5. 5. Goals Improve Quality Tests enable us to specify the behavior in various scenarios captured in a form that we can then execute. Prevent bugs from being introduced. Defect localization. Leo Arias (Canonical) Defect Localization September 24, 2013 5 / 26
  6. 6. Goals Defect Localization Customer tests tell us that some behavior isn’t working as expected, the Unit tests tell us why. We should be able to pinpoint the bug quickly based on which test fails. If a customer test fails but no unit tests fail, it indicates a missing unit test. Leo Arias (Canonical) Defect Localization September 24, 2013 6 / 26
  7. 7. Goals Defect Localization Customer tests tell us that some behavior isn’t working as expected, the Unit tests tell us why. We should be able to pinpoint the bug quickly based on which test fails. If a customer test fails but no unit tests fail, it indicates a missing unit test. Leo Arias (Canonical) Defect Localization September 24, 2013 6 / 26
  8. 8. Isolation Contents ...1 Goals ...2 Isolation ...3 Defect Localization problems ...4 Proposal ...5 Open discussion Leo Arias (Canonical) Defect Localization September 24, 2013 7 / 26
  9. 9. Isolation Isolation We should strive to isolate the system under test as much as possible the parts we choose not to test. Failure to isolate lead to context sensitivity because we end up testing too much software all at once. Leo Arias (Canonical) Defect Localization September 24, 2013 8 / 26
  10. 10. Isolation Isolation We should strive to isolate the system under test as much as possible the parts we choose not to test. Failure to isolate lead to context sensitivity because we end up testing too much software all at once. Leo Arias (Canonical) Defect Localization September 24, 2013 8 / 26
  11. 11. Isolation Isolation, cont. Leo Arias (Canonical) Defect Localization September 24, 2013 9 / 26
  12. 12. Isolation Isolation, cont. Leo Arias (Canonical) Defect Localization September 24, 2013 10 / 26
  13. 13. Isolation Isolation, cont. Leo Arias (Canonical) Defect Localization September 24, 2013 11 / 26
  14. 14. Defect Localization problems Contents ...1 Goals ...2 Isolation ...3 Defect Localization problems ...4 Proposal ...5 Open discussion Leo Arias (Canonical) Defect Localization September 24, 2013 12 / 26
  15. 15. Defect Localization problems Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 13 / 26
  16. 16. Defect Localization problems More Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 14 / 26
  17. 17. Defect Localization problems Canonical Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 15 / 26
  18. 18. Defect Localization problems Canonical Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 16 / 26
  19. 19. Defect Localization problems Canonical Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 17 / 26
  20. 20. Defect Localization problems Canonical Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 18 / 26
  21. 21. Defect Localization problems Canonical Dependencies Leo Arias (Canonical) Defect Localization September 24, 2013 19 / 26
  22. 22. Defect Localization problems Defect Localization We should be able to pinpoint the bug quickly based on which test fails. If a customer test fails on a project B because of a change in project A, it indicates a missing customer test on project A. (and a missing dependency test on project B). Leo Arias (Canonical) Defect Localization September 24, 2013 20 / 26
  23. 23. Defect Localization problems Defect Localization We should be able to pinpoint the bug quickly based on which test fails. If a customer test fails on a project B because of a change in project A, it indicates a missing customer test on project A. (and a missing dependency test on project B). Leo Arias (Canonical) Defect Localization September 24, 2013 20 / 26
  24. 24. Defect Localization problems Defect Localization We should be able to pinpoint the bug quickly based on which test fails. If a customer test fails on a project B because of a change in project A, it indicates a missing customer test on project A. (and a missing dependency test on project B). Leo Arias (Canonical) Defect Localization September 24, 2013 20 / 26
  25. 25. Proposal Contents ...1 Goals ...2 Isolation ...3 Defect Localization problems ...4 Proposal ...5 Open discussion Leo Arias (Canonical) Defect Localization September 24, 2013 21 / 26
  26. 26. Proposal The problems Projects publish an API or behavior without testing it from the point of view of the consumers. Projects start relying on an API or behavior even when it’s not properly tested. Projects change behaviors without notifying the other projects they will affect. Leo Arias (Canonical) Defect Localization September 24, 2013 22 / 26
  27. 27. Proposal The problems Projects publish an API or behavior without testing it from the point of view of the consumers. Projects start relying on an API or behavior even when it’s not properly tested. Projects change behaviors without notifying the other projects they will affect. Leo Arias (Canonical) Defect Localization September 24, 2013 22 / 26
  28. 28. Proposal The problems Projects publish an API or behavior without testing it from the point of view of the consumers. Projects start relying on an API or behavior even when it’s not properly tested. Projects change behaviors without notifying the other projects they will affect. Leo Arias (Canonical) Defect Localization September 24, 2013 22 / 26
  29. 29. Proposal Proposal We should focus on defect localization and isolation while adding test coverage. Leo Arias (Canonical) Defect Localization September 24, 2013 23 / 26
  30. 30. Proposal Short-term actions When an automated test fails because of an error or a behavior change on another project: File a bug describing the problem and the projects affected. <QA> Subscribe to the bug and follow the process to its resolution. Make sure all the people affected are notified. <developers> Add automated customer tests with every fix to prevent regressions. <QA> Make sure no fix comes without a customer test. Check that they are added on the right project. Leo Arias (Canonical) Defect Localization September 24, 2013 24 / 26
  31. 31. Proposal After the release... <QA> Analyze the current API and GUI coverage. <developers> Add isolated tests to fill the holes. <developers> Make sure we only rely on APIs and behaviors with automated customer tests. <QA, developers> Write a test suite specific to test the integration of all the projects on the different Ubuntu images and devices. <design, QA, developers> Start adding user tests since the expected user experience is defined. Establish an effective way for us to communicate between teams. Leo Arias (Canonical) Defect Localization September 24, 2013 25 / 26
  32. 32. Open discussion Open discussion Thank you! Creative Commons - atribución 3.0 Leo Arias (Canonical) Defect Localization September 24, 2013 26 / 26

×