Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

QA Fest 2018. Ярослав Пернеровский. Test Automation Pyramid, how it ruins your life

94 views

Published on

Практически любую проблему человечества можно описать в виде "пирамиды". Про пирамиду Маслоу и финансовые пирамиды слышали все. Тестировщики тоже люди, и они придумали свою священную пирамиду - пирамиду автоматизации тестирования. Эта концепция доминирует на рынке уже многие годы и приносит свои позитивные плоды. Но есть и темная сторона этого процесса, выгорание, уныние и забвение в погоне за пирамидой тоже не редкость. В своем докладе хочу немного покритиковать устоявшиеся устои и разобраться как выжить в мире разработки и автоматизации с пирамидой или без нее, а также пройтись по антипатернам в автоматизации, которые так или иначе касаются пирамиды.

Published in: Education
  • Be the first to comment

  • Be the first to like this

QA Fest 2018. Ярослав Пернеровский. Test Automation Pyramid, how it ruins your life

  1. 1. Holy "Test Automation Pyramid" How it ruins your life
  2. 2. About Yaroslav Pernerovskyy @ysparrow
  3. 3. https://qaguild-slack.herokuapp.com/ About
  4. 4. http://automation-remarks.com/podcast/ About
  5. 5. Unit Integration E2E
  6. 6. Unit Integration E2E Manual
  7. 7. Unit Integration E2E White-box Independent units Mocks/stubs White/Black-box Some deployment External systems Black-box Fully deployed system Real life scenarios
  8. 8. Integration ü uses database ü uses network to call another component/application ü external system (MQ, mail server etc.) ü reads/writes files or performs other I/O ü does not rely on source code, use deployed binary instead http://blog.codepipes.com/testing/software-testing-antipatterns.html
  9. 9. What is wrong with the Pyramid?
  10. 10. It is a theoretical metaphor
  11. 11. It is almost unachievable in real world
  12. 12. Unit Integration E2E Created by Developers Sometimes by Developers Sometimes by Testers Usually by Testers
  13. 13. Contexts differences Communication issues Industry stereotypes
  14. 14. Developers should create all the tests to build the canonical test pyramid
  15. 15. https://seleniumcamp.com/talk/good-tests-change-your-application/
  16. 16. Become a "developer" Talk to each other Brake stereotypes
  17. 17. What is wrong with the Tests?
  18. 18. Having "wrong" kind of tests Unit Integration
  19. 19. Unit Integration Having "wrong" kind of tests
  20. 20. Unit Integration GUI Having "wrong" kind of tests
  21. 21. Testing wrong functionality Core Critical Other
  22. 22. 20%
  23. 23. Code coverage
  24. 24. •http://blog.codepipes.com
  25. 25. Codepipes Testing Metrics (CTM) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70% Anything less than 100% PBCNT % of bugs that create new tests 100% 0%-5% Anything less than 100% PTVB % of tests that verify behavior 100% 10% Anything less than 100% PTD % of tests that are deterministic 100% 50%-80% Anything less than 100% http://blog.codepipes.com/testing/software-testing-antipatterns.html
  26. 26. Codepipes Testing Metrics (CTM) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70% Anything less than 100% PBCNT % of bugs that create new tests 100% 0%-5% Anything less than 100% PTVB % of tests that verify behavior 100% 10% Anything less than 100% PTD % of tests that are deterministic 100% 50%-80% Anything less than 100% http://blog.codepipes.com/testing/software-testing-antipatterns.html
  27. 27. Codepipes Testing Metrics (CTM) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70% Anything less than 100% PBCNT % of bugs that create new tests 100% 0%-5% Anything less than 100% PTVB % of tests that verify behavior 100% 10% Anything less than 100% PTD % of tests that are deterministic 100% 50%-80% Anything less than 100% http://blog.codepipes.com/testing/software-testing-antipatterns.html
  28. 28. Codepipes Testing Metrics (CTM) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70% Anything less than 100% PBCNT % of bugs that create new tests 100% 0%-5% Anything less than 100% PTVB % of tests that verify behavior 100% 10% Anything less than 100% PTD % of tests that are deterministic 100% 50%-80% Anything less than 100% http://blog.codepipes.com/testing/software-testing-antipatterns.html
  29. 29. Codepipes Testing Metrics (CTM) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70% Anything less than 100% PBCNT % of bugs that create new tests 100% 0%-5% Anything less than 100% PTVB % of tests that verify behavior 100% 10% Anything less than 100% PTD % of tests that are deterministic 100% 50%-80% Anything less than 100% http://blog.codepipes.com/testing/software-testing-antipatterns.html
  30. 30. 0% 100%Code coverage effortneeded
  31. 31. 0% 100%Code coverage Valuefromtests
  32. 32. Treating ATDD as a religion ü you can write tests before ü you can write tests at the same time ü you can write tests after ü you can never write test
  33. 33. What else is wrong with the Pyramid?
  34. 34. Context-free geometric analogy It is not about Testing
  35. 35. http://www.satisfice.com/blog/archives/4947
  36. 36. ü The Round Earth model shows testing problems at multiple levels ü The Round Earth model reminds us about data
  37. 37. http://www.satisfice.com/blog/archives/4947
  38. 38. ü The Round Earth model shows testing problems at multiple levels ü The Round Earth model reminds us about data ü The Round Earth model reminds us about testability
  39. 39. Quality above requires quality below
  40. 40. Quality above reduces dependence on expensive high-level testing
  41. 41. Inexpensive low-level testing reduces dependence on expensive high-level testing
  42. 42. Risk grows toward the user
  43. 43. Itogi podvedyom ü Triangle is a mental abstraction ü It should be treated as abstraction ü Think before act ü Don't blame yourself ü Use right metrics ü Improve your programming skills ü Right tests over making triangles ü Do not hesitate to kill shitty tests ü Think about other metaphors

×