Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

QA Fest 2019. Анна Чернышова. Self-healing test automation 2.0. The Future

22 views

Published on

Мы уже разговаривали о self-healing автоматизации, как она работает, какие есть подходы, чем они хороши, плохи и о новом инструменте, который мы разрабатываем в EPAM. Наш продукт завершает стадию POC и настало время поделиться результатами и понять, насколько self-healing автоматизация поможет вашим тестам стать стабильнее? Или наоборот, навредит?... Приходи и узнаешь!

Published in: Education
  • Be the first to comment

  • Be the first to like this

QA Fest 2019. Анна Чернышова. Self-healing test automation 2.0. The Future

  1. 1. SELF-HEALING TEST AUTOMATION 2.0 THE FUTURE • Problem statement • How sha works • Demo • TAF integration • POS results • Future plans
  2. 2. ANNA CHERNYSHOVA Lead Software Test Automation Engineer Testing Competency Center Expert Email: anna_chernyshova@epam.com FB: anna.chernyshova.79
  3. 3. PROBLEMS Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change PROBLEM STATEMENT
  4. 4. PROBLEMS GOALS FEATURES Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change Improve E2E automated test stability PROBLEM STATEMENT
  5. 5. PROBLEMS GOALS FEATURES Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change Improve E2E automated test stability Reduce amount of test cases failed due to non-product issue PROBLEM STATEMENT
  6. 6. PROBLEMS GOALS FEATURES Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change Improve E2E automated test stability Reduce amount of test cases failed due to non-product issue Reduces Tests Maintenance efforts PROBLEM STATEMENT
  7. 7. PROBLEMS GOALS FEATURES Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change Improve E2E automated test stability Reduce amount of test cases failed due to non-product issue Reduces Tests Maintenance efforts More time to cover new functionality with tests PROBLEM STATEMENT
  8. 8. PROBLEMS GOALS FEATURES Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change Improve E2E automated test stability Reduce amount of test cases failed due to non-product issue Reduces Tests Maintenance efforts More time to cover new functionality with tests Shipping better products faster PROBLEM STATEMENT
  9. 9. What we have on the market
  10. 10. Dynamic locators
  11. 11. PROS & CONS • Dynamic DOM state updates • Easy to write and support tests • Tests are stable • Cross-browser, CI, Jira integrations from the box • Only for new projects • Strong dependency from tool ecosystem • May be not secured • May lose tests when project finished
  12. 12. Epam Self-healing Applicable at any stage of project development Minimal dependency from the TAF Replace static locators with dynamic ones
  13. 13. Self-healing library… Gives ability to find controls (new locator) for updated WEB pages Use kinda ML algorithms for Web page changes identification Is a standalone Java jar connected to test cases code base Helps >2x times reduced test fails because of updated UI Provides Html report which handles locators analysis Provides Intellij Idea plugin to make code updates with new locator values
  14. 14. HOW SHA WORKS
  15. 15. LCS ALGORITHM Longest common subsequence problem of finding the longest subsequence common to all sequences in a set of sequences *widely used by revision control systems such as Git
  16. 16. LCS ALGORITHM MODIFICATION Longest common subsequence with weight Added extra weight for tag, Id, class, value, other attributes <button id=btn-1> <button id=btn-1>
  17. 17. Self-healing selenium (jar) Tree-comparing(jar) Includes Tree-comparing dependency Implements Selenium WebDriver Overwrites findElement() method Catch NoSuchElementException Activates LCS algorithm in Tree-comparing library Save reference element path to storage Get reference element path from storage Get current DOM state Search in current state for the best subsequence Generate new CSS locator JAR LIBRARIES
  18. 18. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Test Automation Framework Element Found
  19. 19. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Test Automation Framework Element Found Save successful path Old locator
  20. 20. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework
  21. 21. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework Old locator Successful path Page state
  22. 22. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework Old locator New locator Successful path Page state New locator
  23. 23. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework Old locator New locator Successful path Page state New locator Find New Element
  24. 24. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework Element Found Save successful path Old locator New locator Successful path Page state New locator Find New Element
  25. 25. SelfHealingDriver findElement public WebElement findElement(By by) { if (by instanceof PageAwareBy) { try { Trying to find the element and save its path if success } catch (NoSuchElementException var5) { Find previous reference element path in storage Generate new best matches locator return healedLocator; } } else { return delegate.findElement(by); } }
  26. 26. PageAwareBy extends By PageAwareBy(String pageName, By by, SelfHealingEngine engine) { this.pageName = pageName; this.by = by; this.engine = engine; } PageObject Page Locator New locator processor
  27. 27. Functions supported • By() -> PageAwareBy() • @FindBy -> @PageAwareFindBy • WebDriver -> SelfHealingDriver • Iframe support • Actions support • Remote test run • Parallel test run • Works with Selenium wrappers like Selenide
  28. 28. Html report
  29. 29. Intellij Idea plugin to make code updates HTML REPORT
  30. 30. Intellij Idea plugin to make code updates PLUGIN HTML REPORT Update request { filename: MainPageWithFindBy lineNumber: 49 failedLocatorValue://input[@name='EMAIL’] healedLocatorValue: input.t186__input.js-tilda-rule.t-input }
  31. 31. Intellij Idea plugin to make code updates TAF CODE BASE PLUGIN HTML REPORT Find locator and update Update request { filename: MainPageWithFindBy lineNumber: 49 failedLocatorValue://input[@name='EMAIL’] healedLocatorValue: input.t186__input.js-tilda-rule.t-input }
  32. 32. TAF INTEGRATION
  33. 33. 1. Declare custom EngineConfig with the path to store new locators. For example 'shaselenium' EngineConfig engineConfig = EngineConfig .custom() .setStorage(new FileSystemPathStorage(Paths.get("sha", "selenium"))) .build(); 2. Init driver instance of SelfHealingDriver SelfHealingDriver driver = new SelfHealingDriver(new ChromeDriver(), engineConfig); 3. Locate elements By buttonBy = PageAwareBy.by("MainPage", By.id(testButtonId)); Or @PageAwareFindBy(page="MainPage", findBy = @FindBy(id = "markup-generation-button")) WebElement buttonBy; 4. Interact with elements as usual driver.findElement(buttonBy).click(); Important! Do not delete data from the folder where files with new locators are stored. They are used to perform self-healing in next automation runs Important! "MainPage" is the name of the page to which the WebElement belongs
  34. 34. RESULTS
  35. 35. Projects for pilot TAF based on Java + Selenium (WebDriver) Continuous UI updates UI generated by CMS
  36. 36. Projects for pilot TAF based on Java + Selenium (WebDriver) Continuous UI updates UI generated by CMS >20 replies 6 Web projects for pilot 3 projects are still in focus
  37. 37. 100% tests healed Healed localization testing
  38. 38. Issues Test execution time increase Can’t heal single element from the list Probability of false/positive results Code not updated after healing
  39. 39. FUTURE PLANS
  40. 40. Future plans Idea plugin Mobile support
  41. 41. Idea Plugin ✅ @PageAwareFindBy ❌ PageAwareBy(..) ❌ Local declaring
  42. 42. Thanks to Oleh Khailenko Lead Software Engineer Kostiantyn Morozkin Senior Software Engineer Nikolai Kobzev Senior Software Engineer Vladimir Moshkin SoftwareTest Automation Engineer
  43. 43. Email: anna_chernyshova@epam.com FB: anna.chernyshova.79

×