Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Self healing test automation with Healenium and Minimization of regression suite with Drill4J

396 views

Published on

Joint talk. To cover 2 topics.

Dmitriy will cover 2 topics, 20 minutes each:

self-healing algorithm for selenium based UI tests (Healenium)
Test Gap Analysis and minimization of regression suite with Test-to-code mapping in Drill4J
Self-healing

Selectors in Automated UI is a pain point: tests sooner or later “breaks” due to changes in layout. Which makes UI tests unreliable as a part of testing pipeline and Continuous Testing itself. Layout mutate and as a result test crashes, builds turn “red”.

In this talk Anna will give a word regarding easiest way how to hack this problem in the bud. You will learn about the newly open-sourced tool for self-healing automation called Healenium.

Healenium allows to overcome problem of instability of automation tests automatically and spend less time supporting tests. Now it works for Mobile Automation as well !



Drill4J

Test Gap Analysis is the process of identifying these gaps where new code has been deployed but hasn’t been tested yet. However, often your testing department does not know which parts of code have been changed by the developers. As a result, testers run some unnecessary tests while other crucial tests are overlooked.

With Test Gap Analysis we can find gaps in tests and help you avoid errors made due to recent, untested changes. In doing so, you can optimize the interface between developers and testers and avoid hotfixes after the system’s release.

With this talk Dmitriy will share and unveil new Open Sourced tool Drill4J, describe capabilities of Test-to-Code mapping and how you can minimize your regression time by identifying subset of tests, which should be run, which code have been changed and which changes are not tested after full testing cycle.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Self healing test automation with Healenium and Minimization of regression suite with Drill4J

  1. 1. HEALENIUM - SELF-HEALING AUTOMATION DRILL4J - TEST TO CODE MAPPING
  2. 2. DMITRIY GUMENIUK Senior Delivery Manager, Product Manager ReportPortal.io product owner DelEx Conference organizer
  3. 3. HEALENIUM - SELF-HEALING TEST • Problem statement • How sha works • Demo • TAF integration • POS results • Future plans
  4. 4. PROBLEMS Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change PROBLEM STATEMENT
  5. 5. PROBLEMS GOALS FEATURES Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change Improve E2E automated test stability Reduce amount of test cases failed due to non-product issue Reduces Tests Maintenance efforts More time to cover new functionality with tests Shipping better products faster PROBLEM STATEMENT
  6. 6. What we have on the market
  7. 7. Dynamic locators
  8. 8. PROS & CONS • Dynamic DOM state updates • Easy to write and support tests • Tests are stable • Cross-browser, CI, Jira integrations from the box • Only for new projects • Strong dependency from tool ecosystem • May be not secured • May lose tests when project finished
  9. 9. Healenium Applicable at any stage of project development Minimal dependency from the TAF Replace static locators with dynamic ones
  10. 10. Healenium Gives ability to find controls (new locator) for updated WEB pages Use kinda ML algorithms for Web page changes identification Is a standalone Java jar connected to test cases code base Helps >2x times reduced test fails because of updated UI Provides Html report which handles locators analysis Provides Intellij Idea plugin to make code updates with new locator values
  11. 11. HOW HEALENIUM WORKS
  12. 12. LCS ALGORITHM Longest common subsequence problem of finding the longest subsequence common to all sequences in a set of sequences *widely used by revision control systems such as Git
  13. 13. LCS ALGORITHM MODIFICATION Longest common subsequence with weight Added extra weight for tag, Id, class, value, other attributes <button id=btn-1> <button id=btn-1>
  14. 14. Healenium selenium (jar) Tree-comparing(jar) Includes Tree-comparing dependency Implements Selenium WebDriver Overwrites findElement() method Catch NoSuchElementException Activates LCS algorithm in Tree-comparing library Save reference element path to storage Get reference element path from storage Get current DOM state Search in current state for the best subsequence Generate new CSS locator JAR LIBRARIES
  15. 15. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Test Automation Framework Element Found
  16. 16. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Test Automation Framework Element Found Save successful path Old locator
  17. 17. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework
  18. 18. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework Old locator Successful path Page state
  19. 19. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework Old locator New locator Successful path Page state New locator
  20. 20. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework Old locator New locator Successful path Page state New locator Find New Element
  21. 21. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework Element Found Save successful path Old locator New locator Successful path Page state New locator Find New Element
  22. 22. SelfHealingDriver findElement public WebElement findElement(By by) { if (by instanceof PageAwareBy) { try { Trying to find the element and save its path if success } catch (NoSuchElementException var5) { Find previous reference element path in storage Generate new best matches locator return healedLocator; } } else { return delegate.findElement(by); } }
  23. 23. PageAwareBy extends By PageAwareBy(String pageName, By by, SelfHealingEngine engine) { this.pageName = pageName; this.by = by; this.engine = engine; } PageObject Page Locator New locator processor
  24. 24. Functions supported • By() -> PageAwareBy() • @FindBy -> @PageAwareFindBy • WebDriver -> SelfHealingDriver • Iframe support • Actions support • Remote test run • Parallel test run • Works with Selenium wrappers like Selenide
  25. 25. Html report
  26. 26. Intellij Idea plugin to make code updates HTML REPORT
  27. 27. Intellij Idea plugin to make code updates PLUGIN HTML REPORT Update request { filename: MainPageWithFindBy lineNumber: 49 failedLocatorValue://input[@name='EMAIL’] healedLocatorValue: input.t186__input.js-tilda-rule.t-input }
  28. 28. Intellij Idea plugin to make code updates TAF CODE BASE PLUGIN HTML REPORT Find locator and update Update request { filename: MainPageWithFindBy lineNumber: 49 failedLocatorValue://input[@name='EMAIL’] healedLocatorValue: input.t186__input.js-tilda-rule.t-input }
  29. 29. TAF INTEGRATION
  30. 30. 1. Declare custom EngineConfig with the path to store new locators. For example 'shaselenium' EngineConfig engineConfig = EngineConfig .custom() .setStorage(new FileSystemPathStorage(Paths.get("sha", "selenium"))) .build(); 2. Init driver instance of SelfHealingDriver SelfHealingDriver driver = new SelfHealingDriver(new ChromeDriver(), engineConfig); 3. Locate elements By buttonBy = PageAwareBy.by("MainPage", By.id(testButtonId)); Or @PageAwareFindBy(page="MainPage", findBy = @FindBy(id = "markup-generation-button")) WebElement buttonBy; 4. Interact with elements as usual driver.findElement(buttonBy).click(); Important! Do not delete data from the folder where files with new locators are stored. They are used to perform self-healing in next automation runs Important! "MainPage" is the name of the page to which the WebElement belongs
  31. 31. RESULTS
  32. 32. 100% tests healed Healed localization testing
  33. 33. Implemented Idea plugin Mobile support
  34. 34. github.com/healenium github.com/healenium/healenium-web github.com/healenium/healenium-appium Apache2.0 License
  35. 35. DRILL4J - TEST TO CODE MAPPING
  36. 36. Coverage?
  37. 37. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 Application under test
  38. 38. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18
  39. 39. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6
  40. 40. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6
  41. 41. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1
  42. 42. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7
  43. 43. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5
  44. 44. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5 M1
  45. 45. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5 M1 M7
  46. 46. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5 M1 M7 M1 5
  47. 47. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5
  48. 48. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5
  49. 49. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5
  50. 50. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7
  51. 51. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7
  52. 52. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18
  53. 53. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 Modified by commit
  54. 54. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7
  55. 55. M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC4 TC5 TC6 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 TC1 TC2 TC3 TC6 M1 M7 M1 5 M1 M7 M1 5 M1 M7 M1 5 M1 M7
  56. 56. You find 95% offails in 5% of time
  57. 57. drill4j.github.io t.me/drill4j github.com/Drill4J Drill4J with TestToCode plugin

×