Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Visual Regression Testing

619 views

Published on

vodQA 2016 -

Published in: Technology
  • Be the first to comment

Visual Regression Testing

  1. 1. VISUAL REGRESSION TESTING USING GALEN JYOTHY MARIAM JOHN, RISHABH RAO 16TH JULY 2016
  2. 2. Change Blindness Change blindness is a surprising perceptual phenomenon that occurs when a change in a visual stimulus is introduced and the observer does not notice it. https://www.youtube.com/watch?v=ubNF9QNEQLA
  3. 3. Visual regression tests Tests the layout of a page, the way in which components - texts, images, grids etc are set out on a page. Functional Test will pass for both the cases though layout is broken in the second image. NOTE: Tooltip error message is overlapping text box
  4. 4. Why Visual regression tests? CI / CD Environment - Frequent Releases Responsive Web Design Themes / Skins
  5. 5. Responsive Web Design Responsive web design (RWD) is an approach to web design aimed at crafting sites to provide an optimal viewing and interaction experience—easy reading and navigation with a minimum of resizing, panning, and scrolling—across a wide range of devices (from desktop computer monitors to mobile phones)
  6. 6. Themes / Skins Applying a skin changes a piece of software's look and feel— some skins merely make the program more aesthetically pleasing, but others can rearrange elements of the interface, potentially making the program easier to use.
  7. 7. Automated Visual Regression Tests thus become QA’s lifeline in tackling such complexities
  8. 8. Agenda ✓ Why visual regression tests are important Existing tools & Overview of Galen My journey, challenges faced / learnings
  9. 9. Galen A test framework originally designed for testing layout of web applications. When it comes to testing a responsive layout it works in the following way Open a page in browser Resize it to specified size Test the layout according to user-defined specs It uses Selenium for interacting with elements on page and getting their locations and dimensions. Once it sees that something is wrong – Galen reports the error, makes a screenshot and highlights the misbehaving element on it. http://galenframework.com/docs/all/
  10. 10. Galen Galen unlike existing image comparison tools, provides a way to test components on a page with reference to other components present. Galen also supports image comparisons.
  11. 11. Basics Galen Commands Galen Specification File Galen Reports Galen commands refer spec file for page validation
  12. 12. Spec File Format Object Definitions, for Galen to find elements on test page. Supported locators are id, css & xpath Spec Sections, for describing the page layout for each viewport size using spec reference keywords like near, below, above, left-of, right-of, inside etc. http://galenframework.com/docs/reference-galen-spec- language-guide/#SpecsReference
  13. 13. Sample Spec File
  14. 14. Spec Reference Or be less strict and say near button 5 to 15 px left
  15. 15. CSS Property Test the value of a CSS property for a specific element
  16. 16. Image Supports Image Comparison Reports error if mismatching pixels are more than acceptable range 12% The red pixels on comparison map are the mismatching pixels that are far away from tolerance value, yellow pixels are in the range of 30 to 80 colour difference from the tolerance value, Green pixels are the closest to tolerance value. By default tolerance is set to 30, if we increase this parameter, the amount of mismatching pixels will be decreased
  17. 17. Command Line Tests In order to perform a single page test you can use the command “check”
  18. 18. JavaScript Tests Provides list of API’s to write JavaScript tests E.g load, createDriver, createGridDriver, resize, checkLayout, checkPageSpecLayout, parsePageSpec etc Also provides a lightweight javascript testing framework (GalenPages) to work with page object model test("Home page", function() { var driver = createDriver(“http://galenframework.com","1024x768"); checkLayout(driver, "homePage.gspec", ["all", "desktop"]); }); http://galenframework.com/docs/reference-javascript-tests-guide/
  19. 19. Java Tests Use Galen inside Java tests Java + Maven + TestNG + Galen Integrate visual regression tests into your existing functional regression tests framework and reuse navigation / actions which is already defined for existing tests import com.galenframework.api.Galen;
  20. 20. Agenda ✓ Why visual regression tests are important ✓ Existing tools & Overview of Galen My journey, challenges faced / learnings
  21. 21. Reference Tests
  22. 22. Pixel Tests http://galenframework.com/post/2014-09-28-version- 1.3.0-image-comparison-in-layout-testing/
  23. 23. Image Comparison Configuration File runMode = createBaseline / validateUI
  24. 24. Image Comparison Challenges Dynamic content Varying image colours on different screens
  25. 25. Image Rendering Differences
  26. 26. Image Rendering Differences
  27. 27. Addressing dynamic content Validate @Component Level
  28. 28. Addressing dynamic content Change RunTime Behaviour Of AUT
  29. 29. Addressing dynamic content Mock responses https://github.com/mrak/stubby4node
  30. 30. Addressing dynamic content Inject javascript to modify the text displayed
  31. 31. Reports Inherit all your test class from com.galenframework.testng.GalenTestNgTestBase Add com.galenframework.testng.GalenTestNgReportsL istener in your pom file
  32. 32. Reports Baseline Actual Comparison Map
  33. 33. Reports Baseline Actual Comparison Map
  34. 34. Q&A jyothyj@thoughtworks.com rishabhr@thoughtworks.com
  35. 35. Error Vs Tolerance
  36. 36. Error Vs Tolerance blues.png, tolerance 30 blues.png, tolerance 80 When tolerance is increased red area (in image2) has decreased and less error value is reported More the tolerance, lesser the error..

×