Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Jenkins and visual regression – Exove


Published on

Trainee project about Jenkins and visual regression testing.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Jenkins and visual regression – Exove

  1. 1. Jenkins and Visual Regression Mikko Nirkko, 2019
  2. 2. Table of Contents ● What is Jenkins ● What is visual regression testing ● Case study - Screenshot comparison software in Jenkins
  3. 3. ● Continuous integration Server ● The main goal is to automate all kinds of software builds ● Can be used for ○ Continuous Integration (CI) ○ Continuous Delivery ○ Continuous Deployment ○ Automation ○ Testing ○ … Among many other things! What is Jenkins?
  4. 4. What Can Jenkins Do?
  5. 5. ● Automate software builds and delivery ○ Timed builds ○ Triggered builds ○ Build across different platforms ● Flexible notifications when builds fail ○ Email and Slack mostly used ● Extendable with optional plug-ins to further increase the number of supported technologies
  6. 6. What is Visual Regression Testing?
  7. 7. Visual regression ● Search for differences between two or more images of a common target ● In this case, compare screenshots of a website between builds ● When new updates or other changes cause unwanted visual effects, this means image regression has happened!
  8. 8. Case Study: Visual Regression Testing in Jenkins
  9. 9. ● The main concept was to build a proof-of-concept version of a screenshot comparison software, running entirely within Jenkins ● It was designed to be used by Exove staff, as a simple internal tool ● Since the application operates on (confidential) client sites, the application use is restricted to Exove employees only. Visual Regression Testing Tool
  10. 10. How it works 1. Collect screenshots from a website of choice 2. Save and index the screenshots to a gallery 3. Run the same test again, after a site update (or whenever) 4. This collects a second, identically formed gallery 5. Compare the galleries against each other 6. If notable differences are found, create a detailed report of the findings
  11. 11. Workflow 1. Choose “Build with parameters” from the project page 2. Enter Site Url and press “Build now” (Other parameters are optional) 3. Wait until testing is done. Results appear at the project front page That’s It!
  12. 12. ● Robot Framework ○ for crawling websites and capturing the screenshots ○ Extended with my own Python library and BeautifulSoup4 for processing HTML reports ○ With ImageMagick for screenshot processing and comparison ● Jenkins for: ○ Hosting ○ Screenshot storage ○ Test execution ○ Reporting the test results Technologies Used
  13. 13. PROS & CONS ● Very easy to use, even without technical knowledge ● In most cases, the only required input from the user is the URL of the site they want to test ● No local installations or files needed ● Multiple ways of viewing and distributing test results ● Current proof-of-concept contains primitive methods of difference recognition ● This results in many false positives with things like updated text and image content ● Somewhat slow to execute