Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sauce Labs+Applitools - Automated Visual Testing in the Cloud

8,924 views

Published on

- Reveal expert tips on how to successfully perform large-scale automated visual testing
- Show how to leverage visual testing to increase coverage, while reducing maintenance efforts
- Run a live cross-browser visual test with Sauce Labs and Applitools

Published in: Technology
  • Be the first to comment

Sauce Labs+Applitools - Automated Visual Testing in the Cloud

  1. 1. Adam Carmi Co-Founder and VP R&D at Applitools AUTOMATED VISUAL TESTING IN THE CLOUD Enhance Your Cross-browser Coverage Chris Broesamle Sales Engineer at Sauce Labs
  2. 2. AGENDA  Why automatedvisual testing?  Challengesin visualtestautomation  Demo#1: Visualtestautomationwith ApplitoolsEyes  Demo#2: Cross-browser testingin thecloud with SauceLabs  Demo#3: Runninga cross-browser visualtestinthe cloud  When toperformautomatedvisual testing?  Q&A
  3. 3. WHAT IS VISUAL TESTING? A qualityassurance activity aimed to verify that a Graphical User Interface appears correctly to users
  4. 4. A VISUAL BUG
  5. 5. AND ANOTHER…
  6. 6. WHY SHOULD IT BE AUTOMATED? THE TEST MATRIX IS TOO BIG TO APPROXIMATE MANUALLY  Webbrowsers  Devices  Operating systems  Screen resolutions  Responsive designs  L10n
  7. 7. WHY SHOULD IT BE AUTOMATED?
  8. 8. WHY SHOULD IT BE AUTOMATED? NATIVE / HYBRID MOBILE APPS  Harder to roll back changes  Can’t pushdaily  Updates take battery and data  Higherquality bar
  9. 9. WHY SHOULD IT BE AUTOMATED? Many are already doing it… PhantomCSS Fighting Layout Bugs CSS Critc Wraith Needle Grunt PhotoBox dpdxt WebdriverCSS Eyes Huxley FBSnapshotTestCase GeminiSelenium Visual Diff VisualCeption Specter Snap And Compare
  10. 10. AGENDA  Why automatedvisualtesting?  Challenges invisualtestautomation  Demo#1: Visualtestautomationwith ApplitoolsEyes  Demo#2: Cross-browser testingin thecloud withSauceLabs  Demo#3: Running a cross-browser visualtestin thecloud  Whentoperformvisual testing?  Q&A
  11. 11. THE WORKFLOW Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
  12. 12. THE CHALLENGES Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
  13. 13. SCALABILITY CHALLENGES Image comparison  False positives  Dynamic data Baseline maintenance  Multiplebaselines  Non-localityof changes
  14. 14. WHY NOT? FALSE POSITIVES
  15. 15. ANTI ALIASING 1/2
  16. 16. ANTI ALIASING 2/2
  17. 17. BRIGHTNESS 1/2
  18. 18. BRIGHTNESS 2/2
  19. 19. SUB PIXEL SHIFTS 1/2
  20. 20. SUB PIXEL SHIFTS 2/2
  21. 21. IMAGE SCALING 2/2
  22. 22. IMAGE SCALING 2/2
  23. 23. ANDMORE… 1 pixeloffsets in elementpositioning Dynamic content Movingelements Images ofdifferentsize Performance
  24. 24. AGENDA  Why automatedvisual testing?  Challengesin visual test automation  Demo #1: Visualtest automationwith ApplitoolsEyes  Demo #2: Cross-browser testingin thecloud with SauceLabs  Demo #3: Runninga cross-browser visual test in thecloud  Whento perform visual testing?  Q&A
  25. 25. Chris Broesamle
  26. 26. Agenda • Sauce Labs Overview • Demo of Selenium on Sauce OnDemand
  27. 27. What is Sauce Labs?
  28. 28. We’ve built the worlds largest grid for executing Selenium & Appium tests. We’re open-source at heart and contribute to both projects.
  29. 29. Selenium Grid • A local test script is created using Remote WebDriver, which sends test commands to a grid • A grid consists of a hub server that directs requests and node servers that host OS/browser configurations for execution • Your local hub might be “gridhub.qa.mypage.com” • All you need to do is change that to “ondemand.saucelabs.com”
  30. 30. Local Selenium Grid Hub Receives test requests and relays them to Desired Node Nodes Execute commands on browsers as directed by the hub
  31. 31. Selenium on Sauce
  32. 32. 450+ configurations
  33. 33. Sauce Labs Infrastructure • Environments are kept up to date by our engineering team • Early access to browsers depends on browser developers • Browsers are retired as OS no longer supports it or WebDriver versions are deprecated • Selenium versions are chosen at run time based on compatibility with desired browser
  34. 34. Integrations • Plugins for Jenkins, Bamboo, TeamCity, and Travis CI • Manual testers can open defects in Jira from Sauce • Plugins handle authentication with Sauce, launching Sauce Connect, OS/browser selection in a build, and integration of results
  35. 35. Sauce Labs REST API • Authentication • Provisioning • Jobs (including test results) • Information • Usage • Tunnels • Storage
  36. 36. Selenium on Sauce • Demo
  37. 37. AGENDA  Why automatedvisualtesting?  Challengesin visualtestautomation  Demo#1: Visualtestautomationwith ApplitoolsEyes  Demo#2: Cross-browser testingin thecloud with SauceLabs  Demo#3: Visualtestautomationin thecloud  When toperform visualtesting?  Q&A
  38. 38. WHERE DOES IT FIT? • Component s • Code review • Developers • Designers • QA
  39. 39. WHERE DOES IT FIT? • Pages • Page sections • Developers • Designers • QA • Others
  40. 40. WHERE DOES IT FIT? • Staging vs. Production • Ops • QA
  41. 41. WHERE DOES IT FIT? • Monitoring • Ops • QA
  42. 42. QUESTIONS?

×