The slides I used for my presentation on visual testing at IATI Seminar on Modern Trends In Test Automation

1. Adam Carmi (@carmiadam)
Co-Founder and VP R&D at Applitools
adam.carmi@applitools.com
AUTOMATED VISUAL
TESTING
WHAT, WHEN & HOW

2. AGENDA
 Why automatedvisualtesting?
 Tools& Technology
 Where does it fit?
 Q & A

3. WHAT IS VISUAL TESTING?
A qualityassurance activity aimed to verify that a
Graphical User Interface appears correctly to users

4. A VISUAL BUG

5. AND ANOTHER…

6. AND YET ANOTHER (IE 8 NOT OK)…

7. AND YET ANOTHER (IE 11 OK)…

8. AND THE LAST ONE…

9. WHY SHOULD IT BE AUTOMATED?
THE TEST MATRIX IS TOO BIG TO
APPROXIMATE MANUALLY
 Webbrowsers
 Devices
 Operating systems
 Screen resolutions
 Responsive designs
 L10n

10. WHY SHOULD IT BE AUTOMATED?

11. WHY SHOULD IT BE AUTOMATED?
NATIVE / HYBRID MOBILE APPS
 Harder to roll back changes
 Can’t pushdaily
 Updates take battery and data
 Higherquality bar

12. MANUAL
VISUAL
TESTING
IS
ERROR
PRONE
WHY SHOULD IT BE AUTOMATED?

13. AGENDA
 Why automatedvisualtesting?
 Tools & Technology
 Where does it fit?

14. THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline
images
Report differences
Update the baseline

15. THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline
images
Report differences
Update the baseline

16. DRIVING AND CAPTURING
OTHER
• Wraith (URLs)
• PhantomCSS (CJS)
• dpdxt (URLs)
• Grunt PhotoBox (URLs)
• Snap And Compare
(URLs)
• Specter (JS)
• Gemini (JS)
• Grunt-Vigo (URLs)
• BackstopJS (URLs)
• Applitools Eyes (All)
• WebDriverCSS (JS)
• Fighting Layout Bugs
(Java)
• Wraith-Selenium (Ruby)
• Selenium Visual Diff
(Java)
• Needle (Python)
• VisualCeption (PHP)
• Huxley (RP)
• Gemini (JS)
• Vizregress (.NET)
• Visualci (Java)
• Viff (JS)
• Pix-Diff (JS)
• Applitools Eyes (All)
• CSS Critic (URLs)
• FBSnapshotTestCa
se (XCTest)
• Kobold (folders)
• Applitools Eyes
(All)

17. DRIVING
AND
CAPTURING
WITH
SELENIUM
(WebDriverCSS)

18. MANUALLY WITH EYES EXPRESS

19. THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline
images
Report differences
Update the baseline

21. WHY NOT?
FALSE POSITIVES

22. ANTI ALIASING 1/2

23. ANTI ALIASING 2/2

24. IMAGE SCALING 1/2

25. IMAGE SCALING 2/2

26. ANDMORE…
1 pixeloffsets in elementpositioning
Dynamic content
Movingelements
Images ofdifferentsize
Performance

27. DEMO

28. THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline
images
Report differences
Update the baseline

29. REPORT DIFFERENCES
As files on the file system (combined with source control)

30. REPORT DIFFERENCES
As a Gallery (example from Selenium Visual Diff)

31. THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline
images
Report differences
Update the baseline

32. UPDATE THE BASELINE
 Rename or commit individualimage files
 GUI (GeminiGUI)

33. UPDATE THE BASELINE
 Overwrite mode
 Automatic maintenance (ApplitoolsEyes)

34. AGENDA
 Why automatedvisualtesting?
 Tools& Technology
 Where does it fit?

35. WHERE DOES IT FIT?
• Component
s
• Code
review
• Developers
• Designers
• QA
Visual testing of frontend components by
frontend developers

36. WHERE DOES IT FIT?
• Pages
• Page
sections
• Developers
• Designers
• QA
• Others
Full or partial validation of application
screens
A collaboration tool

37. WHERE DOES IT FIT?
• Staging vs.
Production
• Ops
• QA
Validate your staging deployment using your
production deployment as a baseline

38. WHERE DOES IT FIT?
• Monitoring
• Ops
• QA
No missing resources in production
No corruption due to 3rd party data
No corruption due to browser / OS upgrades

39. QUESTIONS?

40. Adam Carmi (@carmiadam)
Co-Founder and VP R&D at Applitools
adam.carmi@applitools.com
THANK YOU