This document discusses WIX's approach to web automation testing. It outlines two types of automation - for manual testers and developers. WIX believes in automation developed by developers using code. The costs of automation include developers, students, automation farms, and other software providers. Guidelines are provided for integration tests done by developers and end-to-end tests done by automation developers. Image comparison testing and human testing are also discussed.

1. WIX Web Automation
Itay Shmool
itays@wix.com

2. Before We Start - Two Kinds of Automation
• Automation for manual testers (Selenium IDE for
example). Writing tests without coding.
• Automation for developers , code
WIX Believes in Automation for developers

3. Automation costs
• Automation == development
• Wix Automation group:
–13 full time developers
–Students (test analysis and support)
• Automation farm
• Other SW & Solution providers – Sauce Labs ,
Browser Stuck , Applitools Eyes
Now when we know that we can start 

4. Full QA Concept at WIX
Integration
(JS)
E2E
(Selenium)
Human
Code
Image
comparison

5. Integration Vs. E2E
Integration
Tests
E2E
Tests
scope Client only Client + server
Platform Jasmine selenium
Language JS java
Written by Developers /
Automation
developers
Automation
developers
On push Yes (very fast) no
Periodically
(nightly for
example)
Yes Yes
Same Infra structure to run all

6. E2E guidelines
• Should be Fast – think about 5000 tests running 3 times a day:
– Multi Threated
– Grid
– As minimum selenium as possible 
– Use JS code to make things faster (application shortcuts)
• Robustness/quality :
– Design:
• Logic (Tests) Vs. Implementation (Drivers) must be separated (Test doesn’t break unless the feature logic
had changed)
• Object oriented approach - code re-use
– Use junit parametric rule to add more tests easily
– minimum false alarm (wait for events instead of timeouts)
• Availability :
– No automation – no release !
– Redundant
– Cloud (Amazon) , Sauce lab , Browser stuck

7. Integration Tests Guidelines
• Developers as part of the development process
• Automation developers can write as well , depends on the
feature
• Progression – comes together with the feature
• Reduce E2E tests dramatically
• Improved Coverage

8. Image Comparison (Applitools eyes)
• Fast “development” process , can be use for progression
• Large scale (in pilot we are running 500) of visual
production tests
• Using our current setup , the integration of “Eyes” into our
framework was easy
• Mobile native apps ?

9. Human Testing – most important !
• Automation is a robot
• Human testers are the once that takes decisions
• Responsible for the QA E2E process
• Progression
• User Experience
• Left overs (browsers , mobile)