The document discusses the experiences of Airware, a drone company, with utilizing Node.js for full-stack end-to-end test automation over the past year. It highlights the evolution of their testing framework, including the integration of tools like Selenium and Applitools, and emphasizes best practices in developing readable tests and maintaining testability. Key lessons learned from their implementation are shared, along with references to various libraries and the importance of visual testing.

1. Full-Stack End-to-End
Test Automation with
Node.js
One year later
ForwardJS 2017
Chris Clayman
Mek Stittri

2. 2
First things first
Airware, we are a drone company.
We build complete enterprise drone solutions; Cloud Platform, Ground Control
Station, Drone Integration and etc
We are Hiring! - http://www.airware.com/careers
● iOS Software Engineer
● Javascript Software Engineer
Or just come talk to us :)

3. 3
A Bit About Us
The Automation team at Airware, we do drone stuffs.
Continuous Deployment & Test Automation tools aka Quality Ops
Chris Clayman
@chrisclayman
github.com/kidmillions
Mek Stittri
@mekdev
github.com/mekdev

4. Overheard at the Monday workshops
"I don't write tests, testing sucks"
Let's talk About Test Automation
Let us give
you a
warm hug.

5. Why Node.js for Test Automation
QA, Automation engineers
Frontend engineers
Backend engineers
Java
Javascript
Java
Python
Javascript
Java
Ruby
Javascript
Node.js
Company A Company B Company C
Node.js
Javascript
Node.js
Airware
- Node.js adoption, more and more company moving to node.js
- Share code with the whole engineering team
- Play to the engineering team’s strength
True Full-Stack
Delivery Team

6. Functional tests that
interacts with the webapp
as a user would.
Full-Stack End-to-End Automation
Visual
UI Interaction
Http Restful API
Javascript Unit tests
Microservice Integration
Unit tests
DB & Data Schema tests
Martin Fowler - Test Pyramid, Google Testing Blog
2014 Google Test Automation Conference
Tools for tests
● Visual Diff Engine
● Selenium-Webdriver
● REST API Client
Small
Unit tests
Big
E2E tests
Medium
Integration tests
z’ Test
Pyramid
Cost of test
Amount of tests

7. Client implementation that maps to your HTTP
RESTful Backend API
Simulating browser AJAX calls and acts like a user.
Via GET / POST / PUT / DELETE requests and etc..
Request libraries:
- requests - https://github.com/request/request
- request-promise - https://github.com/request/request-promise
- etc...
REST API Client

8. Anyone here use selenium for automation ?
Industry standard for web browser automation
Language-neutral coding interface: C#, JavaScript , Java, Python, Ruby
Official Project - https://github.com/SeleniumHQ/selenium
Selenium Webdriver
$ npm i selenium-webdriver

9. Visual Diff Engine
Bitmap comparison tool for screenshot validation.
Validates a given screenshot against a baseline
image
Available tools:
- Applitools - https://applitools.com/ integrates with selenium JS
- Sikuli, roll your own framework - http://www.sikuli.org/
Baseline Changes

10. Timeline
Q2 2015
The Early Age
Rapid Prototyping Phase
The Middle Age
“The Promise Manager”
Evaluating multiple test frameworks
Test Runner
- Mocha / Karma
- Ava / Jest
API - Http library
- request
- co-request
- request-promise
- Supertest / Chakram
UI - Selenium bindings
- Nightwatch
- WebdriverIO
- selenium-webdriver (WebdriverJS)
Q3 2015 Q3 2016 - Present
- ES6/ES2015
- Promises
- ESLint
- Went with Mocha
- Picked selenium-webdriver
aka WebDriverJs
Built-in Promise Manager
(synchronous like behavior)
- Went with co-request
Generator calls, easier
(synchronous like behavior)
- Applitools visual diff
- ES7/2016+
- Async Await
- Babel
- ESLint + StandardJS
- Flowtype (facebook)
- selenium-webdriver
- Switched to request lib
Async / Await solves a lot of
the previous problems, no
more callbacks or thenables
The Modern Age
“Async/Await” and beyond
Lessons Learnt

11. We made mistakes so you don’t
have to!
Recipe for a
Node/Selenium
Framework

12. - Representation of Pages (Page Objects)
- Readable tests
- Frontend testability
- Robust backend API
- Clients to interact with said API
- De-async testing approach
- Visual tests
- Accurate and descriptive reporting
Automation Framework Ingredients

13. - Representation of a page or
component.
- We use class syntax
- Abstract UI interaction
implementation to a
BasePage or UI utility
outside of the pageobject
- Map page objects 1-to-1 to
pages and components
Page Object Model (POM)
const CHANGE_PASSWORD_BTN = 'button.change-passwd'
class ProfilePage extends BasePage {
clickChangePassword () {
click(CHANGE_PASSWORD_BTN)
}
}

14. Readable Tests
clusterImageGallery.currentSelectedImageAnnotationIndicatorExistsInTrack()
Great work,
Chip!
- Anyone should be able to read or write tests
- Abstract low level webdriver implementation
outside of page objects, e.g. Navigation utility
- Long-as-hell method names are actually OK
as long as they clarify the test

15. - Testability is a product requirement
- Cook data models into your elements
- Stop Xpath in its tracks. Use CSS selectors
Frontend Testability
Xpath Selector
//div[@class='example']//a
CSS Selector
div.example a

16. Frontend Testability
- Data attributes for testing framework - image name, image uuid
- Clear and concise selectors (ideally separate from styling concerns)
data attributes: image id, name
Concise selectors
- class .main-image
- Image track .slick-track
- Selection status .selected

17. - UI tests should limit UI interaction
to only the functionality being
tested
- Build a client to speak to your
backend through HTTP (still act
as an ‘outsider’ user)
- Setup test data, simply navigate
to it, and interact
- If you can’t expose your backend,
bootstrap test data beforehand
Use the Backend API
it('Validate Bob's profile', function () {
const user = apiClient.getUser('bob')
// if you know Bob's user id,
// you can go straight to his profile
driver.goToProfile(user.id)
const title = profilePage.getTitle()
assert.isEqual(title, 'Bob's Killer Profile')
})

18. The gordian knot:
- Selenium actions and the WebDriver
API are asynchronous
- JavaScript is built for asynchronicity
- Tests should run in a synchronous
manner and in a wholly deterministic
order
De-asyncing Tests
How to cleanly write async actions in a sync manner?
….sandwiches?
It’s like we finish each
other’s….

19. - At first, we used the promise manager extensively
- Heavily dependent on selenium-webdriver control flows
The promise manager / control flow
● Implicitly synchronizes asynchronous actions
● Coordinates the scheduling and execution of all commands
● Maintains a queue of scheduled tasks and executes them in order
De-asyncing Via Promise Manager
driver.get('http://www.google.com/ncr');
driver.findElement({name: 'q'}).sendKeys('webdriver');
driver.findElement({name: 'btnGn'}).click();
driver.get('http://www.google.com/ncr')
.then(function() {
return driver.findElement({name: 'q'});
})
.then(function(q) {
return q.sendKeys('webdriver');
})
.then(function() {
return driver.findElement({name: 'btnG'});
})
.then(function(btnG) {
return btnG.click();
});

20. test.it('Verify data from both frontend and backend', function() {
var apiClient = new ApiClient();
var projectFromBackend;
// API Portion of the test
var flow = webdriver.promise.controlFlow();
flow.execute(function *(){
yield webClient.login(USER001_EMAIL, USER_PASSWORD);
var projects = yield apiClient.getProjects();
projectFromBackend = projectutil.getProjectByName(projects, QA_PROJECT);
});
// UI Portion of the test
var login = new LoginPage(driver);
login.enterUserInfo(USER001_EMAIL, USER_PASSWORD);
var topNav = new TopNav(driver);
topNav.getProjects().then(function (projects){
Logger.debug('Projects from backend:', projectsFromBackend);
Logger.debug('Projects from frontend:', projects);
assert.equal(projectsFromBackend.size, projects.size);
});
- Not Readable / flat
- Not testrunner-agnostic
- Non-standard
API part of the test
HTTP Requests
Issues with Promise Manager
Context switch to
REST API calls
UI Part of the test
Selenium & Browser
var test = require('selenium-webdriver/testing');

21. JobsPage.prototype.getJobList = function () {
this.waitForDisplayed(By.css('.job-table'));
const jobNames = [];
const defer = promise.defer();
const flow = promise.controlFlow();
const jobList = this.driver.findElement(By.css('.job-table'));
// get entries
flow.execute(() => {
jobList.findElements(By.css('.show-pointer')).then((jobs) => {
// Get text on all the elements
jobs.forEach((job) => {
let jobName;
flow.execute(function () {
job.findElement(By.css('.job-table-row-name')).then((element) => {
element.getText().then((text) => {
jobName = text;
});
// look up more table cells...
});
}).then(() => {
jobNames.push({
jobName: jobName
});
});
});
});
}).then(() => {
// fulfill results
defer.fulfill(jobNames);
});
return defer.promise;
};
Bad complexity
+ promise chaining =

22. Async / Await
function notAsync () {
foo().then((bar) => {
console.log(bar)
});
}
async function isAsync() {
const bar = await foo();
console.log(bar)
}
Node > 7.6ES5
Given function foo() that returns a promise...

23. C’mon gang, let’s write a test!
https://github.com/airware/webdriver-mocha-async-await-example

24. - When reporting, consider what information you need in
order to be successful
- Failures First
- Flakiness vs. Failure
- Assertion messaging
- Screenshots
- Video
- Screenshots very useful if done right
- Integrate them into test reports
- Use them for visual testing (shoutout to Applitools)
Reporting and Visual Testing

26. Applitools - Automated Visual Testing

27. "I don't write tests, testing sucks"
"Test automation is easy with
JavaScript "
When You Talk About Test Automation
Airware github repo code example:
https://github.com/airware/webdriver-mocha-async-await-example

28. ForwardJS Organizers
Dave, Taylor, Tracy and team
Special Thanks
Saucelabs - browser CI infrastructure
Feedback on selenium and node.js prototyping
● Neil Manvar
● Kristian Meier
● Adam Pilger
● Christian Bromann
Applitools
Automated Visual Diff Engine from Applitools
● Moshe Milman
● Matan Carmi
● Adam Carmi
● Ryan Peterson
MochaJS
Github
#mochajs
Test runner
Selenium Webdriver
Github
Browser
Automation
WebdriverIO
Github
@webdriverio
Easy to use
Selenium API
Mature Mobile APIs
Appium
Github
@AppiumDevs
Mobile & Native Apps
Automation
JavaScript Community and Open Source Projects.
The world runs on OSS - Please help support these projects!

29. Thank you!

30. Q & A