Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Full-Stack End-to-End
Test Automation with
Node.js
One year later
ForwardJS 2017
Chris Clayman
Mek Stittri
2
First things first
Airware, we are a drone company.
We build complete enterprise drone solutions; Cloud Platform, Ground...
3
A Bit About Us
The Automation team at Airware, we do drone stuffs.
Continuous Deployment & Test Automation tools aka Qua...
Overheard at the Monday workshops
"I don't write tests, testing sucks"
Let's talk About Test Automation
Let us give
you a
...
Why Node.js for Test Automation
QA, Automation engineers
Frontend engineers
Backend engineers
Java
Javascript
Java
Python
...
Functional tests that
interacts with the webapp
as a user would.
Full-Stack End-to-End Automation
Visual
UI Interaction
Ht...
Client implementation that maps to your HTTP
RESTful Backend API
Simulating browser AJAX calls and acts like a user.
Via G...
Anyone here use selenium for automation ?
Industry standard for web browser automation
Language-neutral coding interface: ...
Visual Diff Engine
Bitmap comparison tool for screenshot validation.
Validates a given screenshot against a baseline
image...
Timeline
Q2 2015
The Early Age
Rapid Prototyping Phase
The Middle Age
“The Promise Manager”
Evaluating multiple test frame...
We made mistakes so you don’t
have to!
Recipe for a
Node/Selenium
Framework
- Representation of Pages (Page Objects)
- Readable tests
- Frontend testability
- Robust backend API
- Clients to interac...
- Representation of a page or
component.
- We use class syntax
- Abstract UI interaction
implementation to a
BasePage or U...
Readable Tests
clusterImageGallery.currentSelectedImageAnnotationIndicatorExistsInTrack()
Great work,
Chip!
- Anyone shoul...
- Testability is a product requirement
- Cook data models into your elements
- Stop Xpath in its tracks. Use CSS selectors...
Frontend Testability
- Data attributes for testing framework - image name, image uuid
- Clear and concise selectors (ideal...
- UI tests should limit UI interaction
to only the functionality being
tested
- Build a client to speak to your
backend th...
The gordian knot:
- Selenium actions and the WebDriver
API are asynchronous
- JavaScript is built for asynchronicity
- Tes...
- At first, we used the promise manager extensively
- Heavily dependent on selenium-webdriver control flows
The promise ma...
test.it('Verify data from both frontend and backend', function() {
var apiClient = new ApiClient();
var projectFromBackend...
JobsPage.prototype.getJobList = function () {
this.waitForDisplayed(By.css('.job-table'));
const jobNames = [];
const defe...
Async / Await
function notAsync () {
foo().then((bar) => {
console.log(bar)
});
}
async function isAsync() {
const bar = a...
C’mon gang, let’s write a test!
https://github.com/airware/webdriver-mocha-async-await-example
- When reporting, consider what information you need in
order to be successful
- Failures First
- Flakiness vs. Failure
- ...
Applitools - Automated Visual Testing
"I don't write tests, testing sucks"
"Test automation is easy with
JavaScript "
When You Talk About Test Automation
Airwar...
ForwardJS Organizers
Dave, Taylor, Tracy and team
Special Thanks
Saucelabs - browser CI infrastructure
Feedback on seleniu...
Thank you!
Q & A
ForwardJS 2017 -  Fullstack end-to-end Test Automation with node.js
Upcoming SlideShare
Loading in …5
×

ForwardJS 2017 - Fullstack end-to-end Test Automation with node.js

1,265 views

Published on

Slide deck for ForwardJS 2017 in San Francisco - March 1st 2017
https://forwardjs.com/schedule#lecture-224

Airware builds hardware, software and cloud for commercial drones. We have transitioned to Node.js for cloud functional test automation in 2015. The purpose of this is to unite Fullstack developers and Automation engineers to speak in the same language which is JavaScript. With a year worth of lessons learnt, we will share the challenges involved with building a full-stack test automation framework with Node.js while using the latest and greatest in JavaScript tools.

Published in: Technology
  • heal from broken heart" Love spell from dr.Unity brought my husband back" I'm so excited my husband is back after he left me for another woman" After 12years of marriage, me and my husband has been into one quarrel or the other until he finally left me and moved to California to be with another woman. I felt my life was over and my kids thought they would never see their father again. i tried to be strong just for the kids but i could not control the pains that torments my heart, my heart was filled with sorrows and pains because i was really in love with my husband. Every day and night i think of him and always wish he could come back to me, I was really worried and i needed help, so i searched for help online and I came across a website that suggested that Dr Unity can help get ex back fast. So, I felt I should give him a try. I contacted him and he told me what to do and i did it then he did a Love spell for me. 11hours later, my husband really called me and told me that he miss me and the kids so much, So Amazing!! So that was how he came back that same day,with lots of love and joy,and he apologized for his mistake,and for the pain he caused me and the kids. Then from that day,our Marriage was now stronger than how it were before, All thanks to Dr Unity. he is so powerful and i decided to share my story on the internet that Dr.Unity is real spell caster who i will always pray to live long to help his children in the time of trouble, if you are here and you need your ex lover back or save your marriage fast. Do not cry anymore, contact Dr.Unity now. Here’s his contact,Email him at: Unityspelltemple@gmail.com or Call/WhatsApp him: +2348055361568 ,website:https://unityspelltemples.blogspot.com ,your kindness will never be forgotten.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • ===>>>I basically make about $6,000-$8,000 a month online. It’s enough to comfortably replace my old jobs income, especially considering I only work about 10-13 hours a week from home. I was amazed how easy it was after I tried it… ↡↡↡↡●▬▬▬▬ஜHERE► www.jobsthings.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

ForwardJS 2017 - Fullstack end-to-end Test Automation with node.js

  1. 1. Full-Stack End-to-End Test Automation with Node.js One year later ForwardJS 2017 Chris Clayman Mek Stittri
  2. 2. 2 First things first Airware, we are a drone company. We build complete enterprise drone solutions; Cloud Platform, Ground Control Station, Drone Integration and etc We are Hiring! - http://www.airware.com/careers ● iOS Software Engineer ● Javascript Software Engineer Or just come talk to us :)
  3. 3. 3 A Bit About Us The Automation team at Airware, we do drone stuffs. Continuous Deployment & Test Automation tools aka Quality Ops Chris Clayman @chrisclayman github.com/kidmillions Mek Stittri @mekdev github.com/mekdev
  4. 4. Overheard at the Monday workshops "I don't write tests, testing sucks" Let's talk About Test Automation Let us give you a warm hug.
  5. 5. Why Node.js for Test Automation QA, Automation engineers Frontend engineers Backend engineers Java Javascript Java Python Javascript Java Ruby Javascript Node.js Company A Company B Company C Node.js Javascript Node.js Airware - Node.js adoption, more and more company moving to node.js - Share code with the whole engineering team - Play to the engineering team’s strength True Full-Stack Delivery Team
  6. 6. Functional tests that interacts with the webapp as a user would. Full-Stack End-to-End Automation Visual UI Interaction Http Restful API Javascript Unit tests Microservice Integration Unit tests DB & Data Schema tests Martin Fowler - Test Pyramid, Google Testing Blog 2014 Google Test Automation Conference Tools for tests ● Visual Diff Engine ● Selenium-Webdriver ● REST API Client Small Unit tests Big E2E tests Medium Integration tests z’ Test Pyramid Cost of test Amount of tests
  7. 7. Client implementation that maps to your HTTP RESTful Backend API Simulating browser AJAX calls and acts like a user. Via GET / POST / PUT / DELETE requests and etc.. Request libraries: - requests - https://github.com/request/request - request-promise - https://github.com/request/request-promise - etc... REST API Client
  8. 8. Anyone here use selenium for automation ? Industry standard for web browser automation Language-neutral coding interface: C#, JavaScript , Java, Python, Ruby Official Project - https://github.com/SeleniumHQ/selenium Selenium Webdriver $ npm i selenium-webdriver
  9. 9. Visual Diff Engine Bitmap comparison tool for screenshot validation. Validates a given screenshot against a baseline image Available tools: - Applitools - https://applitools.com/ integrates with selenium JS - Sikuli, roll your own framework - http://www.sikuli.org/ Baseline Changes
  10. 10. Timeline Q2 2015 The Early Age Rapid Prototyping Phase The Middle Age “The Promise Manager” Evaluating multiple test frameworks Test Runner - Mocha / Karma - Ava / Jest API - Http library - request - co-request - request-promise - Supertest / Chakram UI - Selenium bindings - Nightwatch - WebdriverIO - selenium-webdriver (WebdriverJS) Q3 2015 Q3 2016 - Present - ES6/ES2015 - Promises - ESLint - Went with Mocha - Picked selenium-webdriver aka WebDriverJs Built-in Promise Manager (synchronous like behavior) - Went with co-request Generator calls, easier (synchronous like behavior) - Applitools visual diff - ES7/2016+ - Async Await - Babel - ESLint + StandardJS - Flowtype (facebook) - selenium-webdriver - Switched to request lib Async / Await solves a lot of the previous problems, no more callbacks or thenables The Modern Age “Async/Await” and beyond Lessons Learnt
  11. 11. We made mistakes so you don’t have to! Recipe for a Node/Selenium Framework
  12. 12. - Representation of Pages (Page Objects) - Readable tests - Frontend testability - Robust backend API - Clients to interact with said API - De-async testing approach - Visual tests - Accurate and descriptive reporting Automation Framework Ingredients
  13. 13. - Representation of a page or component. - We use class syntax - Abstract UI interaction implementation to a BasePage or UI utility outside of the pageobject - Map page objects 1-to-1 to pages and components Page Object Model (POM) const CHANGE_PASSWORD_BTN = 'button.change-passwd' class ProfilePage extends BasePage { clickChangePassword () { click(CHANGE_PASSWORD_BTN) } }
  14. 14. Readable Tests clusterImageGallery.currentSelectedImageAnnotationIndicatorExistsInTrack() Great work, Chip! - Anyone should be able to read or write tests - Abstract low level webdriver implementation outside of page objects, e.g. Navigation utility - Long-as-hell method names are actually OK as long as they clarify the test
  15. 15. - Testability is a product requirement - Cook data models into your elements - Stop Xpath in its tracks. Use CSS selectors Frontend Testability Xpath Selector //div[@class='example']//a CSS Selector div.example a
  16. 16. Frontend Testability - Data attributes for testing framework - image name, image uuid - Clear and concise selectors (ideally separate from styling concerns) data attributes: image id, name Concise selectors - class .main-image - Image track .slick-track - Selection status .selected
  17. 17. - UI tests should limit UI interaction to only the functionality being tested - Build a client to speak to your backend through HTTP (still act as an ‘outsider’ user) - Setup test data, simply navigate to it, and interact - If you can’t expose your backend, bootstrap test data beforehand Use the Backend API it('Validate Bob's profile', function () { const user = apiClient.getUser('bob') // if you know Bob's user id, // you can go straight to his profile driver.goToProfile(user.id) const title = profilePage.getTitle() assert.isEqual(title, 'Bob's Killer Profile') })
  18. 18. The gordian knot: - Selenium actions and the WebDriver API are asynchronous - JavaScript is built for asynchronicity - Tests should run in a synchronous manner and in a wholly deterministic order De-asyncing Tests How to cleanly write async actions in a sync manner? ….sandwiches? It’s like we finish each other’s….
  19. 19. - At first, we used the promise manager extensively - Heavily dependent on selenium-webdriver control flows The promise manager / control flow ● Implicitly synchronizes asynchronous actions ● Coordinates the scheduling and execution of all commands ● Maintains a queue of scheduled tasks and executes them in order De-asyncing Via Promise Manager driver.get('http://www.google.com/ncr'); driver.findElement({name: 'q'}).sendKeys('webdriver'); driver.findElement({name: 'btnGn'}).click(); driver.get('http://www.google.com/ncr') .then(function() { return driver.findElement({name: 'q'}); }) .then(function(q) { return q.sendKeys('webdriver'); }) .then(function() { return driver.findElement({name: 'btnG'}); }) .then(function(btnG) { return btnG.click(); });
  20. 20. test.it('Verify data from both frontend and backend', function() { var apiClient = new ApiClient(); var projectFromBackend; // API Portion of the test var flow = webdriver.promise.controlFlow(); flow.execute(function *(){ yield webClient.login(USER001_EMAIL, USER_PASSWORD); var projects = yield apiClient.getProjects(); projectFromBackend = projectutil.getProjectByName(projects, QA_PROJECT); }); // UI Portion of the test var login = new LoginPage(driver); login.enterUserInfo(USER001_EMAIL, USER_PASSWORD); var topNav = new TopNav(driver); topNav.getProjects().then(function (projects){ Logger.debug('Projects from backend:', projectsFromBackend); Logger.debug('Projects from frontend:', projects); assert.equal(projectsFromBackend.size, projects.size); }); - Not Readable / flat - Not testrunner-agnostic - Non-standard API part of the test HTTP Requests Issues with Promise Manager Context switch to REST API calls UI Part of the test Selenium & Browser var test = require('selenium-webdriver/testing');
  21. 21. JobsPage.prototype.getJobList = function () { this.waitForDisplayed(By.css('.job-table')); const jobNames = []; const defer = promise.defer(); const flow = promise.controlFlow(); const jobList = this.driver.findElement(By.css('.job-table')); // get entries flow.execute(() => { jobList.findElements(By.css('.show-pointer')).then((jobs) => { // Get text on all the elements jobs.forEach((job) => { let jobName; flow.execute(function () { job.findElement(By.css('.job-table-row-name')).then((element) => { element.getText().then((text) => { jobName = text; }); // look up more table cells... }); }).then(() => { jobNames.push({ jobName: jobName }); }); }); }); }).then(() => { // fulfill results defer.fulfill(jobNames); }); return defer.promise; }; Bad complexity + promise chaining =
  22. 22. Async / Await function notAsync () { foo().then((bar) => { console.log(bar) }); } async function isAsync() { const bar = await foo(); console.log(bar) } Node > 7.6ES5 Given function foo() that returns a promise...
  23. 23. C’mon gang, let’s write a test! https://github.com/airware/webdriver-mocha-async-await-example
  24. 24. - When reporting, consider what information you need in order to be successful - Failures First - Flakiness vs. Failure - Assertion messaging - Screenshots - Video - Screenshots very useful if done right - Integrate them into test reports - Use them for visual testing (shoutout to Applitools) Reporting and Visual Testing
  25. 25. Applitools - Automated Visual Testing
  26. 26. "I don't write tests, testing sucks" "Test automation is easy with JavaScript " When You Talk About Test Automation Airware github repo code example: https://github.com/airware/webdriver-mocha-async-await-example
  27. 27. ForwardJS Organizers Dave, Taylor, Tracy and team Special Thanks Saucelabs - browser CI infrastructure Feedback on selenium and node.js prototyping ● Neil Manvar ● Kristian Meier ● Adam Pilger ● Christian Bromann Applitools Automated Visual Diff Engine from Applitools ● Moshe Milman ● Matan Carmi ● Adam Carmi ● Ryan Peterson MochaJS Github #mochajs Test runner Selenium Webdriver Github Browser Automation WebdriverIO Github @webdriverio Easy to use Selenium API Mature Mobile APIs Appium Github @AppiumDevs Mobile & Native Apps Automation JavaScript Community and Open Source Projects. The world runs on OSS - Please help support these projects!
  28. 28. Thank you!
  29. 29. Q & A

×