Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cross browser testing

25 views

Published on

My presentation from DevOps West conference in Vegas, June 2018

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Cross browser testing

  1. 1. Strategy For Selecting the Right Cross—Browser Testing Tools
  2. 2. About Me • Lead Technical Evangelist at Perfecto • Blogger and Speaker • http://continuoustesting.blog • https://www.infoworld.com/author/Eran- Kinsbruner/ • 18+ Years in Development & Testing • Author of The Digital Quality Handbook Weekly Podcast - Testiumpod Twitter: @ek121268 Email: Erank@perfectomobile.com
  3. 3. Agenda  A word about the digital transformation  Cross-Browser Coverage  RWD & PWA Testing As The De-Facto Web App Types  Cross-Browser Testing Landscape and Criteria  Q&A 6/12/2018 3© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  4. 4. Cross-Browser Testing != Desktop Web Testing There is no Web Testing vs. Mobile Testing • 4 out of 10 transactions today take place on multiple devices • 48% of users today complain that the websites they use are not optimized for their smartphones and tablets
  5. 5. Every Browser Has Unique Capabilities 6/12/2018 5© 2015, Perfecto Mobile Ltd. All Rights Reserved. Chrome on Windows 10 - Desktop Edge 17 on Windows 10 - Desktop https://whatwebcando.today/
  6. 6. Coverage Methodology - Web 6/12/2018 6© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  7. 7. Keeping Up 6/12/2018 7© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  8. 8. Fundamentals of RWD Testing © 2015, Perfecto Mobile Ltd. All Rights Reserved.
  9. 9. Responsive Web App – Test Plan Strategy: 6 Steps 6/12/2018 9© 2015, Perfecto Mobile Ltd. All Rights Reserved. Choose The Right Platforms Assure Visual Web Pages Across Platforms Functionality of Navigation Flows and Menus Client Side Performance Testing Test Accessibility for Mobile and Web Test Environment Conditions (Networks, Sensors)
  10. 10. • Identify your object in a robust fashion that fits all digital Platforms • Build object repository and use smart locators Responsive Web Design (RWD) - Objects The average website includes nearly 400 different objects. Now try locating them on each and every DIGITAL platform in your lab…
  11. 11. • Take screenshot and use Visual Checkpoint/assertion to validate responsive aspects Responsive Web Design (RWD) – visual validation w/ Screenshots
  12. 12. Real Life Example – Screen Size & Layout Implications on UX 6/12/2018 12© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  13. 13. Fundamentals of PWA Testing © 2015, Perfecto Mobile Ltd. All Rights Reserved.
  14. 14. The Rise of PWAs – Challenging Native Mobile Apps A Progressive Web App is • Progressive - Works for every user • Responsive - Fits any form factor • Connectivity independent • App-like - Feels like an app • Safe - Served via HTTPS • Search Friendly -allowing search engines to find it. • Re-engageable -Support push notifications. • Linkable - does not require complex installation (store access) 6/12/2018 14© 2015, Perfecto Mobile Ltd. All Rights Reserved. Source :medium.com
  15. 15. Progressive Web App – The Offline Network Benefit 6/12/2018 15© 2015, Perfecto Mobile Ltd. All Rights Reserved. Source: VentureBeat
  16. 16. PWA on iOS is Different than on Android 6/12/2018 16© 2015, Perfecto Mobile Ltd. All Rights Reserved. iOS Android App type shortcut apk Offline data Deleted after idle time No limit Offline storage (browser based) Safari < 50Mb Chrome 6% available storage Sensors support Limited (BT) All access to private information No Yes Payment system No supported Supported Voice support No Yes
  17. 17. Progressive Web App – Test Plan Strategy: 6 Steps 6/12/2018 17© 2015, Perfecto Mobile Ltd. All Rights Reserved. Validate PWA Manifest Validate PWA Service Workers Validate PWA Specific Capabilities Properly Test Across Platforms (RWD) Test Automation and Object Identification Strategy Google’s PWA Checklist Compliance
  18. 18. Progressive Web App – Test Plan Strategy: Step 5 6/12/2018 18© 2015, Perfecto Mobile Ltd. All Rights Reserved. Object Identification Strategy Interact with WebView Elements within the PWA App Launch PWA App Close Driver Interact with Native Elements of the PWA App
  19. 19. Progressive Web App – Test Plan Strategy: Step 6 6/12/2018 19© 2015, Perfecto Mobile Ltd. All Rights Reserved. Validate Your Testing Plans Against Google’s PWA Checklist (Lighthouse) • https://developers.google.com/web/progressive-web-apps/checklist
  20. 20. Tool Selection Strategy © 2015, Perfecto Mobile Ltd. All Rights Reserved.
  21. 21. How to select the right testing framework? 6/12/2018 21© 2015, Perfecto Mobile Ltd. All Rights Reserved. Organizational Fit Technical Fit
  22. 22. Selection Criteria: Organizational Fit Requirements 6/12/2018 22© 2016, Perfecto Mobile Ltd. All Rights Reserved. Project Complexity Quality analysis requirements Test Types & Practices Resources and Skills Availability Test Environment Existing Tech Stack
  23. 23. Selection Criteria: Technical Fit Requirements 6/12/2018 23© 2016, Perfecto Mobile Ltd. All Rights Reserved. SDLC Process Fit (Integrations, Plugins, etc.) Community size, support and Doc’s Feedback Loop and Reporting Automation Coverage Cloud and Automation at Scale Automation Robustness and Maintainability
  24. 24. Web Testing Tools (Open-Source) - Download Trends 6/12/2018 24© 2016, Perfecto Mobile Ltd. All Rights Reserved. JSDom & Mocha Selenium Protractor
  25. 25. Web Market Testing – Selection Criteria 6/12/2018 25© 2016, Perfecto Mobile Ltd. All Rights Reserved. Read More Here
  26. 26. Additional Categorization of JS Tools • Tools that provide a testing structure (BDD/ATDD) (Mocha, Jasmine, Jest, Cucumber) • Tools that provide assertions functions (Chai, Jasmine, Jest, Unexpected) • Tools to generate, display, and watch test results (Mocha, Jasmine, Jest, Karma) • Tools that generate code coverage reports (Istanbul, Jest, Blanket) • Tools that provide a browser or browser-like environment with a control on their scenarios execution, UI testing and more (Protractor, Nightwatch, Phantom, Casper, Selenium, WebDriver.IO, TestCafe) • Provide mocks, spies, and stubs (Sinon, Jasmine, enzyme, Jest, testdouble) 6/12/2018 26© 2015, Perfecto Mobile Ltd. All Rights Reserved. Source: Medium.com
  27. 27. JS Assertions 6/12/2018 27© 2015, Perfecto Mobile Ltd. All Rights Reserved. Source: Medium.com
  28. 28. JS Acceptance Testing with Codecept.JS 6/12/2018 28© 2015, Perfecto Mobile Ltd. All Rights Reserved. Source: Codecept.JS Git
  29. 29. JS Code Coverage (Istanbul with Mocha.JS) $ npm install --save-dev nyc { "scripts": { "test": "nyc mocha" } } 6/12/2018 29© 2015, Perfecto Mobile Ltd. All Rights Reserved. Source: Istanbul.js.org
  30. 30. Headless Browsers Role Motivation • Receive fast feedback (Unit and basic functional tests) – Mostly Dev • Easy environment setup, no IT dependencies • Uses standard JS language • Supported and promoted by browser vendors (Google, Mozilla) • Additional use cases to cover: • HAR File • Performance testing • Basic UI • Security 6/12/2018 30© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  31. 31. Headless Browsers – Puppeteer Example 6/12/2018 31© 2015, Perfecto Mobile Ltd. All Rights Reserved. 'use strict'; const puppeteer = require('puppeteer'); const devices = require('puppeteer/DeviceDescriptors'); (async() => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.emulate(devices['iPhone 6']); await page.goto('https://www.nytimes.com/'); await page.screenshot({path: 'full.png', fullPage: true}); await browser.close(); })(); https://github.com/GoogleChrome/puppeteer
  32. 32. Tool Selection Case Study (USAA) • Define needed capabilities 6/12/2018 32© 2015, Perfecto Mobile Ltd. All Rights Reserved. Selection Criteria End to End Testing BDD/ATDD Friendly Tool Documentation Visual Navigation Testing
  33. 33. USAA Tool Selection • Define needed capabilities • Identify importance (weight capabilities) 6/12/2018 33© 2015, Perfecto Mobile Ltd. All Rights Reserved. Weight Key 1 – Low importance 3 – Medium importance 5 – High importance Weight Selection Criteria 5 (High importance) End to End Testing 3 (Medium importance) BDD/ATDD Friendly 5 (High importance) Tool Documentation 1 (Low importance) Visual Navigation Testing
  34. 34. USAA Tool Selection • Define needed capabilities • Identify importance (weight capabilities) • Define scoring key 6/12/2018 34© 2015, Perfecto Mobile Ltd. All Rights Reserved. Scoring Key 0 – Did not meet expectations 2 – Met expectations 3 – Exceeded expectations Weight Selection Criteria Tool X Tool Y Tool Z 5 (High importance) End to End Testing 3 3 3 3 (Medium importance) BDD/ATDD Friendly 3 2 3 5 (High importance) Tool Documentation 0 2 2 1 (Low importance) Visual Navigation Testing 3 3 2
  35. 35. USAA Tool Selection • Define needed capabilities • Identify importance (weight capabilities) • Define scoring key 6/12/2018 35© 2015, Perfecto Mobile Ltd. All Rights Reserved. Weight Selection Criteria Tool X (weighted) Tool Y (weighted) Tool Z (weighted) 5 (High importance) End to End Testing 3 5 x 3 = 15 3 5 x 3 = 15 3 5 x 3 = 15 3 (Medium importance) BDD/ATDD Friendly 3 3 x 3 = 9 2 3 x 2 = 6 3 3 x 3 = 9 5 (High importance) Tool Documentation 0 5 x 0 = 0 2 5 x 2 = 10 2 5 x 2 = 10 1 (Low importance) Visual Navigation Testing 3 1 x 3 = 3 3 1 x3 = 3 2 1 x 2 = 2 Total 27 34 36 Scoring Key 0 – Did not meet expectations 2 – Met expectations 3 – Exceeded expectations
  36. 36. Infrastructure as a Freeway 6/12/2018 36© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  37. 37. Thank You

×