Proven strategy for testing pw as aus-agile_testers

(Proven) Strategy for Testing
Progressive Web Apps (PWAs)
Eran Kinsbruner
@ek121268, http://continuoustesting.blog
erank@perfecto.io

• A word about Continuous Testing
• PWA Definition & Architecture
• Motivation to Develop PWA
• Testing Strategy for PWA Apps
• Q & A
Twitter: @ek121268 (https://twitter.com/ek121268)
Blog: http://continuoustesting.blog
LinkedIn: https://www.linkedin.com/in/eran-kinsbruner-4b47a81/

About Me
• Lead Software Evangelist at Perfecto
• Blogger and Speaker
• http://continuoustesting.blog
• https://www.infoworld.com/author/Eran-Kinsbruner/
• 18+ Years in Development & Testing
• Author of: The Digital Quality Handbook
Continuous Testing for DevOps Professionals
@ek121268
https://www.linkedin.com/in/eran-kinsbruner-4b47a81/
erank@perfecto.io

Reasons for “not stable” Test Automation
80% of issues have a pattern52% success rate
10% of devices,
causing 80% of lab
issues
Lab
25%
Orches
tration
25%
Scripts
& FW
50%
FAILURE REASON
Objects Codding Time Other
Scripts & FW issues
Device in use
No Device
Orchestration issues
Networking Stability Lock
Other
Lab issues
What’s
wrong
With my
Scripts
What’s wrong
With my Lab
What’s wrong
With my
Executions

Web Testing Tools (Open-Source) - Download Trends
10/27/2018 5© 2016, Perfecto Mobile Ltd. All Rights Reserved.
Source: npmtrends
Protractor
Puppeteer
WebDriver IO

Mobile Capabilities in Web Browser (Incomplete) – Ramping
Up
CameraMic
Device AUTH
BT/BLE
Notification Network

Today’s Mobile Native Apps Reality
Android WebiOS

Progressive Web App -
Definition
Progressive Web Apps (PWAs) are web applications that are
regular web pages or websites, but can appear to the user like
traditional applications or native mobile applications. The
application type attempts to combine features offered by most
modern browsers with the benefits of a mobile experience
(Wikipedia)

The Rise of PWAs – Challenging Native Mobile
Apps
A Progressive Web App is
• Progressive - Works for every user
• Responsive - Fits any form factor
• Connectivity independent
• App-like - Feels like an app
• Safe - Served via HTTPS
• Search Friendly -allowing search engines to find
it.
• Re-engageable -Support push notifications.
• Linkable - does not require complex installation
(store access)
Source :medium.com

When Implemented Properly PWA Is Powerful
PWA Stats: https://www.pwastats.com/

More PWA Stats

And … More PWA Stats
PWA Alibaba: Alibaba PWA

Adoption Trend

Progressive Web App – The Offline Network
Benefit
Source: VentureBeat

Progressive Web App - Architecture
• A PWA App consists of few key components
• Manifest.Json - That's the file within the PWA that describes the
app, provides metadata specific to the app like icons, splash
screens, and more
• Service Workers – A service worker is a script that your browser
runs in the background, separate from a web page, opening the door
to features that don't need a web page or user interaction.
• Capabilities:
• Display and interact with notifications (native OS)
• Subscribe to push services
• Background sync APIs – defer actions until users have stable connection
• Lifecycle
• Registration
• Installation
• Activation

Installation Process
*iOS Hybrid App (not running from Safari + special permissions) vs. Android APK)

PWA on iOS is Different than Android
iOS Android
App type shortcut apk
Offline data Deleted after idle time No limit
Offline storage (browser based) Safari < 50Mb Chrome 6% available
storage
Sensors support Limited (BT) All
access to private information No Yes
Payment system No supported Supported
Voice support No Yes

Instagram/Tinder PWA on iOS
1. Login isn’t saved in between launches
2. Language change/Orientation change
disrupts layout 3rd Party Login

Instagram PWA

Instagram PWA on Desktop Chrome – Inspected by Google
Lighthouse

PWA Test Strategy
© 2018 Perfecto Mobile Ltd. All Rights Reserved.

Roadblocks Ahead
• Young and Growing - Not too many precedents
• Architecture - Not a simple RWD site, but RWD + proper
handling of event management, state, and caching
• Skillset – Frontend developers need to catch up
• Caching Architecture – Not a ‘walk in the park’
• Limited Open-Source Solutions – Expend capabilities,
delight dev.
• Platforms and 3rd Party Integrations (Analytics, Social,
others)
• Maintaining OS updates is a challenge (Mobile, Desktop)

Responsive Web App – Test Plan Strategy: 6
Steps
Choose The
Right Platforms
Assure Visual
Web Pages
Across Platforms
Functionality of
Navigation Flows
and Menus
Client Side
Performance
Testing
Test Accessibility
for Mobile and
Web
Test Environment
Conditions
(Networks,
Sensors)

Progressive Web App – Test Plan Strategy: 6
Steps
Validate PWA
Manifest
Validate PWA
Service
Workers
Validate PWA
Specific
Capabilities
Properly Test
Across
Platforms
(RWD)
Test Automation
and Object
Identification
Strategy
Google’s PWA
Checklist
Compliance
RWD Testing Strategy And


Progressive Web App – Test Plan Strategy: Step 1
icons, splash screen, start URL, background colors, display type, initial
orientation and theme colors, are correctly working and visually properly
displayed across browsers
Validate PWA
Manifest

Validate
PWA Service
Workers

• Test the SW registration
• Test SW install and update functionality
• Test the various SWs Start/Stop/Registration
• chrome://serviceworker-internals/
• chrome://inspect/#service-workers
if ('serviceWorker' in navigator) {
window.addEventListener('load', function() {
navigator.serviceWorker.register('/sw.js').then(function(registration) {
// Registration was successful
console.log('ServiceWorker registration successful with scope: ', registration.scope);
}, function(err) {
// registration failed :(
console.log('ServiceWorker registration failed: ', err);
});
});
}
var CACHE_NAME = 'my-site-cache-v1';
var urlsToCache = [
'/',
'/styles/main.css',
'/script/main.js'
];
self.addEventListener('install', function(event) {
// Perform install steps
event.waitUntil(
caches.open(CACHE_NAME)
.then(function(cache) {
console.log('Opened cache');
return cache.addAll(urlsToCache);
})
);
});
Validate PWA
Service
Workers

Progressive Web App – Test Plan Strategy:
Step 3
• Sensors, Camera, Location capabilities, Push Notification, must be tested across mobile and web
respectively
• PWA vs. Native: Instances, icons, functionality, Gestures
10/27/2018
Google Maps Force Touch (Native App) Google Maps Native vs. PWA ICONS Google Maps PWA Location Prompt
Validate PWA Specific
Capabilities

Progressive Web App – Test Plan Strategy: Step
4
• UI and visual/layout testing across multiple form factors
• Performance and rendering of the content across platforms
• Network related testing – in addition to the offline mode that is covered through service workers,
make sure to cover the app behavior throughout various network conditions (packet loss, flight
mode, latency %, 3G, 4G, 5G etc.)
• Functionality of the entire page user flows across platforms and screen sizes and resolutions
• The differences between browsers, platforms must be covered as well (Chrome, Safari, Android,
Samsung Browser vs. built-in browsers like Facebook - number of instances or copies a PWA
app can have on a device.
Properly Test Across
Platforms (RWD)

Step 5
Object
Identification
Strategy
Interact with WebView Elements within the PWA App
Launch PWA App
Close Driver
Interact with Native Elements of the PWA App

Object
Identification
Strategy

Step 5
Object
Identification
Strategy

Step 5
Object
Identification
Strategy
&

Step 6
• https://developers.google.com/web/progressive-web-apps/checklist
Google’s PWA
Checklist Compliance

Summary & References
• Summary:
• PWA is growing and embraced by
browser vendors and enterprises
• iOS is lagging behind Android
capabilities
• Selenium & Appium (Or Visual
Testing) is the recommended test
automation strategy for PWA
• Leverage what you already know
and have (RWD, Selenium, Mobile
Testing)
• References:
• Service Workers (W3C) -
https://www.w3.org/TR/service-
workers/
• Google Lighthouse - Google
Lighthouse - Web Store
• Debugging SW – Google
Developers
• Chrome SW Inspector - Local
Inspector

Proven strategy for testing pw as aus-agile_testers

Proven strategy for testing pw as aus-agile_testers

More Related Content

What's hot

Similar to Proven strategy for testing pw as aus-agile_testers

More from Perfecto Mobile

Recently uploaded

Proven strategy for testing pw as aus-agile_testers