More Related Content Similar to 5 Mobile App Trends & What They Mean for Dev & Testing (20) More from Perfecto by Perforce (16) 5 Mobile App Trends & What They Mean for Dev & Testing2. Eran Kinsbruner • Chief Evangelist, Product Manager, and Author at Perfecto by Perforce
• Blogger, Inventor, and Speaker
• 20+ years in software development & testing
• Author of:
• The Digital Quality Handbook
• Continuous Testing for DevOps Professionals
• Accelerating Software Quality
• Twitter: @ek121268
Karthik K.K • Testing expert and founder of ExecuteAutomation
• Software testing consultant, researcher, and blogger
• 14+ years of experience in software automation testing
• Passionate about ensuring automation quality is never compromised
• Distinguished reviewer of the leading software testing tools, including TestProject, Katalon
Studio, Testim, and Perfecto
3. perfecto.io3 | Perfecto by Perforce © 2020 Perforce Software, Inc.
1
2
3
The Digital Landscape Overview (Future of Mobile Apps) and Mobile Testing 101
App Clips & APKs
Progressive Web App Testing (PWAs)
4 Appendix (Bonus): Foldable Testing Strategy
5 Q&A
Today’s Agenda
5. perfecto.io5 | Perfecto by Perforce © 2019 Perforce Software, Inc.
• Native apps
• Hybrid apps
Mobile App Types Native Apps Hybrid Apps
• These apps are created based on
specific platforms, such as:
• iOS
• Android
• Tizen, etc.
• E.g. Xamarin, Flutter, React
Native, Ionic, PhoneGap, etc.
• These apps are created for
web and they are responsive
to the platform they run on.
• PWA is another attempt to
make mobile apps more
accessible than browser
mobile based apps.
• E.g. HTML, JavaScript, CSS, etc.
6. perfecto.io6 | Perfecto by Perforce © 2020 Perforce Software, Inc.
The most popular testing tools for these mobile app types include:
• Appium
• Espresso
• XCUITest
• Xamarin.UITest
• Flutter Driver
• Perfecto
Mobile App Testing Tools
7. perfecto.io7 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• Appium is the most popular testing tool for mobile and supports programming languages such as:
• C#/Java/JS/Php/Ruby, etc.
• Runs on Windows/Mac/Linux.
• Appium comes with recorder, server, client libraries, and more.
• Appium is highly extended in many third party tools like:
• TestProject, Katalon Studio, Sauce Labs, and more.
Appium
8. perfecto.io8 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• Flutter Driver is evolving and getting better and better.
• Supports automating applications on Mac/Windows/Linux.
• Written in Dart programming language but close to Java.
Flutter Driver
8
9. perfecto.io9 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• Flutter is a multi-platform framework.
• Was released in Dec 2018 as a stable build.
• Initially supported just mobile platforms (iOS & Android),
but now it supports platforms such as:
Introduction to Flutter
10. perfecto.io10 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Flutter is a free and open-source project developed and maintained by Google.
Flutter uses DART as the programming language.
Flutter is fast, smooth, and capable of running apps in 120fps.
Since it has its own Widgets, it’s highly customizable.
What is Flutter?
11. perfecto.io11 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Flutter is super fast since it has its
own widget and it doesn’t use
JavaScript Bridge — which is used by
many cross-platform app
development SDKs — making the
code execution faster in any platform.
Flutter uses DART and the
programing language is similar
to C#/Java/JS.
Flutter has hot reload option, which
makes the changes made in an app
viewed visually without going
through so many build deploy.
And more …
Advantages of Flutter
12. perfecto.io12 | Perfecto by Perforce © 2020 Perforce Software, Inc.
- Len Wein
In general, shorter is better. If you can
encapsulate your idea into a single
captivating sentence, you're halfway home.
13. perfecto.io13 | Perfecto by Perforce © 2019 Perforce Software, Inc.
App Functionality
Business Flows
UI Testing (Landscape/Portrait, Languages)
Cross Platform Coverage
Real Environment
Condition Testing
Network Conditions
Interruptions (Calls, Alerts, Text Messages)
Background/Foreground
Gestures (Force Touch etc.)
NFT
(Non-Functional Testing)
Security
Accessibility
Performance and Availability
API Testing
Generic Mobile Application Testing Considerations
14. perfecto.io14 | Perfecto by Perforce © 2019 Perforce Software, Inc.
Perfecto Optimizes DevOps Pipelines With
Various Testing Types & Coverage Goals (Real/Virtual Platforms)
Virtual Platforms
Mix of
Virtual Platforms & Real Devices
Only Real Devices
Relevant Unit Testing
High-Value Smoke Testing:
Functional, API, Integration,
Component, and Security Testing
Regression on Real Devices:
Mixed Functional, Non-Functional,
Performance, and Security Testing
15. 15 | How to Guarantee Continuous Value from your Test Automation perfecto.io
APKs and App Clips
16. perfecto.io16 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Digital Apps Are Transforming
Apple AppClips Android APKs
17. perfecto.io17 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• APKs are typically zip files that consist
of multiple, single APK files. Based on
rules, specific devices, and user requests,
these files will get installed and enabled
on the devices in production.
Introduction to APKs
18. perfecto.io18 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• Application binary sizes.
• Overall resource consumption like CPU,
battery, and network by unused features
of the app.
• The ability to gradually expose as well
as enable/disable specific modules and
features based on customers countries,
requirements, and other considerations.
Benefits and Deployment Options for APKs
19. perfecto.io19 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• Developers and testers need better communication to fully understand:
• The release scope and what’s supported in the build to be tested.
• Which devices were added to the supported list.
• Which features and modules are added or deprecate.
Benefits and Deployment Options for APKs
https://developer.android.com/distribute/best-practices/launch/launch-checklist
Learn More
22. perfecto.io22 | Perfecto by Perforce © 2019 Perforce Software, Inc.
Distribution Methods (~10MB Size)
• App Clip Codes
• NFC Tags
• QR Codes
• Safari App Banner
• Links in Messages
• Place Cards in Maps
• Recently Used App Clips
Apple App Clips
Sample Benefits:
1. Privacy protection — fewer permissions required,
after an idle time, content is deleted.
2. Expose app subset.
3. Better UX (skip a que — order and pay from iOS
device with App Clips.)
23. perfecto.io23 | Perfecto by Perforce © 2020 Perforce Software, Inc.
To create and test an iOS App Clip, developers should:
• Build and run the App Clip on a device.
• Archive the App Clip’s corresponding full app, select the archive, and export the App Clip’s binary
for Ad Hoc or Development distribution. Then, add the App Clip to a device, for example, by dragging
the App Clip’s exported .ipa file to a connected device in Xcode’s Devices and Simulators window.
Distribute your App Clips to testers using TestFlight
On the DUT using TestFlight Open the Settings app, navigate to Developer
> Local Experiences, select Register Local Experience and Configure:
• A URL prefix that invokes the App Clip
• Copy for the App Clip card’s title and the subtitle
• A call-to-action verb
• A header image
• Your App Clip’s bundle identifier
Building and Testing App Clips
https://developer.apple.com/documentation/app_clips/testing_your_app_clip_s_launch_experience
Learn More
24. perfecto.io24 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Building and Testing App Clips
https://developer.apple.com/app-clips/
Learn More
https://chibi.app/make
Example:
25. perfecto.io25 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• Consider different device and OS versions for compatibility.
• Test across network conditions and locations.
• Test installation using different supported methods (camera,
NFC/Bluetooth, Safari browser).
• Test UX/UI of the App Clip — first impressions count.
• Appium/XCUITest are the recommended tools.
• After reaching the end of the App Clip snippet, it cannot be continued
without relaunching the browser — design tests accordingly.
• iOS 14 and iPadOS 14 testing varies — screens, browser versions, UX.
• Test incoming event implications on your App Clip.
Testing App Clips
https://firigames.com/phoenix2
Example:
26. 26 | How to Guarantee Continuous Value from your Test Automation perfecto.io
Progressive Web Apps Testing
27. perfecto.io27 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Progressive Web Apps (PWAs) are web applications that are regular web pages or websites but can appear to the user
like traditional applications or native mobile applications. The application type attempts to combine features offered by
most modern browsers with the benefits of a mobile experience (Wikipedia).
Progressive Web App — Definition
28. perfecto.io28 | Perfecto by Perforce © 2020 Perforce Software, Inc.
A Progressive Web App is…
• Progressive — Works for every user.
• Responsive — Fits any form factor.
• Connectivity independent
• App-like — Feels like an app.
• Safe — Served via HTTPS.
• Search Friendly — Allowing search engines to find it.
• Re-engageable — Supports push notifications.
• Linkable — Does not require complex installation (store access).
The Rise of PWAs — Challenging Native Mobile Apps
Source: medium.com
29. perfecto.io29 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• A PWA app consists of few key components:
• Manifest.Json — That's the file within the PWA that describes the app and provides metadata specific to the app
like icons, splash screens, and more.
• Service Workers — A service worker is a script that your browser runs in the background, separate from a web
page, opening the door to features that don't need a web page or user interaction (JS Proxy).
• Capabilities:
• Display and interact with notifications (native OS).
• Subscribe to push services.
• Background sync APIs – defer actions until users have stable connection.
• Lifecycle
• Registration
• Installation
• Activation
Progressive Web App — Architecture
30. perfecto.io30 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Installation Process
*iOS Hybrid App (running from Safari + special permissions vs. Android APK)
31. perfecto.io31 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Top 11 PWA Quality Metrics
Source:https://web.dev/pwa-checklist/#baseline
32. perfecto.io32 | Perfecto by Perforce © 2019 Perforce Software, Inc.
Top 11 PWA Quality Metrics Continued
Source:https://web.dev/pwa-checklist/#baseline
33. perfecto.io33 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Responsive Web App – Test Plan Strategy: 6 Steps
11/16/2020 33
Choose The Right
Platforms
Assure Visual Web
Pages Across
Platforms
Functionality of
Navigation Flows
and Menus
Client-Side
Performance
Testing
Test Accessibility
for Mobile and
Web
Test Environment
Conditions
(Networks,
Sensors)
34. perfecto.io34 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Progressive Web App – Test Plan Strategy: 6 Steps
11/16/2020 34© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Validate PWA
Manifest
Validate PWA
Service Workers
Validate PWA
Specific
Capabilities
Properly Test
Across Platforms
(RWD)
Test Automation
and Object
Identification
Strategy
Google’s PWA
Checklist
Compliance
RWD Testing
Strategy And
35. perfecto.io35 | Perfecto by Perforce © 2019 Perforce Software, Inc.
• Take-Aways:
• PWA is growing and embraced by browser vendors and
enterprises.
• iOS is lagging behind Android capabilities.
• Selenium & Appium (or Visual Testing) is the
recommended test automation strategy for PWA.
• Leverage what you already know and have (RWD,
Selenium, Mobile Testing).
Recommended References
• References:
• Service Workers (W3C) -
https://www.w3.org/TR/service-workers/
• Google Lighthouse - Google Lighthouse - Web Store
• Debugging SW – Google Developers
• Chrome SW Inspector - Local Inspector
• PWA Google YouTube Course - https://bit.ly/2qivQJN
36. 36 | Getting Ready for iOS 13 Testing perfecto.io
Demo — PWA & Appium
38. 38 | How to Guarantee Continuous Value from your Test Automation perfecto.io
Perfecto’s Continuous Testing Platform Overview
39. Perfecto’s Smart Continuous Testing Solution
Smart Execution
Fast and parallel test execution with
multi-team orchestration abilities and
management.
Smart Analytics
"Single pane of glass" provides
visibility and scales to support
millions of test results.
Smart Creation
Automation creation that matches your
team’s skillset (Appium, Espresso,
XCUITest, Quantum BDD, Codeless).
Smart Lab
Always on and stable. Always up
to date. Supports all mobile OS
and platforms.
The Perfecto
human factor
increases your
chances to succeed.
Black Belt
Testing Experts
Training
Dedicated
Success Manager
24/7 VIP
Support
41. 41 | How to Guarantee Continuous Value from your Test Automation perfecto.io
Appendix: Foldable Device Testing
42. perfecto.io42 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Z T E A X O N M H U A W E I M AT E X
L G V 5 0 T H I N Q ( + A C C E S S O R Y ) S A M S U N G G A L A X Y F O L D
$2600
$2000
$725
$1060 $1400
S A M S U N G G A L A X Y Z
Quality Matters More Than Ever
43. perfecto.io43 | Perfecto by Perforce © 2020 Perforce Software, Inc.
A P P C O N T I N U I T Y M U LT I - W I N D O W
• Split View
• Freeform Window
• Multi-Resume
M A I N A N D
C O V E R D I S P L AY S
Important Terms and Features for Foldables
44. perfecto.io44 | Perfecto by Perforce © 2020 Perforce Software, Inc.
A P P S S H O U L D R U N S E A M L E S S L Y W H E N U N F O L D E D A C R O S S
D I F F E R E N T S C R E E N A T T R I B U T E S ( R E S O L U T I O N S E T C . )
App Continuity
45. perfecto.io45 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Android devices
(Above N Below P OS)
Samsung devices
(Above N Below O OS)
Samsung devices
(Above P OS)
Split view
Freeform Window
Multi-Resume
V V V
V V
V
Multi-Window: Capabilities Vary Across OS
46. perfecto.io46 | Perfecto by Perforce © 2020 Perforce Software, Inc.
<meta-data
android:name="android.supports_size_changes" android:value="true" />
Make Your App Resizable
47. perfecto.io47 | Perfecto by Perforce © 2020 Perforce Software, Inc.
S P L I T V I E W 1 S P L I T V I E W 2 - 1 S P L I T V I E W 2 - 2 S P L I T V I E W 3
Multi-Window: Split View (“Open in Split View”)
48. perfecto.io48 | Perfecto by Perforce © 2020 Perforce Software, Inc.
F R E E F O R M W I N D O W 1 F R E E F O R M W I N D O W 2 F R E E F O R M W I N D O W 3 F R E E F O R M W I N D O W 4
Multi-Window: Freeform(“Open Popup View”) View
49. perfecto.io49 | Perfecto by Perforce © 2019 Perforce Software, Inc.
M U L T I - R E S U M E I S E X P E C T E D I S A M A N D A T O R Y B E H A V I O R I N A N D R O I D Q .
R E S U M E D
P A U S E D
P A U S E D
R E S U M E D
R E S U M E D
R E S U M E D
Multi-Window: Multi-Resume View
50. perfecto.io50 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Think About Input Methods
& Responsive/Progressive
Web Testing and Usability
Main and Cover Displays
51. perfecto.io51 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Memory and
battery consumption:
Foldable consumes much
more resources.
Apps, OS , and device
compatibility considerations:
Multi-window functionality,
main/cover operation.
UI and layout testing across
screen sizes, orientation,
landscape/portrait,
and more.
Foldable device support
include changes to onResume
and onPause to support
multi-resume and notify your
app when it has focus.
Key Points to Consider
53. perfecto.io53 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Test Case Category Expected Behavior/Success Criteria
App Launch: Main Display App should open, no crashing, no lag, etc.
App Launch: Cover Display
Working with Keyboard: Main Display Typing text in various languages, symbols, etc. should show up fine on both displays.
Working with Keyboard: Cover Display
Working with Camera: Main Display App opens the camera, and the preview images/videos are scaled and
oriented correctly. The resulting picture taken is scaled and oriented
correctly. Try this in both portrait and landscape mode.
Working with Camera: Cover Display
Transition between main and cover
displays: Screen Layout
After the display is switched, app should be filled in entire full screen.
It should have no blank space and the screen shouldn’t be cut off.
Transition between main and cover display:
App runtime
The display should be switched without app crashing, hanging, or
throwing ANR (app not responding). And, app shouldn’t finish or
be destroyed.
Transition between main and cover display:
UI Components & Location (Weather, Maps, Navigation)
All components (button, text, menu, thumbnails…) should be right location
and maintain same shape as expected. Location values shouldn’t be changed.
Transition between main and cover display:
Screen Drawing Related (Drawing, Handwriting apps)
Try this in drawing/handwriting app. Touch to draw and switch display.
Then, drawing and edited state should be maintained.
Specific Foldable Test Cases To Consider
54. 54 | Getting Ready for iOS 13 Testing perfecto.io
Thank You
Visit perfecto.io or follow us @perfectomobile