iOS 14 is a mature public beta. And yet it is already plagued with quality issues for both developers and testers. The new platform not only introduces major UI/UX changes, but also new capabilities and a completely new line of devices. With features like a redesigned, widget-filled home screen, app clips, a new app library, Mac Catalyst, privacy enhancements, and more, teams need to prepare their apps for iOS 14 by testing on the beta now. From early reviews of the iOS 14, Perfecto and Applitools were able to spot a plethora of severe quality issues, including functional, UI/visual, as well as memory allocation issues. Join Perfecto’s Chief Evangelist and Product Manager, Eran Kinsbruner, together with Applitools Quality Evangelist and Solution Architect, Anand Bagmar, to get a complete overview of what iOS 14 introduces to developers and testers, and how to prevent quality pitfalls ahead of the September release. Key topics covered in the webinar will include: - An overview of iOS 14 features. - Functional and visual testing strategy for new OS version releases. - Test coverage recommendations for iOS and iPadOS across new and legacy versions. - How to combine iOS simulators and real devices in your testing strategy. - How functional and visual testing complement each other. - A live demo of real enterprise app issues with the iOS 14 beta.

1. Perfecto by Perforce © 2020 Perforce Software, Inc.
How to Prepare Your Apps for iOS 14:
Test Strategy, Coverage, & Best Practices

2. 2 | Advanced Codeless Testing for Web Apps
Today’s Speakers:
Eran Kinsbruner
• Chief Evangelist and author at Perfecto
• Blogger and speaker
• 20+ years in development & testing
• Author of “The Digital Quality Handbook”
and “Continuous Testing for DevOps
Professionals”
• @ek121268
• https://www.linkedin.com/in/eran-kinsbruner-4b47a81/
Anand Bagmar
• Software Quality Evangelist at Applitools, blogger, and speaker
• Over 20+ years in software development and testing
• Specializes in product quality strategy & execution, and building
automated testing tools, infrastructure, and frameworks
• Built open source tools related to software testing: WAAT (Web Analytics
Automation Testing Framework), TaaS (for automating the integration
testing in disparate systems) and TTA (Test Trend Analyzer)
• https://in.linkedin.com/in/anandbagmar
• @BagmarAnand

3. perfecto.io3 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Today’s Agenda
1
3
4
Mobile Test Coverage Guidelines
iOS 14 Features Overview and Pitfalls
5
How to Combine Testing on iOS Simulators & Real Devices
Live Demo: iOS 13 vs. iOS 14 Issues (Appium)
6 Q&A
2 Functional & Visual Testing — Recommended Strategy

4. iOS 14: Key Features Overview

5. perfecto.io5 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• New UI Enhancements
• Widgets
• App Library
• Messages
• Notifications
• New translation app
• Enhanced Siri
• New Safari browser (Safari 14)
• Car Key/Car Play
What’s New in iOS 14

6. perfecto.io6 | Perfecto by Perforce © 2020 Perforce Software, Inc.
What’s New in iOS 14 — Continued
• App Clips load within seconds and let users
complete specific tasks, like pay for parking using
Sign In With Apple and Apple Pay. App Clips can be
discovered and accessed by scanning Apple's new
"App Clip codes," or by using NFC and QR codes.
They can also be shared in Messages and Safari.

7. perfecto.io7 | Perfecto by Perforce © 2020 Perforce Software, Inc.
iOS 14 Collected Pitfalls: Functional, Memory, UI Issues
Works on iOS 13,
Crashes on iOS 14
Works on iOS 13,
Crashes on iOS 14

8. perfecto.io8 | Perfecto by Perforce © 2020 Perforce Software, Inc.
iOS 14 Collected Pitfalls: Functional, Memory, UI Issues
Looks Good on iOS 13 Text Field Truncated on iOS 14

9. Functional & Visual Strategy

10. perfecto.io10 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• Quality Strategy = Testing + Automation + App Release Plan
Quality Strategy

11. perfecto.io11 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Quality Strategy

12. perfecto.io12 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• Build in quality by testing early
• Feature kick-off
• Business requirements
• Story kick-off
• User impact
• Dev-Box testing
• Implement what was expected
• Thorough story testing & exploratory testing
• Test was was implemented
• Product owner sign-off per story & feature
• Involve everyone, quality is a team responsibility
Structured Way of Working

13. perfecto.io13 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• Automate as much as you can (what brings relevant value)
• Unit tests
• Integration tests
• Functional tests (E2E) — focused on user-journeys
• Visual testing — user experience should be consistent, and as expected
• Analytics tests
Automation Is Key

14. perfecto.io14 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Value of Automation
Run the tests on every change — capture the
regression as soon as it is introduced in the code.

15. perfecto.io15 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Simulate real-user behavior/interactions.
Run the tests on simulators & real devices.

16. perfecto.io16 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• In-house testing
• On-field testing (sanity)
• Staged releases
• Sentiment analysis
• Review comments (PlayStore/App Store)
• Social media (Twitter, Instagram, Facebook, etc.)
• Extensive monitoring of staged releases
• Analytics
• Crashes
• Support tickets
Release Approach

17. perfecto.io17 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Does your strategy change if a new version
of OS needs to be supported?

18. perfecto.io18 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Eran gave plenty of examples why it is necessary to keep the variables in mind!
YES!

19. perfecto.io19 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• Validate compatibility of your product with the new iOS version, and quickly!
Key aspects to keep in mind in case of iOS 14.

20. perfecto.io20 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• Functionality/gestures
• UI/UX
• Memory
• New installations vs. existing installation vs. upgrade app versions
• Integrations/permissions
• Caching/data storage
Challenges You May Face

21. perfecto.io21 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• Executing automated tests
• Unit tests
• Integration tests
• Functional tests (E2E) — focused on user-journeys
• Visual testing — user experience should be consistent and as expected
• Exploratory testing — finding impact of iOS version change on your app
Strategy for iOS Version Change

22. Test Automation Coverage Guidelines

23. perfecto.io23 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Balancing Test Automation Coverage
TEST SCENARIO ONE
Test Case – 1 Test Case – 2 Test Case – 3 Test Case – N
2 sides of the same coin

24. perfecto.io24 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Test Coverage Fundamentals
Web Traffic Analytics
Selecting the Right Test Cases

25. perfecto.io25 | Perfecto by Perforce © 2020 Perforce Software, Inc.
iOS and iPadOS Testing Coverage

26. perfecto.io26 | Perfecto by Perforce © 2020 Perforce Software, Inc.
• iOS 13 (Latest–1) testing required on
• iPhone 11 series
• iPhone 6 and below
• iPad Air/Pro Series
• iOS 12.4.8 (Latest –2)
• iPhone 6
• iPad Mini 2
iOS and iPadOS Testing Coverage: GA, Latest-1, Latest-2

27. perfecto.io27 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Latest GA Versions:
• Chrome 84
• Firefox 79
• Edge 84
• Safari 13
• Safari 14 Beta*
Global Browser Market Share Update: Adoption Trends Vary

28. perfecto.io28 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Mobile Testing & Desktop Web Testing Are One
iOS Safari
Android Chrome
Desktop Web UI

29. Balancing Virtual Devices and Real Devices

30. perfecto.io30 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Virtual Platform Definitions
• Simulators
• iOS (Apple)
• A simulator is software that imitates the workings of
software on your system that was made for mobile.
• Mobile simulators can be used in app testing and cross-
browser or responsive testing to simulate a mobile device
and running your app or website in it.
• Emulators
• Android (Google)
• An emulator is software that imitates the working of
software on your system that was made for mobile.
• Emulators also take into consideration the hardware part
of the mobile device (memory, processor, battery
performance, etc.).
Real Device Behavior

31. perfecto.io31 | Perfecto by Perforce © 2019 Perforce Software, Inc.
Differences Between Android & iOS Virtual Platforms

32. perfecto.io32 | Perfecto by Perforce © 2019 Perforce Software, Inc.
Considerations
for Using iOS and
Android Virtual
Platforms vs. Real
Devices
Stage and activity in
the software
development cycle
Dev
Test
Debug
Design
Cost avoidance
Coverage
Platforms
Real user
conditions
Test types
Functional/Regression
/Integration Testing
Unit
Non Functional
Production Monitoring

33. perfecto.io33 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Perfecto’s Mobile Testing Pyramid Combo
Testing Pyramid Within Your DevOps Pipeline Requirements
U N I T T E S T S
M A N UA L
T E S T S
Testing isolated units of code.
I N T EG R AT I O N T E S T S
Testing across integrations of more than one system.
U I T E S T S
Testing through a user interface.
Per code commit/ Post commit
Build Acceptance/Regression Testing
E2E Testing
Exploratory/Manual Testing
Dev
Triggered
Mix Real Devices & Emu./Simu.
CI
Triggered+
Real Devices Only
CI
Triggered
Real Devices OnlyProd. Monitoring
Non Functional Testing:
Performance (UX)
Testing/Security
Real Devices Only

34. perfecto.io34 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Virtual Platforms
Mix of
Virtual Platforms & Real Devices
Only Real Devices
Pipeline Testing Coverage Example

35. Demos: Cross Platform Testing With Perfecto and Applitools

36. perfecto.io36 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Continuous Testing – Key Metrics/Categories

37. perfecto.io37 | Perfecto by Perforce © 2020 Perforce Software, Inc.
Perfecto’s 4 Key Pillars of Continuous Testing in DevOps
This is where money is spent (Time, Tools, Resources)
This is where value is being
realized, quality is improved
Unified solution serves DevOps with greater effectiveness.

38. perfecto.io38 | Perfecto by Perforce © 2020 Perforce Software, Inc.
We Invented Visual AI that replicates the human eye and brain, looks at every screen
and page in seconds, and spots app regressions that truly matter.
We Help App Teams with their:
• Functional Testing & Visual Testing
• Web, Mobile, and UI/UX Testing
• Cross Browser & Device Testing
• Content Migration/OS version upgrades/browser version upgrades
• Compliance Testing, Localization, Accessibility, PDF, & More
Applitools Visual AI has analyzed one billion+ images with 99.9999% accuracy.
About Applitools

39. Visual AI
Industry’s only AI powered computer vision
replicates the human eye & brain.
SDKs & Integrations
SDKs to augment all modern test frameworks,
and integrations with popular source control,
CI, and defect tracking systems.
Perfection Across All Screens
Ultrafast Grid ensures visual perfection across
all browsers, screens, and viewports.
Ultrafast Grid Architecture >>
Speed of CI/CD
Execute tests in seconds vs minutes for faster
builds and on-time delivery.
Ultimate Efficiency
Create tests faster & with fewer brittle locators
and labels.
Increase Test Coverage
Open-ended, AI-powered assertions reduce
code and maintenance while increasing test
coverage.
Applitools Cloud
Visual AI Platform
One Intelligent Platform
Adding Applitools Visual AI to your existing test automation framework allows quality teams to release
perfect apps faster and at a reduced cost.

40. perfecto.io40 | Perfecto by Perforce © 2020 Perforce Software, Inc.

41. Thank You!
perfecto.io or follow us @perfectomobile
applitools.com / @applitools