TEST.ALLTHETHINGS() – DIGITAL EDITION Asaf Saar – Director of Product Management, Sauce Labs Testing responsive web sites is a challenging task when it span across Desktop and Mobile browsers. In this session Neil and Asaf will present the strategy and framework around testing responsive web sites using industry standard automation tools like Selenium and Appium against cloud based desktop browsers, real mobile devices, emulators and simulators.

1. test.allTheThings(): Digital Edition
Asaf Saar, Director of Product Management
Neil Manvar, Solutions Architect

2. • Director, Product Management, RDC @
Sauce
• 20+ years in software engineering
– Chief Product Owner at Perfecto Mobile
– Founder and CEO of TenKod
– Area Product Owner of Software
Engineering at SAP
– Agile Software Engineering trainer, coach
& mentor
• 4 Patents in the areas of Test Automation
and CI
About Me
Asaf Saar
asaf@saucelabs.com
@saarasaf

3. • Solutions Architect
• Front End Software Engineer @ Yahoo
• Bachelors in CSE @ UC Davis
About Neil
Neil Manvar
neil@saucelabs.com

4. The World is DIGITAL

5. Mobile Apps Usage Growth
• Mobile apps account for 58% of total time spent consuming digital
media
Source: The 2016 Mobile App Report, ComScore
42%
52%
54%
58%

6. We Live in a World of Screens
Day by day, the number of devices, platforms, and
browsers that need to work with your site grows.
Jeffrey Veen
The Brand Mapping Strategy: Design, Build, and Accelerate Your Brand

7. Share of Web Traffic by Device
Source: Digital in 2017: Global Review
More than half of the world’s web traffic now comes from mobile devices.
But it doesn’t mean computer and tablet access has dropped…
45%
Year-On-Year Change:
-20%
Laptops &
Desktops
50%
+30%
Year-On-Year Change:
Mobile
Phones
5%
Year-On-Year Change:
-5%
Tablet
Devices

8. Web or Native?
Web or Native:
How to respond?

9. Web or Native?
• How important are speed and performance?
• Would you like your app to include any device-specific features?
• Do you want your app to support multiple mobile platforms and devices?
• Budget or time to market?
• Are app stores and marketplaces important?
• Would you like to monetize your app in the future?

10. EMU/SIM or Real Devices?
Emulators/Simulators, or
Real Devices?

11. EMU/SIM or Real Devices?
Device Testing (on Wi-Fi
Networks + Networks
Simulation Tools)
● Use for stable features
● Distribute across device models / form factors
● Important for UX Testing
● Combine with WANem or similar tools for
simulating network connectivity conditions
● Only for network-dependent use cases
● High volume functional tests for large builds
● Frequent builds with high concurrency
Device
Testing
(in network)
Device Testing (on Wi-Fi
Networks + Networks
Simulation Tools)
Device
Testing
(in network)
Device Testing
(on Wi-Fi
Networks + Networks
Simulation Tools)
Emulator & Simulator
Testing

12. EMU/SIM or Real Devices?
Use Real Devices:
● Panel/compatibility testing
● Replicate an issue to match exact model
● Pixel perfect display testing
● Hardware dependencies
● Custom OS
Use EMU/SIM:
● Save costs
● Extremely low error rates
Device Testing (on Wi-Fi
Networks + Networks
Simulation Tools)
Device
Testing
(in network)
Device Testing
(on Wi-Fi
Networks + Networks
Simulation Tools)
Emulator & Simulator
Testing

13. What’s the Testing Strategy?

14. Too Much Stuff!
Web and Native.
Desktops, Emulators, Simulators
and Real Devices.
How do we deal with all of these?

15. test.allTheThings() - Mobile Edition
One
Testing Framework
To Rule Them All!
https://github.com/SauceCon2017-demo/allTheThingsDigital

16. Demo

17. Questions?