In this SauceCon 2019 presentation, Samantha Coffman, Product Manager at Sauce Labs, and Alissa Lydon, Product Marketing Manager at Sauce Labs, discuss Sauce Headless, the industry’s first cloud-based headless testing solution, making it a fast and affordable option for early pipeline testing at scale. They will cover the benefits of using Sauce Headless in conjunction with Sauce Labs traditional VM offering to increase the release velocity, as well as examples of customers who are experiencing upwards of 50% faster test times using Sauce Headless.

1. Automated Testing at the
Speed of Headless
A New Sauce Labs Solution
Alissa Lydon, Sauce Labs
Samantha Coffman, Sauce Labs

2. Agenda
1. Outline the market trends that have lead to
rise of headless testing
2. Discuss best practices for integrating
headless into your testing strategy
3. Introduce Sauce Headless

3. Software Development Is Evolving
And it has changed the way it is tested!

4. Trends That are Accelerating Shift Left
Processes
DevOps → Developers Testing More
React and Angular → Future of Web
Chrome + Firefox as Best First Checks
Tools
JS Frameworks for WebDriver
Components → Integration Testing
Lighter Infrastructure Options

5. Not All Tests are Created Equal
Production
E2E TEST
•Regression
testing
DEPLOY
•Smoke test
•User acceptance
testing
OPERATE
•Production
testing
•Debugging
•Smoke testing
Local Dev Environments
PLAN
•Feature definition
•Test authoring
•TDD
CODE
•Unit test
•Component test
Shared Delivery Pipeline
POST COMMIT
•Component
Tests
•UI Tests
•Sanity Checks
MERGE
New Challenges
● Cultural shift
● Sharing resources
● Context switching
INTEGRATE
•Integration tests

6. We Know This Challenge
• Because Sauce had it
• We needed a way to reliably ship our code faster
• So we built the first iteration of Sauce Headless
• Developers get test feedback much earlier
• As our architecture evolves and development velocity increases,
customers are asking for the same

7. Early pipeline testing means developers need infrastructure that is lightweight, fast, and gives
them quick feedback on the quality of their code
Headless Chrome + Firefox
• Quick feedback for developers
• Accelerate code velocity
Pass / Fail Data
• Complement to full cross browser testing
Affordable Infrastructure (Containers in the Cloud)
• Lower Price Point
• Allows for broad adoption across development teams
Sauce Headless… An Introduction

8. What is a Headless Browser?
Vs.
VM GUI BROWSER

9. Sauce Headless to the Rescue
Production
E2E TEST
•Regression
testing
DEPLOY
•Smoke test
•User acceptance
testing
OPERATE
•Production
testing
•Debugging
•Smoke testing
Local Dev Environments
PLAN
•Feature definition
•Test authoring
•TDD
CODE
•Unit test
•Component test
Shared Delivery Pipeline
POST COMMIT
•Component
Tests
•UI Tests
•Sanity Checks
MERGE
INTEGRATE
•Integration tests

10. Integrate Headless into your Testing Strategy
Which Tests?
Atomic, autonomous, short tests
Parallelize well and quick feedback
Integration test for components
Where in your pipeline?
Every PR in shared dev env.
Sanity check
Scalable infrastructure for “Shift Left”

11. Go Headless with Sauce
Headless Testing on Sauce Labs
Increase build efficiency
● Run early pipeline browser tests quickly
● Improve developer efficiency with faster feedback
Single unified testing experience
● Run your existing Selenium tests on both platforms
● Ability to run tests on either headless for quick feedback,
or cross browser for full debugging capabilities
Security
● With the ability to run through our secure Sauce
Connect tunnel, Sauce Headless is the most secure
solution on the market today
Reduce cost to support and maintain
● Blending headless with cross browser
● Increase testing coverage in overall
development pipeline
● No maintenance, no setup, no cloud to manage

12. Stories From the Beta
“We were able to reduce our UI regression run
from 5 hours to 30 minutes. Such short
feedback cycles are crucial to advance our
continuous delivery pipeline and catch bugs
as early as possible!”
“I have a suite of 6 tests... that runs in less
than 200ms! That is magical. I never
thought Sauce would be this fast.”
“Using Sauce Headless reduced the run
time of a subset of our tests (~110) from 30
mins to 5 mins.”
“Man this platform is awesome, I can run
so many tests so fast! Super stoked!”

13. The Complete Continuous Testing Pipeline
Production
E2E TEST
•Regression
testing
•Performance
testing
DEPLOY
•Smoke test
•User acceptance
testing
•Visual Testing
OPERATE
•Production
testing
•Debugging
•Smoke testing
Local Dev Environments
PLAN
•Feature definition
•Test authoring
•TDD
CODE
•Unit test
•Component test
Shared Delivery Pipeline
POST COMMIT
•Component
Tests
•UI Tests
•Sanity Checks
MERGE
INTEGRATE
•Integration tests
The Sauce Experience
Debugging SecureMost
Reliable
Experience Visual Testing Performance Testing

14. Want to See Headless in Action?
● Stop by the demo theater today
○ 3:00-3:30 in Zilker Terrace Foyer
● Contact your Account Manager to get started
Alissa Lydon
alydon@saucelabs.com
@habes06
Sam Coffman
sam.coffman@saucelabs.com
@samanthacoffma7

15. Thank you.

18. Sauce Headless to the Rescue!
MERGE
E2E TEST
•API testing
•Regression
testing
•Performance
testing
INTEGRATE
•Integration tests
DEPLOY
•Smoke test
•Security testing
•User acceptance
testing
•Visual Testing
OPERATE
•Production
testing
•Monitoring
•Debugging
•Smoke testing
PLAN
•Feature definition
•Test authoring
•TDD
CODE
•Unit test
•Component test
POST COMMIT
•Component
Tests
•UI Tests
•Sanity Checks
Shared Delivery Pipeline Production
RELEASE
Local Dev Environments
RELEASE

19. Not All Tests are Created Equal
Production
E2E TEST
•API testing
•Regression
testing
•Performance
testing
RELEASE
DEPLOY
•Smoke test
•Security testing
•User acceptance
testing
•Visual Testing
OPERATE
•Production
testing
•Monitoring
•Debugging
•Smoke testing
Local Dev Environments
PLAN
•Feature definition
•Test authoring
•TDD
CODE
•Unit test
•Component test
Shared Delivery Pipeline
POST COMMIT
•Component
Tests
•UI Tests
•Sanity Checks
MERGE
New Problems
● Cultural shift - Developers aren’t used to writing E2E tests
● Workflow / sharing resources between QA and Dev
● Context switching is detrimental to productivity
INTEGRATE
•Integration tests

20. We Know This Challenge
• Because Sauce had it
• We needed a way to reliably ship our code faster
• So we built the first iteration of Sauce Headless
• Developers get test feedback much earlier
• As our architecture evolves and development velocity increases,
customers are asking for the same

21. The Sauce Experience
MERGE
E2E TEST
•API testing
•Regression
testing
•Performance
testing
INTEGRATE
•Integration tests
DEPLOY
•Smoke test
•Security testing
•User acceptance
testing
•Visual Testing
OPERATE
•Production
testing
•Monitoring
•Debugging
•Smoke testing
PLAN
•Feature definition
•Test authoring
•TDD
CODE
•Unit test
POST COMMIT
•Component
Tests
•UI Tests
•Sanity Checks
Shared Delivery Pipeline Production
RELEASE
Local Dev Environments
Debugging SecureMost
Reliable
Experience
The Complete Continuous Testing Pipeline
Visual
Testing
Performance
Testing