In this webinar, we’ll look at the current state of cross-browser test automation tools and breakdown the top 10 frameworks in the market. Get an overview of the most downloaded tools, pros and cons of each and best practices for matching tools to your technical requirements. You will learn: -Why JavaScript rules the test framework world in 2017 -How to choose the best tools for your project -How USAA and Panera Bread plan their web testing practices We'll end with live demos Selenium, Protractor and Nightwatch.JS.

1. 10 Emerging Test Frameworks
for Cross-Browser Testing

2. Housekeeping
• Please participate in our live poll and end-of-webinar survey
• Ask us questions in the QA panel
• We will send you the recording and slides
9/20/2017 2© 2015, Perfecto Mobile Ltd. All Rights Reserved.

3. Uzi Eilon
CTO, Perfecto
Eran Kinsbruner
Dir., Product Evangelist
Author: The Digital Quality Handbook
Agenda
How to select the right test automation framework?
Cross browser testing landscape - overview
USAA tool selection strategy
Technical walkthrough demo – Panera RWD Angular site automation
Protractor Integration with Perfecto
Q & A
Bryan Osterkamp
Lead Technical Architect, USAA
Chaley Stanfield
Sr. Analyst, Test Engineer, Panera

4. How to select the right testing
framework?
9/20/2017 4© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Organizational Fit Technical Fit

5. Selection Criteria: Organizational Fit Requirements
9/20/2017 5© 2016, Perfecto Mobile Ltd. All Rights Reserved.
Project
Complexity
Quality
analysis
requirements
Test Types
&
Practices
Resources and
Skills
Availability
Test
Environment
Existing Tech
Stack

6. Selection Criteria: Technical Fit Requirements
9/20/2017 6© 2016, Perfecto Mobile Ltd. All Rights Reserved.
SDLC Process
Fit
(Integrations,
Plugins, etc.)
Community
size, support
and Doc’s
Feedback
Loop and
Reporting
Automation
Coverage
Cloud and
Automation
at Scale
Automation
Robustness and
Maintainability

7. Poll Question
9/20/2017 7© 2015, Perfecto Mobile Ltd. All Rights Reserved.

8. Web Testing Tools - Download Trends
9/20/2017 8© 2016, Perfecto Mobile Ltd. All Rights Reserved.

9. Web Market Testing Landscape
9/20/2017 9© 2016, Perfecto Mobile Ltd. All Rights Reserved.

10. The Biggest Diffs
9/20/2017 10© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Real Browser Testing Headless Browsers
• Interact with a standard browser
• Pros:
• Match your users platforms
• Easy to navigate and control objects
• Cons:
• Slow
• Complicated setup
• Uses Lightweight browser like SW
• Pros:
• Fast and easy to run
• Easy CI setup
• Cons
• Doesn’t fit UI testing
• Limited test coverage capabilities
• Cannot cover diffs between browsers
• Elements management too complex

11. Grouping The Tools
9/20/2017 11© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Real Browser (WebDriver) Headless
Headless browser SW
(can used as standalone)
Used for:
• End to end testing
• Functional testing
Used for:
• Fast CI – Short cycle
• Basic functional testing
• Backend services testing
• Performance / network traffic

12. Headless Example
9/20/2017 12© 2015, Perfecto Mobile Ltd. All Rights Reserved.
The Script
Connect to the Phantom
Use Cases:
• Har
• Java/Selenium
• Join script (headless
validation)

13. WebDriver Example
9/20/2017 13© 2015, Perfecto Mobile Ltd. All Rights Reserved.
The Script:
Connect to the browser

14. WebDriver Grid
9/20/2017 14© 2015, Perfecto Mobile Ltd. All Rights Reserved.
The Script:
Connect to a grid

15. USAA- Bryan Osterkamp
© 2015, Perfecto Mobile Ltd. All Rights Reserved.

16. USAA Tool Selection
• Define needed capabilities
9/20/2017 16© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Selection Criteria
End to End Testing
BDD/ATDD Friendly
Tool Documentation
Visual Navigation Testing

17. USAA Tool Selection
• Define needed capabilities
• Identify importance (weight capabilities)
9/20/2017 17© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Weight Key
1 – Low importance
3 – Medium importance
5 – High importance
Weight Selection Criteria
5 (High importance) End to End Testing
3 (Medium importance) BDD/ATDD Friendly
5 (High importance) Tool Documentation
1 (Low importance) Visual Navigation Testing

18. USAA Tool Selection
• Define needed capabilities
• Identify importance (weight capabilities)
• Define scoring key
9/20/2017 18© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Scoring Key
0 – Did not meet expectations
2 – Met expectations
3 – Exceeded expectations
Weight Selection Criteria Tool X Tool Y Tool Z
5 (High importance) End to End Testing 3 3 3
3 (Medium importance) BDD/ATDD Friendly 3 2 3
5 (High importance) Tool Documentation 0 2 2
1 (Low importance) Visual Navigation Testing 3 3 2

19. USAA Tool Selection
• Define needed capabilities
• Identify importance (weight capabilities)
• Define scoring key
9/20/2017 19© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Weight Selection Criteria Tool X (weighted) Tool Y (weighted) Tool Z (weighted)
5 (High importance) End to End Testing 3 5 x 3 = 15 3 5 x 3 = 15 3 5 x 3 = 15
3 (Medium importance) BDD/ATDD Friendly 3 3 x 3 = 9 2 3 x 2 = 6 3 3 x 3 = 9
5 (High importance) Tool Documentation 0 5 x 0 = 0 2 5 x 2 = 10 2 5 x 2 = 10
1 (Low importance) Visual Navigation Testing 3 1 x 3 = 3 3 1 x3 = 3 2 1 x 2 = 2
Total 27 34 36
Scoring Key
0 – Did not meet expectations
2 – Met expectations
3 – Exceeded expectations

20. Infrastructure as a
Freeway
9/20/2017 20© 2015, Perfecto Mobile Ltd. All Rights Reserved.

21. Panera Bread - Chaley
Stanfield
© 2015, Perfecto Mobile Ltd. All Rights Reserved.

22. Panera Web Testing
Overview
• What is our methodology/strategy?
• TDD
• Agile methodology
• Short sprints
• Planning that involves the full team
• Early delivery
• 2-3 days development, testing starts by 2nd
or 3rd day
• Testing for new functionality wrapped up 2-
3 days before end of sprint to allow time for
bug fixes and regression testing
• Allows for early bug finds and fast
turnaround on fixes
• Short sprints allow teams to assess
processes and adapt quickly
9/20/2017 22© 2015, Perfecto Mobile Ltd. All Rights Reserved.

23. Panera Web Testing
Overview
• How do we decide what to automate?
• Best candidates are:
• Unchanging
• Tested repeatedly
• ‘Tried and true’ functionality
• Our best candidates are regression testing
and data testing
• We normally do not script for new pieces
of functionality until they have been live at
least a couple sprints
• We look to QA for input on what has the
most ROI
9/20/2017 23© 2015, Perfecto Mobile Ltd. All Rights Reserved.

24. Panera Web Testing
Overview
• What were our requirements for an automation framework?
• AngularJS-friendly
• Used for integration and e2e testing
• Support for desktop and MR apps
• Ample, easy-to-understand documentation
• QA-friendly reporting
• Easy to configure
• Multi-threaded testing
• Supports POM
• Similar to Selenium WebDriver (original automation framework)
9/20/2017 24© 2015, Perfecto Mobile Ltd. All Rights Reserved.

25. Panera Web Testing
Overview
• Test Structure (Jasmine)
• conf.js file
• Environments (baseUrl)
• Selenium Address
• Local
• Perfecto
• Suites
• Reporters
• Jasmine HTML Report
• Perfecto Report
• Multicapabilities
• Browser
• Platform (for Perfecto)
• Multi-threaded testing
9/20/2017 25© 2015, Perfecto Mobile Ltd. All Rights Reserved.
• appPage.page.obj.js files
• Locates Web Elements
• appPage.js files
• Logic
• Acts on Web Elements found in
appPage.page.obj.js files
• Assertions
• testSuite.spec.js files
• Tests
• Pieces together methods in appPage.js files to
form a coherent test flow
• Broken down by functionality

26. Panera Web Testing
Overview
• Challenges
• Maintenance
• Can’t check third party
services/our BOH
• Browser updates
• Difficult to change
browser versions on the
fly
• Difficult to run on multiple
platforms/devices
• Making automation
executable by anyone on
the team
9/20/2017 26© 2015, Perfecto Mobile Ltd. All Rights Reserved.

27. Demo
• Panera website testing using Protractor

28. Giveaway for Webinar Attendees/ Q&A

29. Thank You