Automate More With
Selenium your RWD Site
Housekeeping
• Please participate in our live poll and end-of-webinar survey
• Ask us questions in the QA panel
• We will send you the recording and slides
6/23/2017 2© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Uzi Eilon
CTO America
Eran Kinsbruner
Dir., Product Evangelist
@ek121268
http://mobiletestingblog.com
Author: The Digital Quality Handbook
Agenda
• Responsive Web Testing – Key Automation Challenges
• Creating a script for responsive web site
• Understanding The Importance of ‘Correct’ XPATH usage
• Advance RWD Automation Techniques
• From automation script to CI
Drivers & Benefits To Moving to RWD
6/23/2017 4© 2016, Perfecto Mobile Ltd. All Rights Reserved.
Provide consistent DX
and coverage across
all digital platforms
Support end user
work flow across multi
screens
Consolidate
resources & align
business goals
across platforms
Be ready for any new
OS/Platform release
faster
Mobile & Web Test Automation Frameworks Cannot Cover
These Test Cases
Additional Challenges
 How to include visual testing flows with Selenium/Appium?
 Image injections as an example
 None of the listed frameworks can cover such use cases for mobile and web
 Onboard sensors
 Ability to automate TouchID test scenario’s
 How to assess Audio and Video quality (Siri/Google Assistant)?
 None of the listed frameworks can cover such use cases
 Context
 Full system level control (out of the app testing)
 Appium is limited, Espresso & XCTest cannot cover such cases
 Mimic environment
 Testing environment conditions mimicking
 Localization testing for mobile and web testing
 None of the listed frameworks can cover such use cases
 Reporting
 Obtain Side by side reporting and dashboards
 Get production insights of user experience
 None of the listed frameworks provide reach reporting dashboards
 Lab
 Automatically retrieve HAR file and other network capture logs
 None of the listed frameworks can cover such use cases
 Same day support for devices, browsers including beta’s (see below visual)
 These frameworks are highly dependent on 3rd party and cannot commit to immediate support
 Responsiveness
 How to measure UX timers and app performance?
 None of the listed frameworks can cover such use cases
 Reporting
 Obtain Side by side reporting and dashboards
 Get production insights of user experience
 None of the listed frameworks provide reach reporting dashboards
Building Blocks for RWD Testing
6/23/2017 6© 2016, Perfecto Mobile Ltd. All Rights Reserved.
Poll Question
What’s Your Main Challenge In Automating Your RWD Site?
• Available Lab with all permutations on-demand?
• Dealing with flaky tests across platforms (object/test stability)?
• Drilling into RWD specific issues toward fast MTTR?
• Limitation of test framework capabilities leaves too much manual tests?
6/23/2017 7© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Digital Insurance Key Use Cases and Challenges
• Claim submission and processing
• A lot of forms filled across platforms and screens
• Demographics based (Age, Income, Location, Personal Information, etc.)
• Voice assistant customer support
• Location services to match customers geo location
• Visual based cases – upload images as part of claims, user profiles, and more
• Chatbots to enhance customer engagement
• Data processing – retrieve personal records and claims
• Network conditions – Varying network channels (Wi-Fi/Carrier) as part of user journey
• Mobile, Web and IOT are the channels of choice for insurance customers
Automation Challenges
Examples
6/23/2017 9© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Cross Platform
Experiences
6/23/2017 10© 2015, Perfecto Mobile Ltd. All Rights Reserved.
VS
Same code = Same test Code != results
Pillars for Successful Digital Transformation
6/23/2017 11© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Digital Transformation
Digital Winner
4 Pillars
of Success
Value Drivers
Tools * People * Processes
Quality Time to Market Cost
Reduce Risk in
Production
Digital Coverage Automation Fast Feedback
Embedded in
Dev-QA-Ops
Prioritized Test Planning for RWD
6/23/2017 12© 2017,Perfecto Mobile Ltd. All Rights Reserved.
Pillars for Successful Digital Transformation
6/23/2017 13© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Digital Transformation
Digital Winner
4 Pillars
of Success
Value Drivers
Tools * People * Processes
Quality Time to Market Cost
Reduce Risk in
Production
Digital Coverage Automation Fast Feedback
Embedded in
Dev-QA-Ops
Automation Guidelines
6/23/2017 14© 2016, Perfecto Mobile Ltd. All Rights Reserved.
• Easy to create
• Object Recorder
• Language (Java /BDD)
• Easy to maintain
• Object repository
• POM
• Tips
• Runs 10 times on the same device
• Runs on 3 deferent devices
• Handle setup (install app / clear cash for
web)
Demo 1
• Go through an insurance RWD flow manually (5 forms , 10 fields each)
• Identify the objects
• Object Repository & POM
XPATH Validation Examples
/html[1]/body[1]/div[8]/div[11]/div[1]/button[1]
//button[text()='OK']
XPATH Validator
6/23/2017 17© 2015, Perfecto Mobile Ltd. All Rights Reserved.
XPATH Validator: http://xpathvalidator.projectquantum.io/
An efficient way to assure robust objects used as
Part of your test code – Flaky Test Avoidance
Pillars for Successful Digital Transformation
6/23/2017 18© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Digital Transformation
Digital Winner
4 Pillars
of Success
Value Drivers
Tools * People * Processes
Quality Time to Market Cost
Reduce Risk in
Production
Digital Coverage Automation Fast Feedback
Embedded in
Dev-QA-Ops
Unattended Automation Guidelines & Fast Feedback
6/23/2017 19© 2016, Perfecto Mobile Ltd. All Rights Reserved.
• Execution properties
• Parallel execution
• Dynamic device list
• Tags managements
• Execution management
• Three strikes out
• Retry for error
• Execution Environments
• Devices availability
• Devices Q
• Customers based env. (wind tunnel)
Demo 2
• Parallel execution
• Connect to CI
• Reports
Pillars for Successful Digital Transformation
6/23/2017 21© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Digital Transformation
Digital Winner
4 Pillars
of Success
Value Drivers
Tools * People * Processes
Quality Time to Market Cost
Reduce Risk in
Production
Digital Coverage Automation Fast Feedback
Embedded in
Dev-QA-Ops
Demo 3
• HAR
• Dev tunnel / debug on any device !!!!
Giveaway for Webinar Attendees/ Q&A
Chapter 12 Free
Download
Thank You

Automate More with Selenium for your RWD

  • 1.
  • 2.
    Housekeeping • Please participatein our live poll and end-of-webinar survey • Ask us questions in the QA panel • We will send you the recording and slides 6/23/2017 2© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 3.
    Uzi Eilon CTO America EranKinsbruner Dir., Product Evangelist @ek121268 http://mobiletestingblog.com Author: The Digital Quality Handbook Agenda • Responsive Web Testing – Key Automation Challenges • Creating a script for responsive web site • Understanding The Importance of ‘Correct’ XPATH usage • Advance RWD Automation Techniques • From automation script to CI
  • 4.
    Drivers & BenefitsTo Moving to RWD 6/23/2017 4© 2016, Perfecto Mobile Ltd. All Rights Reserved. Provide consistent DX and coverage across all digital platforms Support end user work flow across multi screens Consolidate resources & align business goals across platforms Be ready for any new OS/Platform release faster
  • 5.
    Mobile & WebTest Automation Frameworks Cannot Cover These Test Cases Additional Challenges  How to include visual testing flows with Selenium/Appium?  Image injections as an example  None of the listed frameworks can cover such use cases for mobile and web  Onboard sensors  Ability to automate TouchID test scenario’s  How to assess Audio and Video quality (Siri/Google Assistant)?  None of the listed frameworks can cover such use cases  Context  Full system level control (out of the app testing)  Appium is limited, Espresso & XCTest cannot cover such cases  Mimic environment  Testing environment conditions mimicking  Localization testing for mobile and web testing  None of the listed frameworks can cover such use cases  Reporting  Obtain Side by side reporting and dashboards  Get production insights of user experience  None of the listed frameworks provide reach reporting dashboards  Lab  Automatically retrieve HAR file and other network capture logs  None of the listed frameworks can cover such use cases  Same day support for devices, browsers including beta’s (see below visual)  These frameworks are highly dependent on 3rd party and cannot commit to immediate support  Responsiveness  How to measure UX timers and app performance?  None of the listed frameworks can cover such use cases  Reporting  Obtain Side by side reporting and dashboards  Get production insights of user experience  None of the listed frameworks provide reach reporting dashboards
  • 6.
    Building Blocks forRWD Testing 6/23/2017 6© 2016, Perfecto Mobile Ltd. All Rights Reserved.
  • 7.
    Poll Question What’s YourMain Challenge In Automating Your RWD Site? • Available Lab with all permutations on-demand? • Dealing with flaky tests across platforms (object/test stability)? • Drilling into RWD specific issues toward fast MTTR? • Limitation of test framework capabilities leaves too much manual tests? 6/23/2017 7© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 8.
    Digital Insurance KeyUse Cases and Challenges • Claim submission and processing • A lot of forms filled across platforms and screens • Demographics based (Age, Income, Location, Personal Information, etc.) • Voice assistant customer support • Location services to match customers geo location • Visual based cases – upload images as part of claims, user profiles, and more • Chatbots to enhance customer engagement • Data processing – retrieve personal records and claims • Network conditions – Varying network channels (Wi-Fi/Carrier) as part of user journey • Mobile, Web and IOT are the channels of choice for insurance customers
  • 9.
    Automation Challenges Examples 6/23/2017 9©2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 10.
    Cross Platform Experiences 6/23/2017 10©2015, Perfecto Mobile Ltd. All Rights Reserved. VS Same code = Same test Code != results
  • 11.
    Pillars for SuccessfulDigital Transformation 6/23/2017 11© 2015, Perfecto Mobile Ltd. All Rights Reserved. Digital Transformation Digital Winner 4 Pillars of Success Value Drivers Tools * People * Processes Quality Time to Market Cost Reduce Risk in Production Digital Coverage Automation Fast Feedback Embedded in Dev-QA-Ops
  • 12.
    Prioritized Test Planningfor RWD 6/23/2017 12© 2017,Perfecto Mobile Ltd. All Rights Reserved.
  • 13.
    Pillars for SuccessfulDigital Transformation 6/23/2017 13© 2015, Perfecto Mobile Ltd. All Rights Reserved. Digital Transformation Digital Winner 4 Pillars of Success Value Drivers Tools * People * Processes Quality Time to Market Cost Reduce Risk in Production Digital Coverage Automation Fast Feedback Embedded in Dev-QA-Ops
  • 14.
    Automation Guidelines 6/23/2017 14©2016, Perfecto Mobile Ltd. All Rights Reserved. • Easy to create • Object Recorder • Language (Java /BDD) • Easy to maintain • Object repository • POM • Tips • Runs 10 times on the same device • Runs on 3 deferent devices • Handle setup (install app / clear cash for web)
  • 15.
    Demo 1 • Gothrough an insurance RWD flow manually (5 forms , 10 fields each) • Identify the objects • Object Repository & POM
  • 16.
  • 17.
    XPATH Validator 6/23/2017 17©2015, Perfecto Mobile Ltd. All Rights Reserved. XPATH Validator: http://xpathvalidator.projectquantum.io/ An efficient way to assure robust objects used as Part of your test code – Flaky Test Avoidance
  • 18.
    Pillars for SuccessfulDigital Transformation 6/23/2017 18© 2015, Perfecto Mobile Ltd. All Rights Reserved. Digital Transformation Digital Winner 4 Pillars of Success Value Drivers Tools * People * Processes Quality Time to Market Cost Reduce Risk in Production Digital Coverage Automation Fast Feedback Embedded in Dev-QA-Ops
  • 19.
    Unattended Automation Guidelines& Fast Feedback 6/23/2017 19© 2016, Perfecto Mobile Ltd. All Rights Reserved. • Execution properties • Parallel execution • Dynamic device list • Tags managements • Execution management • Three strikes out • Retry for error • Execution Environments • Devices availability • Devices Q • Customers based env. (wind tunnel)
  • 20.
    Demo 2 • Parallelexecution • Connect to CI • Reports
  • 21.
    Pillars for SuccessfulDigital Transformation 6/23/2017 21© 2015, Perfecto Mobile Ltd. All Rights Reserved. Digital Transformation Digital Winner 4 Pillars of Success Value Drivers Tools * People * Processes Quality Time to Market Cost Reduce Risk in Production Digital Coverage Automation Fast Feedback Embedded in Dev-QA-Ops
  • 22.
    Demo 3 • HAR •Dev tunnel / debug on any device !!!!
  • 23.
    Giveaway for WebinarAttendees/ Q&A Chapter 12 Free Download
  • 24.

Editor's Notes

  • #9 Tzvika: Today’s digital space introduces great challenge to the majority of organizations, that are now required to deal with large number of tests (@Tzvika- from your top of your head, what’s an average # of tests some of our Enterprises are executing in a regression cycle?). With the agile practice, the number f executions and the time in between get shorter – all of these forces organizations to have a quick feedback loop that provides them insights ASAP after each execution. ** lots of scenario
  • #16 Objective: Introduce MRCs tied to their success Example: So we have a recipe for success. Now we need the ingredients, or the minimum required capabilities.
  • #21 Objective: Introduce MRCs tied to their success Example: So we have a recipe for success. Now we need the ingredients, or the minimum required capabilities.
  • #23 Objective: Introduce MRCs tied to their success Example: So we have a recipe for success. Now we need the ingredients, or the minimum required capabilities.