Autom ation Best
Practices
L eo L a s k in , T es t Au t om a t ion Ad v is or
About Me
• Twitter: @LeoLaskin
• LinkedIn: https://www.linkedin.com/in/llaskin
• Email: leo@saucelabs.com
© Sauce Labs, Inc. 2
Manual Tests
© Sauce Labs, Inc. 3
No Manual Tests*
*	Testing	Extreme	Programming	 by	Lisa	Crispin	and	
Tip	House
Start with a Clean System
• Having a clean system every time is paramount for
reproducible results
• SaaS grid based systems can be setup to be fresh
with no system cache, browser cache, cookies, etc.
© Sauce Labs, Inc. 4
Parallelizatio n
• Serialized tests take more time and cost more
money
• Using a Selenium Grid like Sauce Labs,
parallelized tests can spin up multiple
browsers for you or multiple tests
simultaneously
• 300 minute long tests can take either 5 hours
or as little as 1 minute.
© Sauce Labs, Inc. 5
Page Object Pattern
• Encourage reusable code throughout your test app
• Test code more organized for update if test
parameters like selectors change
• Tests are more readable with a natural feeling
Domain-Specific Language(DSL)
© Sauce Labs, Inc. 6
Test Constantly
• Don’t ever stop testing
• Having tests run on every commit is going to provide
significantly higher confidence in your software
• You have unlimited amounts of time to test. Use it!
© Sauce Labs, Inc. 7
Speed things up
• The average attention span of an adult is less then 5
minutes
• Try to provide results before context is switched
• Make the devs work for you not against you
© Sauce Labs, Inc. 8
Start Small
• Don’t rush to automate every test case
• Identify test case priority then automate
• Don’t automate end-to-end
• Divide and conquer
© Sauce Labs, Inc. 9
Divide your testing up
• By Functional Area
• By hosted server
• By backend server
© Sauce Labs, Inc. 10
Automation = Development Activity
• Automation is a real development endeavor, treat it
as such
• Beware of Cowboy Coding
• Copy/Pasting inside your code can be replaced by
reusable methods
• Source control is cheap insurance
• Bad coding practices = bad testing practices (and
vice versa)
© Sauce Labs, Inc. 11
Don’t Forget to Wait!
• Web pages don’t load instantaneously, and not all
elements load at the same time
• A lot of your failures in finding an element will
disappear if you use these
• Explicit or fluent waits are preferred for waiting for
single elements
• Implicit waits for waiting all the time
• Fluent waits for elements that can take variable time
to load
© Sauce Labs, Inc. 12
Avoid Dependencies
• Don’t allow tests to be dependent on each other
• One test’s actions should not drive another tests
assertion criteria
© Sauce Labs, Inc. 13
Prep Your Environment
• Make sure your CI rig can handle the number of jobs
running on it simultaneously
• Make sure your SUT can handle the number of tests
running on it simultaneously
© Sauce Labs, Inc. 14
Collect Metrics
• How long did test runs take before and after
automation?
• How many bugs do automated tests identify per
release?
• How many engineers does a test run require?
© Sauce Labs, Inc. 15
SAUCE LABS RESOURCES
February 29, 2016 © Sauce Labs, Inc.
• Sign up for a free trial of Sauce
https://saucelabs.com/signup/trial
• Check out our docs on Automation and how to use
Sauce at https://wiki.saucelabs.com
16

Automation Best Practices

  • 1.
    Autom ation Best Practices Leo L a s k in , T es t Au t om a t ion Ad v is or
  • 2.
    About Me • Twitter:@LeoLaskin • LinkedIn: https://www.linkedin.com/in/llaskin • Email: leo@saucelabs.com © Sauce Labs, Inc. 2
  • 3.
    Manual Tests © SauceLabs, Inc. 3 No Manual Tests* * Testing Extreme Programming by Lisa Crispin and Tip House
  • 4.
    Start with aClean System • Having a clean system every time is paramount for reproducible results • SaaS grid based systems can be setup to be fresh with no system cache, browser cache, cookies, etc. © Sauce Labs, Inc. 4
  • 5.
    Parallelizatio n • Serializedtests take more time and cost more money • Using a Selenium Grid like Sauce Labs, parallelized tests can spin up multiple browsers for you or multiple tests simultaneously • 300 minute long tests can take either 5 hours or as little as 1 minute. © Sauce Labs, Inc. 5
  • 6.
    Page Object Pattern •Encourage reusable code throughout your test app • Test code more organized for update if test parameters like selectors change • Tests are more readable with a natural feeling Domain-Specific Language(DSL) © Sauce Labs, Inc. 6
  • 7.
    Test Constantly • Don’tever stop testing • Having tests run on every commit is going to provide significantly higher confidence in your software • You have unlimited amounts of time to test. Use it! © Sauce Labs, Inc. 7
  • 8.
    Speed things up •The average attention span of an adult is less then 5 minutes • Try to provide results before context is switched • Make the devs work for you not against you © Sauce Labs, Inc. 8
  • 9.
    Start Small • Don’trush to automate every test case • Identify test case priority then automate • Don’t automate end-to-end • Divide and conquer © Sauce Labs, Inc. 9
  • 10.
    Divide your testingup • By Functional Area • By hosted server • By backend server © Sauce Labs, Inc. 10
  • 11.
    Automation = DevelopmentActivity • Automation is a real development endeavor, treat it as such • Beware of Cowboy Coding • Copy/Pasting inside your code can be replaced by reusable methods • Source control is cheap insurance • Bad coding practices = bad testing practices (and vice versa) © Sauce Labs, Inc. 11
  • 12.
    Don’t Forget toWait! • Web pages don’t load instantaneously, and not all elements load at the same time • A lot of your failures in finding an element will disappear if you use these • Explicit or fluent waits are preferred for waiting for single elements • Implicit waits for waiting all the time • Fluent waits for elements that can take variable time to load © Sauce Labs, Inc. 12
  • 13.
    Avoid Dependencies • Don’tallow tests to be dependent on each other • One test’s actions should not drive another tests assertion criteria © Sauce Labs, Inc. 13
  • 14.
    Prep Your Environment •Make sure your CI rig can handle the number of jobs running on it simultaneously • Make sure your SUT can handle the number of tests running on it simultaneously © Sauce Labs, Inc. 14
  • 15.
    Collect Metrics • Howlong did test runs take before and after automation? • How many bugs do automated tests identify per release? • How many engineers does a test run require? © Sauce Labs, Inc. 15
  • 16.
    SAUCE LABS RESOURCES February29, 2016 © Sauce Labs, Inc. • Sign up for a free trial of Sauce https://saucelabs.com/signup/trial • Check out our docs on Automation and how to use Sauce at https://wiki.saucelabs.com 16