Helpful Automation
Techniques
REST Your Tests / Proxy Validations / Selenium API
Writing a good test!
• Eliminate step duplication.
• Put common steps in your test setup or tear down.
• Never write tests which rely on another test or tests to pass!
• Write just enough steps to get from point A to point B the quickest
way possible to validate your expectation.
• Eliminate steps which have no relation to the final expected result.
REST Your Tests
The Good
• Reduced Runtime.
• Reduced Test Steps.
• Limits Test Flakiness.
• Not failing on steps unrelated
to the test expectation.
• Write Less Code.
• Run Tests More Frequently.
REST Your Tests
The Bad
• Complicated or Limited API.
• Formatting JSON (Params,
Query Strings).
• Difficulty Integrating REST
Library.
Test Example
• Scenario: Validate shared
list notification.
• Case: User B receives list
invite from User A.
100% UI Example* verbose to show all test steps
Now lets see it run…
What’s wrong with this
example?
• Relies on UI actions unrelated to the test expectation.
• Login x 2
• Logout
• Executes several UI steps to create and share list.
• More steps increase chances of failure.
• Longer runtime.
Lets add some API methods
• Created method “api_share_a_list_to_me”.
UI + API Example
Now lets run it…
What did we accomplish?
• Eliminated over 20 UI steps.
• Reduced runtime from 74 to 29 seconds. We can run the
test 2.5 times more now.
• Eliminated steps unrelated to the test expectation.
• What is the expectation or goal of the test?
• Where can I eliminate steps without losing test integrity?
• What if my application doesn’t have an API?
• Ask development to create a private test API for you.
• Develop the API yourself.
• Alternatives:
• Make updates via SQL to a database.
• Setup seed data to run prior to test execution. (requires
maintenance)
Ask Yourself
Proxy Integration Tests
The Good
• Validate Endpoints.
• Validate JSON body.
• Validate Server Response.
• Shorten runtimes without
full end to end test.
• HTTPS can be a pain.
• Requires additional
network setup.
• Cannot proxy everything.
e.g web-socket
connections.
Proxy Integration Tests
The Bad
Test Example
• Scenario: Reset Password.
• Case 1: Validate client sends
“POST” request method.
• Case 2: Validate client sends
request to “/password/reset”
endpoint.
• Case 3: Validate request body
{“email”:”hello@seleniumcamp.com"}
• Case 4: Validate a status code
“200” is returned from server.
Proxy Example
Now lets watch it run…
• Validating app settings when no UI response is displayed.
e.g. Push notifications, Reset Password, User Data etc…
• Validating endpoints and server responses during API
development.
• Regression or Integration testing.
Proxy Use Cases
The Selenium API
• What to do if you encounter an
issue with your test framework or
library?
• Submit a bug.
• Check chat forums.
• Possibly downgrade!?
• Fix the issue & submit a PR.
• Or… All is not lost!
• Use the Selenium API as a
temporary work around!
Getting Started
• We need an HTTP library. HTTParty!
• We need the WebDriver Sever URL. It’s defined in your
CAPS. ChromeDriver it’s defaulted to
http://127.0.0.1:9515.
• We need the WebDriver Session ID e.g. “7018257b-f1db-
4552-a5d5-dbd4b80b0140”. Depending on your bindings,
this can be retrieved in different ways.
• Selenium Wire Protocol:
https://code.google.com/p/selenium/wiki/JsonWireProtocol
• We added API requests similar to the existing selenium methods;
find_element, find_elements, .click, .text, and .send_keys.
Lets add some examples
Selenium API Example - Mobile
Selenium API Example - Web
Lets run an example…
Information & Tips
• Code Examples: https://github.com/isonic1/selenium_camp
• Getting Started with Appium:
• https://saucelabs.com/resources/appium-bootcamp
• https://docs.saucelabs.com/tutorials/appium/
• https://github.com/appium/ruby_lib
• Helpful Cloud Testing Services:
• Sauce Labs (Web and Mobile Automation Service)
• TestObject (Real Mobile Device Automation Service)
• Travis CI (Continuous Integration Service)
Questions?
Thank You!
email: justin.ison@gmail.com
twitter: @isonic1

Helpful Automation Techniques - Selenium Camp 2014

  • 1.
    Helpful Automation Techniques REST YourTests / Proxy Validations / Selenium API
  • 2.
    Writing a goodtest! • Eliminate step duplication. • Put common steps in your test setup or tear down. • Never write tests which rely on another test or tests to pass! • Write just enough steps to get from point A to point B the quickest way possible to validate your expectation. • Eliminate steps which have no relation to the final expected result.
  • 3.
    REST Your Tests TheGood • Reduced Runtime. • Reduced Test Steps. • Limits Test Flakiness. • Not failing on steps unrelated to the test expectation. • Write Less Code. • Run Tests More Frequently.
  • 4.
    REST Your Tests TheBad • Complicated or Limited API. • Formatting JSON (Params, Query Strings). • Difficulty Integrating REST Library.
  • 5.
    Test Example • Scenario:Validate shared list notification. • Case: User B receives list invite from User A.
  • 6.
    100% UI Example*verbose to show all test steps
  • 7.
    Now lets seeit run…
  • 8.
    What’s wrong withthis example? • Relies on UI actions unrelated to the test expectation. • Login x 2 • Logout • Executes several UI steps to create and share list. • More steps increase chances of failure. • Longer runtime.
  • 9.
    Lets add someAPI methods • Created method “api_share_a_list_to_me”.
  • 10.
    UI + APIExample
  • 11.
  • 12.
    What did weaccomplish? • Eliminated over 20 UI steps. • Reduced runtime from 74 to 29 seconds. We can run the test 2.5 times more now. • Eliminated steps unrelated to the test expectation.
  • 13.
    • What isthe expectation or goal of the test? • Where can I eliminate steps without losing test integrity? • What if my application doesn’t have an API? • Ask development to create a private test API for you. • Develop the API yourself. • Alternatives: • Make updates via SQL to a database. • Setup seed data to run prior to test execution. (requires maintenance) Ask Yourself
  • 14.
    Proxy Integration Tests TheGood • Validate Endpoints. • Validate JSON body. • Validate Server Response. • Shorten runtimes without full end to end test.
  • 15.
    • HTTPS canbe a pain. • Requires additional network setup. • Cannot proxy everything. e.g web-socket connections. Proxy Integration Tests The Bad
  • 16.
    Test Example • Scenario:Reset Password. • Case 1: Validate client sends “POST” request method. • Case 2: Validate client sends request to “/password/reset” endpoint. • Case 3: Validate request body {“email”:”hello@seleniumcamp.com"} • Case 4: Validate a status code “200” is returned from server.
  • 17.
  • 18.
    Now lets watchit run…
  • 20.
    • Validating appsettings when no UI response is displayed. e.g. Push notifications, Reset Password, User Data etc… • Validating endpoints and server responses during API development. • Regression or Integration testing. Proxy Use Cases
  • 21.
    The Selenium API •What to do if you encounter an issue with your test framework or library? • Submit a bug. • Check chat forums. • Possibly downgrade!? • Fix the issue & submit a PR. • Or… All is not lost! • Use the Selenium API as a temporary work around!
  • 22.
    Getting Started • Weneed an HTTP library. HTTParty! • We need the WebDriver Sever URL. It’s defined in your CAPS. ChromeDriver it’s defaulted to http://127.0.0.1:9515. • We need the WebDriver Session ID e.g. “7018257b-f1db- 4552-a5d5-dbd4b80b0140”. Depending on your bindings, this can be retrieved in different ways. • Selenium Wire Protocol: https://code.google.com/p/selenium/wiki/JsonWireProtocol
  • 23.
    • We addedAPI requests similar to the existing selenium methods; find_element, find_elements, .click, .text, and .send_keys. Lets add some examples
  • 24.
  • 26.
  • 27.
    Lets run anexample…
  • 29.
    Information & Tips •Code Examples: https://github.com/isonic1/selenium_camp • Getting Started with Appium: • https://saucelabs.com/resources/appium-bootcamp • https://docs.saucelabs.com/tutorials/appium/ • https://github.com/appium/ruby_lib • Helpful Cloud Testing Services: • Sauce Labs (Web and Mobile Automation Service) • TestObject (Real Mobile Device Automation Service) • Travis CI (Continuous Integration Service)
  • 30.
  • 31.