End-to-end web-testing based on
ruby/cucumber/watir/watirsome tech
Alex Mikitenko
Alex Mikitenko, Ukraine
Lead QA Automation, Tech Team Labs
• 10 years in the IT as a developer in test
• Ruby fan
• Anarchist
https://www.linkedin.com/in/omykytenko/
https://www.slideshare.net/korvinua
https://github.com/nonkor
https://www.visualcv.com/alex-mikitenko
[ description ]
A classic task for an automation QA engineer these days:
Developer and automate test coverage for some web or mobile
software solution.
This workshop offers a quick start for anyone who is interested to build a
proper end-to-end test framework for those needs from the scratch.
We will use robust and effective stack of technologies, verified by time and
trusted by community:
RUBY + CUCUMBER + WATIR + WATIRSOME + RSPEC/EXPECTATIONS
Ready? Let’s start then!
[ e2e-testing ]
So, what is end-to-end testing?
End-to-end testing is a methodology used to test whether the flow of an
application is performing as designed from start to finish. The purpose of
carrying out end-to-end tests is to identify system dependencies and to
ensure that the right information is passed between various system
components and systems.
(via) techopedia.com
[ testing pyramid ]
(via) watirmelon.com
[ ruby ]
Why Ruby?
- open source;
- huge ecosystem;
- human oriented standards;
- truly OOP language;
- powerful metaprogramming facilities;
- DSL-oriented;
- easy to learn (hardcode level included, but you need go deep to met it);
- has trusted and robust tech stack for GUI web-automation;
- I use it on a daily basis :)
[ ruby ]
Manage Ruby by rbenv
# install rbenv
$ brew update
$ brew install rbenv
$ rbenv init
$ 'eval "$(rbenv init -)"' >> ~/.bash_profile
# install ruby-build
$ rbenv install -l
# install ruby
$ rbenv install 2.4.1
rbenv is lightweight Ruby version management tool
[ ruby gems ]
Manage Ruby gems by bundler
bundler provides a consistent environment for Ruby projects by tracking
and installing the exact gems and versions that are needed.
# create project and go to it
$ mkdir gui-test-sample
$ cd gui-test-sample
# install bundler
$ gem install bundler
$ bundle init
# install gems
$ ‘gem “cucumber”' >> ~/.bash_profile
$ bundle install
[ cucumber ]
Why Cucumber?
it’s collaboration tool allows you to run automated
acceptance tests written in a BDD style
- illustrates business rules, not just UI;
- scenario writing does not require knowledge of
specific programming language;
- gives a possibility to write Cucumber scenarios
before writing the code.
Your cucumber features should drive your implementation, not reflect it.
Andrew Premdas, one of the first adopters of Cucumber
[ cucumber ]
Initiate cucumber
$ bundle exec cucumber --init
features
steps
support
storage of test scenarios
storage of test step definitions
storage of cucumber env files and hooks preloaded for every test run
[ website sample ]
[ declarative vs. imperative ]
is a programming
paradigm that expresses
the logic of a
computation (what do)
without describing its
control flow (how do).
Declarative
programming
Imperative
programming
is a programming
paradigm that describes
computation in terms of
statements that change a
program state.
[ cucumber features ] Imperative style
[ cucumber features ] Declarative style
[ initiate project ]
env.rb
hooks.rb
[ initiate project ]
helper.rb
settings.yml
[ cucumber features ]
First run
$ bundle exec cucumber features/
sample_shopping_declarative.feature
Using the default profile...
Feature: Sample shopping [declarative]
Scenario: Order a blue t-shirt
…
1 scenario (1 undefined)
19 steps (19 undefined)
0m0.112s
[ cucumber features ]
Pending scenarios
You can implement step definitions for undefined steps with these snippets:
Given(/^I have a valid user account$/) do
pending # Write code here that turns the phrase above into concrete actions
end
…
[ watir ]
Why Watir?
- open source;
- web application testing in ruby;
- interacts with a browser the same way people do;
- is not a record/playback tool;
- has a clear and consistent API;
- based on WebDriver (which is W3C Web Standard);
- many of watir contributors are webdriver contributors, too;
- used as a foundation for cross browser cloud services as saucelabs.com;
- has page-object pattern extensions.
[ watir ]
Watir sample
[ watir ]
plane_steps.rb
[ watir ]
Why Page Objects?
- Page Object pattern represents the screens of your web app as a
series of objects and encapsulates the features represented by a
page;
- it allows us to model the UI in our tests;
- a page object is an object-oriented class that serves as an
interface to a page of your AUT.
[ watir ]
Page Object Model
webdriver
[ watirsome ]
Watirsome sample
[ watirsome ]
tshirts_page.rb
[ watirsome ]
page_steps.rb
[ rspec/expectations ]
rspec/expectations
provides a simple, readable API to express expected outcomes of a code example
expect(actual).to eq(expected)
expect(actual).to eql(expected)
expect(actual).not_to eql(not_expected)
expect(actual).to be > expected
expect(actual).to be < expected
expect(actual).to be_within(delta).of(expected)
expect(actual).to match(/expression/)
expect(actual).to be_an_instance_of(expected)
expect(actual).to be_a(expected)
expect(actual).to be_an(expected)
expect(actual).to be_a_kind_of(expected)
expect { ... }.to raise_error
expect { ... }.to raise_error(ErrorClass)
expect { ... }.to raise_error("message")
expect { ... }.to raise_error(ErrorClass, "message")
Equivalence
Comparisons
Types/classes
Expecting errors
Regular expressions
and much more…
[ rspec/expectations ]
Failed scenario with custom exception
$ DEBUG=1 bundle exec cucumber -p pages
Using the pages and common profiles...
@declarative
Feature: Sample shopping [declarative]
Scenario: Order a blue t-shirt
Given I have a valid user account
But I am an unauthenticated guest
…
Then I see details about chosen t-shirt
Selected color: Orange, but should be: Blue (RuntimeError)
./features/step_definitions/page_steps.rb:23:in `/^I see details about chosen t-shirt$/'
features/sample_shopping_declarative.feature:10:in `Then I see details about chosen t-shirt'
Failing Scenarios:
cucumber -p pages -p common features/sample_shopping_declarative.feature:4 # Scenario: Order a blue t-shirt
[ rspec/expectations ]
$ DEBUG=1 bundle exec cucumber -p pages
Using the pages and common profiles...
@declarative
Feature: Sample shopping [declarative]
Scenario: Order a blue t-shirt
Given I have a valid user account
But I am an unauthenticated guest
…
Then I see details about chosen t-shirt
expected: "Blue"
got: "Orange"
(compared using ==)
(RSpec::Expectations::ExpectationNotMetError)
./features/step_definitions/page_steps.rb:23:in `/^I see details about chosen t-shirt$/'
features/sample_shopping_declarative.feature:10:in `Then I see details about chosen t-shirt'
Failing Scenarios:
cucumber -p pages -p common features/sample_shopping_declarative.feature:4 # Scenario: Order a blue t-shirt
Failed scenario with rspec-expectation
[ helpful links ]
Idea of Cucumber: https://cucumber.io/blog/2014/03/03/the-worlds-most-misunderstood-collaboration-tool
Declarative/Imperative: http://itsadeliverything.com/declarative-vs-imperative-gherkin-scenarios-for-cucumber
Webdriver spec: https://www.w3.org/TR/webdriver/
Browser automation with Watir: https://binarapps.com/blog/browser-automation-with-watir-guide
Page Objects in Webdriver: https://github.com/SeleniumHQ/selenium/wiki/PageObjects
Watirsome gem: https://github.com/p0deje/watirsome
RSpec built-in matchers: https://relishapp.com/rspec/rspec-expectations/docs/built-in-matchers
Another samples of cucumber + watir: https://github.com/spriteCloud/lapis-lazuli
Test Automation Websites #1: http://www.techbeamers.com/websites-to-practice-selenium-webdriver-online/
Test Automation Websites #2: https://www.ultimateqa.com/best-test-automation-websites-to-practice-using-
selenium-webdriver/
Questions
5 minutes.
You can also ask questions for me in the lounge zone

End-to-end web-testing in ruby ecosystem

  • 1.
    End-to-end web-testing basedon ruby/cucumber/watir/watirsome tech Alex Mikitenko
  • 2.
    Alex Mikitenko, Ukraine LeadQA Automation, Tech Team Labs • 10 years in the IT as a developer in test • Ruby fan • Anarchist https://www.linkedin.com/in/omykytenko/ https://www.slideshare.net/korvinua https://github.com/nonkor https://www.visualcv.com/alex-mikitenko
  • 3.
    [ description ] Aclassic task for an automation QA engineer these days: Developer and automate test coverage for some web or mobile software solution. This workshop offers a quick start for anyone who is interested to build a proper end-to-end test framework for those needs from the scratch. We will use robust and effective stack of technologies, verified by time and trusted by community: RUBY + CUCUMBER + WATIR + WATIRSOME + RSPEC/EXPECTATIONS Ready? Let’s start then!
  • 4.
    [ e2e-testing ] So,what is end-to-end testing? End-to-end testing is a methodology used to test whether the flow of an application is performing as designed from start to finish. The purpose of carrying out end-to-end tests is to identify system dependencies and to ensure that the right information is passed between various system components and systems. (via) techopedia.com
  • 5.
    [ testing pyramid] (via) watirmelon.com
  • 6.
    [ ruby ] WhyRuby? - open source; - huge ecosystem; - human oriented standards; - truly OOP language; - powerful metaprogramming facilities; - DSL-oriented; - easy to learn (hardcode level included, but you need go deep to met it); - has trusted and robust tech stack for GUI web-automation; - I use it on a daily basis :)
  • 7.
    [ ruby ] ManageRuby by rbenv # install rbenv $ brew update $ brew install rbenv $ rbenv init $ 'eval "$(rbenv init -)"' >> ~/.bash_profile # install ruby-build $ rbenv install -l # install ruby $ rbenv install 2.4.1 rbenv is lightweight Ruby version management tool
  • 8.
    [ ruby gems] Manage Ruby gems by bundler bundler provides a consistent environment for Ruby projects by tracking and installing the exact gems and versions that are needed. # create project and go to it $ mkdir gui-test-sample $ cd gui-test-sample # install bundler $ gem install bundler $ bundle init # install gems $ ‘gem “cucumber”' >> ~/.bash_profile $ bundle install
  • 9.
    [ cucumber ] WhyCucumber? it’s collaboration tool allows you to run automated acceptance tests written in a BDD style - illustrates business rules, not just UI; - scenario writing does not require knowledge of specific programming language; - gives a possibility to write Cucumber scenarios before writing the code. Your cucumber features should drive your implementation, not reflect it. Andrew Premdas, one of the first adopters of Cucumber
  • 10.
    [ cucumber ] Initiatecucumber $ bundle exec cucumber --init features steps support storage of test scenarios storage of test step definitions storage of cucumber env files and hooks preloaded for every test run
  • 11.
  • 12.
    [ declarative vs.imperative ] is a programming paradigm that expresses the logic of a computation (what do) without describing its control flow (how do). Declarative programming Imperative programming is a programming paradigm that describes computation in terms of statements that change a program state.
  • 13.
    [ cucumber features] Imperative style
  • 14.
    [ cucumber features] Declarative style
  • 15.
    [ initiate project] env.rb hooks.rb
  • 16.
    [ initiate project] helper.rb settings.yml
  • 17.
    [ cucumber features] First run $ bundle exec cucumber features/ sample_shopping_declarative.feature Using the default profile... Feature: Sample shopping [declarative] Scenario: Order a blue t-shirt … 1 scenario (1 undefined) 19 steps (19 undefined) 0m0.112s
  • 18.
    [ cucumber features] Pending scenarios You can implement step definitions for undefined steps with these snippets: Given(/^I have a valid user account$/) do pending # Write code here that turns the phrase above into concrete actions end …
  • 19.
    [ watir ] WhyWatir? - open source; - web application testing in ruby; - interacts with a browser the same way people do; - is not a record/playback tool; - has a clear and consistent API; - based on WebDriver (which is W3C Web Standard); - many of watir contributors are webdriver contributors, too; - used as a foundation for cross browser cloud services as saucelabs.com; - has page-object pattern extensions.
  • 20.
  • 21.
  • 22.
    [ watir ] WhyPage Objects? - Page Object pattern represents the screens of your web app as a series of objects and encapsulates the features represented by a page; - it allows us to model the UI in our tests; - a page object is an object-oriented class that serves as an interface to a page of your AUT.
  • 23.
    [ watir ] PageObject Model webdriver
  • 24.
  • 25.
  • 26.
  • 27.
    [ rspec/expectations ] rspec/expectations providesa simple, readable API to express expected outcomes of a code example expect(actual).to eq(expected) expect(actual).to eql(expected) expect(actual).not_to eql(not_expected) expect(actual).to be > expected expect(actual).to be < expected expect(actual).to be_within(delta).of(expected) expect(actual).to match(/expression/) expect(actual).to be_an_instance_of(expected) expect(actual).to be_a(expected) expect(actual).to be_an(expected) expect(actual).to be_a_kind_of(expected) expect { ... }.to raise_error expect { ... }.to raise_error(ErrorClass) expect { ... }.to raise_error("message") expect { ... }.to raise_error(ErrorClass, "message") Equivalence Comparisons Types/classes Expecting errors Regular expressions and much more…
  • 28.
    [ rspec/expectations ] Failedscenario with custom exception $ DEBUG=1 bundle exec cucumber -p pages Using the pages and common profiles... @declarative Feature: Sample shopping [declarative] Scenario: Order a blue t-shirt Given I have a valid user account But I am an unauthenticated guest … Then I see details about chosen t-shirt Selected color: Orange, but should be: Blue (RuntimeError) ./features/step_definitions/page_steps.rb:23:in `/^I see details about chosen t-shirt$/' features/sample_shopping_declarative.feature:10:in `Then I see details about chosen t-shirt' Failing Scenarios: cucumber -p pages -p common features/sample_shopping_declarative.feature:4 # Scenario: Order a blue t-shirt
  • 29.
    [ rspec/expectations ] $DEBUG=1 bundle exec cucumber -p pages Using the pages and common profiles... @declarative Feature: Sample shopping [declarative] Scenario: Order a blue t-shirt Given I have a valid user account But I am an unauthenticated guest … Then I see details about chosen t-shirt expected: "Blue" got: "Orange" (compared using ==) (RSpec::Expectations::ExpectationNotMetError) ./features/step_definitions/page_steps.rb:23:in `/^I see details about chosen t-shirt$/' features/sample_shopping_declarative.feature:10:in `Then I see details about chosen t-shirt' Failing Scenarios: cucumber -p pages -p common features/sample_shopping_declarative.feature:4 # Scenario: Order a blue t-shirt Failed scenario with rspec-expectation
  • 30.
    [ helpful links] Idea of Cucumber: https://cucumber.io/blog/2014/03/03/the-worlds-most-misunderstood-collaboration-tool Declarative/Imperative: http://itsadeliverything.com/declarative-vs-imperative-gherkin-scenarios-for-cucumber Webdriver spec: https://www.w3.org/TR/webdriver/ Browser automation with Watir: https://binarapps.com/blog/browser-automation-with-watir-guide Page Objects in Webdriver: https://github.com/SeleniumHQ/selenium/wiki/PageObjects Watirsome gem: https://github.com/p0deje/watirsome RSpec built-in matchers: https://relishapp.com/rspec/rspec-expectations/docs/built-in-matchers Another samples of cucumber + watir: https://github.com/spriteCloud/lapis-lazuli Test Automation Websites #1: http://www.techbeamers.com/websites-to-practice-selenium-webdriver-online/ Test Automation Websites #2: https://www.ultimateqa.com/best-test-automation-websites-to-practice-using- selenium-webdriver/
  • 31.
    Questions 5 minutes. You canalso ask questions for me in the lounge zone