Functional testing the_good_the_bad_and_the_ugly

7,584 views
6,399 views

Published on

In the pursuit of continuous delivery, here is our reality:

Unit testing is good, but just not enough. Every Project dedicate a big Functional Testing part at the end to deliver only the top of the testing pyramid.

Then the questions are:
- How can you deliver quickly if you need to retest your entire application manually for each release?
- What automation tools can you use to ensure that your automated tests do not become a maintenance nightmare?
- How can you write automated tests that testers can leverage to make their own manual testing more effective?
The Agile Sydney Community is proud to receive John Ferguson Smart.

Come and join us to challenge John on giving a live demonstration of automated acceptance criteria using JBehave, Selenium 2 and Thucydides.

This will give us the opportunity to discuss a number of open source and commercial automated testing tools, with many examples and stories from real-world projects.

John talks about a range of tools and approaches to automated functional testing, and discuss the place of functional testing in a modern agile project.

He will look at topics such as:

- Where does testing fit into an agile development life-cycle
- Why it's too late to start testing at the end of the project or iteration
- What tests should you automate, and how
- What tools work best for different types of tests.

Published in: Technology
1 Comment
25 Likes
Statistics
Notes
No Downloads
Views
Total views
7,584
On SlideShare
0
From Embeds
0
Number of Embeds
43
Actions
Shares
0
Downloads
138
Comments
1
Likes
25
Embeds 0
No embeds

No notes for slide

Functional testing the_good_the_bad_and_the_ugly

  1. Automated Functional Testing
  2. John Ferguson Smart Consultant Trainer Mentor Author Speaker Coder
  3. Technical Agile Practices Test Driven Development Continuous Integration/Delivery Automated Acceptance Criteria Refactoring Pair Programming “Without  these  technical  prac1ces,  your  agile  adop1on  is  hollow” -­‐  Ma6  Wynne
  4. Technical Agile Practices Test Driven Development Continuous Integration/Delivery Automated Acceptance Criteria Refactoring Pair Programming “Without  these  technical  prac1ces,  your  agile  adop1on  is  hollow” -­‐  Ma6  Wynne
  5. A  good  func<onal  test  tells  a  story
  6. Practices Tools
  7. QA loses its potential if only done at the end of the project
  8. The Three Amigos
  9. Living Documentation - test results that serve everyone
  10. So show me the tools!
  11. The Record-Replay Scam
  12. in theory in practice The Record-Replay Scam
  13. What do ogres and good functional tests have in common?
  14. (This test is not like an ogre)
  15. Remember...good automated acceptance tests have layers
  16. Start off with the business objectives Business-­‐level  requirements
  17. Describe the business flow High-­‐level  steps   (business  flow)
  18. Describe the page interactions Page-­‐level  interac<ons
  19. Page implementation details HTML  details  go  here
  20. Business  Rules Business  Flow Page/Component   interac<ons Page/Component   details Good automated acceptance tests have layers
  21. Come on - can we see the tools now?
  22. Unit  tes<ng  BDD  tools  (39%) Given/When/Then  (40%) Tabular  (13%)
  23. WebDriver / Selenium 2 Mature  browser  automa<on Wide  adop<on Cross-­‐browser  support Cross-­‐browser  support
  24. ...or  use  Selenium  Hub
  25. appium Uses  the  WebDriver  API Can  run  on  SauceLabs  (with  some  limita<ons)
  26. Watir Web Application Testing in Ruby browser.goto 'http://bit.ly/watir-example' browser.text_field(:name => 'entry.0.single').set 'Watir' browser.checkbox(:value => 'Ruby').set Ruby  DSL  for  web  tes<ng WebDriver-­‐based Fluent  and  readable No  na<ve  support  for   Page  Objects?
  27. Geb Groovy Browser Automation Browser.drive  {        go  "http://myapp.com/login"                  assert  $("h1").text()  ==  "Please  Login"                  $("form.login").with  {                username  =  "admin"                password  =  "password"                login().click()        }                  assert  $("h1").text()  ==  "Admin  Section" } Groovy  DSL  for  web  tes<ng WebDriver-­‐based Fluent  and  readable Supports  Page  Objects
  28. 28 High level BDD Many  tools  are  available
  29. 29 Narrative: In order to increase sales of advertised articles As a seller I want buyers to be able to easily find ads for articles they want to buy Scenario: Searching by keyword and location Given Sally wants to buy a puppy for her son When she looks for 'puppy' in the 'Pets and Animals' category Then she should obtain a list of ads for puppies for sale. search_by_keyword_and_location.story Business-­‐friendly  specifica<ons An  example  in  JBehave
  30. 30 Scenario: Searching by keyword and location Given Sally wants to buy a puppy for her son When she looks for 'puppy' in the 'Pets and Animals' category Then she should obtain a list of ads for puppies for sale. search_by_keyword_and_location.story Scenario: Searching by keyword and location Given Sally wants to buy a <present> for her son When she looks for '<present>' in the '<category>' category Then she should obtain a list of ads for <expected> for sale. Examples: |present |category |expected| |puppy |Pets & Animals | puppies| |kitten |Pets & Animals | kittens| |seiko |Jewellery & Watches| watch | Tabular  examples
  31. 31 Scenario: Searching by keyword and location Given Sally wants to buy a puppy for her son When she looks for 'puppy' in the 'Pets and Animals' category Then she should obtain a list of puppy ads search_by_keyword_and_location.story public class SearchAdsSteps {     @Steps     BuyerSteps buyer;     @Given("Sally wants to buy a $present for her son")     public void buyingAPresent(String present) {         buyer.opens_home_page();     }     @When("she looks for $keyword in the $category category")     public void adSearchByCategoryAndKeyword(String category, String keyword) {         buyer.chooses_category_and_keywords(category, keyword);         buyer.performs_search();     }     @Then("she should obtain a list of $keyword ads")     public void shouldOnlySeeAdsContainingKeyword(String keyword) {         buyer.should_only_see_results_with_titles_containing(keyword);     } } Step  implementa<ons  in  your   favorite  language
  32. 32 From web tests to living documentation
  33. 33 (Think “Two-CDs”)
  34. 34 2 Automate your acceptance criteria 1 Discover your acceptance criteria 4 Execute your acceptance tests 3 Implement your acceptance criteria
  35. 35 1 Discover your acceptance criteria Feature: Browse Catalog In order to find items that I would like to buy As a customer I want to be able to browse through the catalog Story: Browse by category In order to find items more easily As a customer I want to be able to browse through the product categories Acceptance Criteria See all the top-level categories Browse through the category hierarchy Should display the correct products for each category Each category should have the correct sub-categories Define  acceptance  criteria  for  each  story
  36. 36 1 Discover your acceptance criteria Acceptance Criteria See all the top-level categories Browse through the category hierarchy Should display the correct products for each category Each category should have the correct sub-categories Scenario: See all top-level categories Given I want to browse the catalog When I am on the home page Then I should see the following product categories: Clothing, Accessories, Shoes Clarify  the  acceptance  criteria  with  examples
  37. 37 2 Automate your acceptance criteria Story: Browse by category In order to find items more easily As a customer I want to be able to browse through the product categories Acceptance Criteria See all the top-level categories Browse through the category hierarchy Should display the correct products for each category Each category should have the correct sub-categories Scenario: See all top-level categories Given I want to browse the catalog When I am on the home page Then I should see the following product categories: Clothing, Accessories, Shoes Narrative: In order to find items more easily As a customer I want to be able to see what product categories exist Scenario: See all top-level categories Given I want to browse the catalog When I am on the home page Then I should see the following product categories: Clothing, Accessories, Shoes We  now  have  executable  specifica<ons
  38. 38 2 Automate your acceptance criteria ...but  they  are  reported   as  ‘pending’
  39. 39 3 Implement your acceptance criteria Narrative: In order to find items more easily As a customer I want to be able to see what product categories exist Scenario: See all top-level categories Given I want to browse the catalog When I am on the home page Then I should see the following product categories: Clothing, Accessories, Shoes
  40. 40 4 Execute your acceptance tests What  tests  have  been  run
  41. 41 What  requirements   have  been  tested? 4 Execute your acceptance tests
  42. 42 4 Execute your acceptance tests How  did  we  show  that  a   par<cular  requirement   worked?
  43. 43 4 Execute your acceptance tests How  did  we  show  that  a   par<cular  requirement   worked?
  44. Demo time!
  45. hYp://wakaleo.com hYp://thucydides.info Want  to  learn  more?
  46. John  Ferguson  Smart Thank you

×