Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Developing a test automation strategy by Brian Bayer

811 views

Published on

Brian Bayer QA or the Highway 2016 talk

Published in: Technology
  • Be the first to comment

Developing a test automation strategy by Brian Bayer

  1. 1. test automation strategy Developing a
  2. 2. why are we here?
  3. 3. regression testing the greatest cost of development
  4. 4. regression testing you have already paid for this at least once
  5. 5. why regression test?
  6. 6. why regression test? quality ensures what was working, still works
  7. 7. why regression test? quality risk money lost revenue, market, contractual ensures what was working, still works deliverables, release, schedule
  8. 8. why regression test? a project without thorough and continuous regression testing is like a road trip where you do not look out the window until the end of the trip
  9. 9. why automate? (vs. manual)
  10. 10. why automate? (vs. manual) TimeMoney
  11. 11. why automate? (vs. manual) Money
  12. 12. why automate? (vs. manual) Time
  13. 13. automation strategy what to consider... tests are living specifications of the system
  14. 14. automation strategy what to consider... ARE THE TESTS GOING TO BE USED?
  15. 15. automation strategy what to consider... business needs return on investment organizational structure and culture
  16. 16. automation strategy business needs... determine areas of focus of the automation
  17. 17. automation strategy return on investment... limits and prioritizes the areas of focus of the automation determines at which level to automate a test determines the automation framework
  18. 18. determines the success or failure determines the speed, ability & openness to change determines the communication determines the ability to learn automation strategy organization and culture...
  19. 19. automation strategy organization and culture... determines who uses and maintains the tests determines the automation framework too
  20. 20. automation strategy what else to consider...
  21. 21. automation strategy what else to consider... development process and lifecycle maintenance of the tests technology stacks
  22. 22. automation strategy development process and lifecycle... legacy (system without tests) to greenfields
  23. 23. automation strategy development process and lifecycle... system under test development process methodology: waterfall agile xp ad hoc is automation development part of the same process?
  24. 24. automation strategy development process and lifecycle... is the automator part of the development team? how much access to the system developers?
  25. 25. automation strategy maintenance of the tests... who and how much can they maintain? how are the system developers involved?
  26. 26. automation strategy maintenance of the tests... how timely can/must tests be maintained what reporting needed to maintain tests
  27. 27. automation strategy finally, technology stacks... languages and technology already used unit and integration testing source code control, continuous integration devops or operations
  28. 28. where to automate?
  29. 29. who is going to use the tests?
  30. 30. developers, maybe... depends
  31. 31. where to automate? https://msdn.microsoft.com “Its essential point is that you should have many more low-level unit tests than high level end-to-end tests running through a GUI.” Martin Fowler
  32. 32. where to automate? UNIT LEVEL INTEGRATION LEVEL GUI INTEGRATION LEVEL XPtest driven development ACCEPTANCE LEVEL BDD
  33. 33. ACCEPTANCE LEVEL BDD where to automate? acceptance tests only if the business is engaged and going to use them ACCEPTANCE LEVEL BDD
  34. 34. where to automate? what to consider... DRY
  35. 35. look for low-hanging fruit risk-based testing what computers do well, humans not so much fragility of test where to automate? what to consider...
  36. 36. which automation framework to use?
  37. 37. which automation framework to use? Gherkin/Cucumber Robot Framework Selenium Webdriver Spock/Geb JUnit Jasmine TestNG
  38. 38. which automation framework to use? what to consider... ORGANIZATIONAL STRUCTURE AND CULTURE
  39. 39. which automation framework to use? what to consider... who is reading the tests? who is writing and maintaining the tests? balance of right tool with existing technology, knowledge, and ability to learn ORGANIZATIONAL STRUCTURE AND CULTURE
  40. 40. which automation framework to use? what else to consider... natural language for high level specification goal is to develop tests NOT framework and...
  41. 41. grids...
  42. 42. integration...
  43. 43. reporting...
  44. 44. teh sexy
  45. 45. gherkin/cucumber/ruby
  46. 46. but JRuby (JVM), Java, Groovy, JavaScript, .NET (using SpecFlow) libraries (Selenium, Watir) Ruby has the richest ecosystem with gems gherkin/cucumber generally ruby based...
  47. 47. Feature: Cash Withdrawal Scenario: Successful withdrawal from an account in credit Given my account has been credited with $100 When I withdraw $20 Then $20 should be dispensed And the balance of my account should be $80 Given /^my account has been credited with (#{CAPTURE_CASH_AMOUNT})$/ do |amount| my_account.credit(amount) end
  48. 48. also runs on Jython (JVM) & IronPython (.NET) libraries (Selenium, Watir, SOAP, etc.) keyword based supports “given when then” supports tabular/data driven robot framework python based...
  49. 49. *** Settings *** Library Selenium Library Suite Setup Start Selenium Server Suite Teardown Stop Selenium Server Test Setup Setup Test Test Teardown Close All Browsers
  50. 50. *** Testcases *** Login Should Succeed When the Correct Username and Password are Entered Enter Username AUser Enter Password TestPass Click the Login Button Login is Successful *** Keywords *** Setup Test Open Browser http://zdiles.chaosnet.org/ googlechrome Maximize Browser Window Enter Username [Arguments] ${username} Input Text uname ${username} Login Is Successful Page Should Contain Welcome
  51. 51. *** Testcases *** Login Should Succeed When the Correct Username and Password are Entered Given I Enter Username AUser And I Enter Password TestPass When I Click the Login Button Then Login is Successful given when then
  52. 52. spock/geb/groovy
  53. 53. developing in java or grails?
  54. 54. groovy-based browser automation tool built on Selenium WebDriver works with JUnit, TestNG, Spock page models and jQuery selector syntax spock/geb/groovy geb...
  55. 55. groovy-based test and specification framework supports “given when then” tabular/data driven spock/geb/groovy spock...
  56. 56. import geb.spock.GebSpec class GebHomepageSpec extends GebSpec { def "can access The Book of Geb via homepage"() { given:”One is on the Geb Home Page” to GebHomePage when: “I click the jQuery Link” highlights.jQueryLikeApi.click() then: “I should be at the jQuery Section” highlights.jQueryLikeApi.isSelected } }
  57. 57. import geb.Page class GebHomePage extends Page { static url = "http://gebish.org" static at = { title == "Geb - Very Groovy Browser Automation" } static content = { highlights { $("#sidebar .sidemenu").module(HighlightsModule) } sectionTitles { $("#main h1")*.text() } } }
  58. 58. FRAMEWORKS
  59. 59. other options...
  60. 60. raw Selenium (Java) excel spreadsheet data driven scripts script capture record/playback flat files in source control other options...
  61. 61. which automation framework to use? the simplest thing that could possibly work
  62. 62. tests are specifications of the system...
  63. 63. how does it fit?
  64. 64. how does it fit... design your approach into the organization
  65. 65. how does it fit... find the gaps and tight squeezes
  66. 66. the specifics...
  67. 67. the mechanics...
  68. 68. so what’s left?
  69. 69. documentation of the strategy...
  70. 70. execution of the strategy...
  71. 71. if project management...
  72. 72. estimation...
  73. 73. resources...

×