Your SlideShare is downloading. ×
0
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Test automation
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Test automation

439

Published on

My ideas for how Test Automation could work.

My ideas for how Test Automation could work.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
439
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Building an effective test automation suiteAs we grow an automated acceptance test suite which we can rely upon will become a valuable resource.
  • 2. Building an effective test automation suite Integrate Identify Run Describe ImplementThe process of building an effective and sustainable test automation suite involves following a simple five step process.
  • 3. Building an effective test automation suite Integrate Identify Run Describe Implement Identify test scenarios to include in our suite.
  • 4. Building an effective test automation suite Integrate Identify Run Describe ImplementDescribe the test scenarios in a way that is easily understood by all product stakeholders.
  • 5. Building an effective test automation suite Integrate Identify Run Describe Implement Implement the test steps using our chosen testing frameworks.
  • 6. Building an effective test automation suite Integrate Identify Run Describe ImplementRun our tests on all supported product development environments.
  • 7. Building an effective test automation suite Integrate Identify Run Describe ImplementIntegrate our completed tests into our Continuous Integration and Testing system.
  • 8. Building an effective test automation suiteTest Identification should take place as an ongoing process throughout the product development lifecycle.
  • 9. Building an effective test automation suite Identify tests during Story creation. Stories should be described in Gherkin/BDD style to develop an ubiquitous language and easily testable deliverables. Identify tests during Acceptance Criteria meetings. After reviewing the initial requirements it may become clear that edge cases can also be defined. Identify tests during development. As the deliverables begin to become fleshed out, additional test cases may become apparent, and should be defined. Identify tests during testing. When the testing process begins, more test scenarios may be discovered, and should be defined and documented. Identify tests when errors appear. Errors/Bugs in any environment, including our live environment, should result in a bug report and tests to prove when its fixed. Identify tests at any other time. Whenever a valuable test case is identified then it should be documented and added to the test automation backlog.Test Identification should take place as an ongoing process throughout the product development lifecycle.
  • 10. Building an effective test automation suite Identify tests during Story creation. Stories should be described in Gherkin/BDD style to develop an ubiquitous language and easily testable deliverables. Identify tests during Acceptance Criteria meetings. After reviewing the initial requirements it may become clear that edge cases can also be defined. Identify tests during development. As the deliverables begin to become fleshed out, additional test cases may become apparent, and should be defined. Identify tests during testing. When the testing process begins, more test scenarios may be discovered, and should be defined and documented. Identify tests when errors appear. Errors/Bugs in any environment, including our live environment, should result in a bug report and tests to prove when its fixed. Identify tests at any other time. Whenever a valuable test case is identified then it should be documented and added to the test automation backlog.We describe our product requirements using the Gherkin syntax – Given, When, Then.
  • 11. Building an effective test automation suite Our BDD scenarios should be kept as short should be – with a in Gherkin/BDD Identify tests during Story creation. Stories as possibledescribedfocus on making them easy to read ubiquitous language and easily testable deliverables. style to develop anand understand in as few sentences as possible. The focus should be Acceptance Criteria meetings. After reviewing the initial Identify tests during to explain the intended behavior of a particular piece of functionality may describe the that required to make it happen. requirements–itnot tobecome clear stepsedge cases can also be defined. Identify tests during development. As the deliverables begin to become fleshed out, additional test cases may become apparent, and should be defined. GIVEN steps during testing. When the testing system state before test Identify testsare used to describe the expectedprocess begins, moreour desired behavior may be scenariosoccurs. discovered, and should be defined and documented. WHEN steps when errors appear. Errors/Bugs in we environment, including our Identify testsare used to describe the action that any would like to carry out on the system. live environment, should result in a bug report and tests to prove when its fixed. THEN steps describe the expected reaction or the state of the is identified then Identify tests at any other time. Whenever a valuable test casesystem after we have carried out the action in the When step. it should be documented and added to the test automation backlog.We describe our product requirements using the Gherkin syntax – Given, When, Then.
  • 12. Building an effective test automation suiteStory: Account Holder withdraws cash Stories should be described in Gherkin/BDD Identify tests during Story creation. style to develop an ubiquitous language and easily testable deliverables.As an Account Holder Acceptance Criteria meetings. After reviewing the initial Identify tests duringI want to withdraw may become clear that edge cases can also be defined. requirements it cash from an ATMSo that I can get during development. As the deliverables begin to become fleshed Identify tests money when the bank is closed out, additional test cases may become apparent, and should be defined.Scenario: Card has beentesting. When the testing process begins, more test Identify tests during disabledGiven the card is disabled scenarios may be discovered, and should be defined and documented.When the Account Holder requests $20 Identify tests when errors appear. Errors/Bugs in any environment, including ourThen the ATM shouldshould the card a bug report and tests to prove when its fixed. live environment, retain result inAnd the ATM should say the card has been retained Identify tests at any other time. Whenever a valuable test case is identified then it should be documented and added to the test automation backlog. http://dannorth.net/whats-in-a-story/We describe our product requirements using the Gherkin syntax – Given, When, Then.
  • 13. Building an effective test automation suiteStory: Account Holder withdraws cash Stories should be described in Gherkin/BDD Identify tests during Story creation. style to develop an ubiquitous language and easily testable deliverables.As an Account Holder Acceptance Criteria meetings. After reviewing the initial Identify tests duringI want to withdraw may become clear that edge cases can also be defined. requirements it cash from an ATMSo that I can get during development. As the deliverables begin to become fleshed Identify tests money when the bank is closed out, additional test cases may become apparent, and should be defined.Scenario: Card has beentesting. When the testing process begins, more test Identify tests during disabledGiven the card is disabled <= Description of state beforeand documented. scenarios may be discovered, and should be defined the actionWhen the Account Holder requests $20 <= The action toenvironment, including our Identify tests when errors appear. Errors/Bugs in any be carried outThen the ATM shouldshould the card a bug reportsystem should reactwhen its fixed. live environment, retain result in <= How the and tests to prove to the actionAnd the ATM should say the card has been retained <= The systems state afterward Identify tests at any other time. Whenever a valuable test case is identified then it should be documented and added to the test automation backlog. http://dannorth.net/whats-in-a-story/We describe our product requirements using the Gherkin syntax – Given, When, Then.
  • 14. Building an effective test automation suite Identify tests during Story creation. Stories should be described in Gherkin/BDD style to develop an ubiquitous language and easily testable deliverables. Identify tests during Acceptance Criteria meetings. After reviewing the initial requirements it may become clear that edge cases can also be defined. Identify tests during development. As the deliverables begin to become fleshed out, additional test cases may become apparent, and should be defined. Identify tests during testing. When the testing process begins, more test scenarios may be discovered, and should be defined and documented. Identify tests when errors appear. Errors/Bugs in any environment, including our live environment, should result in a bug report and tests to prove when its fixed. Identify tests at any other time. Whenever a valuable test case is identified then it should be documented and added to the test automation backlog.Once our scenarios have been Identified and Described then they need to be implementedas automated tests in our chosen test automation tools - currently Cucumber & Capybara.
  • 15. Building an effective test automation suite The Gherkin during Story included Stories should in described in Gherkin/BDD Identify testsscenarios are creation. as feature files be our test automation project develop style to (PACT). an ubiquitous language and easily testable deliverables. All defined tests are Acceptance Criteria meetings. After reviewing the initial Identify tests during initially tagged as @not_started so they are not included in test runs. requirements it may become clear that edge cases can also be defined. When a testsis being development. then the tag changes from @not_started to Identify test during implemented As the deliverables begin to become fleshed @wip. Only one test should ever have apparent, and out, additional test cases may become a @wip tag. should be defined. When the test is successfully implemented then the @wip tag changes to Identify tests during testing. When the testing process begins, more test a @complete tag so it is included in test runs. scenarios may be discovered, and should be defined and documented. Tests may also have other appear. Errors/Bugs in any they belong to. For Identify tests when errors tags to describe what area environment, including our example @messenger or result in a bug report and tests to prove when its fixed. live environment, should @page_manager. This allows selective test runs. Tests should at any other time. run on all a valuable test case is identified then Identify testsbe implemented to Whenever our testable environments or should be tagged to indicate the and added to on test automation backlog. it should be documented environments the which they cannot be run. e.g. @not_staging or @not_production.Once our scenarios have been Identified and Described then they need to be implementedas automated tests in our chosen test automation tools - currently Cucumber & Capybara.
  • 16. Building an effective test automation suiteCompleted tests should be run as often as possible, by as many people as possible, on as many environments, operating systems and browsers as possible.
  • 17. Building an effective test automation suite During tests during of tests, they should be run on the local PC of the test IdentifydevelopmentStory creation. Stories should be described in Gherkin/BDD implementer using ubiquitous language and easily testable deliverables. style to develop an their local OS and Browser configurations. After completion of Acceptance Criteria meetings. After a variety the initial Identify tests during the tests then then should be run onreviewing of supported OS and Browser combinations using our own machines or virtualized systems requirements it may become clear that edge cases can also be defined. such as Saucelabs ondevelopment. As the deliverables begin to become fleshed Identify tests during a regular basis. During our sprint testing may become suites should should be defined. out, additional test casescycles the testapparent, and be run by QA engineers to prove existing functionality When the testing regressions. Identify tests during testing.and catch potentialprocess begins, more test As a regular ongoing process and should be defined and documented. scenarios may be discovered,the test suites should be run against the latest code which is considered appear. Errors/Bugs in any environment, including our Identify tests when errorscomplete by the developers. live environment, should result in a bug report and tests to prove when its fixed. Identify tests at any other time. Whenever a valuable test case is identified then it should be documented and added to the test automation backlog.Completed tests should be run as often as possible, by as many people as possible, on as many environments, operating systems and browsers as possible.
  • 18. Building an effective test automation suite In order to maximize the value of our test automation and to ensure any potentialproblems are caught as early as possible – they should be integrated in a CI environment.
  • 19. Building an effective test automation suite Our tests are during Story creation. Stories should be of environments without Identify tests designed to be run regularly on a varietydescribed in Gherkin/BDD any manual setup or configuration steps. They are testable deliverables. style to develop an ubiquitous language and easily also designed to be quick and independent during other. Identify tests of eachAcceptance Criteria meetings. After reviewing the initial To ensure that the tests are run as that as possible and are constantly requirements it may become clear oftenedge cases can also be defined. proving the correctness of our systems, they will be integrated begin to become fleshed Identify tests during development. As the deliverables into a continuous testing environment, and run on a regular basis against and should be defined. out, additional test cases may become apparent,all our testable environments. We will tests during testing. When integration tool, to automate test runs. Identify use Jenkins, the continuousthe testing process begins, more test These will occur on both a scheduled basis and as new versions of our scenarios may be discovered, and should be defined and documented.code are deployed onto the errors appear. Errors/Bugs in any environment, including our Identify tests whentestable environments. Some environments will receivein a bug report and tests to proveawhen its fixed. live environment, should result more comprehensive testing on more regular basis. For example, other time. Whenever a the Test test case is environment Identify tests at anyall tests would be run on valuable Automationidentified then whenever it is deployed (multiple times every day) but only backlog. it should be documented and added to the test automation a subset of tests would be run on Production (once every 2 weeks). In order to maximize the value of our test automation and to ensure any potentialproblems are caught as early as possible – they should be integrated in a CI environment.
  • 20. Building an effective test automation suite Everyone on the team has a part to play when it comes to ensuring that we have acomprehensive test suite which describes the intended behavior of Wildfire’s software.
  • 21. Building an effective test automation suite Ensure that all the sprints stories have enough Gherkin/BDD scenarios to get the discussion started during planning sessions. The scenarios should be short but concise and written from the perspective of a user of the system. Ideally these scenarios would be written in advance of any planning sessions and circulated to the rest of the team for review. Follow up with the rest of the team both before, during and after work on the story has been undertaken to ensure that your vision has been understood by everyone involved in the development process. Ask to see the automated tests running when the story is marked as Done. Everyone on the team has a part to play when it comes to ensuring that we have acomprehensive test suite which describes the intended behavior of Wildfire’s software.
  • 22. Building an effective test automation suite Review the Gherkin/BDD scenarios of a story prior to the planning and AC meetings to have a clear understanding of what is being asked. Give feedback and ask questions before, during and after the planning sessions. Work with the PM and QA’s to identify any other core scenarios and edge cases which need to be added to the feature to ensure it is complete. Work with the QA’s to ensure that the feature is testable in both a manual and automated fashion on all of our testable environments. When code complete – run the automated acceptance test suite on your environment to ensure that nothing else got broken. Follow up with the test automators to ensure that all environments are tested. Everyone on the team has a part to play when it comes to ensuring that we have acomprehensive test suite which describes the intended behavior of Wildfire’s software.
  • 23. Building an effective test automation suite Review the Gherkin/BDD scenarios of a story prior to the planning and AC meetings to have a clear understanding of what is being asked. Give feedback and ask questions before, during and after the planning sessions. Work with the PM and Dev’s to identify any other core scenarios and edge cases which need to be added to the feature to ensure it is complete. Work with the test automators to create a prioritized backlog of tests which should be added to the automated test suite. Checkout the automated test suite and run it on a regular basis to help catch regressions and identify environmental issues. Work with the automators to ensure your project has good test coverage. Everyone on the team has a part to play when it comes to ensuring that we have acomprehensive test suite which describes the intended behavior of Wildfire’s software.
  • 24. Building an effective test automation suite Review the Gherkin/BDD scenarios of a story prior to the planning and AC meetings to have a clear understanding of what is being asked. Give feedback and ask questions before, during and after the planning sessions. Work with the PM and Dev’s to identify any other core scenarios and edge cases which need to be added to the feature to ensure it is complete. Work with the QA’s to create a prioritized backlog of tests which should be added to the automated test suite. Work with the Developers to ensure that automated tests can be run in any environment and to ensure the test automation environment is suitable for test. Keep the test suite in a well maintained state so that it can be run at any time. Everyone on the team has a part to play when it comes to ensuring that we have acomprehensive test suite which describes the intended behavior of Wildfire’s software.
  • 25. Building an effective test automation suiteFinally we have some goals which we should aim for with our Test Automation.
  • 26. Building an effective test automation suiteOur test automation suite should deliver wide ranging yet concise coverage ofthe existing functionality provided by Wildfire’s applications. This is to helpidentify and fix regressions at the earliest possible opportunity.As new stories are implemented on a sprint by sprint basis we should identifyevery potentially automatable story and add it to the automation backlog.The test suite should be deployable by anyone on the team and should requireno manual intervention or manual setup tasks to run.In collaboration with the developers and system administrators we must worktowards having a completely isolated test automation environment which is freeof dependencies on third party systems.Our ultimate goal should be to have a self testing system which automaticallydeploys and tests itself on various environments and provides feedback onpotential problems and tells us when it is ready to be deployed to ourproduction environment – all without any manual intervention.Finally we have some goals which we should aim for with our Test Automation.
  • 27. Building an effective test automation suite One last thing…
  • 28. Building an effective test automation suiteAn automated test suite is not, and never will be, a replacement for manualtesting activities.The goal of an automated test suite is to provide fast and regular feedback on theknown and predictable elements within our system.We will always require manual testing in order to identify and understand theunknown or unpredictable elements within our system.Our goal should be to find the right balance between manual and automatedtesting which enables us to deliver software in a fast, efficient and confidentmanner. One last thing…
  • 29. Building an effective test automation suite This is still a work in progress. Please give your feedback to make this a more accurate representation of the process we want to achieve. Thanks..

×