Test automation goals, decisions.
Test automation approaches.
Behavior based testing.
Horváth Anita, EPAM
Anita_Horvath@epam.com
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Test automation
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Benefits of test automation
• saving time and workforce with avoiding repeated manual work
• ensure that we always execute tests consistently (elimination of human error)
• reduce the difficult manual tests of multilingual sites
• increase the speed of test execution
• getting faster feedback
• run daily tests (overnight, then get the report every morning)
• test different environments at the same time
3
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Test automation pyramid
• Testing Pyramid is a framework that can help both developers
and QAs create high-quality software.
• It reduces the time required for developers to identify if a
change they introduced breaks the code.
• It can also be helpful in building a more reliable test suite.
• The test automation pyramid operates at three levels:
• Unit tests
• Integration tests
• End-to-End tests
4
ttps://www.browserstack.com/guide/testing-pyramid-for-test-automation
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Test automation approaches
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Main questions
• Can we automate our manual tests?
• Certain test types can only be executed in an automated way (e.g. stress tests, load tests)
• How frequently should the tests be run?
• Another key factor is the complexity of the tests
• How many tests do we have? Should we parallelize them?
• Are the tests dependent of each other?
• What preconditions are required before test execution?
• What kind of tool do we need? (tools for test automation can be expensive)
6
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Time-quality-budget triangle
7
https://www.enterprise-pm.com/pmbasics/triple-constraint/
Quality
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Return of investment
• The return on the investment measures the overall profit on an investment expressed as a
percentage of the amount invested.
8
ps://smartbear.com/resources/ebooks/6-ways-to-measure-the-roi-of-automated-testing/
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Prioritizing
• business critical test cases (with high risk)
• test cases which are executed repeatedly
• tests which are difficult to perform manually
• time consuming tests
W H I C H T E S T S S H O U L D B E A U TO M AT E D
9
T E S T S N O T S U I TA B L E F O R A U TO M AT I O N
• actions which should be performed just a few times
• features which are changing to often
• test cases which are faster to test manually rather than develop automated tests for them
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Behavior Based Testing
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
BDD & BDT Advantages
• It involves getting stakeholders and delivery team with different perspectives onto the same page and
ensuring that all have the same expectations.
• BDD starts with a business goal that translates to features and stories.
• The goal of BDT is a business readable and domain specific language that allows you to describe a
system’s behavior without detailing how that behavior is implemented.
• In BDT, the feature descriptions with scenarios are typically written before anything else and they are
verified by the non-technical stakeholders.
B D D - B E H AV I O R D R I V E N D E V E LO P M E N T
B D T – B E H AV I O R D R I V E N T E S T I N G
11
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Gherkin
H T T P S : / / W W W. G U R U 9 9 . CO M / G H E R K I N - T E S T - C U C U M B E R . H T M L
12
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Tools and testing frameworks
H T T P S : / / W W W. S O F T WA R E T E S T I N G H E L P. C O M / B E H AV I O R- D R I V E N - D E V E LO P M E N T - B D D - TO O L S /
13
1. Hiptest - the only continuous testing tool with native support
2. Cucumber – free to use; as a tool maintains both the requirement statement and the Test conditions
in the same place
3. Easy B - is mostly written with Groovy and is compatible to work with Java or Groovy languages;
mostly used in unit testing
4. Jdave
5. Concordion
6. Jbehave
7. Fitnesse
8. TestLeft
9. BeanSpec
10.SpecFlow
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Gherkin syntax
14
Feature: Short description of the
feature
Scenario: Title of the proper test case
Given [Precondition or Initial Context]
And [Another precondition]
When [Event or Trigger]
Then [Expected output]
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Important terms used in Gherkin
• Feature
• Background
• Scenario
• Given
• When
• Then
• And
• But
• Scenario Outline & Examples (https://cucumber.io/docs/gherkin/reference/ )
15
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Practice
HT TPS://WWW.SPOTIFY.COM/HU/
16
• Gather all test cases for the registration form
• Write feature file for it:
• Pull the framework from Gitlab: https://gitlab.com/unideb-szoftverteszteles/spotify
• Open a cmd or gitbash and go to the right place
• Type: git clone https://gitlab.com/unideb-szoftverteszteles/spotify.git
• Open the project in IntelliJ IDEA
• Create the spotify.feature under the feature-files package
CONFIDENTIAL | © 2020 EPAM Systems, Inc.
THANK YOU FOR YOUR ATTENTION
17

Introduction to Test Automation Notes.pptx

  • 1.
    Test automation goals,decisions. Test automation approaches. Behavior based testing. Horváth Anita, EPAM Anita_Horvath@epam.com CONFIDENTIAL | © 2020 EPAM Systems, Inc.
  • 2.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Test automation
  • 3.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Benefits of test automation • saving time and workforce with avoiding repeated manual work • ensure that we always execute tests consistently (elimination of human error) • reduce the difficult manual tests of multilingual sites • increase the speed of test execution • getting faster feedback • run daily tests (overnight, then get the report every morning) • test different environments at the same time 3
  • 4.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Test automation pyramid • Testing Pyramid is a framework that can help both developers and QAs create high-quality software. • It reduces the time required for developers to identify if a change they introduced breaks the code. • It can also be helpful in building a more reliable test suite. • The test automation pyramid operates at three levels: • Unit tests • Integration tests • End-to-End tests 4 ttps://www.browserstack.com/guide/testing-pyramid-for-test-automation
  • 5.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Test automation approaches
  • 6.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Main questions • Can we automate our manual tests? • Certain test types can only be executed in an automated way (e.g. stress tests, load tests) • How frequently should the tests be run? • Another key factor is the complexity of the tests • How many tests do we have? Should we parallelize them? • Are the tests dependent of each other? • What preconditions are required before test execution? • What kind of tool do we need? (tools for test automation can be expensive) 6
  • 7.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Time-quality-budget triangle 7 https://www.enterprise-pm.com/pmbasics/triple-constraint/ Quality
  • 8.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Return of investment • The return on the investment measures the overall profit on an investment expressed as a percentage of the amount invested. 8 ps://smartbear.com/resources/ebooks/6-ways-to-measure-the-roi-of-automated-testing/
  • 9.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Prioritizing • business critical test cases (with high risk) • test cases which are executed repeatedly • tests which are difficult to perform manually • time consuming tests W H I C H T E S T S S H O U L D B E A U TO M AT E D 9 T E S T S N O T S U I TA B L E F O R A U TO M AT I O N • actions which should be performed just a few times • features which are changing to often • test cases which are faster to test manually rather than develop automated tests for them
  • 10.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Behavior Based Testing
  • 11.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. BDD & BDT Advantages • It involves getting stakeholders and delivery team with different perspectives onto the same page and ensuring that all have the same expectations. • BDD starts with a business goal that translates to features and stories. • The goal of BDT is a business readable and domain specific language that allows you to describe a system’s behavior without detailing how that behavior is implemented. • In BDT, the feature descriptions with scenarios are typically written before anything else and they are verified by the non-technical stakeholders. B D D - B E H AV I O R D R I V E N D E V E LO P M E N T B D T – B E H AV I O R D R I V E N T E S T I N G 11
  • 12.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Gherkin H T T P S : / / W W W. G U R U 9 9 . CO M / G H E R K I N - T E S T - C U C U M B E R . H T M L 12
  • 13.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Tools and testing frameworks H T T P S : / / W W W. S O F T WA R E T E S T I N G H E L P. C O M / B E H AV I O R- D R I V E N - D E V E LO P M E N T - B D D - TO O L S / 13 1. Hiptest - the only continuous testing tool with native support 2. Cucumber – free to use; as a tool maintains both the requirement statement and the Test conditions in the same place 3. Easy B - is mostly written with Groovy and is compatible to work with Java or Groovy languages; mostly used in unit testing 4. Jdave 5. Concordion 6. Jbehave 7. Fitnesse 8. TestLeft 9. BeanSpec 10.SpecFlow
  • 14.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Gherkin syntax 14 Feature: Short description of the feature Scenario: Title of the proper test case Given [Precondition or Initial Context] And [Another precondition] When [Event or Trigger] Then [Expected output]
  • 15.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Important terms used in Gherkin • Feature • Background • Scenario • Given • When • Then • And • But • Scenario Outline & Examples (https://cucumber.io/docs/gherkin/reference/ ) 15
  • 16.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. Practice HT TPS://WWW.SPOTIFY.COM/HU/ 16 • Gather all test cases for the registration form • Write feature file for it: • Pull the framework from Gitlab: https://gitlab.com/unideb-szoftverteszteles/spotify • Open a cmd or gitbash and go to the right place • Type: git clone https://gitlab.com/unideb-szoftverteszteles/spotify.git • Open the project in IntelliJ IDEA • Create the spotify.feature under the feature-files package
  • 17.
    CONFIDENTIAL | ©2020 EPAM Systems, Inc. THANK YOU FOR YOUR ATTENTION 17