Download as PDF, PPTX
Uploaded byThoughtworks
Build the ‘Right’ Regression Suite using Behavior Driven Testing (BDT)
The document discusses using behavior driven testing (BDT) to build an effective regression test suite. It describes challenges with existing testing including brittle, outdated tests. BDT involves identifying business flows and prioritizing them to determine which should be automated. Top flows are automated using BDD tools or programming languages. This provides executable specifications, removes ambiguity, and allows testing in sync with product functionality.
More Related Content
Similar to Build the ‘Right’ Regression Suite using Behavior Driven Testing (BDT)
Build the ‘Right’ Regression Suite using Behavior Driven Testing (BDT)
- 1. BUILD THE “RIGHT” REGRESSIONSUITE USING BEHAVIOR DRIVEN TESTING (BDT) vodQA
- 2.
- 3. WHAT DO YOUEXPECT FROM THIS SESSION?
- 4. CASE STUDY • Multiple,long running projects • Legacy applications • Integrated
- 5. • Limited /flaky / long-running Automation • Long regression cycle • Huge cost of fixing defects FUNCTIONAL TESTING CHALLENGES
- 6. • Brittle • Little/ less value • No visibility into what is tested • Maintenance nightmare • Outdated • Expensive AS A RESULT
- 7. • Trust deficiton the team • In existing Automation • In manual testing • Finger pointing / blame game BIGGEST PROBLEM
- 8.
- 10. Concept #1 Given ->When -> Then
- 11. Expected Functionality: The customershould be able to withdraw money from his account via an ATM machine
- 12. BDD Example: Given theaccount has balance of 5000 When the customer requests 1000 Then the account is debited by 1000
- 13.
- 14. Workshop Begins –Iteration 1 Case Study
- 16. PHASE 1 REQUIREMENTS • Guest User is able to search for a flight for a single traveller • Select option from Search Results page • Specify contact information for person booking the flight
- 17. A Guest Usercan … WORKFLOW Search Search Results Contact Information
- 18.
- 19.
- 20.
- 21.
- 22. PHASE 1 REQUIREMENTS • Guest User is able to search for a flight for a single traveller • Select option from Search Results page • Specify contact information for person booking the flight
- 23. A Guest Usercan … WORKFLOW Search Search Results Contact Information
- 24. LETS GET STARTED… WRITE 2 TEST SCENARIOS
- 25.
- 26.
- 27. • Imperative • Declarative TESTSPECIFICATION STYLES
- 28. A Guest Usercan … AN EXAMPLE Search Search Results Contact Information
- 29. IMPERATIVE STYLE Given Iam a guest user on the home page And I choose “round” trip option And I select “Chicago” from the origin dropdown And I select “San Francisco” from the destination dropdown And I select departure date as “5 December 2013” And I select returning date as “25 December 2013” When I click on Search Then I should see the search results page And I should see at least 1 option for my criteria ...
- 30. IMPERATIVE STYLE … … WhenI select the “first” option Then I am on the Contact Information page When I enter first name as “Anand” And I enter last name as “Bagmar” … … And I click the “Select and Continue” button Then I should be on the next page
- 31. DECLARATIVE STYLE Given Iam a guest user When I search for flight options for a “one-way” trip for “1” “Adult” from “Chicago” to “San Francisco” And I select the “first” flight And I enter “valid” contact details for “traveller1” Then I am able to Save and Continue
- 32. • Ben Mabey- http://benmabey.com/2008/05/19/ imperative-vs-declarative-scenarios-in- user-stories.html • Alister Scott - http://watirmelon.com/2010/12/10/ cucumber-imperative-or-declarative-that- is-the-question/ REFERENCES
- 33. Workshop – Iteration2 Case Study Continues
- 34. • User registrationfrom Contact information page • Registered user can login from home page (below search criteria) • For logged-in user, when booking ticket, contact information is pre-filled with logged-in user information PHASE 2 REQUIREMENTS
- 35.
- 37.
- 39. • User registrationfrom Contact information page • Registered user can login from home page (below search criteria) • For logged-in user, when booking ticket, contact information is pre-filled with logged-in user information PHASE 2 REQUIREMENTS
- 40.
- 41. LETS GET STARTED… WRITE 2 TEST SCENARIOS
- 42.
- 43. WHY IS THISIMPORTANT?
- 44. Concept #3 The TestPyramid
- 45. AUTOMATION – ASAFETY NET
- 46. • “Succeeding withAgile” – Mike Cohn • Martin Fowler – Test Pyramid martinfowler.com/bliki/TestPyramid.html THE TEST PYRAMID
- 47. Time Value Cost IDEAL TEST AUTOMATIONPYRAMID Manual / Exploratory UI Web Service View JavaScript Integration Unit Technology- facing Tests Business-facing Tests
- 48.
- 49. UI Web Service View JavaScript Integration Unit Manual /Exploratory ICE CREAM CONE ANTI PATTERN
- 50. Developers Test PyramidQA Team Test Pyramid Web Service View JavaScript Integration Unit Manual / Exploratory UI Web Service DUAL TEST PYRAMID ANTI-PATTERN
- 51. Concept #4 Behavior DrivenTesting (BDT)
- 52.
- 53. • How isthis functionality going to be used by the end-user? • What is their thought process going to be when using this product? • What “core-business-value” should be delivered to the end-user? THE BIG PICTURE
- 54. Manual / Exploratory UI WebService View JavaScript Integration Unit Technology- facing Tests Business-facing Tests Identify the “right” type of tests that sit on top of the Test Pyramid
- 55.
- 57. HOW TO SOLVETHE PROBLEM?
- 58. • Remove theambiguity • Same visibility for all SOLUTION
- 59. • Whiteboards • Mindmaps • Flow charts • Collaboration!! TOOLS TO ENABLE BDT
- 60. HOW DID WEACCOMPLISH THIS?
- 61.
- 62. • Identify businessflows based on personas, and how the personas use the business functionality, or, • Identify business flows, and how different user personas can use the business flows NOTE: Business flows are different from UI flows HOW TO IDENTIFY BUSINESS FLOWS ?
- 63.
- 64. 1st candidate • Automate! • High Value, Low Cost 2nd candidate • Automate! • High Value, High Cost Can Automate? • Manual Regression? • Low Value, Low Cost Do Not Automate • Manual Regression! • Low Value, High Cost High Value High Cost Low Value Low Cost
- 65. AUTOMATE THE TOPLAYER USING • BDD Tools, or, • Directly in any appropriate programming language / tools
- 66. • Identification ofTests • Automate it! WHAT’S NEXT?
- 67. AUTOMATE TESTS THATARE VALUABLE, NOT BECAUSE YOU CAN OR LIKE TO! CAUTION!
- 68. Who is doingthe action? What is the business functionality? Depicts a user flow / user journey Actions & Verifications happen implicitly Given auctioneer creates an auction And 5 vehicles are added to the auction When auctioneer starts the auction And auctioneer sets a starting bid of 5000 And buyer_1 bids And buyer_2 bids Then auctioneer Sells the vehicle And buyer_2 wins the vehicle EXAMPLE
- 69.
- 70. 1st candidate • Automate! • High Value, Low Cost 2nd candidate • Automate! • High Value, High Cost Can Automate? • Manual Regression? • Low Value, Low Cost Do Not Automate • Manual Regression! • Low Value, High Cost High Value High Cost Low Value Low Cost
- 71. Project Management toolMingle Test case repository (manual + automated) Cucumber .feature files with appropriate tagging Feature files / Scenarios 33 / 65 # of Manual Tests 0 Tests running on CI (Jenkins) Since the 1st test was automated Smoke tests execution time 15 minutes Regression tests execution time 45 minutes FINAL STATE … IN MY CASE STUDY
- 72. • Builds anddeployments were automated • Tests ran on demand, automatically! • Smoke / Regression / Javascript / Unit • Functional Performance • Concurrent Testing ADDITIONALLY …
- 73. How did thisTechnique (BDT) become successful?
- 74. CONTEXT IS KING! Domain Distributed Teams Skills CapabilitiesProduct-under test Tech Stack Process Tools
- 75. NOT work inisolation BDT WILL …
- 76. BDT REQUIRES … Lowerlayers of Test Pyramid to be well automated!
- 77. BDT REQUIRES … Dedicatedand Focused testing (manual / exploratory)
- 79. BDT helps identifythe “right” type of regression tests!
- 80.
- 81. BDT VALUE PROPOSITION… Incrementally build Business Flows Regress Business Value Test in-sync with Product Functionality Effective Tests
- 82. BDT VALUE PROPOSITION… Remove ambiguity Understand new functionality Living Documentation Onboarding
- 83.