Your SlideShare is downloading. ×

Build the Right Regression Suite with Behavior-Driven Testing

460

Published on

Manual functional testing is a slow, tedious, and error prone process. As we continue to incrementally build software, the corresponding regression test suite continues to grow. Rarely is time …

Manual functional testing is a slow, tedious, and error prone process. As we continue to incrementally build software, the corresponding regression test suite continues to grow. Rarely is time allotted to consolidate and keep these test cases in sync with the product under development. If these test cases are used as the basis for automation, the resulting suite is composed of very granular tests that are often quite brittle in nature. Using a case study, Anand Bagmar describes how behavior-driven testing (BDT) can be applied to identify the right type of test cases for manual and automated regression testing. Learn how the BDT technique can be applied in your context and domain, regardless of the tools and technologies used in your project and organization.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
460
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. T14 Test Techniques 5/8/2014 1:30:00 PM Build the Right Regression Suite with Behavior-Driven Testing Presented by: Anand Bagmar ThoughtWorks Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  • 2. Anand Bagmar ThoughtWorks Anand Bagmar is a hands-on and result-oriented software quality evangelist with seventeen years in the IT field. Passionate about shipping quality products, Anand specializes in building automated testing tools, infrastructure, and frameworks. He writes testing-related blogs and has built open-source tools related to software testing-WAAT (Web Analytics Automation Testing Framework), TaaS (for automating integration testing in disparate systems), and TTA (Test Trend Analyzer). Anand is the lead organizer for vodQA, the popular testing conference in India. Follow him on Twitter @BagmarAnand, email him at abagmar@gmail.com, or read his Essence of Testing blog.
  • 3. Building  the  “right”   regression  suite  using   Behavior  Driven  Tes5ng   (BDT)   Anand  Bagmar   So#ware  Quality  Evangelist   Principal  Consultant,  ThoughtWorks  
  • 4. •  “Succeeding  with  Agile”  –  Mike  Cohn   •  MarCn  Fowler  –  Test  Pyramid   –  hFp://marCnfowler.com/bliki/TestPyramid.html   The  Test  Pyramid  
  • 5. Func%onal  Tests  (GUI)   •  Developers  &  /  QA  team   Web  Service  Tests   •  Developers  &  /  QA  team   Integra%on  tests   • Developers   Component  tests   •  Developers   Unit  Tests   •  Developers   Test  AutomaCon  Pyramid   Manual  /   Exploratory   TesCng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   •  Cost   •  Effort   •  Time  
  • 6. Test  Pyramid  reality   Development  team   Test  Pyramid   QA  team   Test  Pyramid  
  • 7. Func%onal  Tests  (GUI)   •  Developers  &  /  QA  team   Web  Service  Tests   •  Developers  &  /  QA  team   Integra%on  tests   • Developers   Component  tests   •  Developers   Unit  Tests   •  Developers   Manual  /  Exploratory  TesCng   Test     Ice-­‐cream  cone   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   Test  Pyramid  –  AnC  PaFern   Product     under  test  
  • 8. Func%onal  Tests  (GUI)   •  Developers  &  /  QA  team   Web  Service  Tests   •  Developers  &  /  QA  team   Integra%on  tests   • Developers   Component  tests   •  Developers   Unit  Tests   •  Developers   Manual  /   Exploratory   TesCng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   IdenCfy  the  right  type  of  tests  that  sit   on  top  of  the  Test  Pyramid  
  • 9. Case  Study   •  MulCple,  long   running  projects   •  Legacy   applicaCons   •  Integrated  
  • 10. •  Limited  AutomaCon     •  Long  regression  cycle   •  Huge  cost  of  fixing  defects   TesCng  Challenges  
  • 11. •  No  visibility  into  what  is  tested   •  Outdated   •  BriFle   •  LiFle  /  less  value   •  Expensive   •  Maintenance  nightmare   As  a  result  
  • 12. Biggest  problem   •  Trust  deficit  on  the  team   – In  exisCng  AutomaCon   – In  manual  tesCng   •  Finger  poinCng  /  blame  game  
  • 13. •  Remove  the  ambiguity   •  Same  visibility  for  all  
  • 14. EvoluCon  of  Test  Frameworks   Behavior   Driven   Development   (BDD)   OpCmized   scripts   Record  &   Playback     Core   Framework  
  • 15. Behavior  Driven  Development     (BDD)  
  • 16. RED   GREEN  REFACTOR  
  • 17. Behavior  Driven  TesCng   (BDT)  
  • 18. The  Big  Picture   •  How  is  this  funcConality  going  to  be  used  by   the  end-­‐user?     •  What  is  their  thought  process  going  to  be   when  using  this  product?     •  What  is  the  “core-­‐business-­‐value”  I  am   delivering  to  the  end-­‐user?  
  • 19. Func%onal  Tests  (GUI)   •  Developers  &  /  QA  team   Web  Service  Tests   •  Developers  &  /  QA  team   Integra%on  tests   • Developers   Component  tests   •  Developers   Unit  Tests   •  Developers   Manual  /   Exploratory   TesCng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   IdenCfy  the  right  type  of  tests  that  sit   on  top  of  the  Test  Pyramid  
  • 20. Expected  Func%onality:     The  customer  should  be  able  to  withdraw   money  from  his  account  via  an  ATM  machine   An  example  
  • 21. Given  the  account  has  balance  of  INR  5000   When  the  customer  requests  INR  1000   Then  the  account  is  debited  by  INR  1000   BDD  Example:  
  • 22. Given  the  account  is  in  credit   And  the  dispenser  contains  cash     When  the  customer  requests  cash     Then  ensure  the  account  is  debited   And  ensure  cash  is  dispensed   And  ensure  the  card  is  returned   BDT  Example:  
  • 23. New  Expected  Func%onality:     The  customer  should  be  able  to  withdraw   money  from  his  account  only  if  his  card  is   valid  
  • 24. Given  the  account  is  in  credit   And  the  card  is  valid   And  the  dispenser  contains  cash     When  the  customer  requests  cash     Then  ensure  the  account  is  debited   And  ensure  cash  is  dispensed   And  ensure  the  card  is  returned   BDT  Example  -­‐  updated:  
  • 25. Test  specificaCon  styles   •  ImperaCve     •  DeclaraCve  
  • 26. •  Write  user  flows  based  on  personas,  and  how   they  use  the  business  funcConality,  or,   •  Write  business  flows,  and  how  different  user   personas  can  use  that  
  • 27. Another  example   For  Indigo  Airlines  website  –   •  Guest  User  is  able  to  search  for  a  flight  for  a   single  traveller   •  Ability  to  specify  contact  informaCon  for   person  booking  the  flight  
  • 28. Search     Search   Results   Contact   Informa%on  
  • 29. ImperaCve  style   Given  I  am  a  guest  user  on  the  Indigo  home  page   And  I  select  “round”  trip  opCon   And  I  select  “Pune”  from  the  origin  dropdown   And  I  select  “Bangalore”  from  the  desCnaCon  drop  down   And  I  select  departure  date  as  “5  December  2013”   And  I  select  returning  date  as  “25  December  2013”     When  I  click  on  Search     Then  I  should  see  the  search  results  page     And  I  should  see  at  least  1  opCon  for  my  criteria     ...  
  • 30. …   When  I  select  the  “first”  opCon   Then  I  am  on  the  Contact  InformaCon  page     When  I  enter  first  name  as  “foo”   And  I  enter  last  name  as  “bar”   …   …   And  I  click  the  “Select  and  Con%nue”  buFon     Then  I  should  be  on  the  next  page    
  • 31. Given  I  am  a  guest  user     When  I  search  for  flight  opCons  for  a  “one-­‐way”  trip   for  “1”  “Adult”  from  “Delhi”  to  “Bangalore”   And  I  select  the  “first”  flight   And  I  enter  “valid”  contact  details  for  “traveller1”     Then  I  am  able  to  Save  and  ConCnue   DeclaraCve  style  
  • 32. What’s  next?  
  • 33. DO  NOT   AUTOMATE   •  Low  Value,  High  Cost   •  Manual   Regression!   2nd  candidate  for   automa%on   •  High  Value,  High  Cost   Automate  ONLY  if   you  have  %me   •  Low  Value,  Low  Cost   •  ?  Manual   Regression?   1st  candidate  for   automa%on   •  High  Value,  Low  Cost   Cost   Value  
  • 34. Func%onal  Tests  (GUI)   •  Developers  &  /  QA  team   Web  Service  Tests   •  Developers  &  /  QA  team   Integra%on  tests   • Developers   Component  tests   •  Developers   Unit  Tests   •  Developers   BDT  helps  iden5fy  the  “right”  type  of   regression  tests!   Manual  /   Exploratory   TesCng   Product   under  test   Ideal  Test   Pyramid  
  • 35. A  real  world  Case  Study  
  • 36. How  did  we  accomplish  this?  
  • 37. Who  is  doing  the   acCon?   What  is  the  business   funcConality?   AcCons  and   verificaCons  happen   implicitly   Depicts  a  user  flow   Given  auc%oneer  creates  a  sale   And  5  items  are  added  to  the   sale             When  auc%oneer  starts  the   sale   And  auc%oneer  sets  a  starCng   bid  of  5000   And  buyer_1  bids   And  buyer_2  bids     Then  auc%oneer  Sells  the  item   And  buyer_2  wins  the  item  
  • 38. Final  state   Tests  running  on  Hudson  /  Jenkins   Since  1st  test  was  automated   Feature  files   33   Scenarios   65   Smoke  tests  execuCon  Cme   15  minutes   Regression  tests  execuCon  Cme   45  minutes   Test  case  repository  (manual  +   automated)   Cucumber  .feature  files  with   appropriate  tagging   Project  Management  tool   Mingle  
  • 39. ?   Incrementally  build   User  Flows   Regress  required   Product  Business   Value   Tests  in-­‐sync   EffecCve  Tests       Remove   ambiguity   Understand   new   funcConality   Living   DocumentaCon   Onboarding  
  • 40. Tools  to  enable  BDD  /  BDT   •  Cucumber   •  JBehave   •  SpecFlow   •  Twist   •  Concordion  
  • 41. abagmar@gmail.com                @BagmarAnand     about.me/anand.bagmar   Thank  you  

×