T14
Test Techniques
5/8/2014 1:30:00 PM
Build the Right Regression Suite
with Behavior-Driven Testing
Presented by:
Anand ...
Anand Bagmar
ThoughtWorks
Anand Bagmar is a hands-on and result-oriented software quality evangelist with seventeen
years ...
Building	
  the	
  “right”	
  
regression	
  suite	
  using	
  
Behavior	
  Driven	
  Tes5ng	
  
(BDT)	
  
Anand	
  Bagmar...
•  “Succeeding	
  with	
  Agile”	
  –	
  Mike	
  Cohn	
  
•  MarCn	
  Fowler	
  –	
  Test	
  Pyramid	
  
–  hFp://marCnfow...
Func%onal	
  Tests	
  (GUI)	
  
•  Developers	
  &	
  /	
  QA	
  team	
  
Web	
  Service	
  Tests	
  
•  Developers	
  &	
...
Test	
  Pyramid	
  reality	
  
Development	
  team	
  
Test	
  Pyramid	
   QA	
  team	
  
Test	
  Pyramid	
  
Func%onal	
  Tests	
  (GUI)	
  
•  Developers	
  &	
  /	
  QA	
  team	
  
Web	
  Service	
  Tests	
  
•  Developers	
  &	
...
Func%onal	
  Tests	
  (GUI)	
  
•  Developers	
  &	
  /	
  QA	
  team	
  
Web	
  Service	
  Tests	
  
•  Developers	
  &	
...
Case	
  Study	
  
•  MulCple,	
  long	
  
running	
  projects	
  
•  Legacy	
  
applicaCons	
  
•  Integrated	
  
•  Limited	
  AutomaCon	
  	
  
•  Long	
  regression	
  cycle	
  
•  Huge	
  cost	
  of	
  fixing	
  defects	
  
TesCng	
 ...
•  No	
  visibility	
  into	
  what	
  is	
  tested	
  
•  Outdated	
  
•  BriFle	
  
•  LiFle	
  /	
  less	
  value	
  
•...
Biggest	
  problem	
  
•  Trust	
  deficit	
  on	
  the	
  team	
  
– In	
  exisCng	
  AutomaCon	
  
– In	
  manual	
  tesC...
•  Remove	
  the	
  ambiguity	
  
•  Same	
  visibility	
  for	
  all	
  
EvoluCon	
  of	
  Test	
  Frameworks	
  
Behavior	
  
Driven	
  
Development	
  
(BDD)	
  
OpCmized	
  
scripts	
  
Record...
Behavior	
  Driven	
  Development	
  	
  
(BDD)	
  
RED	
  
GREEN	
  REFACTOR	
  
Behavior	
  Driven	
  TesCng	
  
(BDT)	
  
The	
  Big	
  Picture	
  
•  How	
  is	
  this	
  funcConality	
  going	
  to	
  be	
  used	
  by	
  
the	
  end-­‐user?	
...
Func%onal	
  Tests	
  (GUI)	
  
•  Developers	
  &	
  /	
  QA	
  team	
  
Web	
  Service	
  Tests	
  
•  Developers	
  &	
...
Expected	
  Func%onality:	
  
	
  
The	
  customer	
  should	
  be	
  able	
  to	
  withdraw	
  
money	
  from	
  his	
  a...
Given	
  the	
  account	
  has	
  balance	
  of	
  INR	
  5000	
  
When	
  the	
  customer	
  requests	
  INR	
  1000	
  
...
Given	
  the	
  account	
  is	
  in	
  credit	
  
And	
  the	
  dispenser	
  contains	
  cash	
  
	
  
When	
  the	
  cust...
New	
  Expected	
  Func%onality:	
  
	
  
The	
  customer	
  should	
  be	
  able	
  to	
  withdraw	
  
money	
  from	
  h...
Given	
  the	
  account	
  is	
  in	
  credit	
  
And	
  the	
  card	
  is	
  valid	
  
And	
  the	
  dispenser	
  contain...
Test	
  specificaCon	
  styles	
  
•  ImperaCve	
  
	
  
•  DeclaraCve	
  
•  Write	
  user	
  flows	
  based	
  on	
  personas,	
  and	
  how	
  
they	
  use	
  the	
  business	
  funcConality,	
  ...
Another	
  example	
  
For	
  Indigo	
  Airlines	
  website	
  –	
  
•  Guest	
  User	
  is	
  able	
  to	
  search	
  for...
Search	
  	
  
Search	
  
Results	
  
Contact	
  
Informa%on	
  
ImperaCve	
  style	
  
Given	
  I	
  am	
  a	
  guest	
  user	
  on	
  the	
  Indigo	
  home	
  page	
  
And	
  I	
  selec...
…	
  
When	
  I	
  select	
  the	
  “first”	
  opCon	
  
Then	
  I	
  am	
  on	
  the	
  Contact	
  InformaCon	
  page	
  
...
Given	
  I	
  am	
  a	
  guest	
  user	
  
	
  
When	
  I	
  search	
  for	
  flight	
  opCons	
  for	
  a	
  “one-­‐way”	
...
What’s	
  next?	
  
DO	
  NOT	
  
AUTOMATE	
  
•  Low	
  Value,	
  High	
  Cost	
  
•  Manual	
  
Regression!	
  
2nd	
  candidate	
  for	
  
...
Func%onal	
  Tests	
  (GUI)	
  
•  Developers	
  &	
  /	
  QA	
  team	
  
Web	
  Service	
  Tests	
  
•  Developers	
  &	
...
A	
  real	
  world	
  Case	
  Study	
  
How	
  did	
  we	
  accomplish	
  this?	
  
Who	
  is	
  doing	
  the	
  
acCon?	
  
What	
  is	
  the	
  business	
  
funcConality?	
  
AcCons	
  and	
  
verificaCons...
Final	
  state	
  
Tests	
  running	
  on	
  Hudson	
  /	
  Jenkins	
   Since	
  1st	
  test	
  was	
  automated	
  
Featu...
?	
  
Incrementally	
  build	
  
User	
  Flows	
  
Regress	
  required	
  
Product	
  Business	
  
Value	
  
Tests	
  in-­...
Tools	
  to	
  enable	
  BDD	
  /	
  BDT	
  
•  Cucumber	
  
•  JBehave	
  
•  SpecFlow	
  
•  Twist	
  
•  Concordion	
  
abagmar@gmail.com	
  	
  
	
  
	
  	
  	
  	
  	
  @BagmarAnand	
  
	
  
about.me/anand.bagmar	
  
Thank	
  you	
  
Build the Right Regression Suite with Behavior-Driven Testing
Build the Right Regression Suite with Behavior-Driven Testing
Build the Right Regression Suite with Behavior-Driven Testing
Build the Right Regression Suite with Behavior-Driven Testing
Build the Right Regression Suite with Behavior-Driven Testing
Upcoming SlideShare
Loading in …5
×

Build the Right Regression Suite with Behavior-Driven Testing

704 views
571 views

Published on

Manual functional testing is a slow, tedious, and error prone process. As we continue to incrementally build software, the corresponding regression test suite continues to grow. Rarely is time allotted to consolidate and keep these test cases in sync with the product under development. If these test cases are used as the basis for automation, the resulting suite is composed of very granular tests that are often quite brittle in nature. Using a case study, Anand Bagmar describes how behavior-driven testing (BDT) can be applied to identify the right type of test cases for manual and automated regression testing. Learn how the BDT technique can be applied in your context and domain, regardless of the tools and technologies used in your project and organization.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
704
On SlideShare
0
From Embeds
0
Number of Embeds
22
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Build the Right Regression Suite with Behavior-Driven Testing

  1. 1. T14 Test Techniques 5/8/2014 1:30:00 PM Build the Right Regression Suite with Behavior-Driven Testing Presented by: Anand Bagmar ThoughtWorks Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  2. 2. Anand Bagmar ThoughtWorks Anand Bagmar is a hands-on and result-oriented software quality evangelist with seventeen years in the IT field. Passionate about shipping quality products, Anand specializes in building automated testing tools, infrastructure, and frameworks. He writes testing-related blogs and has built open-source tools related to software testing-WAAT (Web Analytics Automation Testing Framework), TaaS (for automating integration testing in disparate systems), and TTA (Test Trend Analyzer). Anand is the lead organizer for vodQA, the popular testing conference in India. Follow him on Twitter @BagmarAnand, email him at abagmar@gmail.com, or read his Essence of Testing blog.
  3. 3. Building  the  “right”   regression  suite  using   Behavior  Driven  Tes5ng   (BDT)   Anand  Bagmar   So#ware  Quality  Evangelist   Principal  Consultant,  ThoughtWorks  
  4. 4. •  “Succeeding  with  Agile”  –  Mike  Cohn   •  MarCn  Fowler  –  Test  Pyramid   –  hFp://marCnfowler.com/bliki/TestPyramid.html   The  Test  Pyramid  
  5. 5. Func%onal  Tests  (GUI)   •  Developers  &  /  QA  team   Web  Service  Tests   •  Developers  &  /  QA  team   Integra%on  tests   • Developers   Component  tests   •  Developers   Unit  Tests   •  Developers   Test  AutomaCon  Pyramid   Manual  /   Exploratory   TesCng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   •  Cost   •  Effort   •  Time  
  6. 6. Test  Pyramid  reality   Development  team   Test  Pyramid   QA  team   Test  Pyramid  
  7. 7. Func%onal  Tests  (GUI)   •  Developers  &  /  QA  team   Web  Service  Tests   •  Developers  &  /  QA  team   Integra%on  tests   • Developers   Component  tests   •  Developers   Unit  Tests   •  Developers   Manual  /  Exploratory  TesCng   Test     Ice-­‐cream  cone   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   Test  Pyramid  –  AnC  PaFern   Product     under  test  
  8. 8. Func%onal  Tests  (GUI)   •  Developers  &  /  QA  team   Web  Service  Tests   •  Developers  &  /  QA  team   Integra%on  tests   • Developers   Component  tests   •  Developers   Unit  Tests   •  Developers   Manual  /   Exploratory   TesCng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   IdenCfy  the  right  type  of  tests  that  sit   on  top  of  the  Test  Pyramid  
  9. 9. Case  Study   •  MulCple,  long   running  projects   •  Legacy   applicaCons   •  Integrated  
  10. 10. •  Limited  AutomaCon     •  Long  regression  cycle   •  Huge  cost  of  fixing  defects   TesCng  Challenges  
  11. 11. •  No  visibility  into  what  is  tested   •  Outdated   •  BriFle   •  LiFle  /  less  value   •  Expensive   •  Maintenance  nightmare   As  a  result  
  12. 12. Biggest  problem   •  Trust  deficit  on  the  team   – In  exisCng  AutomaCon   – In  manual  tesCng   •  Finger  poinCng  /  blame  game  
  13. 13. •  Remove  the  ambiguity   •  Same  visibility  for  all  
  14. 14. EvoluCon  of  Test  Frameworks   Behavior   Driven   Development   (BDD)   OpCmized   scripts   Record  &   Playback     Core   Framework  
  15. 15. Behavior  Driven  Development     (BDD)  
  16. 16. RED   GREEN  REFACTOR  
  17. 17. Behavior  Driven  TesCng   (BDT)  
  18. 18. The  Big  Picture   •  How  is  this  funcConality  going  to  be  used  by   the  end-­‐user?     •  What  is  their  thought  process  going  to  be   when  using  this  product?     •  What  is  the  “core-­‐business-­‐value”  I  am   delivering  to  the  end-­‐user?  
  19. 19. Func%onal  Tests  (GUI)   •  Developers  &  /  QA  team   Web  Service  Tests   •  Developers  &  /  QA  team   Integra%on  tests   • Developers   Component  tests   •  Developers   Unit  Tests   •  Developers   Manual  /   Exploratory   TesCng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   IdenCfy  the  right  type  of  tests  that  sit   on  top  of  the  Test  Pyramid  
  20. 20. Expected  Func%onality:     The  customer  should  be  able  to  withdraw   money  from  his  account  via  an  ATM  machine   An  example  
  21. 21. Given  the  account  has  balance  of  INR  5000   When  the  customer  requests  INR  1000   Then  the  account  is  debited  by  INR  1000   BDD  Example:  
  22. 22. Given  the  account  is  in  credit   And  the  dispenser  contains  cash     When  the  customer  requests  cash     Then  ensure  the  account  is  debited   And  ensure  cash  is  dispensed   And  ensure  the  card  is  returned   BDT  Example:  
  23. 23. New  Expected  Func%onality:     The  customer  should  be  able  to  withdraw   money  from  his  account  only  if  his  card  is   valid  
  24. 24. Given  the  account  is  in  credit   And  the  card  is  valid   And  the  dispenser  contains  cash     When  the  customer  requests  cash     Then  ensure  the  account  is  debited   And  ensure  cash  is  dispensed   And  ensure  the  card  is  returned   BDT  Example  -­‐  updated:  
  25. 25. Test  specificaCon  styles   •  ImperaCve     •  DeclaraCve  
  26. 26. •  Write  user  flows  based  on  personas,  and  how   they  use  the  business  funcConality,  or,   •  Write  business  flows,  and  how  different  user   personas  can  use  that  
  27. 27. Another  example   For  Indigo  Airlines  website  –   •  Guest  User  is  able  to  search  for  a  flight  for  a   single  traveller   •  Ability  to  specify  contact  informaCon  for   person  booking  the  flight  
  28. 28. Search     Search   Results   Contact   Informa%on  
  29. 29. ImperaCve  style   Given  I  am  a  guest  user  on  the  Indigo  home  page   And  I  select  “round”  trip  opCon   And  I  select  “Pune”  from  the  origin  dropdown   And  I  select  “Bangalore”  from  the  desCnaCon  drop  down   And  I  select  departure  date  as  “5  December  2013”   And  I  select  returning  date  as  “25  December  2013”     When  I  click  on  Search     Then  I  should  see  the  search  results  page     And  I  should  see  at  least  1  opCon  for  my  criteria     ...  
  30. 30. …   When  I  select  the  “first”  opCon   Then  I  am  on  the  Contact  InformaCon  page     When  I  enter  first  name  as  “foo”   And  I  enter  last  name  as  “bar”   …   …   And  I  click  the  “Select  and  Con%nue”  buFon     Then  I  should  be  on  the  next  page    
  31. 31. Given  I  am  a  guest  user     When  I  search  for  flight  opCons  for  a  “one-­‐way”  trip   for  “1”  “Adult”  from  “Delhi”  to  “Bangalore”   And  I  select  the  “first”  flight   And  I  enter  “valid”  contact  details  for  “traveller1”     Then  I  am  able  to  Save  and  ConCnue   DeclaraCve  style  
  32. 32. What’s  next?  
  33. 33. DO  NOT   AUTOMATE   •  Low  Value,  High  Cost   •  Manual   Regression!   2nd  candidate  for   automa%on   •  High  Value,  High  Cost   Automate  ONLY  if   you  have  %me   •  Low  Value,  Low  Cost   •  ?  Manual   Regression?   1st  candidate  for   automa%on   •  High  Value,  Low  Cost   Cost   Value  
  34. 34. Func%onal  Tests  (GUI)   •  Developers  &  /  QA  team   Web  Service  Tests   •  Developers  &  /  QA  team   Integra%on  tests   • Developers   Component  tests   •  Developers   Unit  Tests   •  Developers   BDT  helps  iden5fy  the  “right”  type  of   regression  tests!   Manual  /   Exploratory   TesCng   Product   under  test   Ideal  Test   Pyramid  
  35. 35. A  real  world  Case  Study  
  36. 36. How  did  we  accomplish  this?  
  37. 37. Who  is  doing  the   acCon?   What  is  the  business   funcConality?   AcCons  and   verificaCons  happen   implicitly   Depicts  a  user  flow   Given  auc%oneer  creates  a  sale   And  5  items  are  added  to  the   sale             When  auc%oneer  starts  the   sale   And  auc%oneer  sets  a  starCng   bid  of  5000   And  buyer_1  bids   And  buyer_2  bids     Then  auc%oneer  Sells  the  item   And  buyer_2  wins  the  item  
  38. 38. Final  state   Tests  running  on  Hudson  /  Jenkins   Since  1st  test  was  automated   Feature  files   33   Scenarios   65   Smoke  tests  execuCon  Cme   15  minutes   Regression  tests  execuCon  Cme   45  minutes   Test  case  repository  (manual  +   automated)   Cucumber  .feature  files  with   appropriate  tagging   Project  Management  tool   Mingle  
  39. 39. ?   Incrementally  build   User  Flows   Regress  required   Product  Business   Value   Tests  in-­‐sync   EffecCve  Tests       Remove   ambiguity   Understand   new   funcConality   Living   DocumentaCon   Onboarding  
  40. 40. Tools  to  enable  BDD  /  BDT   •  Cucumber   •  JBehave   •  SpecFlow   •  Twist   •  Concordion  
  41. 41. abagmar@gmail.com                @BagmarAnand     about.me/anand.bagmar   Thank  you  

×