Test	
  Driven	
  Development	
  
              via	
  
         Agile	
  Tes2ng	
  
What	
  is	
  Tes*ng	
  on	
  Agile	
  projects?	
  
Itera&on	
  0	
            Itera&on	
  1	
          Itera&on	
  2	
          Itera&on	
  3	
   ..	
     ..	
     Itera&on	
  n	
  

    Story	
  test	
                                     Showcase	
               Showcase	
                           Showcase	
  
   analysis	
  for	
               IPM	
               Itera2on	
  1,	
         Itera2on	
  2,	
                    Itera2on	
  n-­‐1,	
  
   Itera2on	
  1	
                                         IPM	
                    IPM	
                                IPM	
  

             Test	
                                      Support	
                 Support	
                          Support	
  
                              Itera&on	
  1	
  
   Automa&on	
                                         Itera2on	
  1	
           Itera2on	
  2	
                   Itera2on	
  n-­‐1	
  
                             Story	
  tes&ng	
  
    -­‐	
  strategy,	
  
   framework	
  
    setup,	
  etc.	
  
                               Story	
  test	
         Itera&on	
  2	
          Itera&on	
  3	
                     Itera&on	
  n	
  
                              analysis	
  for	
       Story	
  tes&ng	
        Story	
  tes&ng	
                   Story	
  tes&ng	
  
                              Itera2on	
  2	
  

                                                          Test	
                   Test	
                              Test	
  
                                                       Automa&on	
              Automa&on	
                         Automa&on	
  
                                                         maint.,	
                maint.,	
                           maint.,	
  
                                                        execu&on	
               execu&on	
                          execu&on	
  

                                                        Story	
  test	
          Story	
  test	
                      Story	
  test	
  
                                                       analysis	
  for	
        analysis	
  for	
                    analysis	
  for	
  
                                                       Itera2on	
  3	
          Itera2on	
  4	
                     Itera2on	
  n+1	
  
Story	
  planning	
  
                                        and	
  wri&ng	
  

               Showcase	
                                        IPM	
  




 Automa&on	
  
  (execu&on,	
                                                             QA	
  kickoff	
  
 maintenance)	
  

                                  Agile	
  Tes2ng	
  	
  
                                 in	
  an	
  Itera2on
 Manual	
  +	
  
Exploratory	
                                                                 Story	
  kickoff	
  
  Tes&ng	
  




             BA	
  /	
  QA	
                                   Test	
  Scenario	
  
            Volleyball	
                                          Wri&ng	
  

                                       Implement	
  Test	
  
                                         Automa&on	
  
What	
  makes	
  Agile	
  Teams	
  	
  
        successful?	
  




Test	
  Automa2on	
  
What	
  makes	
  Agile	
  Tes*ng	
  
       unsuccessful?	
  




Test	
  Automa2on	
  
The	
  Test	
  Pyramid	
  
•  “Succeeding	
  with	
  Agile”	
  –	
  Mike	
  Cohn	
  

•  Mar*n	
  Fowler	
  –	
  Test	
  Pyramid	
  
    –  hCp://mar*nfowler.com/bliki/TestPyramid.html	
  
Ideal	
  Test	
  Pyramid	
  
Test	
  Pyramid	
  reality	
  




Development	
  team	
  
Test	
  Pyramid	
                 QA	
  team	
  
                              Test	
  Pyramid	
  
Test	
  Pyramid	
  –	
  An*	
  PaCern	
  
Case	
  Study	
  


                    •  Mul*ple,	
  long	
  
                       running	
  projects	
  

                    •  Legacy	
  
                       applica*ons	
  

                    •  Integrated	
  
Its	
  not	
  what	
  I	
  
said,	
  its	
  what	
  I	
  
        mean!	
  
•  Remove	
  the	
  ambiguity	
  
•  Same	
  visibility	
  for	
  all	
  
Evolu*on	
  of	
  Test	
  Frameworks	
  
                Behavior	
  
                 Driven	
  
              Development	
  
                 (BDD)	
  

               Op*mized	
  
                scripts	
  

                Record	
  &	
  
                Playback	
  	
  



                  Core	
  
               Framework	
  
Behavior	
  Driven	
  Development	
  	
  
               (BDD)	
  
RED	
  




REFACTOR	
               GREEN	
  
Behavior	
  Driven	
  Tes*ng	
  
          (BDT)	
  
Expected	
  Func&onality:	
  
	
  
The	
  customer	
  should	
  be	
  able	
  to	
  withdraw	
  
money	
  from	
  his	
  account	
  via	
  an	
  ATM	
  machine	
  
BDD	
  Example:	
  

Given	
  the	
  account	
  has	
  balance	
  of	
  INR	
  5000	
  
When	
  the	
  customer	
  requests	
  INR	
  1000	
  
Then	
  the	
  account	
  is	
  debited	
  by	
  INR	
  1000	
  
BDT	
  Example:	
  
Given	
  the	
  account	
  is	
  in	
  credit	
  
And	
  the	
  dispenser	
  contains	
  cash	
  
	
  
When	
  the	
  customer	
  requests	
  cash	
  
	
  
Then	
  ensure	
  the	
  account	
  is	
  debited	
  
And	
  ensure	
  cash	
  is	
  dispensed	
  
And	
  ensure	
  the	
  card	
  is	
  returned	
  
New	
  Expected	
  Func&onality:	
  
	
  
The	
  customer	
  should	
  be	
  able	
  to	
  withdraw	
  
money	
  from	
  his	
  account	
  only	
  if	
  his	
  card	
  is	
  
valid	
  
BDT	
  Example	
  -­‐	
  updated:	
  
Given	
  the	
  account	
  is	
  in	
  credit	
  
And	
  the	
  card	
  is	
  valid	
  
And	
  the	
  dispenser	
  contains	
  cash	
  
	
  
When	
  the	
  customer	
  requests	
  cash	
  
	
  
Then	
  ensure	
  the	
  account	
  is	
  debited	
  
And	
  ensure	
  cash	
  is	
  dispensed	
  
And	
  ensure	
  the	
  card	
  is	
  returned	
  
Test	
  specifica*on	
  styles	
  

Story:	
  Animal	
  Submission	
  
	
  
As	
  a	
  Zoologist	
  
I	
  want	
  to	
  add	
  a	
  new	
  animal	
  to	
  the	
  site	
  
So	
  that	
  I	
  can	
  share	
  my	
  animal	
  knowledge	
  
with	
  the	
  community	
  
Impera*ve	
  Scenario	
  
Given	
  I	
  am	
  on	
  the	
  animal	
  crea*on	
  page	
  
	
  
When	
  I	
  fill	
  in	
  Name	
  with	
  ‘Alligator’	
  
And	
  fill	
  in	
  Genus	
  with	
  ‘Alligator’	
  
And	
  check	
  Lay	
  Eggs	
  
And	
  click	
  the	
  Create	
  buCon	
  
	
  
Then	
  I	
  should	
  see	
  the	
  no*ce	
  ‘Thank	
  you	
  for	
  your	
  animal	
  
submission!’	
  
And	
  the	
  page	
  should	
  include	
  the	
  animal’s	
  name,	
  genus	
  and	
  
lay’s	
  eggs	
  
Reference:	
  
hCp://wa*rmelon.com/2010/12/10/cucumber-­‐impera*ve-­‐or-­‐declara*ve-­‐that-­‐is-­‐the-­‐ques*on/	
  
Declara*ve	
  Scenario	
  

Given	
  I	
  am	
  on	
  the	
  animal	
  crea*on	
  page	
  
	
  
When	
  I	
  add	
  a	
  new	
  animal	
  
	
  
Then	
  I	
  animal	
  should	
  be	
  created	
  successfully	
  




Reference:	
  
hCp://wa*rmelon.com/2010/12/10/cucumber-­‐impera*ve-­‐or-­‐declara*ve-­‐that-­‐is-­‐the-­‐ques*on/	
  
What	
  does	
  this	
  do	
  for	
  me?	
  
How	
  did	
  we	
  accomplish	
  this?	
  
Given	
  auc&oneer	
  creates	
  a	
  sale	
  
And	
  5	
  items	
  are	
  added	
  to	
  the	
  
                                                   Who	
  is	
  doing	
  the	
  
sale	
                                             ac*on?	
  
	
  	
  	
  	
  	
  
When	
  auc&oneer	
  starts	
  the	
               What	
  is	
  the	
  business	
  
sale	
                                             func*onality?	
  
And	
  auc&oneer	
  sets	
  a	
  star*ng	
  
                                                   Ac*ons	
  and	
  
bid	
  of	
  5000	
                                verifica*ons	
  happen	
  
And	
  buyer_1	
  bids	
                           implicitly	
  
And	
  buyer_2	
  bids	
                           Depicts	
  a	
  user	
  flow	
  
	
  
Then	
  auc&oneer	
  Sells	
  the	
  item	
  
And	
  buyer_2	
  wins	
  the	
  item	
  
Current	
  state	
  
Tests	
  running	
  on	
  Hudson	
  /	
  Jenkins	
     Since	
  1st	
  test	
  was	
  automated	
  


Feature	
  files	
                                      33	
  
Scenarios	
                                            65	
  
Smoke	
  tests	
  execu*on	
  *me	
                    15	
  minutes	
  
Regression	
  tests	
  execu*on	
  *me	
               45	
  minutes	
  


Test	
  case	
  repository	
  (manual	
  +	
           Cucumber	
  .feature	
  files	
  with	
  
automated)	
                                           appropriate	
  tagging	
  
Project	
  Management	
  tool	
                        Mingle	
  
Tools	
  to	
  enable	
  BDD	
  /	
  BDT	
  
•    Cucumber	
  
•    JBehave	
  
•    SpecFlow	
  
•    Twist	
  
Anand.Bagmar@thoughtworks.com	
  

      abagmar@gmail.com	
  
            ThoughtWorks®	
  

Test Driven Development via Agile Testing

  • 1.
    Test  Driven  Development   via   Agile  Tes2ng  
  • 2.
    What  is  Tes*ng  on  Agile  projects?  
  • 3.
    Itera&on  0   Itera&on  1   Itera&on  2   Itera&on  3   ..   ..   Itera&on  n   Story  test   Showcase   Showcase   Showcase   analysis  for   IPM   Itera2on  1,   Itera2on  2,   Itera2on  n-­‐1,   Itera2on  1   IPM   IPM   IPM   Test   Support   Support   Support   Itera&on  1   Automa&on   Itera2on  1   Itera2on  2   Itera2on  n-­‐1   Story  tes&ng   -­‐  strategy,   framework   setup,  etc.   Story  test   Itera&on  2   Itera&on  3   Itera&on  n   analysis  for   Story  tes&ng   Story  tes&ng   Story  tes&ng   Itera2on  2   Test   Test   Test   Automa&on   Automa&on   Automa&on   maint.,   maint.,   maint.,   execu&on   execu&on   execu&on   Story  test   Story  test   Story  test   analysis  for   analysis  for   analysis  for   Itera2on  3   Itera2on  4   Itera2on  n+1  
  • 4.
    Story  planning   and  wri&ng   Showcase   IPM   Automa&on   (execu&on,   QA  kickoff   maintenance)   Agile  Tes2ng     in  an  Itera2on Manual  +   Exploratory   Story  kickoff   Tes&ng   BA  /  QA   Test  Scenario   Volleyball   Wri&ng   Implement  Test   Automa&on  
  • 5.
    What  makes  Agile  Teams     successful?   Test  Automa2on  
  • 6.
    What  makes  Agile  Tes*ng   unsuccessful?   Test  Automa2on  
  • 7.
    The  Test  Pyramid   •  “Succeeding  with  Agile”  –  Mike  Cohn   •  Mar*n  Fowler  –  Test  Pyramid   –  hCp://mar*nfowler.com/bliki/TestPyramid.html  
  • 8.
  • 10.
    Test  Pyramid  reality   Development  team   Test  Pyramid   QA  team   Test  Pyramid  
  • 11.
    Test  Pyramid  –  An*  PaCern  
  • 12.
    Case  Study   •  Mul*ple,  long   running  projects   •  Legacy   applica*ons   •  Integrated  
  • 15.
    Its  not  what  I   said,  its  what  I   mean!  
  • 18.
    •  Remove  the  ambiguity   •  Same  visibility  for  all  
  • 20.
    Evolu*on  of  Test  Frameworks   Behavior   Driven   Development   (BDD)   Op*mized   scripts   Record  &   Playback     Core   Framework  
  • 21.
  • 22.
  • 23.
  • 24.
    Expected  Func&onality:     The  customer  should  be  able  to  withdraw   money  from  his  account  via  an  ATM  machine  
  • 25.
    BDD  Example:   Given  the  account  has  balance  of  INR  5000   When  the  customer  requests  INR  1000   Then  the  account  is  debited  by  INR  1000  
  • 26.
    BDT  Example:   Given  the  account  is  in  credit   And  the  dispenser  contains  cash     When  the  customer  requests  cash     Then  ensure  the  account  is  debited   And  ensure  cash  is  dispensed   And  ensure  the  card  is  returned  
  • 27.
    New  Expected  Func&onality:     The  customer  should  be  able  to  withdraw   money  from  his  account  only  if  his  card  is   valid  
  • 28.
    BDT  Example  -­‐  updated:   Given  the  account  is  in  credit   And  the  card  is  valid   And  the  dispenser  contains  cash     When  the  customer  requests  cash     Then  ensure  the  account  is  debited   And  ensure  cash  is  dispensed   And  ensure  the  card  is  returned  
  • 29.
    Test  specifica*on  styles   Story:  Animal  Submission     As  a  Zoologist   I  want  to  add  a  new  animal  to  the  site   So  that  I  can  share  my  animal  knowledge   with  the  community  
  • 30.
    Impera*ve  Scenario   Given  I  am  on  the  animal  crea*on  page     When  I  fill  in  Name  with  ‘Alligator’   And  fill  in  Genus  with  ‘Alligator’   And  check  Lay  Eggs   And  click  the  Create  buCon     Then  I  should  see  the  no*ce  ‘Thank  you  for  your  animal   submission!’   And  the  page  should  include  the  animal’s  name,  genus  and   lay’s  eggs   Reference:   hCp://wa*rmelon.com/2010/12/10/cucumber-­‐impera*ve-­‐or-­‐declara*ve-­‐that-­‐is-­‐the-­‐ques*on/  
  • 31.
    Declara*ve  Scenario   Given  I  am  on  the  animal  crea*on  page     When  I  add  a  new  animal     Then  I  animal  should  be  created  successfully   Reference:   hCp://wa*rmelon.com/2010/12/10/cucumber-­‐impera*ve-­‐or-­‐declara*ve-­‐that-­‐is-­‐the-­‐ques*on/  
  • 32.
    What  does  this  do  for  me?  
  • 35.
    How  did  we  accomplish  this?  
  • 37.
    Given  auc&oneer  creates  a  sale   And  5  items  are  added  to  the   Who  is  doing  the   sale   ac*on?             When  auc&oneer  starts  the   What  is  the  business   sale   func*onality?   And  auc&oneer  sets  a  star*ng   Ac*ons  and   bid  of  5000   verifica*ons  happen   And  buyer_1  bids   implicitly   And  buyer_2  bids   Depicts  a  user  flow     Then  auc&oneer  Sells  the  item   And  buyer_2  wins  the  item  
  • 38.
    Current  state   Tests  running  on  Hudson  /  Jenkins   Since  1st  test  was  automated   Feature  files   33   Scenarios   65   Smoke  tests  execu*on  *me   15  minutes   Regression  tests  execu*on  *me   45  minutes   Test  case  repository  (manual  +   Cucumber  .feature  files  with   automated)   appropriate  tagging   Project  Management  tool   Mingle  
  • 39.
    Tools  to  enable  BDD  /  BDT   •  Cucumber   •  JBehave   •  SpecFlow   •  Twist  
  • 40.
    Anand.Bagmar@thoughtworks.com   abagmar@gmail.com   ThoughtWorks®