Your SlideShare is downloading. ×
0
Executable	
  Specifica/ons
           Documents	
  for	
  So7ware	
  that	
  
                             Chris	
  Sterli...
Executable	
  Specifica/ons	
  -­‐
        Topics	
  to	
  Discuss
        •    What	
  are	
  Executable	
  Specifica/ons?
...
The	
  second-­‐most	
  detailed	
  specifica/on	
  of	
  the	
  customer’s	
  request	
  

             WHAT	
  ARE	
  EXE...
Executable	
  Specifica/ons

        •    Unambiguous	
  defini/on	
  of	
  desired	
  so7ware	
  behavior
        •    Exec...
Building	
  the	
  “right”	
  so7ware

             WHY	
  CREATE	
  EXECUTABLE	
  SPECIFICATIONS?




                   ...
Increasing	
  Cost	
  of	
  Release	
  Stabiliza/on	
  
        Periods

                                                 ...
Costly	
  Regression	
  Test	
  Phases




                                                                               ...
Managing	
  So7ware	
  Debt	
  –	
  an	
  Overview




                                                                   ...
Effect	
  of	
  Project	
  Constraints	
  on	
  Quality




                                                               ...
Effect	
  of	
  Project	
  Constraints	
  on	
  Quality




                                                               ...
How	
  does	
  the	
  Team	
  know	
  they	
  have	
  implemented	
  desired	
  behavior	
  successfully?

             AC...
Agile	
  tes/ng	
  happens	
  at	
  two	
  levels	
  

        • Acceptance	
  Tests	
  tell	
  us	
  whether	
  the	
  
 ...
Agile	
  tes/ng	
  happens	
  at	
  two	
  levels	
  

        • Acceptance	
  Tests	
  tell	
  us	
  whether	
  the	
  
 ...
Acceptance	
  Tests

        • Are	
  also	
  called	
  customer	
  tests	
  or	
  func/onal	
  tests
        • Tell	
  us...
User	
  Stories
                             Story 14
                             As a customer I want to check my
      ...
User	
  Stories
                             Story 14
                             As a customer I want to check my
      ...
What	
  is	
  a	
  User	
  Story?*
        • CARD
              – Token	
  represen4ng	
  the	
  requirement.	
  It's	
  u...
Confirma/on	
  through	
  Acceptance	
  Tests

        • Product	
  Owner	
  makes	
  first	
  pass	
  at	
  Acceptance	
  C...
Confirma/on	
  through	
  Acceptance	
  Tests

        • Product	
  Owner	
  makes	
  first	
  pass	
  at	
  Acceptance	
  C...
Take	
  on	
  the	
  Role	
  of	
  a	
  User	
  -­‐	
  “What	
  
            should	
  the	
  so7ware	
  do	
  next	
  for...
Acceptance	
  Test-­‐Driven	
  Development




                                                                           ...
Legacy	
  so7ware,	
  so7ware	
  test	
  automa/on,	
  and	
  commercial	
  off-­‐the-­‐shelf

             WHEN	
  TO	
  U...
Tradi/onal	
  balance	
  of	
  tests


                             Manual GUI                                            ...
Mike	
  Cohn’s	
  Tes/ng	
  Pyramid

                                                              Small number
          ...
Working	
  with	
  Legacy	
  So7ware

                          Define
                       Acceptance
                 ...
Working	
  with	
  Legacy	
  So7ware

                          Define                 Analyze What
                      ...
Working	
  with	
  Legacy	
  So7ware

                          Define                 Analyze What                       ...
Working	
  with	
  Legacy	
  So7ware

                          Define                 Analyze What                       ...
Working	
  with	
  Legacy	
  So7ware

                          Define                 Analyze What                       ...
Working	
  with	
  Legacy	
  So7ware

                          Define                 Analyze What                       ...
Working	
  with	
  Legacy	
  So7ware

        • Focus	
  on	
  Wri/ng	
  Executable	
  Specifica/ons	
  for	
  New	
  
    ...
So7ware	
  Test	
  Automa/on


                       Manual Test Script
                                  Login as
      ...
So7ware	
  Test	
  Automa/on


                       Manual Test Script
                              Login as
          ...
Commercial	
  Off-­‐the-­‐Shelf	
  (COTS)

                                                                            Vers...
Commercial	
  Off-­‐the-­‐Shelf	
  (COTS)

                                 Create                                         ...
Commercial	
  Off-­‐the-­‐Shelf	
  (COTS)

                                 Create                                         ...
Commercial	
  Off-­‐the-­‐Shelf	
  (COTS)

                                 Create                                         ...
Commercial	
  Off-­‐the-­‐Shelf	
  (COTS)

                                 Create                                         ...
Walk	
  through	
  the	
  crea/on	
  of	
  executable	
  specifica/ons	
  and	
  how	
  to	
  execute	
  them	
  effec/vely
...
A	
  simple	
  example	
  of	
  a	
  FIT	
  test




                                                                     ...
Fit	
  Workflow*

        • Starts	
  with	
  whiteboard	
  conversa/on
        • Customer	
  (SME,	
  business	
  person,	...
Fit	
  Workflow*

        • Customer,	
  working	
  with	
  delivery	
  team,	
  refines	
  
          examples	
  into	
  t...
Fit	
  Workflow*

        • Delivery	
  team	
  suggest	
  addi/onal	
  areas	
  to	
  cover	
  




                      ...
Fit	
  Workflow*

        • Delivery	
  team	
  format	
  tables	
  for	
  use	
  with	
  Fit




                         ...
Fit	
  Overview




                                                               *Source: http://fit.c2.com/


         ...
Fit	
  Workflow*

        • Delivery	
  team	
  creates	
  Fit	
  “fixtures”	
  (small	
  piece	
  of	
  code	
  
          ...
Fit	
  Workflow*

        • Execute	
  Fit	
  document	
                               • Click	
  to	
  edit	
  Master	
  t...
Fit	
  Workflow*

        • Delivery	
  team	
                                   • Click	
  to	
  edit	
  Master	
  text	
 ...
Fit	
  Workflow*

        • Document	
  is	
  kept	
  for	
  regression	
  tes/ng
        • Document	
  is	
  included	
  i...
Con/nuous	
  Integra/on




                                                                                      36
     ...
Organizing	
  FIT	
  tests

        • Maintain	
  suite	
  of	
  regression	
  tests	
  from	
  past	
  itera/ons	
  
    ...
Automated	
  Regression	
  Tes/ng

                              Application
                                Under
       ...
Automated	
  Regression	
  Tes/ng

                              Application
                                Under
       ...
Automated	
  Regression	
  Tes/ng

                               Application
                                 Under
     ...
Automated	
  Regression	
  Tes/ng

                               Application
                                 Under
     ...
Automated	
  Regression	
  Tes/ng

                               Application
                                 Under
     ...
Cost	
  reduc/on	
  using	
  Fit	
  for	
  test	
  automa/on	
  and	
  data	
  conversion

             A	
  FIT	
  CASE	
...
Manual	
  Regression	
  Tes/ng

        • Tes/ng	
  was	
  taking	
  75	
  person	
  hours	
  during	
  2	
  full	
  test	...
Introducing	
  Fit	
  into	
  Tes/ng	
  Process

        • A7er	
  8	
  itera/ons	
  team	
  had	
  introduced	
  healthy	...
Tools	
  that	
  Teams	
  can	
  use	
  and	
  what	
  context	
  they	
  are	
  best	
  used	
  in

             AVAILABL...
Table-­‐based	
  Acceptance	
  Tes/ng	
  Tools

        • Fit	
  –	
  hlp://fit.c2.com/	
  
        • FitNesse	
  –	
  hlp:...
Programmer	
  Acceptance	
  Tes/ng	
  Tools

        •    Cucumber	
  -­‐	
  hlp://cukes.info/	
  
        •    Rspec	
  -...
Web	
  UI	
  Agile	
  Tes/ng	
  Tools
        • Open	
  Source	
  Web	
  UI	
  Tes/ng	
  tools
              – StoryTestIQ...
What	
  about	
  tradi/onal	
  tes/ng	
  tools?

        • Why	
  don’t	
  we	
  use	
  commercial	
  products	
  such	
  ...
Ques/ons	
  and	
  Answers

             THANK	
  YOU




                                                                ...
Presenter:	
  Chris	
  Sterling
        Partner	
  at	
  Sterling	
  Barton
        • Technology	
  Consultant,	
  Cer/fied...
Upcoming SlideShare
Loading in...5
×

Executable Specifications Agile Palooza

2,488

Published on

Published in: Technology, Business
2 Comments
3 Likes
Statistics
Notes
No Downloads
Views
Total Views
2,488
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
86
Comments
2
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "Executable Specifications Agile Palooza"

  1. 1. Executable  Specifica/ons Documents  for  So7ware  that   Chris  Sterling Partner  at  Sterling  Barton Cer/fied  Scrum  Trainer Agile  Consultant   Thursday, January 14, 2010 1
  2. 2. Executable  Specifica/ons  -­‐ Topics  to  Discuss • What  are  Executable  Specifica/ons? • Why  Create  Them? • Acceptance  Tests • When  to  Use • An  Approach  with  Example • Available  Tools 2 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 2
  3. 3. The  second-­‐most  detailed  specifica/on  of  the  customer’s  request   WHAT  ARE  EXECUTABLE  SPECIFICATIONS? 3 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 3
  4. 4. Executable  Specifica/ons • Unambiguous  defini/on  of  desired  so7ware  behavior • Executable  documenta/on • Repeatable  and  specific  tests • Regression  tests  to  validate  exis/ng  so7ware  behavior • The  second-­‐most  detailed  specifica/on  of  the   customer’s  request   4 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 4
  5. 5. Building  the  “right”  so7ware WHY  CREATE  EXECUTABLE  SPECIFICATIONS? 5 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 5
  6. 6. Increasing  Cost  of  Release  Stabiliza/on   Periods 500,000 375,000 250,000 125,000 Release 1 Release 2 Release 3 0 Release 4 Release 5 Release 6 Cost of Fixing Defects Cost for Feature Dev 6 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 6
  7. 7. Costly  Regression  Test  Phases 7 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 7
  8. 8. Managing  So7ware  Debt  –  an  Overview 8 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 8
  9. 9. Effect  of  Project  Constraints  on  Quality 9 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 9
  10. 10. Effect  of  Project  Constraints  on  Quality 9 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 9
  11. 11. How  does  the  Team  know  they  have  implemented  desired  behavior  successfully? ACCEPTANCE  TESTS 10 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 10
  12. 12. Agile  tes/ng  happens  at  two  levels   • Acceptance  Tests  tell  us  whether  the   system  does  what  the  customer   expects  (“building  the  right  code”) • Programmer  Tests  define  whether  the   system  does  what  the  developers   expect  (“building  the  code  right”) 11 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 11
  13. 13. Agile  tes/ng  happens  at  two  levels   • Acceptance  Tests  tell  us  whether  the   system  does  what  the  customer   expects  (“building  the  right  code”) • Programmer  Tests  define  whether  the   system  does  what  the  developers   expect  (“building  the  code  right”) 11 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 11
  14. 14. Acceptance  Tests • Are  also  called  customer  tests  or  func/onal  tests • Tell  us  whether  the  system  does  what  the  customer   expects • Enable  Developers  to  know  they’ve  sa/sfied   requirements • Can  be  run  automa/cally  at  any  /me  by  anyone • Helps  us  build  the  “right”  so7ware • “Running,  Tested  Features”* * “A Metric Leading to Agility” – Ron Jeffries http://www.xprogramming.com/xpmag/jatRtsMetric.htm 12 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 12
  15. 15. User  Stories Story 14 As a customer I want to check my order status online so that I can know when to expect my package 13 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 13
  16. 16. User  Stories Story 14 As a customer I want to check my order status online so that I can know when to expect my package A small piece of business value that can be delivered in an iteration 13 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 13
  17. 17. What  is  a  User  Story?* • CARD – Token  represen4ng  the  requirement.  It's  used  in  planning.   Notes  are  wriDen  on  it,  reflec4ng  priority  and  cost • CONVERSATION – The  requirement  itself  is  communicated  from  customer  to   programmers  through  conversa4on  (The  conversa4on  is   largely  verbal,  but  is  oMen  supplemented  with  documents) • CONFIRMATION – The  confirma4on  provided  by  the  acceptance  test  is  what   makes  possible  the  simple  approach  of  card  and  conversa4on – When  the  conversa4on  about  a  card  gets  down  to  the  details   * “Essential XP: Card, Conversation, Confirmation” – Ron Jeffries http://www.xprogramming.com/xpmag/EXPCardConversationConfirmation.htm 14 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 14
  18. 18. Confirma/on  through  Acceptance  Tests • Product  Owner  makes  first  pass  at  Acceptance  Criteria   • During  Itera/on  Planning  Acceptance  Criteria  are  discussed • Final  Acceptance  Criteria  for  each  User  Story  is  product  of   nego/a/on  between  Delivery  Team  and  Product  Manager • Should  be  short,  easy  to  understand  statements Story 14 As a customer, I want to check my order status online so that I can know when to expect my package 15 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 15
  19. 19. Confirma/on  through  Acceptance  Tests • Product  Owner  makes  first  pass  at  Acceptance  Criteria   • During  Itera/on  Planning  Acceptance  Criteria  are  discussed • Final  Acceptance  Criteria  for  each  User  Story  is  product  of   nego/a/on  between  Delivery  Team  and  Product  Manager • Should  be  short,  easy  to  understand  statements Story 14 Acceptance Criteria •View status as “waiting for pickup”, As a customer, I want to check “en route” or “delivered” my order status online so that I •Date of each step in route can know when to expect my •Estimated time of delivery package 15 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 15
  20. 20. Take  on  the  Role  of  a  User  -­‐  “What   should  the  so7ware  do  next  for  me?” This  ques/on  helps  you  to  decide  what  the   next  acceptance  test  should  model 16 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 16
  21. 21. Acceptance  Test-­‐Driven  Development 17 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 17
  22. 22. Legacy  so7ware,  so7ware  test  automa/on,  and  commercial  off-­‐the-­‐shelf WHEN  TO  USE  EXECUTABLE  SPECIFICATIONS? 18 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 18
  23. 23. Tradi/onal  balance  of  tests Manual GUI Easy to Create Acceptance Very familiar – what we always do Tests Typically tedious How do we know coverage? Need Automation specialists Automated Automation good for performance GUI Tests Seems like we always rewrite Sometimes fragile Unit Tests What is Development testing? How do we know what these are? How do we know when they fail? (Originally discussed by Mike Cohn) 19 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 19
  24. 24. Mike  Cohn’s  Tes/ng  Pyramid Small number Automate most GUI Acceptance Tests Drive development and acceptance Executable Specifications Do the most Create Test Driven Design Unit Tests (Also “borrowed” from Mike Cohn) 20 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 20
  25. 25. Working  with  Legacy  So7ware Define Acceptance Criteria for Requirement 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21
  26. 26. Working  with  Legacy  So7ware Define Analyze What Acceptance Might Be Criteria for Affected by Requirement Requirement 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21
  27. 27. Working  with  Legacy  So7ware Define Analyze What Write Acceptance Might Be Executable Criteria for Affected by Specifications Requirement Requirement for How it Works Now 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21
  28. 28. Working  with  Legacy  So7ware Define Analyze What Write Acceptance Might Be Executable Criteria for Affected by Specifications Requirement Requirement for How it Works Now Write Failing Executable Specifications for How it Should Work 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21
  29. 29. Working  with  Legacy  So7ware Define Analyze What Write Acceptance Might Be Executable Criteria for Affected by Specifications Requirement Requirement for How it Works Now Carefully Write Failing Modify Source Executable Code to Specifications Implement for How it Requirement Should Work 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21
  30. 30. Working  with  Legacy  So7ware Define Analyze What Write Acceptance Might Be Executable Criteria for Affected by Specifications Requirement Requirement for How it Works Now Execute Carefully Write Failing Executable Modify Source Executable Specifications Code to Specifications to Verify Implement for How it Change Requirement Should Work 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21
  31. 31. Working  with  Legacy  So7ware • Focus  on  Wri/ng  Executable  Specifica/ons  for  New   Requirements  and  Func/onality  that  may  be  Affected • Stabilize  Legacy  So7ware  with  Black-­‐box  Tests   (Executable  Specifica/ons) • Incrementally  Improve  the  Internal  So7ware  Design • Try  to  Iden/fy  where  Programmer  Tests  can  be  Added • Execute  Regression  Test  Suite  that  Includes  Executable   Specifica/ons  Frequently  (more  than  once  per  day  if   possible) 22 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 22
  32. 32. So7ware  Test  Automa/on Manual Test Script Login as Step 1 Administrato r Create Step 2 Order with 1 Item Click “Check Step 3 Out” Button Verify Item in Step 4 Shopping Cart 23 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 23
  33. 33. So7ware  Test  Automa/on Manual Test Script Login as Step 1 Administrato Automate r Create Step 2 Order with 1 Item Click “Check Step 3 Out” Button Verify Item in Step 4 Shopping Cart 23 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 23
  34. 34. Commercial  Off-­‐the-­‐Shelf  (COTS) Version 1: COTS Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24
  35. 35. Commercial  Off-­‐the-­‐Shelf  (COTS) Create Version 1: Executable Specifications for COTS Configurations & Customizations Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24
  36. 36. Commercial  Off-­‐the-­‐Shelf  (COTS) Create Version 1: Executable Specifications for COTS Configurations & Customizations Software Version 2: COTS Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24
  37. 37. Commercial  Off-­‐the-­‐Shelf  (COTS) Create Version 1: Executable Specifications for COTS Configurations & Customizations Software Execute Existing Version 2: Executable Specifications to COTS Find Upgrade Path Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24
  38. 38. Commercial  Off-­‐the-­‐Shelf  (COTS) Create Version 2: Executable Specifications for COTS Configurations & Customizations Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24
  39. 39. Walk  through  the  crea/on  of  executable  specifica/ons  and  how  to  execute  them  effec/vely AN  EXAMPLE  APPROACH 25 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 25
  40. 40. A  simple  example  of  a  FIT  test *Source: http://fit.c2.com/ 26 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 26
  41. 41. Fit  Workflow* • Starts  with  whiteboard  conversa/on • Customer  (SME,  business  person,  product  manager,   etc.)  informally  describe  new  feature • Programmers  and  testers  ask  ques/ons *Source: http://fit.c2.com/ 27 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 27
  42. 42. Fit  Workflow* • Customer,  working  with  delivery  team,  refines   examples  into  tables • Use  business-­‐friendly  tools,  such  as  Microso7  Excel  and   Word,  to  capture  test  tables *Source: http://fit.c2.com/ 28 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 28
  43. 43. Fit  Workflow* • Delivery  team  suggest  addi/onal  areas  to  cover   *Source: http://fit.c2.com/ 29 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 29
  44. 44. Fit  Workflow* • Delivery  team  format  tables  for  use  with  Fit *Source: http://fit.c2.com/ 30 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 30
  45. 45. Fit  Overview *Source: http://fit.c2.com/ 31 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 31
  46. 46. Fit  Workflow* • Delivery  team  creates  Fit  “fixtures”  (small  piece  of  code   that  translates  test  tables  into  execu/on  tests  against   the  so7ware) *Source: http://fit.c2.com/ 32 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 32
  47. 47. Fit  Workflow* • Execute  Fit  document   • Click  to  edit  Master  text  styles against  so7ware • At  first  some  tests  should   be  failing  (red) *Source: http://fit.c2.com/ 33 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 33
  48. 48. Fit  Workflow* • Delivery  team   • Click  to  edit  Master  text  styles collaborates  with   customer  to  incrementally   enhance  test  tables • Delivery  team  implements   so7ware  to  meet  test   execu/on  (green) *Source: http://fit.c2.com/ 34 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 34
  49. 49. Fit  Workflow* • Document  is  kept  for  regression  tes/ng • Document  is  included  in  automated  build  to  ensure   everything  keeps  working   • As  ques/ons  arise  about  func/onality  an  example  is   added  and  Fit  reports  the  answer   *Source: http://fit.c2.com/ 35 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 35
  50. 50. Con/nuous  Integra/on 36 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 36
  51. 51. Organizing  FIT  tests • Maintain  suite  of  regression  tests  from  past  itera/ons   that  always  pass • Run  “regression”  tests  with  build. • Maintain  a  suite  of  “in  progress”  tests  for  the  current   itera/on – Begin  the  itera4on  with  all  tests  failing – End  the  itera4on  with  most  tests  passing • At  the  end  of  the  itera/on,  move  newly  passing  tests   into  regression  suite – Beware  the  Fitnesse  “Refactor/Move”  command 37 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 37
  52. 52. Automated  Regression  Tes/ng Application Under Development 38 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 38
  53. 53. Automated  Regression  Tes/ng Application Under Development New Feature 38 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 38
  54. 54. Automated  Regression  Tes/ng Application Under Development New Design Changes Feature 38 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 38
  55. 55. Automated  Regression  Tes/ng Application Under Development Automated Design Changes Regression New Test Run Feature 38 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 38
  56. 56. Automated  Regression  Tes/ng Application Under Development Automated Design Changes Regression New Test Run Feature 38 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 38
  57. 57. Cost  reduc/on  using  Fit  for  test  automa/on  and  data  conversion A  FIT  CASE  STUDY 39 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 39
  58. 58. Manual  Regression  Tes/ng • Tes/ng  was  taking  75  person  hours  during  2  full  test   runs  consis/ng  of: – Comprehensive  manual  regression  tes4ng – Data  conversion  and  valida4on • Cost  for  tes/ng  was  $17,000  each  itera/on 40 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 40
  59. 59. Introducing  Fit  into  Tes/ng  Process • A7er  8  itera/ons  team  had  introduced  healthy  amount   of  Fit  fixtures  and  automated  tests • Reduced  70+  hour  test  run/me  down  to  6  hours  which   now  included: – Fit  automated  regression  tes4ng   – Data  conversion  and  valida4on  automated  with  Fit  fixtures   • Reduced  cost  of  tes/ng  each  itera/on  from  $17,000  to   $7,000 41 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 41
  60. 60. Tools  that  Teams  can  use  and  what  context  they  are  best  used  in AVAILABLE  TOOLS 42 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 42
  61. 61. Table-­‐based  Acceptance  Tes/ng  Tools • Fit  –  hlp://fit.c2.com/   • FitNesse  –  hlp://www.fitnesse.org/ • RobotFramework  -­‐  hlp://code.google.com/p/ robonramework/   43 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 43
  62. 62. Programmer  Acceptance  Tes/ng  Tools • Cucumber  -­‐  hlp://cukes.info/   • Rspec  -­‐  hlp://rspec.info/   • JBehave  -­‐  hlp://jbehave.org/   • JMeter  -­‐  hlp://jakarta.apache.org/jmeter/   • xUnit  (JUnit/NUnit/CppUnit/etc…) 44 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 44
  63. 63. Web  UI  Agile  Tes/ng  Tools • Open  Source  Web  UI  Tes/ng  tools – StoryTestIQ – WATiR – Selenium – Canoo  Web  Test • GUI  tests  tend  to  be  more  brille • Tests  wrilen  and  maintained  incrementally • Most  of  these  tools  have  code  integra/on 45 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 45
  64. 64. What  about  tradi/onal  tes/ng  tools? • Why  don’t  we  use  commercial  products  such  as  SilkTest,   TestDirector,  QuickTest  Pro,  etc.  for  Agile  tes/ng? – Expensive – Automated  tools  are  record-­‐and-­‐playback;  briDle – Ties  our  tests  to  the  UI  implementa4on – Manual  tools  (TestDirector)  take  too  long  to  run. – Work  fine  as  an  interim  strategy  (especially  if  you  already   have  the  licenses) – Consider  adding  open  source  Agile  Tes4ng  tool  as  a   component  of  your  tes4ng  strategy 46 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 46
  65. 65. Ques/ons  and  Answers THANK  YOU 47 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 47
  66. 66. Presenter:  Chris  Sterling Partner  at  Sterling  Barton • Technology  Consultant,  Cer/fied  Scrum   Trainer  and  Agile  Coach • Consults  on  so7ware  technology  across  a   spectrum  of  industries • Consults  organiza/ons  on  Agile   development,  management,  and   enterprise  prac/ces • Founder  of  Interna/onal  Associa/on  of   So7ware  Architects  (IASA)  Puget  Sound   chapter • University  of  Washington  Lecturer:  Agile   Developer  Cer/ficate  Program Email:  chris@SterlingBarton.com Our  Blog:  hlp://www.getngagile.com Follow  me  on  Twiler:  @csterwa 48 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 48
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×