Test Strategy

            Zbyszek Moćkun

                 © 2010 Cognifide Limited. In commercial confidence only.
Agenda

1.   V Model
2.   Agile Way
3.   Test Planing
4.   Control & Execution
5.   Metrics
6.   Demo


                           © 2010 Cognifide Limited. In commercial confidence only.
V Model

Perfect world




                © 2010 Cognifide Limited. In commercial confidence only.
V Model



 • Coverage
 • Do not duplicate tests
   during regression
 • Test specific
   functionality




                            © 2010 Cognifide Limited. In commercial confidence only.
Preventive or reactive




  • Coverage
  • Do not duplicate tests
    during regression
  • Test specific
    functionality




                             © 2010 Cognifide Limited. In commercial confidence only.
Test Pyramid




  • Coverage
  • Do not duplicate tests
    during regression
  • Test specific
    functionality




                             © 2010 Cognifide Limited. In commercial confidence only.
Agile

New way?




           © 2010 Cognifide Limited. In commercial confidence only.
Agile Manifesto




  • Coverage
  • Do not duplicate tests
    during regression
  • Test specific
    functionality




                             Source: blog.twg.ca
                                  © 2010 Cognifide Limited. In commercial confidence only.
Agile against waterfall


Coding, new functionality


                                      Requirements

        Bug fixing
                                                     Design


                                                              Coding

       Testing
                                                                         Testing


                            Release                                                             Release


       Iteration


                                                                       © 2010 Cognifide Limited. In commercial confidence only.
Agile ways

                     Processes   Methodologies




  • Coverage
                     ATDD        Kanban
  • Do not duplicate tests
    during regression
  • Test specific
                      BDD         Scrum
    functionality
                      TDD            XP



                                                 © 2010 Cognifide Limited. In commercial confidence only.
Continuous Integration




  • Coverage
  • Do not duplicate tests
    during regression
  • Test specific
    functionality




                             Source: http://wiki.mxunit.org


                                                    © 2010 Cognifide Limited. In commercial confidence only.
Reactive approach?




  • Coverage
  • Do not duplicate tests
    during regression
  • Test specific
    functionality




                             © 2010 Cognifide Limited. In commercial confidence only.
Scripting issues
• Scripted testing report
 phase               All              Executed   passed                      failed
 Test design phase   80               0          0                           0
 First day           90 (4 updated)   20         16                          4
 Second day          95 (10)          75         65                          10
 Third day           120 (14)         75         73                          2
 Fourth day          125 (16)         110        98                          12

• Not all founded issues are against test case
• Test case number means nothing (James Bach)
• New test scripts added almost each day
• Old onces are updated

                                                          © 2010 Cognifide Limited. In commercial confidence only.
Test Planning




                © 2010 Cognifide Limited. In commercial confidence only.
What am I doing here?




                                                               Source: http://arborosa.org/tag/test-planning/
  • Coverage
  • Do not duplicate tests
    during regression
  • Test specific
    functionality




                             © 2010 Cognifide Limited. In commercial confidence only.
Test Strategy / Test Plan




                                                            Source: http://technet.microsoft.com
  • Coverage
  • Do not duplicate tests
    during regression
  • Test specific
    functionality




                             © 2010 Cognifide Limited. In commercial confidence only.
Control and Execution




                 © 2010 Cognifide Limited. In commercial confidence only.
Coverage

                            Requirements




 • Coverage
                            Manaul Tests
 • Do not duplicate tests
   during regression
 • Test specific
   functionality

                             Automated
                               Tests



                                           © 2010 Cognifide Limited. In commercial confidence only.
Execution
                                 Test Suite



  • Coverage                 Test Run
  • Do not duplicate tests
    during regression
  • Test specific
    functionality
                                 Test Case
                                   status


                                              © 2010 Cognifide Limited. In commercial confidence only.
Traceability




  • Coverage
  • Do not duplicate tests
    during regression
  • Test specific
    functionality




                             Source: http://www.tuleap.com
                                               © 2010 Cognifide Limited. In commercial confidence only.
Entry and Exit Criteria




  • Coverage
  • Do not duplicate tests
    during regression
  • Test specific
    functionality




                             © 2010 Cognifide Limited. In commercial confidence only.
Reports




  • Coverage
  • Do not duplicate tests
    during regression
  • Test specific
    functionality




                             © 2010 Cognifide Limited. In commercial confidence only.
Regression




             © 2010 Cognifide Limited. In commercial confidence only.
Retest or regression?




                 Source: http://www.softwaretestinghelp.com

                                                        © 2010 Cognifide Limited. In commercial confidence only.
Automation




             Source: http://blog.jugnoo.com

                                              © 2010 Cognifide Limited. In commercial confidence only.
Metrics

Find what you need




                     © 2010 Cognifide Limited. In commercial confidence only.
Compare and improve




                      © 2010 Cognifide Limited. In commercial confidence only.
Q&A




      © 2010 Cognifide Limited. In commercial confidence only.
PTaQ

Poznań Testing and Quality Group




                              © 2010 Cognifide Limited. In commercial confidence only.
Demo

Test Case Management




                       © 2010 Cognifide Limited. In commercial confidence only.

Test Strategy

  • 1.
    Test Strategy Zbyszek Moćkun © 2010 Cognifide Limited. In commercial confidence only.
  • 2.
    Agenda 1. V Model 2. Agile Way 3. Test Planing 4. Control & Execution 5. Metrics 6. Demo © 2010 Cognifide Limited. In commercial confidence only.
  • 3.
    V Model Perfect world © 2010 Cognifide Limited. In commercial confidence only.
  • 4.
    V Model •Coverage • Do not duplicate tests during regression • Test specific functionality © 2010 Cognifide Limited. In commercial confidence only.
  • 5.
    Preventive or reactive • Coverage • Do not duplicate tests during regression • Test specific functionality © 2010 Cognifide Limited. In commercial confidence only.
  • 6.
    Test Pyramid • Coverage • Do not duplicate tests during regression • Test specific functionality © 2010 Cognifide Limited. In commercial confidence only.
  • 7.
    Agile New way? © 2010 Cognifide Limited. In commercial confidence only.
  • 8.
    Agile Manifesto • Coverage • Do not duplicate tests during regression • Test specific functionality Source: blog.twg.ca © 2010 Cognifide Limited. In commercial confidence only.
  • 9.
    Agile against waterfall Coding,new functionality Requirements Bug fixing Design Coding Testing Testing Release Release Iteration © 2010 Cognifide Limited. In commercial confidence only.
  • 10.
    Agile ways Processes Methodologies • Coverage ATDD Kanban • Do not duplicate tests during regression • Test specific BDD Scrum functionality TDD XP © 2010 Cognifide Limited. In commercial confidence only.
  • 11.
    Continuous Integration • Coverage • Do not duplicate tests during regression • Test specific functionality Source: http://wiki.mxunit.org © 2010 Cognifide Limited. In commercial confidence only.
  • 12.
    Reactive approach? • Coverage • Do not duplicate tests during regression • Test specific functionality © 2010 Cognifide Limited. In commercial confidence only.
  • 13.
    Scripting issues • Scriptedtesting report phase All Executed passed failed Test design phase 80 0 0 0 First day 90 (4 updated) 20 16 4 Second day 95 (10) 75 65 10 Third day 120 (14) 75 73 2 Fourth day 125 (16) 110 98 12 • Not all founded issues are against test case • Test case number means nothing (James Bach) • New test scripts added almost each day • Old onces are updated © 2010 Cognifide Limited. In commercial confidence only.
  • 14.
    Test Planning © 2010 Cognifide Limited. In commercial confidence only.
  • 15.
    What am Idoing here? Source: http://arborosa.org/tag/test-planning/ • Coverage • Do not duplicate tests during regression • Test specific functionality © 2010 Cognifide Limited. In commercial confidence only.
  • 16.
    Test Strategy /Test Plan Source: http://technet.microsoft.com • Coverage • Do not duplicate tests during regression • Test specific functionality © 2010 Cognifide Limited. In commercial confidence only.
  • 17.
    Control and Execution © 2010 Cognifide Limited. In commercial confidence only.
  • 18.
    Coverage Requirements • Coverage Manaul Tests • Do not duplicate tests during regression • Test specific functionality Automated Tests © 2010 Cognifide Limited. In commercial confidence only.
  • 19.
    Execution Test Suite • Coverage Test Run • Do not duplicate tests during regression • Test specific functionality Test Case status © 2010 Cognifide Limited. In commercial confidence only.
  • 20.
    Traceability •Coverage • Do not duplicate tests during regression • Test specific functionality Source: http://www.tuleap.com © 2010 Cognifide Limited. In commercial confidence only.
  • 21.
    Entry and ExitCriteria • Coverage • Do not duplicate tests during regression • Test specific functionality © 2010 Cognifide Limited. In commercial confidence only.
  • 22.
    Reports •Coverage • Do not duplicate tests during regression • Test specific functionality © 2010 Cognifide Limited. In commercial confidence only.
  • 23.
    Regression © 2010 Cognifide Limited. In commercial confidence only.
  • 24.
    Retest or regression? Source: http://www.softwaretestinghelp.com © 2010 Cognifide Limited. In commercial confidence only.
  • 25.
    Automation Source: http://blog.jugnoo.com © 2010 Cognifide Limited. In commercial confidence only.
  • 26.
    Metrics Find what youneed © 2010 Cognifide Limited. In commercial confidence only.
  • 27.
    Compare and improve © 2010 Cognifide Limited. In commercial confidence only.
  • 28.
    Q&A © 2010 Cognifide Limited. In commercial confidence only.
  • 29.
    PTaQ Poznań Testing andQuality Group © 2010 Cognifide Limited. In commercial confidence only.
  • 30.
    Demo Test Case Management © 2010 Cognifide Limited. In commercial confidence only.

Editor's Notes

  • #11 Given a customer previously bought a black sweater from me And I currently have three black sweaters left in stock When he returns the sweater for a refund Then I should have four black sweaters in stock
  • #28 Qa do not improve quality, so measure on qa level .