Agile Testing Best Practices



       Stephen Ritchie
       DCAST March Meeting
       19-Mar-2013
Agenda
  1   Motivation

  2   Principles

  3   Obstacles
         Excella Consulting   -2-
Motivation




             Why Test
             Software?


               Excella Consulting
Chrysler New Yorker




                      Excella Consulting
Motivation




             Why Write
             Automated
               Tests?

               Excella Consulting
Achieve Desired Results




                            •   Software costs too much
                            •   Users demand quality
                            •   Testing takes too long
                            •   New features take too long




                          Excella Consulting
Microscope: Visibility and Insight




                               Excella Consulting
Smoke Detector: Problem Detection




                           Excella Consulting
Weather Satellite: Advanced Warning




                            Excella Consulting
Automated Tests
  Make Sure
Software Works
  As Intended
Agenda
  1   Motivation

  2   Principles

  3   Obstacles
         Excella Consulting   - 12 -
Agile Test Pyramid: Vision




Testers




                                                                   Automated
Engineers



       Google “Agile Test Pyramid”         http://bit.ly/10Qubsy

                             Excella Consulting
Agile Test Pyramid: Reality




       Ice-Cream Cone Anti-Pattern         http://bit.ly/10QpUFG

                              Excella Consulting
Agile Test Quadrants




                       Excella Consulting
Principles




             1. Unit Test

    Developer Tests
 “Intention Checking”

                Excella Consulting
Principles




             2. Automate
                  • Always running
                  • Can't be bargained with
                  • Can't be reasoned with
                  • Tests don't feel pity or
                    remorse or fear
                  • Absolutely will not stop,
                    ever, until you deliver a
                    quality software product.



                Excella Consulting
Principles




             2. Automate
                  • CI Server
                     • Per Commit
                     • Overnight Functional
                     • Smoke Testing
                     • Stability Testing
                     • Performance/Load
                  • Automate Deployments
                  • Project “Dashboard”



                Excella Consulting
Principles




   3. Zero Configuration*

 I can run your tests,
 You can run mine.
       * One-time configuration is okay
                   Excella Consulting
Principles




                  4. Fast

             All tests run
             time efficiently

                   Excella Consulting
Principles




             5. Clear Results
                Pass/Fail
             Focused Test
             Isolates Issue
                  Excella Consulting
Principles




        6. Easy to Maintain

             Conventional
                Brief

                 Excella Consulting
Principles




             7. Timely
Test-Driven Development (TDD)

        Test As-It’s-Built
        Test As-It’s-Fixed
               Excella Consulting
Agenda
  1   Motivation

  2   Principles

  3   Obstacles
         Excella Consulting   - 24 -
Requirements, Requirements, Requirements




                  Obstacle 1
        •Too Abstract
        •Changing
        •Over Specification

                           Excella Consulting
Specification by Example




Examples & Feature Files




                           Excella Consulting
One Primary Assert To Rule Them All




      One Primary Assert




                            Excella Consulting
Time Crunch




               Obstacle 2
              Time Crunch



                  Excella Consulting
Four Ways to Fake Time




                  Fake Time
            • Virtual Machine
            • Isolate The System
            • Inspect The Clock
            • Dummy IClock

                         Excella Consulting
Dummy               Real System
Clock               Clock



        IClock
        Interface




  System Under Test
Database Killed The Integration Test




                   Obstacle 3
                   Database
                   Integration


                             Excella Consulting
Fixture Setup and Data Arrangement




                           Excella Consulting
Fixture Setup and Data Arrangement


• Schema Migration
   – RoundhousE
   – Flyway Db
   – Liquibase
• Data Arrangement
   – DbUnit & NDbUnit




                           Excella Consulting
Now We Have Too Much Test Code




                 Obstacle 4
                 Explosion of
                 Test Code


                         Excella Consulting
Monitor Test Code Growth Rate




            Control Growth
            • Monitor The Ratio
            • Data-Drive Tests
            • Boundary Analysis


                           Excella Consulting
Bad Coupling




      Obstacle 5
  Unhelpful Reuse
  Unhelpful Repetition
  Unhelpful Coupling
               Excella Consulting
In Test Code, Do Repeat Yourself ... Do Repeat Yourself




   Test Code Is Different
    • Avoid Inheritance Reuse
    • Use “Context” Helpers
    • Repetition Is Better Than
      Coupling

                             Excella Consulting
Agenda
  1   Motivation

  2   Principles

  3   Obstacles
         Excella Consulting   - 41 -
Of Course It’s Safe … After You




                             Excella Consulting
Shameless Self Promotion Time!


                                        • “Best Practices” Initiative
                                        • Tips and Tricks
                                        • Generate Ideas
                                           • Quantifying Value
                                           • Biases & Aversions
                                           • Tools & Technologies
                                        • Avoid Making the Same
                                          Mistake Once.




                            Excella Consulting
Code and Slides




• Code:           https://github.com/ruthlesshelp

• Slides:         http://www.slideshare.net/ruthlesshelp




                              Excella Consulting
Contact Me




      •   Email:      stephen.ritchie@excella.com
      •   Twitter:    @ruthlesshelp
      •   Blog:       http://ruthlesslyhelpful.net
      •   LinkedIn:   http://www.linkedin.com/in/sritchie




                           Excella Consulting

Agile Testing Best Practices

  • 1.
    Agile Testing BestPractices Stephen Ritchie DCAST March Meeting 19-Mar-2013
  • 2.
    Agenda 1 Motivation 2 Principles 3 Obstacles Excella Consulting -2-
  • 3.
    Motivation Why Test Software? Excella Consulting
  • 5.
    Chrysler New Yorker Excella Consulting
  • 6.
    Motivation Why Write Automated Tests? Excella Consulting
  • 7.
    Achieve Desired Results • Software costs too much • Users demand quality • Testing takes too long • New features take too long Excella Consulting
  • 8.
    Microscope: Visibility andInsight Excella Consulting
  • 9.
    Smoke Detector: ProblemDetection Excella Consulting
  • 10.
    Weather Satellite: AdvancedWarning Excella Consulting
  • 11.
    Automated Tests Make Sure Software Works As Intended
  • 12.
    Agenda 1 Motivation 2 Principles 3 Obstacles Excella Consulting - 12 -
  • 13.
    Agile Test Pyramid:Vision Testers Automated Engineers Google “Agile Test Pyramid” http://bit.ly/10Qubsy Excella Consulting
  • 14.
    Agile Test Pyramid:Reality Ice-Cream Cone Anti-Pattern http://bit.ly/10QpUFG Excella Consulting
  • 15.
    Agile Test Quadrants Excella Consulting
  • 16.
    Principles 1. Unit Test Developer Tests “Intention Checking” Excella Consulting
  • 17.
    Principles 2. Automate • Always running • Can't be bargained with • Can't be reasoned with • Tests don't feel pity or remorse or fear • Absolutely will not stop, ever, until you deliver a quality software product. Excella Consulting
  • 18.
    Principles 2. Automate • CI Server • Per Commit • Overnight Functional • Smoke Testing • Stability Testing • Performance/Load • Automate Deployments • Project “Dashboard” Excella Consulting
  • 19.
    Principles 3. Zero Configuration* I can run your tests, You can run mine. * One-time configuration is okay Excella Consulting
  • 20.
    Principles 4. Fast All tests run time efficiently Excella Consulting
  • 21.
    Principles 5. Clear Results Pass/Fail Focused Test Isolates Issue Excella Consulting
  • 22.
    Principles 6. Easy to Maintain Conventional Brief Excella Consulting
  • 23.
    Principles 7. Timely Test-Driven Development (TDD) Test As-It’s-Built Test As-It’s-Fixed Excella Consulting
  • 24.
    Agenda 1 Motivation 2 Principles 3 Obstacles Excella Consulting - 24 -
  • 25.
    Requirements, Requirements, Requirements Obstacle 1 •Too Abstract •Changing •Over Specification Excella Consulting
  • 27.
    Specification by Example Examples& Feature Files Excella Consulting
  • 28.
    One Primary AssertTo Rule Them All One Primary Assert Excella Consulting
  • 29.
    Time Crunch Obstacle 2 Time Crunch Excella Consulting
  • 30.
    Four Ways toFake Time Fake Time • Virtual Machine • Isolate The System • Inspect The Clock • Dummy IClock Excella Consulting
  • 31.
    Dummy Real System Clock Clock IClock Interface System Under Test
  • 32.
    Database Killed TheIntegration Test Obstacle 3 Database Integration Excella Consulting
  • 33.
    Fixture Setup andData Arrangement Excella Consulting
  • 34.
    Fixture Setup andData Arrangement • Schema Migration – RoundhousE – Flyway Db – Liquibase • Data Arrangement – DbUnit & NDbUnit Excella Consulting
  • 35.
    Now We HaveToo Much Test Code Obstacle 4 Explosion of Test Code Excella Consulting
  • 36.
    Monitor Test CodeGrowth Rate Control Growth • Monitor The Ratio • Data-Drive Tests • Boundary Analysis Excella Consulting
  • 38.
    Bad Coupling Obstacle 5 Unhelpful Reuse Unhelpful Repetition Unhelpful Coupling Excella Consulting
  • 39.
    In Test Code,Do Repeat Yourself ... Do Repeat Yourself Test Code Is Different • Avoid Inheritance Reuse • Use “Context” Helpers • Repetition Is Better Than Coupling Excella Consulting
  • 41.
    Agenda 1 Motivation 2 Principles 3 Obstacles Excella Consulting - 41 -
  • 42.
    Of Course It’sSafe … After You Excella Consulting
  • 43.
    Shameless Self PromotionTime! • “Best Practices” Initiative • Tips and Tricks • Generate Ideas • Quantifying Value • Biases & Aversions • Tools & Technologies • Avoid Making the Same Mistake Once. Excella Consulting
  • 44.
    Code and Slides •Code: https://github.com/ruthlesshelp • Slides: http://www.slideshare.net/ruthlesshelp Excella Consulting
  • 45.
    Contact Me • Email: stephen.ritchie@excella.com • Twitter: @ruthlesshelp • Blog: http://ruthlesslyhelpful.net • LinkedIn: http://www.linkedin.com/in/sritchie Excella Consulting